Disqus - Latest Comments for kramerica

Re: Building for OS X - MongoDB - 10gen Confluence

kramerica — Thu, 20 Oct 2011 18:27:20 -0000

Here is how I built on Mac OS 10.6.8

brew install boost pcre++ scons v8
cd Workspaces
git clone git://github.com/mongodb/mo...
cd mongo
scons . --usev8 --prefix=$HOME/Installs install
export PATH=$HOME/Installs/bin:$PATH
mongod

Re: Brizzled: A Scala build tool

kramerica — Thu, 23 Jul 2009 23:35:50 -0000

I've been thinking more about this topic over the past few days, and I wanted to toss out this thought and see what other folks thought. Ivy is a good example of a focused component that can be used in conjunction with other tools (would like to have a better interface other than XML). What if all the components in the build system were actually independent components. Not plugins, but actual independent components. If this approach was used for build systems, continuous integration systems, issue tracking systems, and wiki, I think we'd wind up with a collection of power single focused components that can be combined in many different ways to solve a far wider range of problems and people would be able to assemble solutions that are much more effective than lock in to a specific language and description model. In a proof-of-concept project I was working on during 2000-2002 (off and on), I was applying the same concept but with web apps, and that each application is independent but I could assemble a custom system per the organizations requirements: http://qbal.mozdev.org/oldQ... and http://qbal.mozdev.org/ (I shut this effort down as other business activities got in the way). I'm going to restart it again, but this time, I'm going to start with these 2 areas. First will be a release definition tool that is generalized and its role is to provide a mechanism to define a release, its components, branches, automation properties, build properties, etc... from this automation is configuration, initialized and started-up. The next problem area I want to start on is the build system problem. What I'm looking to do is integration, and developer where needed a very loosely couple tool box of build components, i.e Ivy is a very good starting point, although I'd love for it to have an external DSL other than XML as the interface.

Re: Valley job-clingers: Why you need to get fired

kramerica — Wed, 22 Jul 2009 10:03:31 -0000

the grass may not be greener on the other side, but the sky may be bluer

Re: Brizzled: A Scala build tool

kramerica — Mon, 20 Jul 2009 12:27:27 -0000

Great statements. But those are the symptoms that Martin talks about, not root cause. In fact many of the continuous integration things Martin writes about... are too late in the game for many commercial software companies. What Martin proposes in his writings is to create a symptom identification process via continuous integration, when in fact what you want to do is create a scenario where integration rarely fails. To do this, requires the right source code control tools, engineering discipline in the form of pre-integration practices, and a well designed build and test system. We were doing things like this back in the mid-80's on the HP-UX OS teams (specifically IND). At Sun in the early to mid-90's on some teams we ran a similar model. I've been lucky enough to work with distributed source code control tools in one form or another since the about 1987. They allow you to have workflows that are much simpler to use than the old-school cvs, subversion, and perforce. I've been creating java build systems since 1995, initially using make, then Ant, and more recently Maven. I thought nothing could get worse than Ant until I started working with Maven (i started drinking heavily). Scons I thought was going in the right direction, but really is no different than using groovy. I've read about a number of other approach, but they too fall into the same trap of too much coding in the build system. What is needed is a good external DSL with different thinking about the fundamental problems than in previous build systems

But build systems are generally a mess because of a lack of good examples available to learn from and people not dealing with the fact of a bad data model requires oddities to occur to operate on that data model

1) bad data model. generally speaking native source trees in commercial companies are poorly designed and rarely take into account the entire product life cycle. this leads to the problems Martin and others run into needing all kinds of 1-off solutions to produce a build. For example, suppose nothing changes in the source code, but the dependencies change, can the build system handle that without branching or modifying the build system

2) Ant and Maven are weakness is due to an very very narrow usage focus. Because of this focus and because they are the status quo, people are not learning much by using them, only that they do not do what is needed (and mostly what is needed is a reaction to a symptom). So in many ways because our build tools are weak so is our understanding of build systems.

3) Many java projects cannot fit in the maven model and the ant stuff is way to brutal in xml and also its really a poor design to begin with. I worked on one team that developing software to run on set top boxes (interestingly enough... javas initial target) and it was so painful using Ant to implement the build system. There is one good thing I can say about Maven and Ivy... the dependency management is good.

In regards to your latest statement about having to use actual code, I agree 100%. How that is done is the difference between what generally ends up being very sloppy build system or elegant (which should translate into easy to maintain and extend). I like to work with teams to completely separate targets, properties, and code which allows users and maintainers to focus on the specific logic of the build system versus, configuration of the build system, or supporting tasks (code snippers), instead of unraveling

In summary, when I define/implement build system, my main focus is removing the odd, local build weirdness. Mostly this oddness occurs for these reasons:

* the build system was difficult to understand and the developer needed to deliver their technology (primary focus) and in a flash put in something to keep progress moving forward

* the build system is much like a wiki and that is it often not owned by anyone, no specs, no best-practices, its just a unmanaged container that somehow produces a result

* the person assigned to developing the build system operated in much the same way agile teams do, and that is with a narrow focus on the goals of the team in the immediate and near future, and then iteratively add support. but generally fundamental flaws are established with initial instance and we pile on more tasks for the build system in rapid fire.

* garbage in, process garbage, maybe something useful as output.

The best thing we could do, is as a community establish a place on the web to describe the problems, symptoms, exceptions and approach to this problem. The solution would not be a build system, but instead of collection if libraries, utils (a toolbox), from that build systems would be implemented.

How simple can a native build system be... All targets are dynamic as follows and maintain in its own file lets call that targets.mk. Then we'll need another file to contain all the properties, properties.mk. The approach with that file is to define 3 categories of properties: little tuple, big tuple and common properties. The tuple type properties are used to group properties along platform, toolchain and other variants. The next file we'll next is responsible from resloving the properties into the variables used in the tarket.mk file. What we've done is separate the build system into 3 major pieces (targets, properties, resolver). This approach requires very few 'make' statements/commands. It support unlimited toolchains, chipsets, languages, etc... This is what I call a variant/relocatable based build system. Almost impossible to do in a simple manner with Ant and Maven.

$(OBJDIR)%.c: (SRCDIR)%.o
$(DOT_C_DOT_O_RULE)

$(LIBRARY): $(OBJS)
$(LIBRARY_RULE)

$(EXECUTABLE): $(OBJS)
$(EXECUTABLE_RULE)

As soon as I see a target like libfoo.a I no that build system is rigid and will not respond to have the company does business, where they do it, etc.. without someone hacking on it.

So... A build system implementers, why don't we remove the odd and build weirdness? And why don't we work with build technology designers to have a better perspective of the problem?

Regards -- Kramer

Re: Brizzled: A Scala build tool

kramerica — Mon, 20 Jul 2009 11:11:28 -0000

dang nabit -- i was hoping to see some progress in build systems. in my opinion what the java community did was throw the baby out with the bath water when the ignored the essence of 'make'. what is really missing is a good approach to describe a build recipe. Unfortunately the java world is web centric when it comes to build system, so if you company is developing suites of products, unbundled products, products targeted at multiple channels: OEMs, VARs, ISV, IHV, Web, Direct these modern build tools are actually poor engineering projects. I just glanced at SBT and I'll need to muck with it. As soon as I see a build system that is not able to describe lists in a simple manner, I wonder what hoops the author will come up with to deal with variants. For example in 'make' I could do things like

LIST = foo
LIST += bar

when I start to see folks implementing build systems like scons, and rake that use the native language syntax to express the recipe, I become concerned that they are really scripting the recipe and not defining the recipe

One of my favorite things about 'make' is the multiple pass thru the recipe to support interpolated macros, and exactly the opposite of ant/maven and that is mutable properties.

Every build system I've worked on where the developers started going outside the declarative and into procedural coding, the build system turned into a rats nest that nobody else on the team wanted to have anything to deal with.

having said all that, I'd love to see a build system implemented in scala, but designed/implemented with right reasons behind them. Often all I see from the author is a description of symptoms they are solving. What is needed is a paradigm change, not just more rehashing of the same old junk