Brian "Krow" Aker (krow) wrote,
Brian "Krow" Aker

Release Criteria, Open Source, Thoughts On...

MySQL internally has been debating "what are release criteria" internally. This has me thinking about release criteria and what it means in open source.

Right now I am working my way toward stable versions of the Memecache Engine, and libmemcached.

libmemcached is moving forward very rapidly at this point. Releases occur about once a week at the moment, and I've got enough feedback to know that a number of users (more then 10, less then 50) are pulling heavily off the Mercurial download server. Jedi build their own light-sabers, and Jedi pull open source directly from revision control systems.

A few observations on this:
  • My users are using RSS. This is the first time I have taken a brand new open source project from start to release in years where I have leveraged this. It is making a huge difference. Those users who are looking/needing latest and greatest are tracking this and building off of it.

  • No nightly tarballs. Each "commit" can be pulled as a tarball. I try to keep each push build-able, so you can trust that these will work (though I am sure I am breaking releases from time to time). I've got a number of thoughts on how to improve this (mainly through automated push/test/reject loops)

  • Release often. Watching the graphs I know that each release is carrying more users then before. The rate of growth is about 20% (which is also telling me that there are more memcached users out there then what I think anyone realizes). Here is a hint... it is far from just being Web 2.0.

    I am not worried about stable,ga, or any similar release terms. Right now the current features feel baked. I hit a point where it was useable a while ago, and bugs have come in at a rate that makes me feel I should fix them before extending the library again. Is there anything half baked right now? No, the features feel baked in usage. No complaints have come in for the last couple of weeks.

    So how to pick the time to call it stable? When I find that releases have slowed down because the bug flow has tapered off. Since this is not a mature project with a lot of users, I have to make a call at some point based on the feedback I have (and I am in luck because several large sites have put it to use so I some confidence). When mod_layout went stable for 3.0, I had to let it sit for weeks waiting to see what bugs came in. What day did I pick to call it stable? The day I almost forgot about making it stable. It had been long enough since I had gotten a bug that I had nearly forgotten about it.

    Will I use the terms alpha, beta, and ga? Not for this version. I am going to use the term "stable". If I had a lot more users, or I did not have a couple of users who I felt were representative I would. These users I call bellweathers. Before I ever tag a tree "stable" I ask the bellweathers how it is going. Identifying these people are easy, then send me the most email. As they drop off my radar, I know their needs are met (so why write them? Because if they stopped using it, I want to know why...).

    If you have a project which has a lot of moving parts I do not believe you can use simple terms.


    Because you will not be able to pin point the bellweathers, and there are less features for developers to "spin" on (and if you don't get "spin", then you have not been writing a lot of locking code lately).

    In these situations you have to stick to bug inflow and outflow. You also need "beta" just to give a warning to the developers to stop adding new features. Developers will refine things over and over if you do not send a clear message that it is time to move on. Want to make them move on more quickly? Keep your version cycles short enough that they feel they will also have the ability to get their code in the hands of users.

    In the closed source world you try to seed beta customers, in the open source world seed the community. Finding bellweathers versus letting them find you.

    Another key piece? Completely automate the building of a new release. Count how many steps you had to do with each build, and for every build reduce this by one. Testing is of course the same, but you should solve the testing problem by not writing a line of code that does not come with a test case.

    Release Early, Release Often.
  • Subscribe
    • Post a new comment


      Comments allowed for friends only

      Anonymous comments are disabled in this journal

      default userpic

      Your reply will be screened

      Your IP address will be recorded