Brian "Krow" Aker's Idle Thoughts - mysqlslap

Today was fairly productive, I pushed the last piece of code into 5.1 for mysqlslap that I wanted to make sure would make the 5.1 beta freeze date (January 4th for the those playing at home).

So what does mysqlslap do? Basically its a load generation application. It has three stages, create schema, load data, and a general purpose run.

If can take queries from the command line for each stage or it can read in queries from a file. Patrick Galbraith added a primitive operation for it to generate data for all three if you need be.

What you then do is specify bits likes --concurrency="1,8,64,1024" and it will run the tests at each concurrency level (--concurrency handles the general purpose stage while --concurrency-load will handle the load stage). You are not required to run all of the stages so its safe to just run general queries against existing data sets.

You can also run --iterations and run tests multiple times.

Options like --engine="mysiam, archive" can be used to run the entire framework against multiple engines. This was the main reason I wrote it, because I have been looking at concurrency scaling issues among the different engines.

The feature I added today lets me take three computers and run a timed test against a server using multiple machines. One mysqlslap client acts as the master. That master communicates with many slaves using shared disk to make sure the operations occur at the same time. All of the slaves use a --slave to set themselves up for the run. Its far from foolproof and I need to look at coming up with a better two phase protocol to make sure that the tests happen correctly. If I get a better idea in the next day or so I may modify it to make it work a bit better.

I should really add a --csv option so that I can just copy and paste the output of the application to a spreadsheet. Maybe I will just do that during some phone call tomorrow. Should be a fairly brainless operation to add.

And the nice thing about all of this is that since I put it in the server's repository I won't have a to dig up a tool like this every time I need to run a concurrency test. Much of the work on this started because I got tired of trying to make other people's products/ projects work. I went through :

1) A windows tool that even once I installed windows on a backup laptop worked poorly.
2) Some Java application that crashed constantly.
3) A perl application that during the install of the packages tried to completely overwrite my perl install (way to go CPAN tool!)
4) An application written in C that even once I finished fixing all of the Makefiles it leaked memory all over the place. Wasn't that stable for large loads :(

Open Source, the mother of necessity.