?

Log in

No account? Create an account

Archive, AIO, First Runs

« previous entry | next entry »
Sep. 4th, 2007 | 08:34 am

What you see in the graph is a comparison of the read() versus
aio_read() in Archive. This is using Fedora Core 7. The test is
pretty simple. Read 10,000 rows, 1000 times per user. Increase the
number of users. No aggregate function was run, I did not see where
that was really needed for this test (aggregate functions/queries are
the main SELECT type used against Archive tables).

From the graph it is pretty obvious that aio_read() makes a
difference. I am going to run this later on the T1000 and see what
happens there (I've got other tests running on it right now so no
comparison for the moment...).

The open question for me now is how this compares to my hand written
AIO method. This code is pretty much ready to go (it won't be in MySQL 5.1,
it will got in the next version (which is either 5.2 or 6.0)). The
burnin test has been running overnight so I am pretty sure at this
point that it is safe. I've added a new program in the storage/
archive/ directory called archive_performance. You can run it to get
an idea of whether or not AIO or read() is faster on your own system
(though I would really recommend just using mysqlslap so that you can
see what happens in concurrency).

I have a suspicion that AIO is broken on OS X. When trying to
compile my test case there I get all sorts of warnings from the aio.h
header.

Why does the graph look different from what I usually post? I
upgraded to iWork 08 over the weekend. It has a new spreadsheet
called Numbers. I've got a couple of annoyances with it already (like
the graph is not one object and that it didn't install itself so
that .csv opens to it). Otherwise it feels much faster then Excel,
and it is visually a lot prettier. Good first 1.0 product.

archive_aio.jpg

Link | Leave a comment | Share

Comments {4}

Поисковик-затейник

(no subject)

from: itman
date: Sep. 4th, 2007 03:55 pm (UTC)
Link

Phew, it is a double increase. But, how large was the database in relation to the memory size? Was it cached or is it more like a "cold-start" test when most data is read directly from hard drive?

Reply | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Sep. 4th, 2007 03:59 pm (UTC)
Link

Cold start. I blow the entire database between runs (I designed slap to do that... I don't believe in warm starts).

Archive has no block or row cache (it is a long explanation but basically I don't believe in them for stream formats). The OS filecache is used though if the OS is capable of it.

Reply | Parent | Thread

Поисковик-затейник

(no subject)

from: itman
date: Sep. 4th, 2007 04:13 pm (UTC)
Link

You said that the program "reads 10,000 rows, 1000 times per user". Does it read the same 10 thousand rows in each trial?

Reply | Parent | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Sep. 4th, 2007 04:32 pm (UTC)
Link

Yep, so more then likely the OS has the file in its own cache (though there is no way to be sure). In either case, what the improvement is about is fetching while the other block is being decompressed and used.

Reply | Parent | Thread