Brian "Krow" Aker (krow) wrote,
Brian "Krow" Aker
krow

AIO, Sparc/Intel, Archive, maybe posix after all...

I'm looking at the final patch for adding Asynchronous IO to the
Archive engine in MySQL. The current patch relies on an AIO library I
wrote myself a while ago to play around with the problem. It works,
but it adds more ongoing threads to MySQL, and I have mixed feelings
about doing that in user space (though its all the rage among engine
writers).

Last night I got it in my head to try out the posix AIO callers. In
part because I've never used them, and in part because I like the
thought of reusing code that someone else wrote (aka support my own
AIO means well... supporting it...).

I decided to do a test using 1024 byte buffers on a 1 gig file:

First Intel:

read()
real 0m0.789s
user 0m0.041s
sys 0m0.744s

aio()
real 0m2.487s
user 0m3.067s
sys 0m1.798s


Now Sun:


read()
real 0m21.764s
user 0m1.597s
sys 0m20.121s

aio()
real 0m31.098s
user 0m38.586s
sys 0m42.436s


Eww.... AIO is slower, much slower with 1024 byte buffers. Archive
uses a much larger read buffer, using its default (48K) things change
the other way around:

Intel:

read()
real 0m0.789s
user 0m0.053s
sys 0m0.733s

aio()
real 0m0.360s
user 0m0.382s
sys 0m0.335s


Sparc:

read()
real 0m15.493s
user 0m0.065s
sys 0m8.493s

aio()
real 0m15.242s
user 0m15.511s
sys 0m9.625s


So the AIO system does perform better with a larger buffer on Linux,
and for some reason the Intel 8way is faster then the Sparc T1000
(though they have nearly identical disks). I reran the test several
times, and for reads Linux was always faster with AIO, and for
Solaris it was sometimes faster and sometimes slower. This surprises
me and makes me think there must be some secret to enabling AIO on
Solaris that I am not aware of.

For Solaris I tried upping the buffer to 128K, and the AIO and normal
tied. Moving to something ridiculous like 512K, just made it slower
again (which is not a surprise). Something has to be wrong, or AIO on
Solaris must just suck. Most likely we can blame this on me though.

I am aware of the aio_read() posix calls over the kernel
implementation in Linux. I hate writing to OS specifics though. I
know that each vendor likes to trick out their own particular calls
because they disagree with the standard, but for application design
this creates a portability nightmare.

I need to benchmark my homebuilt solution now against the above. If
it turns out that my homebuilt is either slower, or only a little
faster, I am just going to go with the posix AIO calls and assume
vendors are putting effort into making AIO calls faster. I've got
better things to do with my time then hack AIO implementations. I can
make it an option that can be enabled/disabled by users so they can
try it out themselves.
Subscribe
  • Post a new comment

    Error

    Comments allowed for friends only

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 10 comments