Comparisons, Mark's blog on global mutexes

« previous entry | next entry »
Jan. 28th, 2009 | 11:22 am

It is not hard to agree with a statement about No new global mutexes!. This has been something we have been preaching since day one when Drizzle was started. I have lost count on how many we have been able to get rid of along the way.

Looking through his blog entry I thought I would comment a bit on some of his points in context for Drizzle:

LOCK_mdl We never had this lock in the first place. It must have been added in the last year. I suspect this is some sort of meta-data lock. If we had a good MVCC in-memory engine sitting around I suspect you could get rid of this lock and similar cache locks (Drizzle's design doesn't currently require this sort of shared information, but we are looking for an in-memory style engine for when we do).

LOCK_plugin We don't have this lock. We also cannot load new modules into the database, and then eject them while being online. Since none of the current modules can do this... we will wait to add this sort functionality when we think it is needed. The way we consider how we will be deployed I am not so sure this matters. I have hard time seeing people load new code to production servers on the fly. I would think you would plan these sorts of upgrades on your network (and if you have scaled at all, you can probably pick times for pieces to come on and offline). A good feature for doing development of modules though, I just don't see how we want to incur this cost for production systems.

LOCK_event_loop We have this one. Recently we took our scheduler and turned it into a plugin. Want to get rid of this lock? Write your own solution. We have abstracted out this problem now so you can focus on solving this problem if you want. While our solution is a little more refined then what is in the original server, we know there is a long way to go on this.

LOCK_open This one is better currently, but we are on the road to removing it completely. This sort of lock exists because of the lack of clear boundaries between some of the legacy engines, like MyISAM and the rest of the kernel. We have stripped a lot of this from the server, but it took a long time to put it in... so it is taking quite some effort to toss it out.

Its good that he brings up tcmalloc, we compile with it by default.

MyISAM... what to do with MyISAM is a question I ask a lot. There is still a group of folks to want it, but its design is pretty old at this point. We have thought to relegate it to just being used for internal temporary tables, but for the moment it is still user accessible. We aren't sure when Maria will be done and we may have to take some action on it before then. Its current design does not fit well with us moving toward using a value object over the current Field object design. We need to push its locking design out of the kernel, and down into the engine.

The HEAP engine needs a rewrite. We breathed a little more light into ours by using a combination of Google and Ebay derived patches, but the truth is... it needs to be replaced. Paul from PBXT has had a number of good ideas on this, and I am hoping he explores them.

When it comes to pool of threads I think the current design misses the point of using libevent. Currently is does not yield on IO block, so in essence all it is doing it keeping you from overwhelming the operating system's scheduler and providing a completion for a given action. For small queries this is fine, but for longer running queries this is not very good (though... most queries we see are pretty short so this part is not a huge concern). It needs to be redesigned to make better use of IO, and this is something we will work on soon. Our replication will not run on the same port as our clients, so we don't have the sort of IO usage clash Mark is referring to in MySQL (we will be using a different port for our replication, which is a structured set of Google Protobuffs BTW with its own library for communication).

I am not really sure what sort of point he is making on GET_LOCK() or LOCK TABLES. Yes they lock... that is the point to both of them. Personally I would prefer to just get rid of LOCK TABLES, but that is because I have never had much use for it (I've been using Innodb ever since we helped Heikki get it stable for Slashdot). GET_LOCK() was a cute hack for turning a database into a network locking daemon. Personally? I would rather pick an open source locking daemon and use it (and no... I am not one of the people who are a fan of adding this to Memcached... but maybe Gearman...).

It is good to see more people talking about the problems around scaling open source databases. The current state of open source databases running on SMP hardware is poor at the moment (with Postgres being the only production database doing it well right now).

We are at an inflection point for open source databases, where all of the projects will need to learn to evolve quickly to really work well on the hardware that will be delivered in the next few years.

Link | Leave a comment | Share

Comments {4}

non-transactional engines

from: anonymous
date: Jan. 28th, 2009 08:26 pm (UTC)
Link

Speaking just for myself, I would strongly prefer if Drizzle could completely abandon the concept of supporting non-transactional engines like MyISAM and HEAP as first-tier. It would simplify a lot of the "weird" MySQL concepts that make life more difficult for people living in a transactional world:
* locking issues such as what you mentioned
* non-transactional DDL
* non-transactional aspects of certain DML
* crash recovery

In my opinion the use cases for MyISAM and non-transactional engines don't fit with the principles and goals of the Drizzle project as laid out on your Launchpad. Not reliable, not scalable for massive concurrency, and adds management complexity. To play devil's advocate... why not for the people who have valid use cases for MyISAM, why not let them continue on with MySQL?

-Ryan Thiessen

Reply | Thread

Brian "Krow" Aker

Re: non-transactional engines

from: krow
date: Jan. 28th, 2009 09:42 pm (UTC)
Link

We can drop HEAP as soon as we can get a replacement for it. Paul has an idea for one from the PBXT design that I am interested in.

MyISAM is one of the list issues... if we can disconnect it from our needs (or hide it), then we can allow someone to bring an analytic engine in that is row locking, or at the very least contains its own locking. We will have an in person summit in April, and I am hoping to bring this up.

Your point about MyISAM and MySQL use is valid...

Reply | Parent | Thread

tcmalloc and drizzle

from: anonymous
date: Jan. 28th, 2009 09:10 pm (UTC)
Link

> Its good that he brings up tcmalloc, we compile with it by default.

Two questions:

Do you leave reasonable support for other mallocs? jemalloc on FreeBSD comes to mind.

Does tcmalloc really help in cases where there are few long lived connections in MySQL/Drizzle? AFAICT, MySQL malloc()ed it's memory at startup and then only in situations where there are a lot of short lived connections.

Sorry if these questions are uninformed.

Best,

-pjr

Reply | Thread

Brian "Krow" Aker

Re: tcmalloc and drizzle

from: krow
date: Jan. 28th, 2009 09:45 pm (UTC)
Link

We will happily take a patch for a jemalloc if someone provides it. If you go to our IRC channel, I suspect someone would be willing to right it up (it should be simple enough to do).

MySQL never had a good memory policy. In the beginning, in 3.23 there was a policy, but malloc is now used a lot in the server design. A memory policy is something we are still looking at, but we have no conclusions at the moment as to how we should be doing it. Once we have a better set of regression benchmarks, we will do more with it.

Reply | Parent | Thread