libmemached, Replication for nodes...

« previous entry | next entry »
Feb. 27th, 2008 | 02:44 am

This was the idea.

With consistent hashing we have data spread out over servers. A loss of a single server removed 1/N of the available cache.

Not bad, but it is also not perfect for everyone. Some users would rather use more hardware and take an approach of fewer losses.

What needed to be done was to replicate the data to multiple node, and handle node failure. This has been on the list for a while :)

Did I get to it? Nope.

Did someone else? Yep.

I got a patch for this a few days ago from a user using memcached that needed it.

So now:

memcached_return enable_replication(memcached_st *memc)
{
uint64_t value;
value= 2;

memcached_behavior_set(memc, MEMCACHED_BEHAVIOR_REPLICAS, &value);
}


All you now need to do is set the number of servers you want to replicate to and you are in business. There is the problem of the split brain/error but not crash of the first node. If users are particularly worried an asynchronous fetch could be done with a compared result. I'm not sure this matters, but I'll probably consider it.

You can pull it from:

http://hg.tangent.org/libmemcached

hg update -C replication

I have it in a separate branch for the moment. 0.17 will be released in the next day, and I do not want this going out without more review.

Thanks to a cross Atlantic flight and a weird sleep schedule I finished this up tonight :)

Link | Leave a comment | Add to Memories | Tell a Friend

Comments {5}

couple other cool things

from: [info]mike503
date: Feb. 27th, 2008 11:25 am (UTC)
Link

As of 3.0.0 in PHP's PECL memcache module they do this same thing (it appears) on the client side by writing to N nodes at once. Pretty cool idea....

http://pecl.php.net/package-changelog.php?package=memcache&release=3.0.0

there's also repcached which is a (new?) daemon that is basically a replicating memcached...

Reply | Thread

Brian "Krow" Aker

Re: couple other cool things

from: [info]krow
date: Feb. 27th, 2008 12:31 pm (UTC)
Link

I saw the daemon... I don't like the extra hop count it will cost me if I used it. I've thought about building a proxy, but I have not made up my mind on it yet.

Reply | Parent | Thread

Kevin Allen Burton

in parallel?

from: [info]burtonator
date: Feb. 27th, 2008 05:40 pm (UTC)
Link

You haven't implemented setMulti yet have you?

I've been meaning to do this with the Java client. I don't use memcached anymore so this really isn't a priority for me.

setMulti would use one frame and send multiple sets to the server at once.

This speeds removes gigabit ethernet latency for multiple sets/puts.

Also, does your replica support work in parallel? if you have many keys you need to set this is going to raise latency up a lot.

I've been known to underestimate gigabit latency in the past and I think it's a common mistake now.

If you implement both of these features it will fall from an O(N) function to O(1)....

The latency for N is small but if you have a large N then you might be screwed ;)

Kevin

Reply | Thread

Re: in parallel?

from: [info]jamesd
date: Feb. 28th, 2008 01:49 am (UTC)
Link

There's also the annoying problem of switches with maximum packet rates that won't support gigabit if you send lots of small packets through them. Combining with setMulti is good to overcome limitations like this before they bite.

Reply | Parent | Thread

Brian "Krow" Aker

Re: in parallel?

from: [info]krow
date: Feb. 28th, 2008 10:29 am (UTC)
Link

There is a set multi if you use the queue'ing option. This was a very early optimization in the library.

Replica is in parallel for send if you are in async mode.

For fetch, it is not yet.

Yet :)


Reply | Parent | Thread