Log in

No account? Create an account

libmemached, Replication for nodes...

« previous entry | next entry »
Feb. 27th, 2008 | 02:44 am

This was the idea.

With consistent hashing we have data spread out over servers. A loss of a single server removed 1/N of the available cache.

Not bad, but it is also not perfect for everyone. Some users would rather use more hardware and take an approach of fewer losses.

What needed to be done was to replicate the data to multiple node, and handle node failure. This has been on the list for a while :)

Did I get to it? Nope.

Did someone else? Yep.

I got a patch for this a few days ago from a user using memcached that needed it.

So now:

memcached_return enable_replication(memcached_st *memc)
uint64_t value;
value= 2;

memcached_behavior_set(memc, MEMCACHED_BEHAVIOR_REPLICAS, &value);

All you now need to do is set the number of servers you want to replicate to and you are in business. There is the problem of the split brain/error but not crash of the first node. If users are particularly worried an asynchronous fetch could be done with a compared result. I'm not sure this matters, but I'll probably consider it.

You can pull it from:


hg update -C replication

I have it in a separate branch for the moment. 0.17 will be released in the next day, and I do not want this going out without more review.

Thanks to a cross Atlantic flight and a weird sleep schedule I finished this up tonight :)

Link | Leave a comment |

Comments {5}

in parallel?

from: burtonator
date: Feb. 27th, 2008 05:40 pm (UTC)

You haven't implemented setMulti yet have you?

I've been meaning to do this with the Java client. I don't use memcached anymore so this really isn't a priority for me.

setMulti would use one frame and send multiple sets to the server at once.

This speeds removes gigabit ethernet latency for multiple sets/puts.

Also, does your replica support work in parallel? if you have many keys you need to set this is going to raise latency up a lot.

I've been known to underestimate gigabit latency in the past and I think it's a common mistake now.

If you implement both of these features it will fall from an O(N) function to O(1)....

The latency for N is small but if you have a large N then you might be screwed ;)


Reply | Thread

Re: in parallel?

from: jamesd
date: Feb. 28th, 2008 01:49 am (UTC)

There's also the annoying problem of switches with maximum packet rates that won't support gigabit if you send lots of small packets through them. Combining with setMulti is good to overcome limitations like this before they bite.

Reply | Parent | Thread

Brian "Krow" Aker

Re: in parallel?

from: krow
date: Feb. 28th, 2008 10:29 am (UTC)

There is a set multi if you use the queue'ing option. This was a very early optimization in the library.

Replica is in parallel for send if you are in async mode.

For fetch, it is not yet.

Yet :)

Reply | Parent | Thread