What does that mean?
There are a lot of forks of memcached. Some commercial, some open source, some that are in house at a variety of companies. They all share a few common concepts though.
The recognition of this lead me to want to create a set of libraries that anyone, could use to build these types of servers quickly. After talking to a number of vendors and users it became pretty apparent that something like this needed to be done. The goal is to make it easy for people to write protocol compatible servers in a way that they can focus on the problems that they are concerned with, without getting bogged down in the details.
Also, for those of us who are client writers, we didn't want to see a tower of babble pop up around the protocol. memcapable, our protocol certification tool, works hand in hand with this to make sure that services that are being built today are compatible with the clients that have been released. No one benefits from a tower of babel.
Users want to know they can use different servers with existing applications, client writers want to be able to support users, and vendors of all variety want end user confidence in their solutions.
In 0.34 "libmemcachedprotocol" has now been added to libmemcached. Here is an example of the structure:
void initialize_interface_v0_handler(void)
{
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_GET]= get_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_SET]= set_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_ADD]= add_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_REPLACE]= replace_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_DELETE]= delete_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_INCREMENT]= arithmetic_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_DECREMENT]= arithmetic_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_QUIT]= quit_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_FLUSH]= flush_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_GETQ]= get_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_NOOP]= noop_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_VERSION]= version_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_GETK]= get_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_GETKQ]= get_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_APPEND]= concat_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_PREPEND]= concat_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_STAT]= stat_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_SETQ]= set_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_ADDQ]= add_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_REPLACEQ]= replace_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_DELETEQ]= delete_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_INCREMENTQ]= arithmetic_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_DECREMENTQ]= arithmetic_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_QUITQ]= quit_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_FLUSHQ]= flush_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_APPENDQ]= concat_command_handler;
interface_v0_impl.interface.v0.comcode[PROTOCOL_BINARY_CMD_PREPENDQ]= concat_command_handler;
}
As you can see, the interface is a designed around callbacks. You supply the callbacks, we handle the protocol parsing, both binary and text. The idea is to let you hook up the memcached protocol to anything. If you download the latest code, or just grab via CMS, aka "bzr clone lp:libmemcached", you can get a feel for it.
Take a look inside of example/interface_v0.c and you will find a simple single threaded memcached server that uses just a simple linked list to store items held in memcached.
Want something more complex?
Take a look inside of example/storage_innodb.c and you can see where we have an example memcached storage system using the Innodb Embedded Engine. Coming soon, a longer article on this!
Wanting to contribute one for Tokyo Cabinet, BDB, SQLite? Please send patches :)
Right now we are still accepting feedback on the interface. While we have shopped it around and gotten some great feedback, I would love to see more. We have not finished writing the encapsulation functions just yet, so we haven't generated API for Ruby, Perl, Python, etc just yet, but I expect to see this happen over the next couple of months.
Have fun with it!