libmemcached, Callbacks faster...

« previous entry | next entry »
Dec. 21st, 2007 | 04:45 pm

So current benchmark run:

generate_hsieh_consistent

Testing generate_data					 6.983 [ ok ]
Testing get_read					 5.545 [ ok ]
Testing delete_generate					 5.143 [ ok ]
Testing generate_data					 7.173 [ ok ]
Testing mget_read					 1.369 [ ok ]
Testing mget_read_result					 1.379 [ ok ]
Testing mget_read_function					 1.308 [ ok ]
Testing mdelete_generate					 1.001 [ ok ]
Testing cleanup					 0.047 [ ok ]



I've added this function:
memcached_return memcached_fetch_execute(memcached_st *ptr,
                                              unsigned int (*callback[])(memcached_st *ptr, memcached_result_st *result, void *context),
                                              void *context,
                                              unsigned int number_of_callbacks
                                              );


Instead of fetching a result, you can specify a callback function to run on each result. It is up to you as to what you do. Sean Chittenden at the hackathon suggested the function array interface. Typically I just have a single callback, but he had a couple of cases where it was more useful to handle it this way. In the callback you can create threads... do whatever you like really. The context param just allows you to keep state. This method of handling results is benchmarking slightly better then the "fetch and use while()".

As an example of how to use it:
unsigned int callback_counter(memcached_st *ptr, memcached_result_st *result, void *context)
{
   unsigned int *counter= (unsigned int *)context;

   *counter= *counter + 1;

   return 0;
}


By passing a pointer to the memcahced_st you are given the option of cloning the memcached_st, and then executing whatever commands you want against the memcached cluster.

Hopefully Patrick will read this and come up with an idea on how to use this from within UDF to extend the callback references in side of MySQL. Without Antony's current work on functions, I can't think of a way to make this reissue SQL commands in the DB (ok... I can... but it would be ugggllly...).

Link | Leave a comment | Add to Memories | Share

Comments {4}

Tanjent

(no subject)

from: tanjent
date: Dec. 22nd, 2007 01:33 am (UTC)
Link

Interface has to compile under plain C, I'm guessing?

Reply | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Dec. 22nd, 2007 01:41 am (UTC)
Link

What language do you need?

Reply | Parent | Thread

Tanjent

C++

from: tanjent
date: Dec. 22nd, 2007 01:56 am (UTC)
Link

I saw the array of function pointers and thought

struct ICallback
{
	unsigned int operator () ( memcached_st *ptr, memcached_result_st *result ) = 0;
};

memcached_return memcached_fetch_execute(memcached_st *ptr, ICallback * callback[], unsigned int number_of_callbacks);

Reply | Parent | Thread

Brian "Krow" Aker

Re: C++

from: krow
date: Dec. 22nd, 2007 03:15 am (UTC)
Link

Is it worth it or not to create a C++ interface? I go back and forth on this one. It is easy enough to just use the C interface in C++ so I don't know if there is enough value or not.

Simple for me to do though.

Reply | Parent | Thread