?

Log in

No account? Create an account

Queue Engine, A little more effort...

« previous entry | next entry »
Nov. 12th, 2007 | 10:21 am

This is not one of my big focuses, but I did put together a queue engine many months ago (http://krow.livejournal.com/530752.html).

The semantic idea was that when you SELECT data the person who selects data gets a limited period of time to see the data.

An example is if I inserted 5 records, and then did a "SELECT ... LIMIT 3" I would get three rows. If another user selected data after my data and did a "SELECT ... LIMIT 3" they would only get two records (and this would be the 4th and 5th record).

Right now the period of "exclusion" for reading a record is 60 seconds. Anyone can always fetch and delete a record based on primary key (and you would want to delete records before the 60 seconds are up if you want them removed from the queue). Using triggers you can use a queue table to control for instance the data in an Innodb table.

For those who need a queue service this should hopefully make a lot of sense, for anyone else it probably sounds pretty confusing :)

Here is the tarball I put together:
http://download.tangent.org/queue_engine-0.2.tar.gz

From revision control:
http://hg.tangent.org/queue_engine/

This is not a project I am planning on putting a lot of energy in. It is highly useful though, so I will happily except patches (at the moment the code is only safe on a single CPU system BTW). I've had a number of people comment on it/play with it, so I thought it would be good to clean up the code a little for others.

While I have a need for this, it is not a pressing need.

One final thought, think of this as a service for other engines. I do not see a reason at the moment to make the engine durable, with triggers you really can use any engine as the durable store. The original plan for "partitioning" was to make it a similar service. Not all new storage engines need to be stand alone "engines", I believe there is market place for extensions to current engines.

Link | Leave a comment | Share

Comments {3}

...

from: burtonator
date: Nov. 15th, 2007 06:04 am (UTC)
Link

I think the thing that bothers me here is that as long as no additional DML statements have been executed any additional SELECTs should be deterministic.

Maybe this shouldn't be a storage engine?

Reply | Thread

Brian "Krow" Aker

Re: ...

from: krow
date: Nov. 15th, 2007 03:48 pm (UTC)
Link

I've got two on going ideas:

1) Like partitioning, this could be wrapped around any engine (I call these engine services).

2) Perhaps use hints in the SQL longterm.

At the moment I am just playing with this, I don't know that it will become real.

Reply | Parent | Thread

Re: ...

from: burtonator
date: Nov. 16th, 2007 12:11 am (UTC)
Link

I hear ya about the experiment.....

Also, I blogged about the idea of "storage engine services" as I've been thinking about the same thing lately.

http://feedblog.org/2007/11/15/mysql-storage-engine-services/

Reply | Parent | Thread