Tokyo Cabinet == Tokyo Engine!

« previous entry | next entry »
Nov. 4th, 2007 | 03:31 pm

Mikio Hirabayash released his TokyoCabinet DB. So I took a minute this morning to play around with it!


mysql> INSTALL PLUGIN tokyo SONAME 'libtokyo_engine.so';
Query OK, 0 rows affected (0.00 sec)

mysql> show plugins;
+------------------+--------+--------------------+-----------------------+---------+
| Name | Status | Type | Library | License |
+------------------+--------+--------------------+-----------------------+---------+
| binlog | ACTIVE | STORAGE ENGINE | NULL | GPL |
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
| ARCHIVE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| CSV | ACTIVE | STORAGE ENGINE | NULL | GPL |
| FEDERATED | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MEMORY | ACTIVE | STORAGE ENGINE | NULL | GPL |
| InnoDB | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MRG_MYISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MyISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| SKELETON | ACTIVE | STORAGE ENGINE | libskeleton_engine.so | BSD |
| MEMCACHE | ACTIVE | STORAGE ENGINE | libmemcache_engine.so | GPL |
| memcache_servers | ACTIVE | INFORMATION SCHEMA | libmemcache_engine.so | GPL |
| TOKYO | ACTIVE | STORAGE ENGINE | libtokyo_engine.so | BSD |
+------------------+--------+--------------------+-----------------------+---------+
14 rows in set (0.00 sec)

mysql>

:)

TokyoCabinet:
http://tokyocabinet.sourceforge.net/

My morning's hack:
http://hg.tangent.org/TokyoEngine

I had to modify his engine to make this work. There were no C++ declaration around the include headers, and the tokyocabinet.pc file has a dangling variable in its libs.
I know nothing about performance at the moment... this was just me playing around for a bit. Tokyo Cabinet is his successor to QDBM and the work was sponsored by Mixi.jp (aka... think Facebook for Japan fro the ones of you who never study what the rest of the world is doing).

Link | Leave a comment | Share

Comments {9}

Egor Egorov

(no subject)

from: egorfine
date: Nov. 5th, 2007 07:46 am (UTC)
Link

What is good about TokyoCabinet?

Reply | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Nov. 5th, 2007 07:54 am (UTC)
Link

The one interesting thing about it is that it will work in memory... and it will be able to store blobs. An in memory blob store might be nice. I'll benchmark it against the built in memory engine to see if it is any faster.

This only took me an hour to do, and I did it more to see if someone else would pick it up.

For in memory blob store, I am starting to use the memcached engine (and a new version of it will be out this week).

Reply | Parent | Thread

(no subject)

from: tmaesaka
date: Nov. 5th, 2007 09:24 am (UTC)
Link

Phew, you take things into action pretty quick. Didn't we talk about TC just last night? heh.

Reply | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Nov. 5th, 2007 03:37 pm (UTC)
Link

This stuff is pretty easy for me to write, and the TokyoCabinet code is very easy to follow. You are looking at less then two hours of my time putting together engines does not really take a lot out out me.


Reply | Parent | Thread

(no subject)

from: jamesd
date: Nov. 6th, 2007 12:57 am (UTC)
Link

OK, you're a storage engine pimp. :) So, what is the optimum combination of storage engines for this (insert query with 15 unions and 40 tables)? :) The Support team is going to have real fun with this someday. :)

Reply | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Nov. 6th, 2007 08:21 am (UTC)
Link

Yeah well... that is why we will need to get a bit more exotic :)

Reply | Parent | Thread

shared access to a storage engine

from: uzikalfaddim
date: Oct. 25th, 2009 09:26 pm (UTC)
Link

I've played with qdbm. It worked very well but the code was pretty grungy. Hope his new stuff is better.

I'm interested in having all the speed advantages of qdbm/cabinet for a high performance application and also being able to use mysql to poke around/report/maintain my data. Any chance it would work if you did this?

tokyo cabinet -> tokyo tyrant -> your storage engine plugin -> mysql

I could also send requests to tokyo tyrant from my other processes.

Reply | Thread

Brian "Krow" Aker

Re: shared access to a storage engine

from: krow
date: Oct. 26th, 2009 03:44 am (UTC)
Link

Google for Blitzdb.

Reply | Parent | Thread

Re: shared access to a storage engine

from: uzikalfaddim
date: Oct. 26th, 2009 12:23 pm (UTC)
Link

Thanks Brian :)
No downloads available yet though... :(

Reply | Parent | Thread