Brian "Krow" Aker (krow) wrote,
Brian "Krow" Aker

Building a Storage Engine: Reading Data

Now that you can load an engine, we are going to look at reading data. For this we will need to implement three methods. We will also need the following schema:

CREATE TABLE `services` (
`a` varchar(125) NOT NULL DEFAULT '',
`b` text

Storage engines provide "handler" objects that are used to read/write/update tables. They inherit from the handler class defined in sql/handler.h.
The file holds the implementation of the handler for the Skeleton engine. Once a handler object is created it is cached and can be used for different tables that a particular storage engine controls. MySQL uses the open() and close() methods of handler object to tell the handler what table it should currently work with.

open() and close() are not called for each usage of a table, or each usage of a transaction.

For reading data we are going to skip worrying about open() and close() and just concentrate on reading data. For this we will implement just the rnd_init(), rnd_next, and rnd_end() methods. These are used to handle table scans.

For rnd_init() we are going to open up /etc/services and read from it:

int ha_skeleton::rnd_init(bool scan)

if (!(file_stream= my_fopen("/etc/services", O_RDONLY, MYF(0))))
line_number= 0;


The my_fopen() comes from mysys, which is the MySQL portable runtime library (think APR, but for MySQL). For a complete description of it, you can look in the mysys/ directory of any MySQL distribution (there is an ongoing effort to add Doxygen to the server's source code so hopefully these will get documented in detail in the future). In the example above we use my_fopen() to open the /etc/services file. We also set line_number to equal zero. We will increment it with each read.

The variables file_stream and line_number are private variables that are declared in ha_skeleton.h.

Now we read our data:

int ha_skeleton::rnd_next(byte *buf)
char buffer[2048];
Field **field=table->field;

my_bitmap_map *org_bitmap= dbug_tmp_use_all_columns(table, table- >read_set);


if (!(fgets(buffer, 2048, file_stream)))


(*field)->store(buffer, strlen(buffer), system_charset_info);

dbug_tmp_restore_column_map(table->read_set, org_bitmap);


By doing an fgets() we will grab one line of the file at a time. We then take that line and store it in a Field object.

Field objects represent the columns in the database. What we do in the example is walk through the array of Field objects. The array of Field objects are stored in the order of creation. In our case we are just looking at the first and second object. We store data via the store() method on the field object. We also make sure the object is set not null (and yes... this is odd and will be fixed... it would make a better interface if when you called store() you just set the notnull method directly).

DBUG_ENTER() and DBUG_RETURN() are used in the creation of debug trace files, and are only enabled in debug builds. The my_bitmap_map can be used to determine if particular fields are needed to be accessed. The main purpose for this is to allow engines to not return costly attributes, like blobs, if they are not needed.

The rnd_next() method will be called until we have exhausted the lines in the files. At that point we will return HA_ERR_END_OF_FILE to signal that we have no more rows to return.

Then we will close the file in rnd_end():

int ha_skeleton::rnd_end()

my_fclose(file_stream, MYF(0));


While it is required to implement rnd_init() and rnd_next(), it is not required to implement rnd_end(). In our case we need to close the file we opened, so we implement it.

Finally to be safe we are going to throw an error during a create() method call if more then two fields were declared:

int ha_skeleton::create(const char *name, TABLE *table_arg,
HA_CREATE_INFO *create_info)

if (table_arg->s->fields != 2)


Notice that we are using table_arg. It contains the definition of the table as we created it. The variable "name" holds the name of the tables (or an encoded version depending on the type of engine that was declared (more on this later)). "create_info" will contain the arguments from the "CREATE TABLE (...)" that are MySQL extensions.

At this point you should know how to open a file and return its contents via a scan on the table.

You can find the example source code to play with here under "Chapter 2":

The previous entries in this series:
Getting The Skeleton to Compile
Tags: mysql, storage engines
  • Post a new comment


    Comments allowed for friends only

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 1 comment