Fields, rows, and just how do I save data in my engine? - Brian "Krow" Aker's Idle Thoughts

Today I was asked about this bit of code that appears in some of the examples MySQL has for engine designers:

memcpy(record, ptr, table->s->null_bytes);

What is exactly going on here?

Whenever we read a row, or write a row the method looks something like this:

int ha_example::write_row(byte *record)

"record" is a byte array of data that makes up the row. The first bytes of it store whether fields are null or not. Values for columns are then stored in order after the null bytes. Most engines never look at "record" directly, and instead use Field objects to look at, or set the current record. To do this you typically setup a for loop like so:

for (Field **field=table->field ; *field ; field++)

Field has both store and value functions that allow you to manipulate the data. When storing a field you typically call:
(*field)->store(string, length, system_charset_info);
(*field)->set_notnull();

Setting a value against field is not enough, you also need to explicitly state that the field is not NULL.

For getting a value out of a Field you can do this:
(*field)->val_str(&attribute,&attribute)

In this example attribute is a MySQL String object (there are also methods for getting the value as numbers, time structures, and geometry types).

So why directly manipulate the buffer? Some engines have no concept of NULL, so they copy the NULL bytes from the record to some sort of internal storage when writing a row. When they need to supply the row, they just copy the NULL back into the record in a wholesale method. Today there is no engine that directly stores "record" as is from Memory.

If you want to see how this is done, I'd recommend looks at the CSV engine. It is by far the most straight forward engine to use when you have a question on how to store and retrieve values from field objects. If you want to look at the basic methods for a Field object, look at field.h inside of the sql directory for the server's source