I've written another library, Catalina. It started as an example for using the threading library Iris and turned into what I think is a useful library. Catalina is an object data-store for glib and gobject. It provides access through a natural key/value pair interface.
Transparent serialization is supported to and from storage for types that can be stored in GValue's. A tight binary format is provided with the library. It supports basic types such as integers, doubles, floats and strings as well as GObjects in an endian-safe manner. However someone should go double check to call my bluff (and verify its correctness). A JSON serializer would be a quick hack if someone was interested.
In addition to serialization, Catalina supports buffer transformations to and from storage. Included is CatalinaZlibTransform which can apply compression using zlib. It will avoid compression on buffers smaller than the watermark property. This will help on data-sets that are occasionally small and compression would in fact enlarge them.
Catalina is an asynchronous data-store by design. The optimal way of accessing it is the same.
Everything is built upon Trivial DB (TDB) from the samba project. It was chosen over Berkeley DB because of its license. Like Catalina, it is LGPL and does not impose extra restrictions on linking applications such as BDB.
However, the one downside to using TDB is its lack of concurrent transactions. This means that if you have multiple threads doing work and updating storage the transactions would interleave. Since we are using iris, we can use message passing as a way to manage concurrent transactions. (This is done by queuing messages until the commit phase.)
Here is a short example using Vala to asynchronously open, serialize and store a bunch of "Person" GObjects. All the while compressing each buffer with zlib. Don't be scared by the mutex/cond, it's there to negate the need of a main loop.
I intend to add indexes soon, however that is going to take a bit of planning.
So there you have it, my newest hack.
git clone git://git.dronelabs.com/catalina
-- Christian Hergert 2009-06-21