Comments on: 5 more essentials for your programming toolbox

By: Mr. Developer

Mr. Developer — Sat, 22 Mar 2008 02:13:27 +0000

There is a van Emde Boas cache oblivious stratified tree implementation available for Download from http://complearn.org/downloads/libcoveb-1.0.3.tar.gz

By: Tom

Tom — Sat, 15 Mar 2008 17:40:35 +0000

@Peter: I hadn’t thought about the disk IO benefits of skip lists, but that is interesting. In general, when I think about putting a tree on a disk I think about B-Trees. But, I’ll keep skip lists in mind for that.

@pidedings & Christopher: It seems like SQLite is a frequent response to any mention of Berkeley DB. I’ll definitely check it out next time I’m thinking about doing something with the disk. How big of a database have you guys used it for?

By: Ajay

Ajay — Sat, 15 Mar 2008 11:22:26 +0000

Wanted to mention SkipDB http://directory.fsf.org/project/SkipDB/,
which is based on skip lists

By: Ben

Ben — Sat, 15 Mar 2008 11:17:50 +0000

Even more data structures I hadn’t heard of! Looking forward to some more of these programming toolbox posts in the future.

By: pipedings

pipedings — Sat, 15 Mar 2008 05:13:49 +0000

I have used db_recover far too many times. Berkeley DB dies easily with messy results on the stored data.

SQLite seems like a good idea, for now it’s flat files or PostgreSQL for me.

By: Bron

Bron — Sat, 15 Mar 2008 04:47:26 +0000

Cyrus IMAP uses skiplists for a lot of databases (and BDB for others) - I know the skiplist code pretty well now, having debugged it pretty extensively. It is a good format for infrequently changed files, though you do want to rebuild it every so often.

Yes, it’s not particularly cache friendly for sequential access, but it’s nicer than a hash still, and you can iterate through the keys cheaply.

By: paigow

paigow — Sat, 15 Mar 2008 03:53:11 +0000

Diese seite ist genauso interessant wie informativ. Viele Grüße!

By: JN

JN — Sat, 15 Mar 2008 01:51:32 +0000

Another alternative to Berkeley DB for some tasks is cdb, and also Judy for some in memory lookups.

By: Peter

Peter — Fri, 14 Mar 2008 22:23:13 +0000

I’ve used skip lists–they’re great when what you want is like a hash table except that you have to persist it to disk and you want to ‘even out’ the number of disk hits on update.

That is, the problem with a hash table or binary tree is that when you resize it you have to (essentially) recreate it. Even balanced binary trees, when you insert and delete a lot have to be rebalanced. Each rebalance can potentially mean shifting lots of pointers.

When what you really want is for most data to stay inert on disk for each and every operation, this is important. It’s even more the case when the boss says, ‘never read the whole thing into memory’ (we wanted to be good citizens and not swap the user’s memory out).

By: Christopher Cashell

Christopher Cashell — Sat, 08 Mar 2008 06:14:56 +0000

I’ve used Berkeley DB for a number of things in the past, and while it can be extremely useful in certain circumstances, I’ve found myself moving it less and less, and using SQLite where I previously might have used Berkeley DB. And I find myself using it lots of places that I previously wouldn’t used any database just because of the hassle.

If you haven’t used it, I’d highly recommend checking it out.