SuperServer future – think about it

From Jim Starkey:
I said in a post this morning that I thought a single code base for
superserver and classic was a mistake. I thought it might be useful to
elaborate.

Classic is defined as a set of independent server processors sharing a
database file (or files) synchronized by a lock manager. The lock
manager, in turn, is built out of shared memory and system level mutexes
or semaphores. Superserver is a single process that has exclusive
control of the database file or files.

Classic is a difficult and demanding environment because of the limited
communication between the database processes. For the system to work at
all, database pages must be written in strict “careful write” order with
deadlock free update algorithms. Changes in metadata must be carefully
propagated among the servers by either indicators either in the lock
table or the on-disk structure to force an engine to refresh portions of
its internal data structures from system tables. This make some
operations impossible, the release of pointer pages, for example, and
others awkward, like the propagation of primary keys. The lock table is
itself difficult and demanding since all pointers must be self relative
and every process must be capable of repairing damage caused by a failed
process. The lock table must be shared globally, and is very difficult
— or impossible — to extend dynamically.

Superserver, in theory, can relax almost all of constraints. There is
no need for an external lock table, hence no need to system wide shared
memory and global mutexes or semaphores or their administration. And,
unlike classic, there is only one set of internal metadata, so there
needn’t be any constraint on metadata update. The synchronization
between in-memory structures should be trivial. However, since the
algorithms and strategies of the superserver are common to classic, the
superserver inherits most of the warts and limitations of classic.

More importantly, superserver should be able to use a serial log
strategy where transaction specific data is written to a serial log
flushed at commit time. The serial log, though itself non-buffered
writes, eliminates the tradeoff between performance (database in a
buffered file system) and robustness (database in a non-buffered file
system), though at the expense of a post-crash recovery mechanism. Most
modern database systems are built around serial log. A serial log
implementation is unthinkable for classic, but could be adapted to
superserver.

Firebird is currently caught in an uncomfortable niche. Purpose built
embedded databases are smaller, more reliable, and less subject to
expert administration than Firebird. On the other than, the major
transaction databases can pump transactions must faster than Firebird.
Firebird is good in the middle, but the middle get smaller every year as
the embedded database engines get better and the big guys come down in
cost and administrative complexity.

I suggest that Firebird start giving serious thought to the future of
superserver. If superserver is going to have a place in the future,
perhaps the time is coming to fork the engine into core modules,
classic, and superserver. Much could be shared, but freed from the
constraints of classic, superserver could evolve into a high performance
transaction engine with a gratifyingly small administrative footprint.