-
Notifications
You must be signed in to change notification settings - Fork 8
Milestone plan
The following sections describe milestones for the past and upcoming releases.
The upcoming release is marked with [CURRENT].
Every release has a summary header, followed by milestone list.
Each milestone has a title and a description.
This version should be implemented as an external library and should be used in-process. For now, only writes should be supported, because querying requires indices, which are too complex for the first release. However, write-ahead logging and crash recovery should be supported in this version, because those mechanisms are essential for database integrity and don't require indices. To sum up, this version will serve as a basis for additional functionality.
This action should create consistent, empty database with a valid file format. The caller should supply a logical database name, which, for now, should be a path to the local directory.
The database should be openable by specifying the same logical database name which was used when it has been created.
A method call should take an opaque record from its caller and save the record to the database.
This opaque record will actually be in the FlatBuffers format, but the internal structure of that record is not important in this version.
Writing should not touch the database file directly because WAL will be used.
Each record should be numbered with an internal record ID, which should be a distinct numeric type.
For now, this type should be defined as using RecordId = uint64_t.
Record IDs should start from 1.
The write-ahead log should temporarily store records that are not yet flushed to the database file. The WAL should be kept in-memory and also on-disk. Disk sync period for the WAL should be externally configurable (this provides speed-durability tradeoff).
After a certain number of writes to the WAL has been completed, the WAL should be replayed onto the database file, and then deleted.
During the database startup sequence, database file consistency should be checked before moving to the ready state. That is, any records that are partially-written should be either re-replayed or played pack.
The config file should be read before every other action takes place (even before crash check and recovery). The config parameters should be read by the main database class. Then, this class should use those values for instantiating and configuring other classes. Configuration file should not be passed downstream as-is because parameters should be defined at the class interface level.