OidaDB

OidaDB is a project to reinvent the modern database. Supporting dynamic typing, geospatial R-Trees, arbitrary B-Trees, hardware accelerated queries, horizontal scalling, Machine Learning and AI interfacing, and much, much, much more.

OidaDB “v0” was recently built tested and now OidaGroup inteneds to approach the “v1” of the database. v1 will have much of the same expectations although with a unique (patent pending) layered design.

Specification

• See Version v0 file specification
• …And its companion process locking structure
• Finally, the the v1 specification

Product

OidaDB, at the end of the day, is just a library (liboidadb.so). It is not a stand-alone executable unlike many other database software. Software you build will link against this library and that software will become the “host” depending on how you use this database.

Building

Building the library, documentation, and other support software is done via makefile for releasing builds. But Cmake is used for a easier development cycle with jetbrains products, so these 2 build systems exist in parallel for now. With that being said, use the makefiles to start out with. How the makefiles function is sharply defined in Developing but as a quick overview:

• All build products are put in folders with the sacred name of build/.
• This repository has more than just the library to worry about, so multiple folders have makefiles inside of them.

Now to the actual commands (you can execute these in any order, no difference):

• testing the library - make test
• building the library - make liboidadb/build/liboidadb.so
• packaging the manual - make man/build/man.tar.xz

Note: there’s a chance that the above commands are out of date if they’re not working, see =makefile= in the repo’s root for an exact break down of the build targets

Developing

C source repository conventions

• all files that are in the same directory are files that are known to share a Namespace.
• A Namespace can add a header file into include/oidadb-internal so they can export their symbols to be used by other Namespaces, or, include/oidadb to export their symbols to the library at whole (with much scrutaniy).
• All other header files (that are not named after the directory) are known as utility headers, or private headers. These files cannot be included in any file outside of the namespace.
• At no point should a .c file include another .c file.
• Each namespace may have its own rules as to what other namespaces its allowed to include so keep your eyes open.

makefiles

You’ll see makefile laid out throughout the repository. Makefiles will typically execute their immeidiate subdirectories’ makefiles. All makefiles must follow these standard rules:

• makefiles must be designed to run independantly: in otherwords, a given makefile is agnostic to its depth in the repository.
• All files generated by all make files must be done inside a sacred build folder of the same directory.

CMake

CMake is used exclusively for local developing. All packaging, releasing, and serious testing is always done through make.

Array conventions

If we are dealing with arrays, we have some special naming conventions to make things a bit easier to understand. Take the below example of a structure:

struct items {

    int   itemc;
	int   itemq;
	item *itemv;
	item *itemm;

};

As you can see we name everything that deals with an array of objects the same, but with different suffixes depending on its function:

• -c - Array count, this is the amount of valid elements in the array.
• -q - Array capacity, sometimes arrays are allocated beyond the amount they are currently holding to allow for growth. capacities will always be larger or equal to counts. Most cases, the capacity value is not needed as the count will only be needed.
• -v - Normal array pointer. This is the pointer to where you can find the array of objects.
• -m - this is the the same thing as -v except the pointer is page-aligned. This means this pointer can be used as the first argument with mmap(2) utilizing MAP_FIXED. I can’t think of any practical reason you’d have both -v and -m for a given array, so the above example is unrealistic in that sense.

make test

To test OidaDB, you must only need to run make test. This will do the following:

1. It will compile all C code in the OidaDB as if it were building the library, however it will not link the object files. These object files will be piled into build/tests.
2. It will then glob all files using the c.src/tests/*_t.c syntax. Each one of these files is expected to be a build target. For each one of these files, it will then compile it and link it against the object files that were generated in step 1. The resulting build for each *_t.c file will be built into an executable found in build/tests, these files are known as testexecs.
3. For each testexec that is built, it will then run that testexec. If the testexec does not return a 0 error code, its considered a pass. If a testexec returns non-0, or fails to build, the test is considered a failure.

All test files in c.src/tests that have the scheme ##-(namespace)_##_t.c will have a main defined in them and ready to build. The first set of numbers is the priority of which it should be tested (the lower the higher the priority), the default value is 20. The second set of numbers is arbitary (to seperate test with same namespace/priority).

Most of these test do NOT require a full library build. They will pull in the .h files on their own in some casees for unit testing.

Each test will have a exit code of 1 if they failed and and all errornous ouput will be exclusively through stderr. stdout will be used liberally.

TEST IDEA: Create 2 database files: execute the exact same jobs between them. Then close the databases’ hosts. 0-out the database id (in their respective header) and check to make sure their sha1s are equal. The purpose is to prove that the same jobs will always provide the same changes to the file. And the databases’ only difference is their unique db id.

Development/Debug builds

Right now all development and debug builds are handled through CMake. This is because the tool I use to develop and test is the clion IDE, and clion is retarded with makefiles so I’m forced to use cmake.

make release - Release Builds

Non-debug, release, live, whatever builds are handled through makefile. Run make release to attempt to build, test, and package a release.

Links

• Kanban: https://oidadb.youtrack.cloud/

Dependencies

Here I will list what packages you need to develop this project as well as what packages you need to deploy this into a release.

Each dependency needs a justification as well as a version that it has been tested with. Be ready to pop open git blame on these items to deduce the age of each dependency if it so happens not to work anymore.

Release

• libopenssl - cryptographic functions.

Development

• emacs-nox - for compiling manual .org files. emacs 27.1 seems to work.
• gcc - for compiling C. gcc 10.2.1-6 seems to work fine.
• m4 - for various macro stuff for generating some summeries. m4 is never used in actual code generation. 1.4.18.
• make - for building production-ready builds. 4.3
• cmake - for making development-ready builds. 3.22.1
• libopenssl-dev - headers and libraries for compiling against.
• ldd - glibc. 2.35.