Robj/indirection support

[rtjohnso]

No description provided.

[netlify]

✅ Deploy Preview for splinterdb canceled.

Name	Link
🔨 Latest commit	9db4655
🔍 Latest deploy log	https://app.netlify.com/sites/splinterdb/deploys/6447980f1f7a760008f1207e

[rtjohnso]

Reviewing guide

This is a big PR, but it can be logically broken down into a few layers. Feel free to do reviews for each layer separately.

Recommended order to review files:

The mini_allocator layer:

1. mini_allocator.[hc]. The main thing is to add support for sub-page allocations, multi-page allocations, and extent sharing. It also adds support for having a different page type for each batch. It incidentally eliminates pinning support, since the pinning flag didn't actually do anything. Also reorg the mini_allocator struct to cleanly separate constant state (e.g. num_batches) from dynamic state (e.g. num_extents).
2. Optionally: check out routing_filter.c, where the only changes are to match the new mini_allocator APIs.
3. trunk.[hc] are also only minor changes, except that it also forces splits to never split more than the parent can handle.

The blob code:

5. blob.[hc]. Code for accessing blobs.
6. blob_build.[hc]. Code for building and cloning blobs. Blob building code is broken out into a separate file to avoid a circular dependency blob_build -> mini_allocator -> data_internal -> blob. If blob and blob_build were all in a single source file, this would be a cycle.
7. Detour to blob_test.c.

The data adapter layer:

8. data_internal.[hc] and data_blob_build.[hc]. Deblobify things before passing them to user-provided functions. Also various utilities for (de-)blobifying messages, merge_accumulators, etc. data_blob_build is broken out for the same circular dependency reason.

Uses of the data adapter layer to add blob support in the rest of the code:

9. shard_log.[hc]. Handles blobs. Also, has a function to build a blob in the log's mini_allocator. The intention here is that, when the user inserts a large value, the trunk will blobify it in the log's mini_allocator, and then insert the blob into the btree, and then insert a log record with the blob and the generation number from the btree.
10. btree.[hc]. Handles big values in insert and pack.
11. splinterdb.c. De-blobify things before returning them to the user.

The rest:

12. Miscellaneous other files and tests.