-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs(store/v2): update store v2 docs (#19502)
Co-authored-by: cool-developer <[email protected]>
- Loading branch information
1 parent
6c9a785
commit 8b83a2e
Showing
4 changed files
with
188 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Store | ||
|
||
The `store` package contains the implementation of store/v2, which is the SDK's | ||
abstraction around managing historical and committed state. See [ADR-065](../docs/architecture/adr-065-store-v2.md) | ||
and [Store v2 Design](https://docs.google.com/document/d/1l6uXIjTPHOOWM5N4sUUmUfCZvePoa5SNfIEtmgvgQSU/edit#heading=h.nz8dqy6wa4g1) for a high-level overview of the design and rationale. | ||
|
||
## Migration | ||
|
||
<!-- TODO --> | ||
|
||
## Pruning | ||
|
||
The `root.Store` is NOT responsible for pruning. Rather, pruning is the responsibility | ||
of the underlying SS and SC layers. This means pruning can be implementation specific, | ||
such as being synchronous or asynchronous. | ||
|
||
## Usage | ||
|
||
The `store` package contains a `root.Store` type which is intended to act as an | ||
abstraction layer around it's two primary constituent components - state storage (SS) | ||
and state commitment (SC). It acts as the main entry point into storage for an | ||
application to use in server/v2. Through `root.Store`, an application can query | ||
and iterate over both current and historical data, commit new state, perform state | ||
sync, and fetch commitment proofs. | ||
|
||
A `root.Store` is intended to be initialized with already constructed SS and SC | ||
backends (see relevant package documentation for instantiation details). Note, | ||
from the perspective of `root.Store`, there is no notion of multi or single tree/store, | ||
rather these are implementation details of SS and SC. For SS, we utilize store keys | ||
to namespace raw key/value pairs. For SC, we utilize an abstraction, `commitment.CommitStore`, | ||
to map store keys to a commitment trees. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,47 @@ | ||
# State Commitment (SC) | ||
|
||
TODO | ||
The `commitment` package contains the state commitment (SC) implementation. | ||
Specifically, it contains an IAVL v1 implementation of SC and the necessary types | ||
and abstractions to support other SC backends, as well as supporting general integration | ||
into store/v2, specifically the `RootStore` type. | ||
|
||
A foremost design goal is that SC backends should be easily swappable, i.e. not | ||
necessarily IAVL. To this end, the scope of SC has been reduced, it must only: | ||
|
||
* Provide a stateful root app hash for height h resulting from applying a batch | ||
of key-value set/deletes to height h-1. | ||
* Fulfill (though not necessarily provide) historical proofs for all heights < `h`. | ||
* Provide an API for snapshot create/restore to fulfill state sync requests. | ||
|
||
Notably, SC is not required to provide key iteration or value retrieval for either | ||
queries or state machine execution, this now being the responsibility of state | ||
storage. | ||
|
||
An SC implementation may choose not to provide historical proofs past height `h - n` | ||
(`n` can be 0) due to the time and space constraints, but since store/v2 defines | ||
an API for historical proofs there should be at least one configuration of a | ||
given SC backend which supports this. | ||
|
||
## Benchmarks | ||
|
||
See this [section](https://docs.google.com/document/d/1l6uXIjTPHOOWM5N4sUUmUfCZvePoa5SNfIEtmgvgQSU/edit#heading=h.7l0i621y5vgm) for specifics on SC benchmarks on various implementations. | ||
|
||
## Pruning | ||
|
||
<!-- TODO --> | ||
|
||
## State Sync | ||
|
||
State commitment (SC) does not have a direct notion of state sync. Rather, | ||
`snapshots.Manager` is responsible for creating and restoring snapshots of the | ||
entire state. The `snapshots.Manager` has a `CommitSnapshotter` field which is | ||
fulfilled by the `CommitStore` type, specifically it implements the `Snapshot` | ||
and `Restore` methods. | ||
|
||
## Usage | ||
|
||
Similar to the `storage` package, the `commitment` package is designed to be used | ||
in a broader store implementation, i.e. it fulfills the role of the SC backend. | ||
Specifically, it provides a `CommitStore` type which accepts a `store.RawDB` and | ||
a mapping from store key, a string meant to represent a single module, to a `Tree`, | ||
which reflects the commitment structure. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,110 @@ | ||
# State Storage (SS) | ||
|
||
TODO | ||
The `storage` package contains the state storage (SS) implementation. Specifically, | ||
it contains RocksDB, PebbleDB, and SQLite (Btree) backend implementations of the | ||
`VersionedDatabase` interface. | ||
|
||
The goal of SS is to provide a modular storage backend, i.e. multiple implementations, | ||
to facilitate storing versioned raw key/value pairs in a fast embedded database, | ||
although an embedded database is not required, i.e. you could use a replicated | ||
RDBMS system. | ||
|
||
The responsibility and functions of SS include the following: | ||
|
||
* Provide fast and efficient queries for versioned raw key/value pairs | ||
* Provide versioned CRUD operations | ||
* Provide versioned batching functionality | ||
* Provide versioned iteration (forward and reverse) functionality | ||
* Provide pruning functionality | ||
|
||
All of the functionality provided by an SS backend should work under a versioned | ||
scheme, i.e. a user should be able to get, store, and iterate over keys for the | ||
latest and historical versions efficiently. | ||
|
||
## Backends | ||
|
||
### RocksDB | ||
|
||
The RocksDB implementation is a CGO-based SS implementation. It fully supports | ||
the `VersionedDatabase` API and is arguably the most efficient implementation. It | ||
also supports versioning out-of-the-box using User-defined Timestamps in | ||
ColumnFamilies (CF). However, it requires the CGO dependency which can complicate | ||
an app’s build process. | ||
|
||
### PebbleDB | ||
|
||
The PebbleDB implementation is a native Go SS implementation that is primarily an | ||
alternative to RocksDB. Since it does not support CF, results in the fact that we | ||
need to implement versioning (MVCC) ourselves. This comes with added implementation | ||
complexity and potential performance overhead. However, it is a pure Go implementation | ||
and does not require CGO. | ||
|
||
### SQLite (Btree) | ||
|
||
The SQLite implementation is another CGO-based SS implementation. It fully supports | ||
the `VersionedDatabase` API. The implementation is relatively straightforward and | ||
easy to understand as it’s entirely SQL-based. However, benchmarks show that this | ||
options is least performant, even for reads. This SS backend has a lot of promise, | ||
but needs more benchmarking and potential SQL optimizations, like dedicated tables | ||
for certain aspects of state, e.g. latest state, to be extremely performant. | ||
|
||
## Benchmarks | ||
|
||
Benchmarks for basic operations on all supported native SS implementations can | ||
be found in `store/storage/storage_bench_test.go`. | ||
|
||
At the time of writing, the following benchmarks were performed: | ||
|
||
```shell | ||
name time/op | ||
Get/backend_rocksdb_versiondb_opts-10 7.41µs ± 0% | ||
Get/backend_pebbledb_default_opts-10 6.17µs ± 0% | ||
Get/backend_btree_sqlite-10 29.1µs ± 0% | ||
ApplyChangeset/backend_pebbledb_default_opts-10 5.73ms ± 0% | ||
ApplyChangeset/backend_btree_sqlite-10 56.9ms ± 0% | ||
ApplyChangeset/backend_rocksdb_versiondb_opts-10 4.07ms ± 0% | ||
Iterate/backend_pebbledb_default_opts-10 1.04s ± 0% | ||
Iterate/backend_btree_sqlite-10 1.59s ± 0% | ||
Iterate/backend_rocksdb_versiondb_opts-10 778ms ± 0% | ||
``` | ||
|
||
## Pruning | ||
|
||
Pruning is an implementation and responsibility of the underlying SS backend. | ||
Specifically, the `StorageStore` accepts `store.PruneOptions` which defines the | ||
pruning configuration. During `ApplyChangeset`, the `StorageStore` will check if | ||
pruning should occur based on the current height being committed. If so, it will | ||
delegate a `Prune` call on the underlying SS backend, which can be defined specific | ||
to the implementation, e.g. asynchronous or synchronous. | ||
|
||
|
||
## State Sync | ||
|
||
State storage (SS) does not have a direct notion of state sync. Rather, `snapshots.Manager` | ||
is responsible for creating and restoring snapshots of the entire state. The | ||
`snapshots.Manager` has a `StorageSnapshotter` field which is fulfilled by the | ||
`StorageStore` type, specifically it implements the `Restore` method. The `Restore` | ||
method reads off of a provided channel and writes key/value pairs directly to a | ||
batch object which is committed to the underlying SS engine. | ||
|
||
## Non-Consensus Data | ||
|
||
<!-- TODO --> | ||
|
||
## Usage | ||
|
||
An SS backend is meant to be used within a broader store implementation, as it | ||
only stores data for direct and historical query purposes. We define a `Database` | ||
interface in the `storage` package which is mean to be represent a `VersionedDatabase` | ||
with only the necessary methods. The `StorageStore` interface is meant to wrap or | ||
accept this `Database` type, e.g. RocksDB. | ||
|
||
The `StorageStore` interface is an abstraction or wrapper around the backing SS | ||
engine can be seen as the the main entry point to using SS. | ||
|
||
Higher up the stack, there should exist a `root.Store` implementation. The `root.Store` | ||
is meant to encapsulate both an SS backend and an SC backend. The SS backend is | ||
defined by this `StorageStore` implementation. | ||
|
||
In short, initialize your SS engine of choice and then provide that to `NewStorageStore` | ||
which will further be provided to `root.Store` as the SS backend. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters