From 1230f93affeec72b12f9091a0636132e6e5ddb51 Mon Sep 17 00:00:00 2001 From: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Date: Sun, 10 Nov 2024 18:16:12 -0500 Subject: [PATCH 1/4] improve docs related to storage --- src/SUMMARY.md | 2 - .../advanced_topics/persistent_storage.md | 58 ------------------- src/developers/advanced_topics/views.md | 26 --------- src/developers/sdk/state.md | 56 ++++++++++-------- 4 files changed, 32 insertions(+), 110 deletions(-) delete mode 100644 src/developers/advanced_topics/persistent_storage.md delete mode 100644 src/developers/advanced_topics/views.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 7f203a74..ec528d66 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -33,8 +33,6 @@ - [Advanced Topics](developers/advanced_topics.md) - - [Views](developers/advanced_topics/views.md) - - [Persistent Storage](developers/advanced_topics/persistent_storage.md) - [Contract Finalization](developers/advanced_topics/contract_finalize.md) - [Validators](developers/advanced_topics/validators.md) - [Creating New Blocks](developers/advanced_topics/block_creation.md) diff --git a/src/developers/advanced_topics/persistent_storage.md b/src/developers/advanced_topics/persistent_storage.md deleted file mode 100644 index 01b5e345..00000000 --- a/src/developers/advanced_topics/persistent_storage.md +++ /dev/null @@ -1,58 +0,0 @@ -# Persistent storage - -Validators run the servers and the data is stored in persistent storage. As a -consequence we need a tool for working with persistent storage and so we have -added `linera-db` for that purpose. - -## Available persistent storage - -The persistent storage that is available right now includes `RocksDB`, -`DynamoDB`, and `ScyllaDB`. Each has its own strengths and weaknesses. - -- [`RocksDB`](https://rocksdb.org/): Data is stored on disk and cannot be shared - between shards but is very fast. - -- [`DynamoDB`](https://aws.amazon.com/dynamodb/): Data is stored on a remote - storage, that has to be on AWS. Data can be shared between shards. - -- [`ScyllaDB`](https://www.scylladb.com/): Data is stored on a remote storage. - Data can be shared between shards. - -There is no fundamental obstacle to the addition of other persistent storage -solutions. - -In addition, the `DynamoDB` and `ScyllaDB` have the notion of a table which -means that a given remote location can be used for several completely -independent purposes. - -## The `linera-db` tool - -When operating on a persistent storage some global operations can be required. -The command line tool `linera-db` helps in making them work. - -The functionalities are the following: - -- `list_tables`(`DynamoDB` and `ScyllaDB`): It lists all the tables that have - been created on the persistent storage - -- `initialize`(`RocksDB`, `DynamoDB` and `ScyllaDB`): It initializes a - persistent storage. - -- `check_existence`(`RocksDB`, `DynamoDB` and `ScyllaDB`): It tests the - existence of a persistent storage. If the error code is 0 then the table - exists, if the error code is 1 then the table is absent. - -- `check_absence`(`RocksDB`, `DynamoDB` and `ScyllaDB`): It tests the absence of - a persistent storage. If the error code is 0 then the table is absent, if the - error code is 1 then the table does not exist. - -- `delete_all`(`RocksDB`, `DynamoDB` and `ScyllaDB`): It deletes all the table - of a persistent storage. - -- `delete_single`(`DynamoDB` and `ScyllaDB`): It deletes a single table of a - persistent storage. - -If some error occurs during the operation, then the error code 2 is returned and -0 if everything went fine with the exception of `check_existence` and -`check_absence` for which the value 1 can occur if the connection with the -database was established correctly but the result is not what we expected. diff --git a/src/developers/advanced_topics/views.md b/src/developers/advanced_topics/views.md deleted file mode 100644 index a6cefdc7..00000000 --- a/src/developers/advanced_topics/views.md +++ /dev/null @@ -1,26 +0,0 @@ -# Views - -> Views are a specific functionality of the Linera system that allow to have -> data in memory and then seamlessly flush it to an underlying persistent -> datastore. - -The [full documentation](https://docs.rs/linera-views/latest/linera_views/) is -available on the crate documentation with all functions having examples. - -Concretely, what is provided is the following: - -- A trait `View` that provides `load`, `rollback`, `clear`, `flush`, `delete`. - The idea is that we can do operation on the data and then flush it to the - database storing them. - -- Several other traits `HashableView`, `RootView`, `CryptoHashView`, - `CryptoHashRootView` that are important for computing hash. - -- A number of standard containers: `MapView`, `SetView`, `LogView`, `QueueView`, - `RegisterView` that implement the `View` and `HashableView` traits. - -- Two containers `CollectionView` and `ReentrantCollectionView` that are similar - to `MapView` but whose values are views themselves. - -- Derive macros that allow to implement the above mentioned traits on struct - data types whose entries are views. diff --git a/src/developers/sdk/state.md b/src/developers/sdk/state.md index 9555671e..1e8f96f2 100644 --- a/src/developers/sdk/state.md +++ b/src/developers/sdk/state.md @@ -1,28 +1,32 @@ # Creating the Application State -The `struct` which defines your application's state can be found in -`src/state.rs`. An application state is the data that is persisted on storage +The state of a Linera application consists of onchain data that are persisted between transactions. -To represent our counter, we're going to need a single `u64`. To persist the -counter we'll be using Linera's [view](../advanced_topics/views.md) paradigm. - -Views are a little like an -[ORM](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping), however -instead of mapping data structures to a relational database like Postgres, they -are instead mapped onto key-value stores like [RocksDB](https://rocksdb.org/). +The `struct` which defines your application's state can be found in +`src/state.rs`. To represent our counter, we're going to use a `u64` integer. -In vanilla Rust, we might represent our Counter as so: +While we could use a plain data-structure for the entire application state: ```rust,ignore -// do not use this struct Counter { value: u64 } ``` -However, to persist your data, you'll need to replace the existing `Application` -state struct in `src/state.rs` with the following view: +in general, we prefer to manage persistent data using the concept of "views": + +> [Views](https://docs.rs/linera-views/latest/linera_views/) allow an +> application to load persistent data in memory and stage modifications in a +> flexible way. +> +> Views resemble the persistent objects of an +> [ORM](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) +> framework, except that they are stored as a set of key-values (instead of a +> SQL row). + +In this case, the placeholder `Application` struct in `src/state.rs` should be +replaced by ```rust,ignore /// The application state. @@ -30,21 +34,25 @@ state struct in `src/state.rs` with the following view: #[view(context = "ViewStorageContext")] pub struct Counter { pub value: RegisterView, + // Additional fields here will get their own key in storage. } ``` -and all other occurrences of `Application` in your app. +and the occurrences of `Application` in the rest of the project should be +replaced by `Counter`. -The `RegisterView` supports modifying a single value of type `T`. There are -different types of views for different use-cases, but the majority of common -data structures have already been implemented: +The derive macro `async_graphql::SimpleObject` is related to GraphQL queries +discussed in the [next section](service.md). -- A `Vec` or `VecDeque` corresponds to a `LogView` -- A `BTreeMap` corresponds to a `MapView` if its values are primitive, or to - `CollectionView` if its values are other views; -- A `Queue` corresponds to a `QueueView` +A `RegisterView` supports modifying a single value of type `T`. Other data +structures available in the library +[`linera_views`](https://docs.rs/linera-views/latest/linera_views/) include: -For an exhaustive list refer to the Views -[documentation](../advanced_topics/views.md). +- `LogView` for a growing vector of values; +- `QueueView` for queues; +- `MapView` and `CollectionView` for associative maps; specifically, `MapView` + in the case of static values, and `CollectionView` when values are other + views. -Finally, run `cargo check` to ensure that your changes compile. +For an exhaustive list of the different constructions, refer to the crate +[documentation](https://docs.rs/linera-views/latest/linera_views/). From ab7d0e4dc83a9400e265dc69d1235ab98461797d Mon Sep 17 00:00:00 2001 From: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Date: Sun, 10 Nov 2024 18:20:05 -0500 Subject: [PATCH 2/4] fix outdated API --- src/developers/advanced_topics/contract_finalize.md | 2 +- src/developers/sdk/contract.md | 2 +- src/developers/sdk/service.md | 2 +- src/developers/sdk/testing.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/developers/advanced_topics/contract_finalize.md b/src/developers/advanced_topics/contract_finalize.md index fdb3deb0..d22ea2ec 100644 --- a/src/developers/advanced_topics/contract_finalize.md +++ b/src/developers/advanced_topics/contract_finalize.md @@ -31,7 +31,7 @@ impl Contract for MyContract { type Parameters = (); async fn load(runtime: ContractRuntime) -> Self { - let state = MyState::load(ViewStorageContext::from(runtime.key_value_store())) + let state = MyState::load(runtime.root_view_storage_context()) .await .expect("Failed to load state"); diff --git a/src/developers/sdk/contract.md b/src/developers/sdk/contract.md index 4eeaf4d3..d2e3e0c5 100644 --- a/src/developers/sdk/contract.md +++ b/src/developers/sdk/contract.md @@ -71,7 +71,7 @@ instance: ```rust,ignore async fn load(runtime: ContractRuntime) -> Self { - let state = Counter::load(ViewStorageContext::from(runtime.key_value_store())) + let state = Counter::load(runtime.root_view_storage_context()) .await .expect("Failed to load state"); CounterContract { state, runtime } diff --git a/src/developers/sdk/service.md b/src/developers/sdk/service.md index 41fc4f09..ede1b73e 100644 --- a/src/developers/sdk/service.md +++ b/src/developers/sdk/service.md @@ -73,7 +73,7 @@ it to load the application state: ```rust,ignore async fn load(runtime: ServiceRuntime) -> Self { - let state = Counter::load(ViewStorageContext::from(runtime.key_value_store())) + let state = Counter::load(runtime.root_view_storage_context()) .await .expect("Failed to load state"); Ok(CounterService { state }) diff --git a/src/developers/sdk/testing.md b/src/developers/sdk/testing.md index ba8a0934..d8e96b1b 100644 --- a/src/developers/sdk/testing.md +++ b/src/developers/sdk/testing.md @@ -52,7 +52,7 @@ mod tests { // Check that the state in memory is different from the state in storage assert_ne!( contract.state, - ApplicationState::load(ViewStorageContext::from(runtime.key_value_store())) + ApplicationState::load(runtime.root_view_storage_context()) ); } } From 918830fb0e68c6b5f4333b6aa83003f274ccba3b Mon Sep 17 00:00:00 2001 From: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Date: Mon, 11 Nov 2024 11:59:51 -0500 Subject: [PATCH 3/4] Update src/developers/sdk/state.md Co-authored-by: Andreas Fackler --- src/developers/sdk/state.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/developers/sdk/state.md b/src/developers/sdk/state.md index 1e8f96f2..9370a4e6 100644 --- a/src/developers/sdk/state.md +++ b/src/developers/sdk/state.md @@ -22,7 +22,7 @@ in general, we prefer to manage persistent data using the concept of "views": > > Views resemble the persistent objects of an > [ORM](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) -> framework, except that they are stored as a set of key-values (instead of a +> framework, except that they are stored as a set of key-value pairs (instead of a > SQL row). In this case, the placeholder `Application` struct in `src/state.rs` should be From 7d344fc382833f2beb31f3c4ffe20fb0ff2eb609 Mon Sep 17 00:00:00 2001 From: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Date: Mon, 11 Nov 2024 12:02:24 -0500 Subject: [PATCH 4/4] formatting --- src/developers/sdk/state.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/developers/sdk/state.md b/src/developers/sdk/state.md index 9370a4e6..6ac6f95d 100644 --- a/src/developers/sdk/state.md +++ b/src/developers/sdk/state.md @@ -22,8 +22,8 @@ in general, we prefer to manage persistent data using the concept of "views": > > Views resemble the persistent objects of an > [ORM](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) -> framework, except that they are stored as a set of key-value pairs (instead of a -> SQL row). +> framework, except that they are stored as a set of key-value pairs (instead of +> a SQL row). In this case, the placeholder `Application` struct in `src/state.rs` should be replaced by