diff --git a/.github/workflows/md-link-checker.yml b/.github/workflows/md-link-checker.yml index dbc16dbd1de7..38560ea8c80c 100644 --- a/.github/workflows/md-link-checker.yml +++ b/.github/workflows/md-link-checker.yml @@ -7,12 +7,8 @@ jobs: markdown-link-check: runs-on: ubuntu-latest steps: -<<<<<<< HEAD - - uses: actions/checkout@v3 -======= - uses: actions/checkout@v4 - run: cd docs && sh ./pre.sh ->>>>>>> 2efafee65 (chore: rename develop to learn (#17821)) - uses: gaurav-nelson/github-action-markdown-link-check@1.0.15 with: folder-path: "docs" diff --git a/docs/.gitignore b/docs/.gitignore index 8e5b072eb088..28443e3b7b15 100644 --- a/docs/.gitignore +++ b/docs/.gitignore @@ -24,8 +24,7 @@ docs/docs/packages/02-collections.md docs/docs/packages/03-orm.md docs/user/run-node/04-rosetta.md docs/build/migrations/02-upgrading.md -<<<<<<< HEAD -docs/develop/advanced/17-autocli.md +docs/learn/advanced/17-autocli.md # Misc .DS_Store @@ -37,6 +36,3 @@ docs/develop/advanced/17-autocli.md npm-debug.log* yarn-debug.log* yarn-error.log* -======= -docs/learn/advanced/17-autocli.md ->>>>>>> 2efafee65 (chore: rename develop to learn (#17821)) diff --git a/docs/docs/build/building-modules/06-keeper.md b/docs/docs/build/building-modules/06-keeper.md index d98a94a2f0ea..399ec648cac8 100644 --- a/docs/docs/build/building-modules/06-keeper.md +++ b/docs/docs/build/building-modules/06-keeper.md @@ -47,14 +47,9 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keepe Let us go through the different parameters: * An expected `keeper` is a `keeper` external to a module that is required by the internal `keeper` of said module. External `keeper`s are listed in the internal `keeper`'s type definition as interfaces. These interfaces are themselves defined in an `expected_keepers.go` file in the root of the module's folder. In this context, interfaces are used to reduce the number of dependencies, as well as to facilitate the maintenance of the module itself. -<<<<<<< HEAD:docs/docs/build/building-modules/06-keeper.md -* `storeKey`s grant access to the store(s) of the [multistore](../../develop/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. -* `cdc` is the [codec](../../build/develop/advanced/05-encoding.md) used to marshall and unmarshall structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated. -======= * `storeKey`s grant access to the store(s) of the [multistore](../../learn/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. * `cdc` is the [codec](../../learn/advanced/05-encoding.md) used to marshall and unmarshall structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. * The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated. ->>>>>>> 2efafee65 (chore: rename develop to learn (#17821)):docs/build/building-modules/06-keeper.md Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../learn/beginner/00-app-anatomy.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them. diff --git a/docs/docs/develop/intro/00-what-is-sdk.md b/docs/docs/develop/intro/00-what-is-sdk.md deleted file mode 100644 index 616d3316921f..000000000000 --- a/docs/docs/develop/intro/00-what-is-sdk.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -sidebar_position: 1 ---- - -# What is the Cosmos SDK - -The [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is an open-source framework for building multi-asset public Proof-of-Stake (PoS) blockchains, like the Cosmos Hub, as well as permissioned Proof-of-Authority (PoA) blockchains. Blockchains built with the Cosmos SDK are generally referred to as **application-specific blockchains**. - -The goal of the Cosmos SDK is to allow developers to easily create custom blockchains from scratch that can natively interoperate with other blockchains. We envision the Cosmos SDK as the npm-like framework to build secure blockchain applications on top of [CometBFT](https://github.com/cometbft/cometbft). SDK-based blockchains are built out of composable [modules](../../build/building-modules/00-intro.md), most of which are open-source and readily available for any developers to use. Anyone can create a module for the Cosmos SDK, and integrating already-built modules is as simple as importing them into your blockchain application. What's more, the Cosmos SDK is a capabilities-based system that allows developers to better reason about the security of interactions between modules. For a deeper look at capabilities, jump to [Object-Capability Model](../advanced/10-ocap.md). - -## What are Application-Specific Blockchains - -One development paradigm in the blockchain world today is that of virtual-machine blockchains like Ethereum, where development generally revolves around building decentralized applications on top of an existing blockchain as a set of smart contracts. While smart contracts can be very good for some use cases like single-use applications (e.g. ICOs), they often fall short for building complex decentralized platforms. More generally, smart contracts can be limiting in terms of flexibility, sovereignty and performance. - -Application-specific blockchains offer a radically different development paradigm than virtual-machine blockchains. An application-specific blockchain is a blockchain customized to operate a single application: developers have all the freedom to make the design decisions required for the application to run optimally. They can also provide better sovereignty, security and performance. - -Learn more about [application-specific blockchains](./01-why-app-specific.md). - -## Why the Cosmos SDK - -The Cosmos SDK is the most advanced framework for building custom application-specific blockchains today. Here are a few reasons why you might want to consider building your decentralized application with the Cosmos SDK: - -* The default consensus engine available within the Cosmos SDK is [CometBFT](https://github.com/cometbft/cometbft). CometBFT is the most (and only) mature BFT consensus engine in existence. It is widely used across the industry and is considered the gold standard consensus engine for building Proof-of-Stake systems. -* The Cosmos SDK is open-source and designed to make it easy to build blockchains out of composable [modules](../modules). As the ecosystem of open-source Cosmos SDK modules grows, it will become increasingly easier to build complex decentralized platforms with it. -* The Cosmos SDK is inspired by capabilities-based security, and informed by years of wrestling with blockchain state-machines. This makes the Cosmos SDK a very secure environment to build blockchains. -* Most importantly, the Cosmos SDK has already been used to build many application-specific blockchains that are already in production. Among others, we can cite [Cosmos Hub](https://hub.cosmos.network), [IRIS Hub](https://irisnet.org), [Binance Chain](https://docs.binance.org/), [Terra](https://terra.money/) or [Kava](https://www.kava.io/). [Many more](https://cosmos.network/ecosystem) are building on the Cosmos SDK. - -## Getting started with the Cosmos SDK - -* Learn more about the [architecture of a Cosmos SDK application](./02-sdk-app-architecture.md) -* Learn how to build an application-specific blockchain from scratch with the [Cosmos SDK Tutorial](https://cosmos.network/docs/tutorial) diff --git a/docs/docs/develop/advanced/00-baseapp.md b/docs/docs/learn/advanced/00-baseapp.md similarity index 100% rename from docs/docs/develop/advanced/00-baseapp.md rename to docs/docs/learn/advanced/00-baseapp.md diff --git a/docs/docs/develop/advanced/01-transactions.md b/docs/docs/learn/advanced/01-transactions.md similarity index 100% rename from docs/docs/develop/advanced/01-transactions.md rename to docs/docs/learn/advanced/01-transactions.md diff --git a/docs/docs/develop/advanced/02-context.md b/docs/docs/learn/advanced/02-context.md similarity index 100% rename from docs/docs/develop/advanced/02-context.md rename to docs/docs/learn/advanced/02-context.md diff --git a/docs/docs/develop/advanced/03-node.md b/docs/docs/learn/advanced/03-node.md similarity index 100% rename from docs/docs/develop/advanced/03-node.md rename to docs/docs/learn/advanced/03-node.md diff --git a/docs/docs/develop/advanced/04-store.md b/docs/docs/learn/advanced/04-store.md similarity index 100% rename from docs/docs/develop/advanced/04-store.md rename to docs/docs/learn/advanced/04-store.md diff --git a/docs/docs/develop/advanced/05-encoding.md b/docs/docs/learn/advanced/05-encoding.md similarity index 100% rename from docs/docs/develop/advanced/05-encoding.md rename to docs/docs/learn/advanced/05-encoding.md diff --git a/docs/docs/develop/advanced/06-grpc_rest.md b/docs/docs/learn/advanced/06-grpc_rest.md similarity index 100% rename from docs/docs/develop/advanced/06-grpc_rest.md rename to docs/docs/learn/advanced/06-grpc_rest.md diff --git a/docs/docs/develop/advanced/07-cli.md b/docs/docs/learn/advanced/07-cli.md similarity index 100% rename from docs/docs/develop/advanced/07-cli.md rename to docs/docs/learn/advanced/07-cli.md diff --git a/docs/docs/develop/advanced/08-events.md b/docs/docs/learn/advanced/08-events.md similarity index 100% rename from docs/docs/develop/advanced/08-events.md rename to docs/docs/learn/advanced/08-events.md diff --git a/docs/docs/develop/advanced/09-telemetry.md b/docs/docs/learn/advanced/09-telemetry.md similarity index 100% rename from docs/docs/develop/advanced/09-telemetry.md rename to docs/docs/learn/advanced/09-telemetry.md diff --git a/docs/docs/develop/advanced/10-ocap.md b/docs/docs/learn/advanced/10-ocap.md similarity index 100% rename from docs/docs/develop/advanced/10-ocap.md rename to docs/docs/learn/advanced/10-ocap.md diff --git a/docs/docs/develop/advanced/11-runtx_middleware.md b/docs/docs/learn/advanced/11-runtx_middleware.md similarity index 100% rename from docs/docs/develop/advanced/11-runtx_middleware.md rename to docs/docs/learn/advanced/11-runtx_middleware.md diff --git a/docs/docs/develop/advanced/12-simulation.md b/docs/docs/learn/advanced/12-simulation.md similarity index 100% rename from docs/docs/develop/advanced/12-simulation.md rename to docs/docs/learn/advanced/12-simulation.md diff --git a/docs/docs/develop/advanced/13-proto-docs.md b/docs/docs/learn/advanced/13-proto-docs.md similarity index 100% rename from docs/docs/develop/advanced/13-proto-docs.md rename to docs/docs/learn/advanced/13-proto-docs.md diff --git a/docs/docs/develop/advanced/15-upgrade.md b/docs/docs/learn/advanced/15-upgrade.md similarity index 100% rename from docs/docs/develop/advanced/15-upgrade.md rename to docs/docs/learn/advanced/15-upgrade.md diff --git a/docs/docs/develop/advanced/16-config.md b/docs/docs/learn/advanced/16-config.md similarity index 100% rename from docs/docs/develop/advanced/16-config.md rename to docs/docs/learn/advanced/16-config.md diff --git a/docs/docs/develop/advanced/_category_.json b/docs/docs/learn/advanced/_category_.json similarity index 100% rename from docs/docs/develop/advanced/_category_.json rename to docs/docs/learn/advanced/_category_.json diff --git a/docs/docs/develop/advanced/baseapp_state-begin_block.png b/docs/docs/learn/advanced/baseapp_state-begin_block.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state-begin_block.png rename to docs/docs/learn/advanced/baseapp_state-begin_block.png diff --git a/docs/docs/develop/advanced/baseapp_state-checktx.png b/docs/docs/learn/advanced/baseapp_state-checktx.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state-checktx.png rename to docs/docs/learn/advanced/baseapp_state-checktx.png diff --git a/docs/docs/develop/advanced/baseapp_state-commit.png b/docs/docs/learn/advanced/baseapp_state-commit.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state-commit.png rename to docs/docs/learn/advanced/baseapp_state-commit.png diff --git a/docs/docs/develop/advanced/baseapp_state-deliver_tx.png b/docs/docs/learn/advanced/baseapp_state-deliver_tx.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state-deliver_tx.png rename to docs/docs/learn/advanced/baseapp_state-deliver_tx.png diff --git a/docs/docs/develop/advanced/baseapp_state-initchain.png b/docs/docs/learn/advanced/baseapp_state-initchain.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state-initchain.png rename to docs/docs/learn/advanced/baseapp_state-initchain.png diff --git a/docs/docs/develop/advanced/baseapp_state-prepareproposal.png b/docs/docs/learn/advanced/baseapp_state-prepareproposal.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state-prepareproposal.png rename to docs/docs/learn/advanced/baseapp_state-prepareproposal.png diff --git a/docs/docs/develop/advanced/baseapp_state-processproposal.png b/docs/docs/learn/advanced/baseapp_state-processproposal.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state-processproposal.png rename to docs/docs/learn/advanced/baseapp_state-processproposal.png diff --git a/docs/docs/develop/advanced/baseapp_state.png b/docs/docs/learn/advanced/baseapp_state.png similarity index 100% rename from docs/docs/develop/advanced/baseapp_state.png rename to docs/docs/learn/advanced/baseapp_state.png diff --git a/docs/docs/develop/beginner/00-app-anatomy.md b/docs/docs/learn/beginner/00-app-anatomy.md similarity index 100% rename from docs/docs/develop/beginner/00-app-anatomy.md rename to docs/docs/learn/beginner/00-app-anatomy.md diff --git a/docs/docs/develop/beginner/01-tx-lifecycle.md b/docs/docs/learn/beginner/01-tx-lifecycle.md similarity index 100% rename from docs/docs/develop/beginner/01-tx-lifecycle.md rename to docs/docs/learn/beginner/01-tx-lifecycle.md diff --git a/docs/docs/develop/beginner/02-query-lifecycle.md b/docs/docs/learn/beginner/02-query-lifecycle.md similarity index 100% rename from docs/docs/develop/beginner/02-query-lifecycle.md rename to docs/docs/learn/beginner/02-query-lifecycle.md diff --git a/docs/docs/develop/beginner/03-accounts.md b/docs/docs/learn/beginner/03-accounts.md similarity index 100% rename from docs/docs/develop/beginner/03-accounts.md rename to docs/docs/learn/beginner/03-accounts.md diff --git a/docs/docs/develop/beginner/04-gas-fees.md b/docs/docs/learn/beginner/04-gas-fees.md similarity index 100% rename from docs/docs/develop/beginner/04-gas-fees.md rename to docs/docs/learn/beginner/04-gas-fees.md diff --git a/docs/docs/develop/beginner/_category_.json b/docs/docs/learn/beginner/_category_.json similarity index 100% rename from docs/docs/develop/beginner/_category_.json rename to docs/docs/learn/beginner/_category_.json diff --git a/docs/learn/intro/00-overview.md b/docs/docs/learn/intro/00-overview.md similarity index 100% rename from docs/learn/intro/00-overview.md rename to docs/docs/learn/intro/00-overview.md diff --git a/docs/docs/develop/intro/01-why-app-specific.md b/docs/docs/learn/intro/01-why-app-specific.md similarity index 100% rename from docs/docs/develop/intro/01-why-app-specific.md rename to docs/docs/learn/intro/01-why-app-specific.md diff --git a/docs/docs/develop/intro/02-sdk-app-architecture.md b/docs/docs/learn/intro/02-sdk-app-architecture.md similarity index 100% rename from docs/docs/develop/intro/02-sdk-app-architecture.md rename to docs/docs/learn/intro/02-sdk-app-architecture.md diff --git a/docs/docs/develop/intro/03-sdk-design.md b/docs/docs/learn/intro/03-sdk-design.md similarity index 100% rename from docs/docs/develop/intro/03-sdk-design.md rename to docs/docs/learn/intro/03-sdk-design.md diff --git a/docs/docs/develop/intro/_category_.json b/docs/docs/learn/intro/_category_.json similarity index 100% rename from docs/docs/develop/intro/_category_.json rename to docs/docs/learn/intro/_category_.json diff --git a/docs/docusaurus.config.js b/docs/docusaurus.config.js index 19e52494a462..ce1aff326752 100644 --- a/docs/docusaurus.config.js +++ b/docs/docusaurus.config.js @@ -217,9 +217,9 @@ const config = { toExtensions: ["html"], createRedirects(existingPath) { return [ - existingPath.replace('/core', '/develop/advanced'), - existingPath.replace('/basics', '/develop/beginner'), - existingPath.replace('/intro', '/develop/intro'), + existingPath.replace('/core', '/learn/advanced'), + existingPath.replace('/basics', '/learn/beginner'), + existingPath.replace('/intro', '/learn/intro'), existingPath.replace('/architecture', '/build/architecture/'), existingPath.replace('/building-apps', '/build/building-apps'), existingPath.replace('/building-modules', '/build/building-modules'), @@ -432,7 +432,7 @@ const config = { }, { from: ["/main/building-modules/autocli"], - to: "/main/develop/advanced/autocli", + to: "/main/learn/advanced/autocli", }, ], }, diff --git a/docs/learn/advanced/00-baseapp.md b/docs/learn/advanced/00-baseapp.md deleted file mode 100644 index 7121991e9777..000000000000 --- a/docs/learn/advanced/00-baseapp.md +++ /dev/null @@ -1,547 +0,0 @@ ---- -sidebar_position: 1 ---- - -# BaseApp - -:::note Synopsis -This document describes `BaseApp`, the abstraction that implements the core functionalities of a Cosmos SDK application. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) -* [Lifecycle of a Cosmos SDK transaction](../beginner/01-tx-lifecycle.md) - -::: - -## Introduction - -`BaseApp` is a base type that implements the core of a Cosmos SDK application, namely: - -* The [Application Blockchain Interface](#main-abci-messages), for the state-machine to communicate with the underlying consensus engine (e.g. CometBFT). -* [Service Routers](#service-routers), to route messages and queries to the appropriate module. -* Different [states](#state-updates), as the state-machine can have different volatile states updated based on the ABCI message received. - -The goal of `BaseApp` is to provide the fundamental layer of a Cosmos SDK application -that developers can easily extend to build their own custom application. Usually, -developers will create a custom type for their application, like so: - -```go -type App struct { - // reference to a BaseApp - *baseapp.BaseApp - - // list of application store keys - - // list of application keepers - - // module manager -} -``` - -Extending the application with `BaseApp` gives the former access to all of `BaseApp`'s methods. -This allows developers to compose their custom application with the modules they want, while not -having to concern themselves with the hard work of implementing the ABCI, the service routers and state -management logic. - -## Type Definition - -The `BaseApp` type holds many important parameters for any Cosmos SDK based application. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/baseapp.go#L58-L182 -``` - -Let us go through the most important components. - -> **Note**: Not all parameters are described, only the most important ones. Refer to the -> type definition for the full list. - -First, the important parameters that are initialized during the bootstrapping of the application: - -* [`CommitMultiStore`](./04-store.md#commitmultistore): This is the main store of the application, - which holds the canonical state that is committed at the [end of each block](#commit). This store - is **not** cached, meaning it is not used to update the application's volatile (un-committed) states. - The `CommitMultiStore` is a multi-store, meaning a store of stores. Each module of the application - uses one or multiple `KVStores` in the multi-store to persist their subset of the state. -* Database: The `db` is used by the `CommitMultiStore` to handle data persistence. -* [`Msg` Service Router](#msg-service-router): The `msgServiceRouter` facilitates the routing of `sdk.Msg` requests to the appropriate - module `Msg` service for processing. Here a `sdk.Msg` refers to the transaction component that needs to be - processed by a service in order to update the application state, and not to ABCI message which implements - the interface between the application and the underlying consensus engine. -* [gRPC Query Router](#grpc-query-router): The `grpcQueryRouter` facilitates the routing of gRPC queries to the - appropriate module for it to be processed. These queries are not ABCI messages themselves, but they - are relayed to the relevant module's gRPC `Query` service. -* [`TxDecoder`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/types#TxDecoder): It is used to decode - raw transaction bytes relayed by the underlying CometBFT engine. -* [`AnteHandler`](#antehandler): This handler is used to handle signature verification, fee payment, - and other pre-message execution checks when a transaction is received. It's executed during - [`CheckTx/RecheckTx`](#checktx) and [`FinalizeBlock`](#finalizeblock). -* [`InitChainer`](../beginner/00-app-anatomy.md#initchainer), [`PreBlocker`](../beginner/00-app-anatomy.md#preblocker), [`BeginBlocker` and `EndBlocker`](../beginner/00-app-anatomy.md#beginblocker-and-endblocker): These are - the functions executed when the application receives the `InitChain` and `FinalizeBlock` - ABCI messages from the underlying CometBFT engine. - -Then, parameters used to define [volatile states](#state-updates) (i.e. cached states): - -* `checkState`: This state is updated during [`CheckTx`](#checktx), and reset on [`Commit`](#commit). -* `finalizeBlockState`: This state is updated during [`FinalizeBlock`](#finalizeblock), and set to `nil` on - [`Commit`](#commit) and gets re-initialized on `FinalizeBlock`. -* `processProposalState`: This state is updated during [`ProcessProposal`](#process-proposal). -* `prepareProposalState`: This state is updated during [`PrepareProposal`](#prepare-proposal). - -Finally, a few more important parameters: - -* `voteInfos`: This parameter carries the list of validators whose precommit is missing, either - because they did not vote or because the proposer did not include their vote. This information is - carried by the [Context](./02-context.md) and can be used by the application for various things like - punishing absent validators. -* `minGasPrices`: This parameter defines the minimum gas prices accepted by the node. This is a - **local** parameter, meaning each full-node can set a different `minGasPrices`. It is used in the - `AnteHandler` during [`CheckTx`](#checktx), mainly as a spam protection mechanism. The transaction - enters the [mempool](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_basic_concepts.md#mempool-methods) - only if the gas prices of the transaction are greater than one of the minimum gas price in - `minGasPrices` (e.g. if `minGasPrices == 1uatom,1photon`, the `gas-price` of the transaction must be - greater than `1uatom` OR `1photon`). -* `appVersion`: Version of the application. It is set in the - [application's constructor function](../beginner/00-app-anatomy.md#constructor-function). - -## Constructor - -```go -func NewBaseApp( - name string, logger log.Logger, db dbm.DB, txDecoder sdk.TxDecoder, options ...func(*BaseApp), -) *BaseApp { - - // ... -} -``` - -The `BaseApp` constructor function is pretty straightforward. The only thing worth noting is the -possibility to provide additional [`options`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/options.go) -to the `BaseApp`, which will execute them in order. The `options` are generally `setter` functions -for important parameters, like `SetPruning()` to set pruning options or `SetMinGasPrices()` to set -the node's `min-gas-prices`. - -Naturally, developers can add additional `options` based on their application's needs. - -## State Updates - -The `BaseApp` maintains four primary volatile states and a root or main state. The main state -is the canonical state of the application and the volatile states, `checkState`, `prepareProposalState`, `processProposalState` and `finalizeBlockState` -are used to handle state transitions in-between the main state made during [`Commit`](#commit). - -Internally, there is only a single `CommitMultiStore` which we refer to as the main or root state. -From this root state, we derive four volatile states by using a mechanism called _store branching_ (performed by `CacheWrap` function). -The types can be illustrated as follows: - -![Types](./baseapp_state.png) - -### InitChain State Updates - -During `InitChain`, the four volatile states, `checkState`, `prepareProposalState`, `processProposalState` -and `finalizeBlockState` are set by branching the root `CommitMultiStore`. Any subsequent reads and writes happen -on branched versions of the `CommitMultiStore`. -To avoid unnecessary roundtrip to the main state, all reads to the branched store are cached. - -![InitChain](./baseapp_state-initchain.png) - -### CheckTx State Updates - -During `CheckTx`, the `checkState`, which is based off of the last committed state from the root -store, is used for any reads and writes. Here we only execute the `AnteHandler` and verify a service router -exists for every message in the transaction. Note, when we execute the `AnteHandler`, we branch -the already branched `checkState`. -This has the side effect that if the `AnteHandler` fails, the state transitions won't be reflected in the `checkState` --- i.e. `checkState` is only updated on success. - -![CheckTx](./baseapp_state-checktx.png) - -### PrepareProposal State Updates - -During `PrepareProposal`, the `prepareProposalState` is set by branching the root `CommitMultiStore`. -The `prepareProposalState` is used for any reads and writes that occur during the `PrepareProposal` phase. -The function uses the `Select()` method of the mempool to iterate over the transactions. `runTx` is then called, -which encodes and validates each transaction and from there the `AnteHandler` is executed. -If successful, valid transactions are returned inclusive of the events, tags, and data generated -during the execution of the proposal. -The described behavior is that of the default handler, applications have the flexibility to define their own -[custom mempool handlers](https://docs.cosmos.network/main/building-apps/app-mempool#custom-mempool-handlers). - -![ProcessProposal](./baseapp_state-prepareproposal.png) - -### ProcessProposal State Updates - -During `ProcessProposal`, the `processProposalState` is set based off of the last committed state -from the root store and is used to process a signed proposal received from a validator. -In this state, `runTx` is called and the `AnteHandler` is executed and the context used in this state is built with information -from the header and the main state, including the minimum gas prices, which are also set. -Again we want to highlight that the described behavior is that of the default handler and applications have the flexibility to define their own -[custom mempool handlers](https://docs.cosmos.network/main/building-apps/app-mempool#custom-mempool-handlers). - -![ProcessProposal](./baseapp_state-processproposal.png) - -### FinalizeBlock State Updates - -During `FinalizeBlock`, the `finalizeBlockState` is set for use during transaction execution and endblock. The -`finalizeBlockState` is based off of the last committed state from the root store and is branched. -Note, the `finalizeBlockState` is set to `nil` on [`Commit`](#commit). - -The state flow for transcation execution is nearly identical to `CheckTx` except state transitions occur on -the `finalizeBlockState` and messages in a transaction are executed. Similarly to `CheckTx`, state transitions -occur on a doubly branched state -- `finalizeBlockState`. Successful message execution results in -writes being committed to `finalizeBlockState`. Note, if message execution fails, state transitions from -the AnteHandler are persisted. - -### Commit State Updates - -During `Commit` all the state transitions that occurred in the `finalizeBlockState` are finally written to -the root `CommitMultiStore` which in turn is committed to disk and results in a new application -root hash. These state transitions are now considered final. Finally, the `checkState` is set to the -newly committed state and `finalizeBlockState` is set to `nil` to be reset on `FinalizeBlock`. - -![Commit](./baseapp_state-commit.png) - -## ParamStore - -During `InitChain`, the `RequestInitChain` provides `ConsensusParams` which contains parameters -related to block execution such as maximum gas and size in addition to evidence parameters. If these -parameters are non-nil, they are set in the BaseApp's `ParamStore`. Behind the scenes, the `ParamStore` -is managed by an `x/consensus_params` module. This allows the parameters to be tweaked via - on-chain governance. - -## Service Routers - -When messages and queries are received by the application, they must be routed to the appropriate module in order to be processed. Routing is done via `BaseApp`, which holds a `msgServiceRouter` for messages, and a `grpcQueryRouter` for queries. - -### `Msg` Service Router - -[`sdk.Msg`s](../../build/building-modules/02-messages-and-queries.md#messages) need to be routed after they are extracted from transactions, which are sent from the underlying CometBFT engine via the [`CheckTx`](#checktx) and [`FinalizeBlock`](#finalizeblock) ABCI messages. To do so, `BaseApp` holds a `msgServiceRouter` which maps fully-qualified service methods (`string`, defined in each module's Protobuf `Msg` service) to the appropriate module's `MsgServer` implementation. - -The [default `msgServiceRouter` included in `BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/msg_service_router.go) is stateless. However, some applications may want to make use of more stateful routing mechanisms such as allowing governance to disable certain routes or point them to new modules for upgrade purposes. For this reason, the `sdk.Context` is also passed into each [route handler inside `msgServiceRouter`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/msg_service_router.go#L31-L32). For a stateless router that doesn't want to make use of this, you can just ignore the `ctx`. - -The application's `msgServiceRouter` is initialized with all the routes using the application's [module manager](../../build/building-modules/01-module-manager.md#manager) (via the `RegisterServices` method), which itself is initialized with all the application's modules in the application's [constructor](../beginner/00-app-anatomy.md#constructor-function). - -### gRPC Query Router - -Similar to `sdk.Msg`s, [`queries`](../../build/building-modules/02-messages-and-queries.md#queries) need to be routed to the appropriate module's [`Query` service](../../build/building-modules/04-query-services.md). To do so, `BaseApp` holds a `grpcQueryRouter`, which maps modules' fully-qualified service methods (`string`, defined in their Protobuf `Query` gRPC) to their `QueryServer` implementation. The `grpcQueryRouter` is called during the initial stages of query processing, which can be either by directly sending a gRPC query to the gRPC endpoint, or via the [`Query` ABCI message](#query) on the CometBFT RPC endpoint. - -Just like the `msgServiceRouter`, the `grpcQueryRouter` is initialized with all the query routes using the application's [module manager](../../build/building-modules/01-module-manager.md) (via the `RegisterServices` method), which itself is initialized with all the application's modules in the application's [constructor](../beginner/00-app-anatomy.md#app-constructor). - -## Main ABCI 2.0 Messages - -The [Application-Blockchain Interface](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_basic_concepts.md) (ABCI) is a generic interface that connects a state-machine with a consensus engine to form a functional full-node. It can be wrapped in any language, and needs to be implemented by each application-specific blockchain built on top of an ABCI-compatible consensus engine like CometBFT. - -The consensus engine handles two main tasks: - -* The networking logic, which mainly consists in gossiping block parts, transactions and consensus votes. -* The consensus logic, which results in the deterministic ordering of transactions in the form of blocks. - -It is **not** the role of the consensus engine to define the state or the validity of transactions. Generally, transactions are handled by the consensus engine in the form of `[]bytes`, and relayed to the application via the ABCI to be decoded and processed. At keys moments in the networking and consensus processes (e.g. beginning of a block, commit of a block, reception of an unconfirmed transaction, ...), the consensus engine emits ABCI messages for the state-machine to act on. - -Developers building on top of the Cosmos SDK need not implement the ABCI themselves, as `BaseApp` comes with a built-in implementation of the interface. Let us go through the main ABCI messages that `BaseApp` implements: - -* [`Prepare Proposal`](#prepare-proposal) -* [`Process Proposal`](#process-proposal) -* [`CheckTx`](#checktx) -* [`FinalizeBlock`](#finalizeblock) -* [`ExtendVote`](#extendvote) -* [`VerifyVoteExtension`](#verifyvoteextension) - - -### Prepare Proposal - -The `PrepareProposal` function is part of the new methods introduced in Application Blockchain Interface (ABCI++) in CometBFT and is an important part of the application's overall governance system. In the Cosmos SDK, it allows the application to have more fine-grained control over the transactions that are processed, and ensures that only valid transactions are committed to the blockchain. - -Here is how the `PrepareProposal` function can be implemented: - -1. Extract the `sdk.Msg`s from the transaction. -2. Perform _stateful_ checks by calling `Validate()` on each of the `sdk.Msg`'s. This is done after _stateless_ checks as _stateful_ checks are more computationally expensive. If `Validate()` fails, `PrepareProposal` returns before running further checks, which saves resources. -3. Perform any additional checks that are specific to the application, such as checking account balances, or ensuring that certain conditions are met before a transaction is proposed.hey are processed by the consensus engine, if necessary. -4. Return the updated transactions to be processed by the consensus engine - -Note that, unlike `CheckTx()`, `PrepareProposal` process `sdk.Msg`s, so it can directly update the state. However, unlike `FinalizeBlock()`, it does not commit the state updates. It's important to exercise caution when using `PrepareProposal` as incorrect coding could affect the overall liveness of the network. - -It's important to note that `PrepareProposal` complements the `ProcessProposal` method which is executed after this method. The combination of these two methods means that it is possible to guarantee that no invalid transactions are ever committed. Furthermore, such a setup can give rise to other interesting use cases such as Oracles, threshold decryption and more. - -`PrepareProposal` returns a response to the underlying consensus engine of type [`abci.ResponseCheckTx`](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_methods.md#processproposal). The response contains: - -* `Code (uint32)`: Response Code. `0` if successful. -* `Data ([]byte)`: Result bytes, if any. -* `Log (string):` The output of the application's logger. May be non-deterministic. -* `Info (string):` Additional information. May be non-deterministic. - - -### Process Proposal - -The `ProcessProposal` function is called by the BaseApp as part of the ABCI message flow, and is executed during the `FinalizeBlock` phase of the consensus process. The purpose of this function is to give more control to the application for block validation, allowing it to check all transactions in a proposed block before the validator sends the prevote for the block. It allows a validator to perform application-dependent work in a proposed block, enabling features such as immediate block execution, and allows the Application to reject invalid blocks. - -The `ProcessProposal` function performs several key tasks, including: - -1. Validating the proposed block by checking all transactions in it. -2. Checking the proposed block against the current state of the application, to ensure that it is valid and that it can be executed. -3. Updating the application's state based on the proposal, if it is valid and passes all checks. -4. Returning a response to CometBFT indicating the result of the proposal processing. - -The `ProcessProposal` is an important part of the application's overall governance system. It is used to manage the network's parameters and other key aspects of its operation. It also ensures that the coherence property is adhered to i.e. all honest validators must accept a proposal by an honest proposer. - -It's important to note that `ProcessProposal` complements the `PrepareProposal` method which enables the application to have more fine-grained transaction control by allowing it to reorder, drop, delay, modify, and even add transactions as they see necessary. The combination of these two methods means that it is possible to guarantee that no invalid transactions are ever committed. Furthermore, such a setup can give rise to other interesting use cases such as Oracles, threshold decryption and more. - -CometBFT calls it when it receives a proposal and the CometBFT algorithm has not locked on a value. The Application cannot modify the proposal at this point but can reject it if it is invalid. If that is the case, CometBFT will prevote `nil` on the proposal, which has strong liveness implications for CometBFT. As a general rule, the Application SHOULD accept a prepared proposal passed via `ProcessProposal`, even if a part of the proposal is invalid (e.g., an invalid transaction); the Application can ignore the invalid part of the prepared proposal at block execution time. - -However, developers must exercise greater caution when using these methods. Incorrectly coding these methods could affect liveness as CometBFT is unable to receive 2/3 valid precommits to finalize a block. - -`ProcessProposal` returns a response to the underlying consensus engine of type [`abci.ResponseCheckTx`](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_methods.md#processproposal). The response contains: - -* `Code (uint32)`: Response Code. `0` if successful. -* `Data ([]byte)`: Result bytes, if any. -* `Log (string):` The output of the application's logger. May be non-deterministic. -* `Info (string):` Additional information. May be non-deterministic. - - -### CheckTx - -`CheckTx` is sent by the underlying consensus engine when a new unconfirmed (i.e. not yet included in a valid block) -transaction is received by a full-node. The role of `CheckTx` is to guard the full-node's mempool -(where unconfirmed transactions are stored until they are included in a block) from spam transactions. -Unconfirmed transactions are relayed to peers only if they pass `CheckTx`. - -`CheckTx()` can perform both _stateful_ and _stateless_ checks, but developers should strive to -make the checks **lightweight** because gas fees are not charged for the resources (CPU, data load...) used during the `CheckTx`. - -In the Cosmos SDK, after [decoding transactions](./05-encoding.md), `CheckTx()` is implemented -to do the following checks: - -1. Extract the `sdk.Msg`s from the transaction. -2. **Optionally** perform _stateless_ checks by calling `ValidateBasic()` on each of the `sdk.Msg`s. This is done - first, as _stateless_ checks are less computationally expensive than _stateful_ checks. If - `ValidateBasic()` fail, `CheckTx` returns before running _stateful_ checks, which saves resources. - This check is still performed for messages that have not yet migrated to the new message validation mechanism defined in [RFC 001](https://docs.cosmos.network/main/rfc/rfc-001-tx-validation) and still have a `ValidateBasic()` method. -3. Perform non-module related _stateful_ checks on the [account](../beginner/03-accounts.md). This step is mainly about checking - that the `sdk.Msg` signatures are valid, that enough fees are provided and that the sending account - has enough funds to pay for said fees. Note that no precise [`gas`](../beginner/04-gas-fees.md) counting occurs here, - as `sdk.Msg`s are not processed. Usually, the [`AnteHandler`](../beginner/04-gas-fees.md#antehandler) will check that the `gas` provided - with the transaction is superior to a minimum reference gas amount based on the raw transaction size, - in order to avoid spam with transactions that provide 0 gas. - -`CheckTx` does **not** process `sdk.Msg`s - they only need to be processed when the canonical state need to be updated, which happens during `FinalizeBlock`. - -Steps 2. and 3. are performed by the [`AnteHandler`](../beginner/04-gas-fees.md#antehandler) in the [`RunTx()`](#runtx-antehandler-and-runmsgs) -function, which `CheckTx()` calls with the `runTxModeCheck` mode. During each step of `CheckTx()`, a -special [volatile state](#state-updates) called `checkState` is updated. This state is used to keep -track of the temporary changes triggered by the `CheckTx()` calls of each transaction without modifying -the [main canonical state](#main-state). For example, when a transaction goes through `CheckTx()`, the -transaction's fees are deducted from the sender's account in `checkState`. If a second transaction is -received from the same account before the first is processed, and the account has consumed all its -funds in `checkState` during the first transaction, the second transaction will fail `CheckTx`() and -be rejected. In any case, the sender's account will not actually pay the fees until the transaction -is actually included in a block, because `checkState` never gets committed to the main state. The -`checkState` is reset to the latest state of the main state each time a blocks gets [committed](#commit). - -`CheckTx` returns a response to the underlying consensus engine of type [`abci.ResponseCheckTx`](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_methods.md#checktx). -The response contains: - -* `Code (uint32)`: Response Code. `0` if successful. -* `Data ([]byte)`: Result bytes, if any. -* `Log (string):` The output of the application's logger. May be non-deterministic. -* `Info (string):` Additional information. May be non-deterministic. -* `GasWanted (int64)`: Amount of gas requested for transaction. It is provided by users when they generate the transaction. -* `GasUsed (int64)`: Amount of gas consumed by transaction. During `CheckTx`, this value is computed by multiplying the standard cost of a transaction byte by the size of the raw transaction. Next is an example: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/ante/basic.go#L102 -``` - -* `Events ([]cmn.KVPair)`: Key-Value tags for filtering and indexing transactions (eg. by account). See [`event`s](./08-events.md) for more. -* `Codespace (string)`: Namespace for the Code. - -#### RecheckTx - -After `Commit`, `CheckTx` is run again on all transactions that remain in the node's local mempool -excluding the transactions that are included in the block. To prevent the mempool from rechecking all transactions -every time a block is committed, the configuration option `mempool.recheck=false` can be set. As of -Tendermint v0.32.1, an additional `Type` parameter is made available to the `CheckTx` function that -indicates whether an incoming transaction is new (`CheckTxType_New`), or a recheck (`CheckTxType_Recheck`). -This allows certain checks like signature verification can be skipped during `CheckTxType_Recheck`. - -## RunTx, AnteHandler, RunMsgs, PostHandler - -### RunTx - -`RunTx` is called from `CheckTx`/`Finalizeblock` to handle the transaction, with `execModeCheck` or `execModeFinalize` as parameter to differentiate between the two modes of execution. Note that when `RunTx` receives a transaction, it has already been decoded. - -The first thing `RunTx` does upon being called is to retrieve the `context`'s `CacheMultiStore` by calling the `getContextForTx()` function with the appropriate mode (either `runTxModeCheck` or `execModeFinalize`). This `CacheMultiStore` is a branch of the main store, with cache functionality (for query requests), instantiated during `FinalizeBlock` for transaction execution and during the `Commit` of the previous block for `CheckTx`. After that, two `defer func()` are called for [`gas`](../beginner/04-gas-fees.md) management. They are executed when `runTx` returns and make sure `gas` is actually consumed, and will throw errors, if any. - -After that, `RunTx()` calls `ValidateBasic()`, when available and for backward compatibility, on each `sdk.Msg`in the `Tx`, which runs preliminary _stateless_ validity checks. If any `sdk.Msg` fails to pass `ValidateBasic()`, `RunTx()` returns with an error. - -Then, the [`anteHandler`](#antehandler) of the application is run (if it exists). In preparation of this step, both the `checkState`/`finalizeBlockState`'s `context` and `context`'s `CacheMultiStore` are branched using the `cacheTxContext()` function. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/baseapp.go#L663-L680 -``` - -This allows `RunTx` not to commit the changes made to the state during the execution of `anteHandler` if it ends up failing. It also prevents the module implementing the `anteHandler` from writing to state, which is an important part of the [object-capabilities](./10-ocap.md) of the Cosmos SDK. - -Finally, the [`RunMsgs()`](#runmsgs) function is called to process the `sdk.Msg`s in the `Tx`. In preparation of this step, just like with the `anteHandler`, both the `checkState`/`finalizeBlockState`'s `context` and `context`'s `CacheMultiStore` are branched using the `cacheTxContext()` function. - -### AnteHandler - -The `AnteHandler` is a special handler that implements the `AnteHandler` interface and is used to authenticate the transaction before the transaction's internal messages are processed. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/handler.go#L6-L8 -``` - -The `AnteHandler` is theoretically optional, but still a very important component of public blockchain networks. It serves 3 primary purposes: - -* Be a primary line of defense against spam and second line of defense (the first one being the mempool) against transaction replay with fees deduction and [`sequence`](./01-transactions.md#transaction-generation) checking. -* Perform preliminary _stateful_ validity checks like ensuring signatures are valid or that the sender has enough funds to pay for fees. -* Play a role in the incentivisation of stakeholders via the collection of transaction fees. - -`BaseApp` holds an `anteHandler` as parameter that is initialized in the [application's constructor](../beginner/00-app-anatomy.md#application-constructor). The most widely used `anteHandler` is the [`auth` module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/ante/ante.go). - -Click [here](../beginner/04-gas-fees.md#antehandler) for more on the `anteHandler`. - -### RunMsgs - -`RunMsgs` is called from `RunTx` with `runTxModeCheck` as parameter to check the existence of a route for each message the transaction, and with `execModeFinalize` to actually process the `sdk.Msg`s. - -First, it retrieves the `sdk.Msg`'s fully-qualified type name, by checking the `type_url` of the Protobuf `Any` representing the `sdk.Msg`. Then, using the application's [`msgServiceRouter`](#msg-service-router), it checks for the existence of `Msg` service method related to that `type_url`. At this point, if `mode == runTxModeCheck`, `RunMsgs` returns. Otherwise, if `mode == execModeFinalize`, the [`Msg` service](../../build/building-modules/03-msg-services.md) RPC is executed, before `RunMsgs` returns. - -### PostHandler - -`PostHandler` is similar to `AnteHandler`, but it, as the name suggests, executes custom post tx processing logic after [`RunMsgs`](#runmsgs) is called. `PostHandler` receives the `Result` of the the `RunMsgs` in order to enable this customizable behavior. - -Like `AnteHandler`s, `PostHandler`s are theoretically optional. - -Other use cases like unused gas refund can also be enabled by `PostHandler`s. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/posthandler/post.go#L1-L15 -``` - -Note, when `PostHandler`s fail, the state from `runMsgs` is also reverted, effectively making the transaction fail. - -## Other ABCI Messages - -### InitChain - -The [`InitChain` ABCI message](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_basic_concepts.md#method-overview) is sent from the underlying CometBFT engine when the chain is first started. It is mainly used to **initialize** parameters and state like: - -* [Consensus Parameters](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_app_requirements.md#consensus-parameters) via `setConsensusParams`. -* [`checkState` and `finalizeBlockState`](#state-updates) via `setState`. -* The [block gas meter](../beginner/04-gas-fees.md#block-gas-meter), with infinite gas to process genesis transactions. - -Finally, the `InitChain(req abci.RequestInitChain)` method of `BaseApp` calls the [`initChainer()`](../beginner/00-app-anatomy.md#initchainer) of the application in order to initialize the main state of the application from the `genesis file` and, if defined, call the [`InitGenesis`](../../build/building-modules/08-genesis.md#initgenesis) function of each of the application's modules. - - -### FinalizeBlock - -The [`FinalizeBlock` ABCI message](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci++_basic_concepts.md#method-overview) is sent from the underlying CometBFT engine when a block proposal created by the correct proposer is received. The previous `BeginBlock, DeliverTx and Endblock` calls are private methods on the BaseApp struct. - - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci.go#L623 -``` - -#### PreBlock - -* Run the application's [`preBlocker()`](../beginner/00-app-anatomy.md#preblocker), which mainly runs the [`PreBlocker()`](../../build/building-modules/17-preblock.md#preblock) method of each of the modules. - -#### BeginBlock - -* Initialize [`finalizeBlockState`](#state-updates) with the latest header using the `req abci.RequestFinalizeBlock` passed as parameter via the `setState` function. - - ```go reference - https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/baseapp.go#L682-L706 - ``` - - This function also resets the [main gas meter](../beginner/04-gas-fees.md#main-gas-meter). - -* Initialize the [block gas meter](../beginner/04-gas-fees.md#block-gas-meter) with the `maxGas` limit. The `gas` consumed within the block cannot go above `maxGas`. This parameter is defined in the application's consensus parameters. -* Run the application's [`beginBlocker()`](../beginner/00-app-anatomy.md#beginblocker-and-endblocker), which mainly runs the [`BeginBlocker()`](../../build/building-modules/06-beginblock-endblock.md#beginblock) method of each of the modules. -* Set the [`VoteInfos`](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_methods.md#voteinfo) of the application, i.e. the list of validators whose _precommit_ for the previous block was included by the proposer of the current block. This information is carried into the [`Context`](./02-context.md) so that it can be used during transaction execution and EndBlock. - -#### Transaction Execution - -When the underlying consensus engine receives a block proposal, each transaction in the block needs to be processed by the application. To that end, the underlying consensus engine sends the transactions in FinalizeBlock message to the application for each transaction in a sequential order. - -Before the first transaction of a given block is processed, a [volatile state](#state-updates) called `finalizeBlockState` is initialized during FinalizeBlock. This state is updated each time a transaction is processed via `FinalizeBlock`, and committed to the [main state](#main-state) when the block is [committed](#commit), after what it is set to `nil`. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/baseapp.go#LL708-L743 -``` - -Transaction execution within `FinalizeBlock` performs the **exact same steps as `CheckTx`**, with a little caveat at step 3 and the addition of a fifth step: - -1. The `AnteHandler` does **not** check that the transaction's `gas-prices` is sufficient. That is because the `min-gas-prices` value `gas-prices` is checked against is local to the node, and therefore what is enough for one full-node might not be for another. This means that the proposer can potentially include transactions for free, although they are not incentivised to do so, as they earn a bonus on the total fee of the block they propose. -2. For each `sdk.Msg` in the transaction, route to the appropriate module's Protobuf [`Msg` service](../../build/building-modules/03-msg-services.md). Additional _stateful_ checks are performed, and the branched multistore held in `finalizeBlockState`'s `context` is updated by the module's `keeper`. If the `Msg` service returns successfully, the branched multistore held in `context` is written to `finalizeBlockState` `CacheMultiStore`. - -During the additional fifth step outlined in (2), each read/write to the store increases the value of `GasConsumed`. You can find the default cost of each operation: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/gas.go#L230-L241 -``` - -At any point, if `GasConsumed > GasWanted`, the function returns with `Code != 0` and the execution fails. - -Each transactions returns a response to the underlying consensus engine of type [`abci.ExecTxResult`](https://github.com/cometbft/cometbft/blob/v0.38.0-rc1/spec/abci/abci%2B%2B_methods.md#exectxresult). The response contains: - -* `Code (uint32)`: Response Code. `0` if successful. -* `Data ([]byte)`: Result bytes, if any. -* `Log (string):` The output of the application's logger. May be non-deterministic. -* `Info (string):` Additional information. May be non-deterministic. -* `GasWanted (int64)`: Amount of gas requested for transaction. It is provided by users when they generate the transaction. -* `GasUsed (int64)`: Amount of gas consumed by transaction. During transaction execution, this value is computed by multiplying the standard cost of a transaction byte by the size of the raw transaction, and by adding gas each time a read/write to the store occurs. -* `Events ([]cmn.KVPair)`: Key-Value tags for filtering and indexing transactions (eg. by account). See [`event`s](./08-events.md) for more. -* `Codespace (string)`: Namespace for the Code. - -#### EndBlock - -EndBlock is run after transaction execution completes. It allows developers to have logic be executed at the end of each block. In the Cosmos SDK, the bulk EndBlock() method is to run the application's EndBlocker(), which mainly runs the EndBlocker() method of each of the application's modules. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/baseapp.go#L747-L769 -``` - -### Commit - -The [`Commit` ABCI message](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_basic_concepts.md#method-overview) is sent from the underlying CometBFT engine after the full-node has received _precommits_ from 2/3+ of validators (weighted by voting power). On the `BaseApp` end, the `Commit(res abci.ResponseCommit)` function is implemented to commit all the valid state transitions that occurred during `FinalizeBlock` and to reset state for the next block. - -To commit state-transitions, the `Commit` function calls the `Write()` function on `finalizeBlockState.ms`, where `finalizeBlockState.ms` is a branched multistore of the main store `app.cms`. Then, the `Commit` function sets `checkState` to the latest header (obtained from `finalizeBlockState.ctx.BlockHeader`) and `finalizeBlockState` to `nil`. - -Finally, `Commit` returns the hash of the commitment of `app.cms` back to the underlying consensus engine. This hash is used as a reference in the header of the next block. - -### Info - -The [`Info` ABCI message](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_basic_concepts.md#info-methods) is a simple query from the underlying consensus engine, notably used to sync the latter with the application during a handshake that happens on startup. When called, the `Info(res abci.ResponseInfo)` function from `BaseApp` will return the application's name, version and the hash of the last commit of `app.cms`. - -### Query - -The [`Query` ABCI message](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_basic_concepts.md#info-methods) is used to serve queries received from the underlying consensus engine, including queries received via RPC like CometBFT RPC. It used to be the main entrypoint to build interfaces with the application, but with the introduction of [gRPC queries](../../build/building-modules/04-query-services.md) in Cosmos SDK v0.40, its usage is more limited. The application must respect a few rules when implementing the `Query` method, which are outlined [here](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_app_requirements.md#query). - -Each CometBFT `query` comes with a `path`, which is a `string` which denotes what to query. If the `path` matches a gRPC fully-qualified service method, then `BaseApp` will defer the query to the `grpcQueryRouter` and let it handle it like explained [above](#grpc-query-router). Otherwise, the `path` represents a query that is not (yet) handled by the gRPC router. `BaseApp` splits the `path` string with the `/` delimiter. By convention, the first element of the split string (`split[0]`) contains the category of `query` (`app`, `p2p`, `store` or `custom` ). The `BaseApp` implementation of the `Query(req abci.RequestQuery)` method is a simple dispatcher serving these 4 main categories of queries: - -* Application-related queries like querying the application's version, which are served via the `handleQueryApp` method. -* Direct queries to the multistore, which are served by the `handlerQueryStore` method. These direct queries are different from custom queries which go through `app.queryRouter`, and are mainly used by third-party service provider like block explorers. -* P2P queries, which are served via the `handleQueryP2P` method. These queries return either `app.addrPeerFilter` or `app.ipPeerFilter` that contain the list of peers filtered by address or IP respectively. These lists are first initialized via `options` in `BaseApp`'s [constructor](#constructor). - -### ExtendVote - -`ExtendVote` allows an application to extend a pre-commit vote with arbitrary data. This process does NOT have be deterministic and the data returned can be unique to the validator process. - -In the Cosmos-SDK this is implemented as a NoOp: - -``` go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go#L274-L281 -``` - -### VerifyVoteExtension - -`VerifyVoteExtension` allows an application to verify that the data returned by `ExtendVote` is valid. This process does NOT have be deterministic and the data returned can be unique to the validator process. - -In the Cosmos-SDK this is implemented as a NoOp: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go#L282-L288 -``` diff --git a/docs/learn/advanced/01-transactions.md b/docs/learn/advanced/01-transactions.md deleted file mode 100644 index 1901675fbcb2..000000000000 --- a/docs/learn/advanced/01-transactions.md +++ /dev/null @@ -1,200 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Transactions - -:::note Synopsis -`Transactions` are objects created by end-users to trigger state changes in the application. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK Application](../beginner/00-app-anatomy.md) - -::: - -## Transactions - -Transactions are comprised of metadata held in [contexts](./02-context.md) and [`sdk.Msg`s](../../build/building-modules/02-messages-and-queries.md) that trigger state changes within a module through the module's Protobuf [`Msg` service](../../build/building-modules/03-msg-services.md). - -When users want to interact with an application and make state changes (e.g. sending coins), they create transactions. Each of a transaction's `sdk.Msg` must be signed using the private key associated with the appropriate account(s), before the transaction is broadcasted to the network. A transaction must then be included in a block, validated, and approved by the network through the consensus process. To read more about the lifecycle of a transaction, click [here](../beginner/01-tx-lifecycle.md). - -## Type Definition - -Transaction objects are Cosmos SDK types that implement the `Tx` interface - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/tx_msg.go#L51-L56 -``` - -It contains the following methods: - -* **GetMsgs:** unwraps the transaction and returns a list of contained `sdk.Msg`s - one transaction may have one or multiple messages, which are defined by module developers. -* **ValidateBasic:** lightweight, [_stateless_](../beginner/01-tx-lifecycle.md#types-of-checks) checks used by ABCI messages [`CheckTx`](./00-baseapp.md#checktx) and [`DeliverTx`](./00-baseapp.md#delivertx) to make sure transactions are not invalid. For example, the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/main/x/auth) module's `ValidateBasic` function checks that its transactions are signed by the correct number of signers and that the fees do not exceed what the user's maximum. When [`runTx`](./00-baseapp.md#runtx) is checking a transaction created from the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/main/x/auth/spec) module, it first runs `ValidateBasic` on each message, then runs the `auth` module AnteHandler which calls `ValidateBasic` for the transaction itself. - - :::note - This function is different from the deprecated `sdk.Msg` [`ValidateBasic`](../beginner/01-tx-lifecycle.md#ValidateBasic) methods, which was performing basic validity checks on messages only. - ::: - -As a developer, you should rarely manipulate `Tx` directly, as `Tx` is really an intermediate type used for transaction generation. Instead, developers should prefer the `TxBuilder` interface, which you can learn more about [below](#transaction-generation). - -### Signing Transactions - -Every message in a transaction must be signed by the addresses specified by its `GetSigners`. The Cosmos SDK currently allows signing transactions in two different ways. - -#### `SIGN_MODE_DIRECT` (preferred) - -The most used implementation of the `Tx` interface is the Protobuf `Tx` message, which is used in `SIGN_MODE_DIRECT`: - -```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L13-L26 -``` - -Because Protobuf serialization is not deterministic, the Cosmos SDK uses an additional `TxRaw` type to denote the pinned bytes over which a transaction is signed. Any user can generate a valid `body` and `auth_info` for a transaction, and serialize these two messages using Protobuf. `TxRaw` then pins the user's exact binary representation of `body` and `auth_info`, called respectively `body_bytes` and `auth_info_bytes`. The document that is signed by all signers of the transaction is `SignDoc` (deterministically serialized using [ADR-027](../../build/architecture/adr-027-deterministic-protobuf-serialization.md)): - -```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L48-L65 -``` - -Once signed by all signers, the `body_bytes`, `auth_info_bytes` and `signatures` are gathered into `TxRaw`, whose serialized bytes are broadcasted over the network. - -#### `SIGN_MODE_LEGACY_AMINO_JSON` - -The legacy implementation of the `Tx` interface is the `StdTx` struct from `x/auth`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/migrations/legacytx/stdtx.go#L83-L90 -``` - -The document signed by all signers is `StdSignDoc`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/migrations/legacytx/stdsign.go#L31-L45 -``` - -which is encoded into bytes using Amino JSON. Once all signatures are gathered into `StdTx`, `StdTx` is serialized using Amino JSON, and these bytes are broadcasted over the network. - -#### Other Sign Modes - -The Cosmos SDK also provides a couple of other sign modes for particular use cases. - -#### `SIGN_MODE_DIRECT_AUX` - -`SIGN_MODE_DIRECT_AUX` is a sign mode released in the Cosmos SDK v0.46 which targets transactions with multiple signers. Whereas `SIGN_MODE_DIRECT` expects each signer to sign over both `TxBody` and `AuthInfo` (which includes all other signers' signer infos, i.e. their account sequence, public key and mode info), `SIGN_MODE_DIRECT_AUX` allows N-1 signers to only sign over `TxBody` and _their own_ signer info. Morever, each auxiliary signer (i.e. a signer using `SIGN_MODE_DIRECT_AUX`) doesn't -need to sign over the fees: - -```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L67-L98 -``` - -The use case is a multi-signer transaction, where one of the signers is appointed to gather all signatures, broadcast the signature and pay for fees, and the others only care about the transaction body. This generally allows for a better multi-signing UX. If Alice, Bob and Charlie are part of a 3-signer transaction, then Alice and Bob can both use `SIGN_MODE_DIRECT_AUX` to sign over the `TxBody` and their own signer info (no need an additional step to gather other signers' ones, like in `SIGN_MODE_DIRECT`), without specifying a fee in their SignDoc. Charlie can then gather both signatures from Alice and Bob, and -create the final transaction by appending a fee. Note that the fee payer of the transaction (in our case Charlie) must sign over the fees, so must use `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. - - -#### `SIGN_MODE_TEXTUAL` - -`SIGN_MODE_TEXTUAL` is a new sign mode for delivering a better signing experience on hardware wallets, it is currently still under implementation. If you wish to learn more, please refer to [ADR-050](https://github.com/cosmos/cosmos-sdk/pull/10701). - -#### Custom Sign modes - -There is the the opportunity to add your own custom sign mode to the Cosmos-SDK. While we can not accept the implementation of the sign mode to the repository, we can accept a pull request to add the custom signmode to the SignMode enum located [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/signing/v1beta1/signing.proto#L17) - -## Transaction Process - -The process of an end-user sending a transaction is: - -* decide on the messages to put into the transaction, -* generate the transaction using the Cosmos SDK's `TxBuilder`, -* broadcast the transaction using one of the available interfaces. - -The next paragraphs will describe each of these components, in this order. - -### Messages - -:::tip -Module `sdk.Msg`s are not to be confused with [ABCI Messages](https://docs.cometbft.com/v0.37/spec/abci/) which define interactions between the CometBFT and application layers. -::: - -**Messages** (or `sdk.Msg`s) are module-specific objects that trigger state transitions within the scope of the module they belong to. Module developers define the messages for their module by adding methods to the Protobuf [`Msg` service](../../build/building-modules/03-msg-services.md), and also implement the corresponding `MsgServer`. - -Each `sdk.Msg`s is related to exactly one Protobuf [`Msg` service](../../build/building-modules/03-msg-services.md) RPC, defined inside each module's `tx.proto` file. A SDK app router automatically maps every `sdk.Msg` to a corresponding RPC. Protobuf generates a `MsgServer` interface for each module `Msg` service, and the module developer needs to implement this interface. -This design puts more responsibility on module developers, allowing application developers to reuse common functionalities without having to implement state transition logic repetitively. - -To learn more about Protobuf `Msg` services and how to implement `MsgServer`, click [here](../../build/building-modules/03-msg-services.md). - -While messages contain the information for state transition logic, a transaction's other metadata and relevant information are stored in the `TxBuilder` and `Context`. - -### Transaction Generation - -The `TxBuilder` interface contains data closely related with the generation of transactions, which an end-user can freely set to generate the desired transaction: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/tx_config.go#L40-L53 -``` - -* `Msg`s, the array of [messages](#messages) included in the transaction. -* `GasLimit`, option chosen by the users for how to calculate how much gas they will need to pay. -* `Memo`, a note or comment to send with the transaction. -* `FeeAmount`, the maximum amount the user is willing to pay in fees. -* `TimeoutHeight`, block height until which the transaction is valid. -* `Signatures`, the array of signatures from all signers of the transaction. - -As there are currently two sign modes for signing transactions, there are also two implementations of `TxBuilder`: - -* [wrapper](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/builder.go#L26-L43) for creating transactions for `SIGN_MODE_DIRECT`, -* [StdTxBuilder](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/migrations/legacytx/stdtx_builder.go#L14-L17) for `SIGN_MODE_LEGACY_AMINO_JSON`. - -However, the two implementation of `TxBuilder` should be hidden away from end-users, as they should prefer using the overarching `TxConfig` interface: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/tx_config.go#L24-L34 -``` - -`TxConfig` is an app-wide configuration for managing transactions. Most importantly, it holds the information about whether to sign each transaction with `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. By calling `txBuilder := txConfig.NewTxBuilder()`, a new `TxBuilder` will be created with the appropriate sign mode. - -Once `TxBuilder` is correctly populated with the setters exposed above, `TxConfig` will also take care of correctly encoding the bytes (again, either using `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`). Here's a pseudo-code snippet of how to generate and encode a transaction, using the `TxEncoder()` method: - -```go -txBuilder := txConfig.NewTxBuilder() -txBuilder.SetMsgs(...) // and other setters on txBuilder - -bz, err := txConfig.TxEncoder()(txBuilder.GetTx()) -// bz are bytes to be broadcasted over the network -``` - -### Broadcasting the Transaction - -Once the transaction bytes are generated, there are currently three ways of broadcasting it. - -#### CLI - -Application developers create entry points to the application by creating a [command-line interface](./07-cli.md), [gRPC and/or REST interface](./06-grpc_rest.md), typically found in the application's `./cmd` folder. These interfaces allow users to interact with the application through command-line. - -For the [command-line interface](../../build/building-modules/09-module-interfaces.md#cli), module developers create subcommands to add as children to the application top-level transaction command `TxCmd`. CLI commands actually bundle all the steps of transaction processing into one simple command: creating messages, generating transactions and broadcasting. For concrete examples, see the [Interacting with a Node](../../user/run-node/02-interact-node.md) section. An example transaction made using CLI looks like: - -```bash -simd tx send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000stake -``` - -#### gRPC - -[gRPC](https://grpc.io) is the main component for the Cosmos SDK's RPC layer. Its principal usage is in the context of modules' [`Query` services](../../build/building-modules/04-query-services.md). However, the Cosmos SDK also exposes a few other module-agnostic gRPC services, one of them being the `Tx` service: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/service.proto -``` - -The `Tx` service exposes a handful of utility functions, such as simulating a transaction or querying a transaction, and also one method to broadcast transactions. - -Examples of broadcasting and simulating a transaction are shown [here](../../user/run-node/03-txs.md#programmatically-with-go). - -#### REST - -Each gRPC method has its corresponding REST endpoint, generated using [gRPC-gateway](https://github.com/grpc-ecosystem/grpc-gateway). Therefore, instead of using gRPC, you can also use HTTP to broadcast the same transaction, on the `POST /cosmos/tx/v1beta1/txs` endpoint. - -An example can be seen [here](../../user/run-node/03-txs.md#using-rest) - -#### CometBFT RPC - -The three methods presented above are actually higher abstractions over the CometBFT RPC `/broadcast_tx_{async,sync,commit}` endpoints, documented [here](https://docs.cometbft.com/v0.37/core/rpc). This means that you can use the CometBFT RPC endpoints directly to broadcast the transaction, if you wish so. diff --git a/docs/learn/advanced/02-context.md b/docs/learn/advanced/02-context.md deleted file mode 100644 index 4d9ecdf690ff..000000000000 --- a/docs/learn/advanced/02-context.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Context - -:::note Synopsis -The `context` is a data structure intended to be passed from function to function that carries information about the current state of the application. It provides access to a branched storage (a safe branch of the entire state) as well as useful objects and information like `gasMeter`, `block height`, `consensus parameters` and more. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK Application](../beginner/00-app-anatomy.md) -* [Lifecycle of a Transaction](../beginner/01-tx-lifecycle.md) - -::: - -## Context Definition - -The Cosmos SDK `Context` is a custom data structure that contains Go's stdlib [`context`](https://pkg.go.dev/context) as its base, and has many additional types within its definition that are specific to the Cosmos SDK. The `Context` is integral to transaction processing in that it allows modules to easily access their respective [store](./04-store.md#base-layer-kvstores) in the [`multistore`](./04-store.md#multistore) and retrieve transactional context such as the block header and gas meter. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/context.go#L41-L67 -``` - -* **Base Context:** The base type is a Go [Context](https://pkg.go.dev/context), which is explained further in the [Go Context Package](#go-context-package) section below. -* **Multistore:** Every application's `BaseApp` contains a [`CommitMultiStore`](./04-store.md#multistore) which is provided when a `Context` is created. Calling the `KVStore()` and `TransientStore()` methods allows modules to fetch their respective [`KVStore`](./04-store.md#base-layer-kvstores) using their unique `StoreKey`. -* **Header:** The [header](https://docs.cometbft.com/v0.37/spec/core/data_structures#header) is a Blockchain type. It carries important information about the state of the blockchain, such as block height and proposer of the current block. -* **Header Hash:** The current block header hash, obtained during `abci.FinalizeBlock`. -* **Chain ID:** The unique identification number of the blockchain a block pertains to. -* **Transaction Bytes:** The `[]byte` representation of a transaction being processed using the context. Every transaction is processed by various parts of the Cosmos SDK and consensus engine (e.g. CometBFT) throughout its [lifecycle](../beginner/01-tx-lifecycle.md), some of which do not have any understanding of transaction types. Thus, transactions are marshaled into the generic `[]byte` type using some kind of [encoding format](./05-encoding.md) such as [Amino](./05-encoding.md). -* **Logger:** A `logger` from the CometBFT libraries. Learn more about logs [here](https://docs.cometbft.com/v0.37/core/configuration). Modules call this method to create their own unique module-specific logger. -* **VoteInfo:** A list of the ABCI type [`VoteInfo`](https://docs.cometbft.com/master/spec/abci/abci.html#voteinfo), which includes the name of a validator and a boolean indicating whether they have signed the block. -* **Gas Meters:** Specifically, a [`gasMeter`](../beginner/04-gas-fees.md#main-gas-meter) for the transaction currently being processed using the context and a [`blockGasMeter`](../beginner/04-gas-fees.md#block-gas-meter) for the entire block it belongs to. Users specify how much in fees they wish to pay for the execution of their transaction; these gas meters keep track of how much [gas](../beginner/04-gas-fees.md) has been used in the transaction or block so far. If the gas meter runs out, execution halts. -* **CheckTx Mode:** A boolean value indicating whether a transaction should be processed in `CheckTx` or `DeliverTx` mode. -* **Min Gas Price:** The minimum [gas](../beginner/04-gas-fees.md) price a node is willing to take in order to include a transaction in its block. This price is a local value configured by each node individually, and should therefore **not be used in any functions used in sequences leading to state-transitions**. -* **Consensus Params:** The ABCI type [Consensus Parameters](https://docs.cometbft.com/master/spec/abci/apps.html#consensus-parameters), which specify certain limits for the blockchain, such as maximum gas for a block. -* **Event Manager:** The event manager allows any caller with access to a `Context` to emit [`Events`](./08-events.md). Modules may define module specific - `Events` by defining various `Types` and `Attributes` or use the common definitions found in `types/`. Clients can subscribe or query for these `Events`. These `Events` are collected throughout `FinalizeBlock` and are returned to CometBFT for indexing. -* **Priority:** The transaction priority, only relevant in `CheckTx`. -* **KV `GasConfig`:** Enables applications to set a custom `GasConfig` for the `KVStore`. -* **Transient KV `GasConfig`:** Enables applications to set a custom `GasConfig` for the transiant `KVStore`. -* **StreamingManager:** The streamingManager field provides access to the streaming manager, which allows modules to subscribe to state changes emitted by the blockchain. The streaming manager is used by the state listening API, which is described in [ADR 038](https://docs.cosmos.network/main/architecture/adr-038-state-listening). -* **CometInfo:** A lightweight field that contains information about the current block, such as the block height, time, and hash. This information can be used for validating evidence, providing historical data, and enhancing the user experience. For further details see [here](https://github.com/cosmos/cosmos-sdk/blob/main/core/comet/service.go#L14). -* **HeaderInfo:** The `headerInfo` field contains information about the current block header, such as the chain ID, gas limit, and timestamp. For further details see [here](https://github.com/cosmos/cosmos-sdk/blob/main/core/header/service.go#L14). - -## Go Context Package - -A basic `Context` is defined in the [Golang Context Package](https://pkg.go.dev/context). A `Context` -is an immutable data structure that carries request-scoped data across APIs and processes. Contexts -are also designed to enable concurrency and to be used in goroutines. - -Contexts are intended to be **immutable**; they should never be edited. Instead, the convention is -to create a child context from its parent using a `With` function. For example: - -```go -childCtx = parentCtx.WithBlockHeader(header) -``` - -The [Golang Context Package](https://pkg.go.dev/context) documentation instructs developers to -explicitly pass a context `ctx` as the first argument of a process. - -## Store branching - -The `Context` contains a `MultiStore`, which allows for branchinig and caching functionality using `CacheMultiStore` -(queries in `CacheMultiStore` are cached to avoid future round trips). -Each `KVStore` is branched in a safe and isolated ephemeral storage. Processes are free to write changes to -the `CacheMultiStore`. If a state-transition sequence is performed without issue, the store branch can -be committed to the underlying store at the end of the sequence or disregard them if something -goes wrong. The pattern of usage for a Context is as follows: - -1. A process receives a Context `ctx` from its parent process, which provides information needed to - perform the process. -2. The `ctx.ms` is a **branched store**, i.e. a branch of the [multistore](./04-store.md#multistore) is made so that the process can make changes to the state as it executes, without changing the original`ctx.ms`. This is useful to protect the underlying multistore in case the changes need to be reverted at some point in the execution. -3. The process may read and write from `ctx` as it is executing. It may call a subprocess and pass - `ctx` to it as needed. -4. When a subprocess returns, it checks if the result is a success or failure. If a failure, nothing - needs to be done - the branch `ctx` is simply discarded. If successful, the changes made to - the `CacheMultiStore` can be committed to the original `ctx.ms` via `Write()`. - -For example, here is a snippet from the [`runTx`](./00-baseapp.md#runtx-antehandler-runmsgs-posthandler) function in [`baseapp`](./00-baseapp.md): - -```go -runMsgCtx, msCache := app.cacheTxContext(ctx, txBytes) -result = app.runMsgs(runMsgCtx, msgs, mode) -result.GasWanted = gasWanted -if mode != runTxModeDeliver { - return result -} -if result.IsOK() { - msCache.Write() -} -``` - -Here is the process: - -1. Prior to calling `runMsgs` on the message(s) in the transaction, it uses `app.cacheTxContext()` - to branch and cache the context and multistore. -2. `runMsgCtx` - the context with branched store, is used in `runMsgs` to return a result. -3. If the process is running in [`checkTxMode`](./00-baseapp.md#checktx), there is no need to write the - changes - the result is returned immediately. -4. If the process is running in [`deliverTxMode`](./00-baseapp.md#delivertx) and the result indicates - a successful run over all the messages, the branched multistore is written back to the original. diff --git a/docs/learn/advanced/03-node.md b/docs/learn/advanced/03-node.md deleted file mode 100644 index 47b691b32c6b..000000000000 --- a/docs/learn/advanced/03-node.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Node Client (Daemon) - -:::note Synopsis -The main endpoint of a Cosmos SDK application is the daemon client, otherwise known as the full-node client. The full-node runs the state-machine, starting from a genesis file. It connects to peers running the same client in order to receive and relay transactions, block proposals and signatures. The full-node is constituted of the application, defined with the Cosmos SDK, and of a consensus engine connected to the application via the ABCI. -::: - -:::note Pre-requisite Readings - -* [Anatomy of an SDK application](../beginner/00-app-anatomy.md) - -::: - -## `main` function - -The full-node client of any Cosmos SDK application is built by running a `main` function. The client is generally named by appending the `-d` suffix to the application name (e.g. `appd` for an application named `app`), and the `main` function is defined in a `./appd/cmd/main.go` file. Running this function creates an executable `appd` that comes with a set of commands. For an app named `app`, the main command is [`appd start`](#start-command), which starts the full-node. - -In general, developers will implement the `main.go` function with the following structure: - -* First, an [`encodingCodec`](./05-encoding.md) is instantiated for the application. -* Then, the `config` is retrieved and config parameters are set. This mainly involves setting the Bech32 prefixes for [addresses](../beginner/03-accounts.md#addresses). - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/config.go#L14-L29 -``` - -* Using [cobra](https://github.com/spf13/cobra), the root command of the full-node client is created. After that, all the custom commands of the application are added using the `AddCommand()` method of `rootCmd`. -* Add default server commands to `rootCmd` using the `server.AddCommands()` method. These commands are separated from the ones added above since they are standard and defined at Cosmos SDK level. They should be shared by all Cosmos SDK-based applications. They include the most important command: the [`start` command](#start-command). -* Prepare and execute the `executor`. - -```go reference -https://github.com/cometbft/cometbft/blob/v0.37.0/libs/cli/setup.go#L74-L78 -``` - -See an example of `main` function from the `simapp` application, the Cosmos SDK's application for demo purposes: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/main.go -``` - -## `start` command - -The `start` command is defined in the `/server` folder of the Cosmos SDK. It is added to the root command of the full-node client in the [`main` function](#main-function) and called by the end-user to start their node: - -```bash -# For an example app named "app", the following command starts the full-node. -appd start - -# Using the Cosmos SDK's own simapp, the following commands start the simapp node. -simd start -``` - -As a reminder, the full-node is composed of three conceptual layers: the networking layer, the consensus layer and the application layer. The first two are generally bundled together in an entity called the consensus engine (CometBFT by default), while the third is the state-machine defined with the help of the Cosmos SDK. Currently, the Cosmos SDK uses CometBFT as the default consensus engine, meaning the start command is implemented to boot up a CometBFT node. - -The flow of the `start` command is pretty straightforward. First, it retrieves the `config` from the `context` in order to open the `db` (a [`leveldb`](https://github.com/syndtr/goleveldb) instance by default). This `db` contains the latest known state of the application (empty if the application is started from the first time. - -With the `db`, the `start` command creates a new instance of the application using an `appCreator` function: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/start.go#L220 -``` - -Note that an `appCreator` is a function that fulfills the `AppCreator` signature: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/types/app.go#L68 -``` - -In practice, the [constructor of the application](../beginner/00-app-anatomy.md#constructor-function) is passed as the `appCreator`. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L294-L308 -``` - -Then, the instance of `app` is used to instantiate a new CometBFT node: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/start.go#L341-L378 -``` - -The CometBFT node can be created with `app` because the latter satisfies the [`abci.Application` interface](https://github.com/cometbft/cometbft/blob/v0.37.0/abci/types/application.go#L9-L35) (given that `app` extends [`baseapp`](./00-baseapp.md)). As part of the `node.New` method, CometBFT makes sure that the height of the application (i.e. number of blocks since genesis) is equal to the height of the CometBFT node. The difference between these two heights should always be negative or null. If it is strictly negative, `node.New` will replay blocks until the height of the application reaches the height of the CometBFT node. Finally, if the height of the application is `0`, the CometBFT node will call [`InitChain`](./00-baseapp.md#initchain) on the application to initialize the state from the genesis file. - -Once the CometBFT node is instantiated and in sync with the application, the node can be started: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/start.go#L350-L352 -``` - -Upon starting, the node will bootstrap its RPC and P2P server and start dialing peers. During handshake with its peers, if the node realizes they are ahead, it will query all the blocks sequentially in order to catch up. Then, it will wait for new block proposals and block signatures from validators in order to make progress. - -## Other commands - -To discover how to concretely run a node and interact with it, please refer to our [Running a Node, API and CLI](../../user/run-node/01-run-node.md) guide. diff --git a/docs/learn/advanced/04-store.md b/docs/learn/advanced/04-store.md deleted file mode 100644 index 35f67665752a..000000000000 --- a/docs/learn/advanced/04-store.md +++ /dev/null @@ -1,288 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Store - -:::note Synopsis -A store is a data structure that holds the state of the application. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) - -::: - -## Introduction to Cosmos SDK Stores - -The Cosmos SDK comes with a large set of stores to persist the state of applications. By default, the main store of Cosmos SDK applications is a `multistore`, i.e. a store of stores. Developers can add any number of key-value stores to the multistore, depending on their application needs. The multistore exists to support the modularity of the Cosmos SDK, as it lets each module declare and manage their own subset of the state. Key-value stores in the multistore can only be accessed with a specific capability `key`, which is typically held in the [`keeper`](../../build/building-modules/06-keeper.md) of the module that declared the store. - -```text -+-----------------------------------------------------+ -| | -| +--------------------------------------------+ | -| | | | -| | KVStore 1 - Manage by keeper of Module 1 | -| | | | -| +--------------------------------------------+ | -| | -| +--------------------------------------------+ | -| | | | -| | KVStore 2 - Manage by keeper of Module 2 | | -| | | | -| +--------------------------------------------+ | -| | -| +--------------------------------------------+ | -| | | | -| | KVStore 3 - Manage by keeper of Module 2 | | -| | | | -| +--------------------------------------------+ | -| | -| +--------------------------------------------+ | -| | | | -| | KVStore 4 - Manage by keeper of Module 3 | | -| | | | -| +--------------------------------------------+ | -| | -| +--------------------------------------------+ | -| | | | -| | KVStore 5 - Manage by keeper of Module 4 | | -| | | | -| +--------------------------------------------+ | -| | -| Main Multistore | -| | -+-----------------------------------------------------+ - - Application's State -``` - -### Store Interface - -At its very core, a Cosmos SDK `store` is an object that holds a `CacheWrapper` and has a `GetStoreType()` method: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L15-L18 -``` - -The `GetStoreType` is a simple method that returns the type of store, whereas a `CacheWrapper` is a simple interface that implements store read caching and write branching through `Write` method: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L287-L320 -``` - -Branching and cache is used ubiquitously in the Cosmos SDK and required to be implemented on every store type. A storage branch creates an isolated, ephemeral branch of a store that can be passed around and updated without affecting the main underlying store. This is used to trigger temporary state-transitions that may be reverted later should an error occur. Read more about it in [context](./02-context.md#Store-branching) - -### Commit Store - -A commit store is a store that has the ability to commit changes made to the underlying tree or db. The Cosmos SDK differentiates simple stores from commit stores by extending the basic store interfaces with a `Committer`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L32-L37 -``` - -The `Committer` is an interface that defines methods to persist changes to disk: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L20-L30 -``` - -The `CommitID` is a deterministic commit of the state tree. Its hash is returned to the underlying consensus engine and stored in the block header. Note that commit store interfaces exist for various purposes, one of which is to make sure not every object can commit the store. As part of the [object-capabilities model](./10-ocap.md) of the Cosmos SDK, only `baseapp` should have the ability to commit stores. For example, this is the reason why the `ctx.KVStore()` method by which modules typically access stores returns a `KVStore` and not a `CommitKVStore`. - -The Cosmos SDK comes with many types of stores, the most used being [`CommitMultiStore`](#multistore), [`KVStore`](#kvstore) and [`GasKv` store](#gaskv-store). [Other types of stores](#other-stores) include `Transient` and `TraceKV` stores. - -## Multistore - -### Multistore Interface - -Each Cosmos SDK application holds a multistore at its root to persist its state. The multistore is a store of `KVStores` that follows the `Multistore` interface: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L123-L155 -``` - -If tracing is enabled, then branching the multistore will firstly wrap all the underlying `KVStore` in [`TraceKv.Store`](#tracekv-store). - -### CommitMultiStore - -The main type of `Multistore` used in the Cosmos SDK is `CommitMultiStore`, which is an extension of the `Multistore` interface: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L164-L227 -``` - -As for concrete implementation, the [`rootMulti.Store`] is the go-to implementation of the `CommitMultiStore` interface. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/rootmulti/store.go#L53-L77 -``` - -The `rootMulti.Store` is a base-layer multistore built around a `db` on top of which multiple `KVStores` can be mounted, and is the default multistore store used in [`baseapp`](./00-baseapp.md). - -### CacheMultiStore - -Whenever the `rootMulti.Store` needs to be branched, a [`cachemulti.Store`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/cachemulti/store.go) is used. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/cachemulti/store.go#L19-L33 -``` - -`cachemulti.Store` branches all substores (creates a virtual store for each substore) in its constructor and hold them in `Store.stores`. Moreover caches all read queries. `Store.GetKVStore()` returns the store from `Store.stores`, and `Store.Write()` recursively calls `CacheWrap.Write()` on all the substores. - -## Base-layer KVStores - -### `KVStore` and `CommitKVStore` Interfaces - -A `KVStore` is a simple key-value store used to store and retrieve data. A `CommitKVStore` is a `KVStore` that also implements a `Committer`. By default, stores mounted in `baseapp`'s main `CommitMultiStore` are `CommitKVStore`s. The `KVStore` interface is primarily used to restrict modules from accessing the committer. - -Individual `KVStore`s are used by modules to manage a subset of the global state. `KVStores` can be accessed by objects that hold a specific key. This `key` should only be exposed to the [`keeper`](../../build/building-modules/06-keeper.md) of the module that defines the store. - -`CommitKVStore`s are declared by proxy of their respective `key` and mounted on the application's [multistore](#multistore) in the [main application file](../beginner/00-app-anatomy.md#core-application-file). In the same file, the `key` is also passed to the module's `keeper` that is responsible for managing the store. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L229-L266 -``` - -Apart from the traditional `Get` and `Set` methods, that a `KVStore` must implement via the `BasicKVStore` interface; a `KVStore` must provide an `Iterator(start, end)` method which returns an `Iterator` object. It is used to iterate over a range of keys, typically keys that share a common prefix. Below is an example from the bank's module keeper, used to iterate over all account balances: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/keeper/view.go#L125-L140 -``` - -### `IAVL` Store - -The default implementation of `KVStore` and `CommitKVStore` used in `baseapp` is the `iavl.Store`. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/iavl/store.go#L35-L40 -``` - -`iavl` stores are based around an [IAVL Tree](https://github.com/cosmos/iavl), a self-balancing binary tree which guarantees that: - -* `Get` and `Set` operations are O(log n), where n is the number of elements in the tree. -* Iteration efficiently returns the sorted elements within the range. -* Each tree version is immutable and can be retrieved even after a commit (depending on the pruning settings). - -The documentation on the IAVL Tree is located [here](https://github.com/cosmos/iavl/blob/master/docs/overview.md). - -### `DbAdapter` Store - -`dbadapter.Store` is a adapter for `dbm.DB` making it fulfilling the `KVStore` interface. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/dbadapter/store.go#L13-L16 -``` - -`dbadapter.Store` embeds `dbm.DB`, meaning most of the `KVStore` interface functions are implemented. The other functions (mostly miscellaneous) are manually implemented. This store is primarily used within [Transient Stores](#transient-store) - -### `Transient` Store - -`Transient.Store` is a base-layer `KVStore` which is automatically discarded at the end of the block. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/transient/store.go#L16-L19 -``` - -`Transient.Store` is a `dbadapter.Store` with a `dbm.NewMemDB()`. All `KVStore` methods are reused. When `Store.Commit()` is called, a new `dbadapter.Store` is assigned, discarding previous reference and making it garbage collected. - -This type of store is useful to persist information that is only relevant per-block. One example would be to store parameter changes (i.e. a bool set to `true` if a parameter changed in a block). - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/params/types/subspace.go#L21-L31 -``` - -Transient stores are typically accessed via the [`context`](./02-context.md) via the `TransientStore()` method: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/context.go#L340-L343 -``` - -## KVStore Wrappers - -### CacheKVStore - -`cachekv.Store` is a wrapper `KVStore` which provides buffered writing / cached reading functionalities over the underlying `KVStore`. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/cachekv/store.go#L26-L36 -``` - -This is the type used whenever an IAVL Store needs to be branched to create an isolated store (typically when we need to mutate a state that might be reverted later). - -#### `Get` - -`Store.Get()` firstly checks if `Store.cache` has an associated value with the key. If the value exists, the function returns it. If not, the function calls `Store.parent.Get()`, caches the result in `Store.cache`, and returns it. - -#### `Set` - -`Store.Set()` sets the key-value pair to the `Store.cache`. `cValue` has the field dirty bool which indicates whether the cached value is different from the underlying value. When `Store.Set()` caches a new pair, the `cValue.dirty` is set `true` so when `Store.Write()` is called it can be written to the underlying store. - -#### `Iterator` - -`Store.Iterator()` have to traverse on both cached items and the original items. In `Store.iterator()`, two iterators are generated for each of them, and merged. `memIterator` is essentially a slice of the `KVPairs`, used for cached items. `mergeIterator` is a combination of two iterators, where traverse happens ordered on both iterators. - -### `GasKv` Store - -Cosmos SDK applications use [`gas`](../beginner/04-gas-fees.md) to track resources usage and prevent spam. [`GasKv.Store`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/gaskv/store.go) is a `KVStore` wrapper that enables automatic gas consumption each time a read or write to the store is made. It is the solution of choice to track storage usage in Cosmos SDK applications. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/gaskv/store.go#L11-L17 -``` - -When methods of the parent `KVStore` are called, `GasKv.Store` automatically consumes appropriate amount of gas depending on the `Store.gasConfig`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/gas.go#L219-L228 -``` - -By default, all `KVStores` are wrapped in `GasKv.Stores` when retrieved. This is done in the `KVStore()` method of the [`context`](./02-context.md): - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/context.go#L335-L338 -``` - -In this case, the gas configuration set in the `context` is used. The gas configuration can be set using the `WithKVGasConfig` method of the `context`. -Otherwise it uses the following default: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/gas.go#L230-L241 -``` - -### `TraceKv` Store - -`tracekv.Store` is a wrapper `KVStore` which provides operation tracing functionalities over the underlying `KVStore`. It is applied automatically by the Cosmos SDK on all `KVStore` if tracing is enabled on the parent `MultiStore`. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/tracekv/store.go#L20-L43 -``` - -When each `KVStore` methods are called, `tracekv.Store` automatically logs `traceOperation` to the `Store.writer`. `traceOperation.Metadata` is filled with `Store.context` when it is not nil. `TraceContext` is a `map[string]interface{}`. - -### `Prefix` Store - -`prefix.Store` is a wrapper `KVStore` which provides automatic key-prefixing functionalities over the underlying `KVStore`. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/prefix/store.go#L15-L21 -``` - -When `Store.{Get, Set}()` is called, the store forwards the call to its parent, with the key prefixed with the `Store.prefix`. - -When `Store.Iterator()` is called, it does not simply prefix the `Store.prefix`, since it does not work as intended. In that case, some of the elements are traversed even they are not starting with the prefix. - -### `ListenKv` Store - -`listenkv.Store` is a wrapper `KVStore` which provides state listening capabilities over the underlying `KVStore`. -It is applied automatically by the Cosmos SDK on any `KVStore` whose `StoreKey` is specified during state streaming configuration. -Additional information about state streaming configuration can be found in the [store/streaming/README.md](https://github.com/cosmos/cosmos-sdk/tree/v0.50.0-alpha.0/store/streaming). - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/listenkv/store.go#L11-L18 -``` - -When `KVStore.Set` or `KVStore.Delete` methods are called, `listenkv.Store` automatically writes the operations to the set of `Store.listeners`. - -## `BasicKVStore` interface - -An interface providing only the basic CRUD functionality (`Get`, `Set`, `Has`, and `Delete` methods), without iteration or caching. This is used to partially expose components of a larger store. diff --git a/docs/learn/advanced/05-encoding.md b/docs/learn/advanced/05-encoding.md deleted file mode 100644 index 097767d97d7e..000000000000 --- a/docs/learn/advanced/05-encoding.md +++ /dev/null @@ -1,287 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Encoding - -:::note Synopsis -While encoding in the Cosmos SDK used to be mainly handled by `go-amino` codec, the Cosmos SDK is moving towards using `gogoprotobuf` for both state and client-side encoding. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) - -::: - -## Encoding - -The Cosmos SDK utilizes two binary wire encoding protocols, [Amino](https://github.com/tendermint/go-amino/) which is an object encoding specification and [Protocol Buffers](https://developers.google.com/protocol-buffers), a subset of Proto3 with an extension for -interface support. See the [Proto3 spec](https://developers.google.com/protocol-buffers/docs/proto3) -for more information on Proto3, which Amino is largely compatible with (but not with Proto2). - -Due to Amino having significant performance drawbacks, being reflection-based, and -not having any meaningful cross-language/client support, Protocol Buffers, specifically -[gogoprotobuf](https://github.com/cosmos/gogoproto/), is being used in place of Amino. -Note, this process of using Protocol Buffers over Amino is still an ongoing process. - -Binary wire encoding of types in the Cosmos SDK can be broken down into two main -categories, client encoding and store encoding. Client encoding mainly revolves -around transaction processing and signing, whereas store encoding revolves around -types used in state-machine transitions and what is ultimately stored in the Merkle -tree. - -For store encoding, protobuf definitions can exist for any type and will typically -have an Amino-based "intermediary" type. Specifically, the protobuf-based type -definition is used for serialization and persistence, whereas the Amino-based type -is used for business logic in the state-machine where they may convert back-n-forth. -Note, the Amino-based types may slowly be phased-out in the future, so developers -should take note to use the protobuf message definitions where possible. - -In the `codec` package, there exists two core interfaces, `BinaryCodec` and `JSONCodec`, -where the former encapsulates the current Amino interface except it operates on -types implementing the latter instead of generic `interface{}` types. - -The `ProtoCodec`, where both binary and JSON serialization is handled -via Protobuf. This means that modules may use Protobuf encoding, but the types must -implement `ProtoMarshaler`. If modules wish to avoid implementing this interface -for their types, this is autogenerated via [buf](https://buf.build/) - -If modules use [Collections](../../build/packages/02-collections.md) or [ORM](../../build/packages/03-orm.md), encoding and decoding are handled, marshal and unmarshal should not be handled manually unless for specific cases identified by the developer. - -```go reference - -### Gogoproto - -Modules are encouraged to utilize Protobuf encoding for their respective types. In the Cosmos SDK, we use the [Gogoproto](https://github.com/cosmos/gogoproto) specific implementation of the Protobuf spec that offers speed and DX improvements compared to the official [Google protobuf implementation](https://github.com/protocolbuffers/protobuf). - -### Guidelines for protobuf message definitions - -In addition to [following official Protocol Buffer guidelines](https://developers.google.com/protocol-buffers/docs/proto3#simple), we recommend using these annotations in .proto files when dealing with interfaces: - -* use `cosmos_proto.accepts_interface` to annote `Any` fields that accept interfaces - * pass the same fully qualified name as `protoName` to `InterfaceRegistry.RegisterInterface` - * example: `(cosmos_proto.accepts_interface) = "cosmos.gov.v1beta1.Content"` (and not just `Content`) -* annotate interface implementations with `cosmos_proto.implements_interface` - * pass the same fully qualified name as `protoName` to `InterfaceRegistry.RegisterInterface` - * example: `(cosmos_proto.implements_interface) = "cosmos.authz.v1beta1.Authorization"` (and not just `Authorization`) - -Code generators can then match the `accepts_interface` and `implements_interface` annotations to know whether some Protobuf messages are allowed to be packed in a given `Any` field or not. - -### Transaction Encoding - -Another important use of Protobuf is the encoding and decoding of -[transactions](./01-transactions.md). Transactions are defined by the application or -the Cosmos SDK but are then passed to the underlying consensus engine to be relayed to -other peers. Since the underlying consensus engine is agnostic to the application, -the consensus engine accepts only transactions in the form of raw bytes. - -* The `TxEncoder` object performs the encoding. -* The `TxDecoder` object performs the decoding. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/tx_msg.go#L91-L95 -``` - -A standard implementation of both these objects can be found in the [`auth/tx` module](../../build/modules/auth/2-tx.md): - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/decoder.go -``` - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/encoder.go -``` - -See [ADR-020](../../build/architecture/adr-020-protobuf-transaction-encoding.md) for details of how a transaction is encoded. - -### Interface Encoding and Usage of `Any` - -The Protobuf DSL is strongly typed, which can make inserting variable-typed fields difficult. Imagine we want to create a `Profile` protobuf message that serves as a wrapper over [an account](../beginner/03-accounts.md): - -```protobuf -message Profile { - // account is the account associated to a profile. - cosmos.auth.v1beta1.BaseAccount account = 1; - // bio is a short description of the account. - string bio = 4; -} -``` - -In this `Profile` example, we hardcoded `account` as a `BaseAccount`. However, there are several other types of [user accounts related to vesting](../../build/modules/auth/1-vesting.md), such as `BaseVestingAccount` or `ContinuousVestingAccount`. All of these accounts are different, but they all implement the `AccountI` interface. How would you create a `Profile` that allows all these types of accounts with an `account` field that accepts an `AccountI` interface? - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/account.go#L15-L32 -``` - -In [ADR-019](../../build/architecture/adr-019-protobuf-state-encoding.md), it has been decided to use [`Any`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto)s to encode interfaces in protobuf. An `Any` contains an arbitrary serialized message as bytes, along with a URL that acts as a globally unique identifier for and resolves to that message's type. This strategy allows us to pack arbitrary Go types inside protobuf messages. Our new `Profile` then looks like: - -```protobuf -message Profile { - // account is the account associated to a profile. - google.protobuf.Any account = 1 [ - (cosmos_proto.accepts_interface) = "cosmos.auth.v1beta1.AccountI"; // Asserts that this field only accepts Go types implementing `AccountI`. It is purely informational for now. - ]; - // bio is a short description of the account. - string bio = 4; -} -``` - -To add an account inside a profile, we need to "pack" it inside an `Any` first, using `codectypes.NewAnyWithValue`: - -```go -var myAccount AccountI -myAccount = ... // Can be a BaseAccount, a ContinuousVestingAccount or any struct implementing `AccountI` - -// Pack the account into an Any -accAny, err := codectypes.NewAnyWithValue(myAccount) -if err != nil { - return nil, err -} - -// Create a new Profile with the any. -profile := Profile { - Account: accAny, - Bio: "some bio", -} - -// We can then marshal the profile as usual. -bz, err := cdc.Marshal(profile) -jsonBz, err := cdc.MarshalJSON(profile) -``` - -To summarize, to encode an interface, you must 1/ pack the interface into an `Any` and 2/ marshal the `Any`. For convenience, the Cosmos SDK provides a `MarshalInterface` method to bundle these two steps. Have a look at [a real-life example in the x/auth module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/keeper/keeper.go#L240-L243). - -The reverse operation of retrieving the concrete Go type from inside an `Any`, called "unpacking", is done with the `GetCachedValue()` on `Any`. - -```go -profileBz := ... // The proto-encoded bytes of a Profile, e.g. retrieved through gRPC. -var myProfile Profile -// Unmarshal the bytes into the myProfile struct. -err := cdc.Unmarshal(profilebz, &myProfile) - -// Let's see the types of the Account field. -fmt.Printf("%T\n", myProfile.Account) // Prints "Any" -fmt.Printf("%T\n", myProfile.Account.GetCachedValue()) // Prints "BaseAccount", "ContinuousVestingAccount" or whatever was initially packed in the Any. - -// Get the address of the accountt. -accAddr := myProfile.Account.GetCachedValue().(AccountI).GetAddress() -``` - -It is important to note that for `GetCachedValue()` to work, `Profile` (and any other structs embedding `Profile`) must implement the `UnpackInterfaces` method: - -```go -func (p *Profile) UnpackInterfaces(unpacker codectypes.AnyUnpacker) error { - if p.Account != nil { - var account AccountI - return unpacker.UnpackAny(p.Account, &account) - } - - return nil -} -``` - -The `UnpackInterfaces` gets called recursively on all structs implementing this method, to allow all `Any`s to have their `GetCachedValue()` correctly populated. - -For more information about interface encoding, and especially on `UnpackInterfaces` and how the `Any`'s `type_url` gets resolved using the `InterfaceRegistry`, please refer to [ADR-019](../../build/architecture/adr-019-protobuf-state-encoding.md). - -#### `Any` Encoding in the Cosmos SDK - -The above `Profile` example is a fictive example used for educational purposes. In the Cosmos SDK, we use `Any` encoding in several places (non-exhaustive list): - -* the `cryptotypes.PubKey` interface for encoding different types of public keys, -* the `sdk.Msg` interface for encoding different `Msg`s in a transaction, -* the `AccountI` interface for encodinig different types of accounts (similar to the above example) in the x/auth query responses, -* the `Evidencei` interface for encoding different types of evidences in the x/evidence module, -* the `AuthorizationI` interface for encoding different types of x/authz authorizations, -* the [`Validator`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/types/staking.pb.go#L340-L377) struct that contains information about a validator. - -A real-life example of encoding the pubkey as `Any` inside the Validator struct in x/staking is shown in the following example: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/types/validator.go#L41-L64 -``` - -#### `Any`'s TypeURL - -When packing a protobuf message inside an `Any`, the message's type is uniquely defined by its type URL, which is the message's fully qualified name prefixed by a `/` (slash) character. In some implementations of `Any`, like the gogoproto one, there's generally [a resolvable prefix, e.g. `type.googleapis.com`](https://github.com/gogo/protobuf/blob/b03c65ea87cdc3521ede29f62fe3ce239267c1bc/protobuf/google/protobuf/any.proto#L87-L91). However, in the Cosmos SDK, we made the decision to not include such prefix, to have shorter type URLs. The Cosmos SDK's own `Any` implementation can be found in `github.com/cosmos/cosmos-sdk/codec/types`. - -The Cosmos SDK is also switching away from gogoproto to the official `google.golang.org/protobuf` (known as the Protobuf API v2). Its default `Any` implementation also contains the [`type.googleapis.com`](https://github.com/protocolbuffers/protobuf-go/blob/v1.28.1/types/known/anypb/any.pb.go#L266) prefix. To maintain compatibility with the SDK, the following methods from `"google.golang.org/protobuf/types/known/anypb"` should not be used: - -* `anypb.New` -* `anypb.MarshalFrom` -* `anypb.Any#MarshalFrom` - -Instead, the Cosmos SDK provides helper functions in `"github.com/cosmos/cosmos-proto/anyutil"`, which create an official `anypb.Any` without inserting the prefixes: - -* `anyutil.New` -* `anyutil.MarshalFrom` - -For example, to pack a `sdk.Msg` called `internalMsg`, use: - -```diff -import ( -- "google.golang.org/protobuf/types/known/anypb" -+ "github.com/cosmos/cosmos-proto/anyutil" -) - -- anyMsg, err := anypb.New(internalMsg.Message().Interface()) -+ anyMsg, err := anyutil.New(internalMsg.Message().Interface()) - -- fmt.Println(anyMsg.TypeURL) // type.googleapis.com/cosmos.bank.v1beta1.MsgSend -+ fmt.Println(anyMsg.TypeURL) // /cosmos.bank.v1beta1.MsgSend -``` - -## FAQ - -### How to create modules using protobuf encoding - -#### Defining module types - -Protobuf types can be defined to encode: - -* state -* [`Msg`s](../../build/building-modules/02-messages-and-queries.md#messages) -* [Query services](../../build/building-modules/04-query-services.md) -* [genesis](../../build/building-modules/08-genesis.md) - -#### Naming and conventions - -We encourage developers to follow industry guidelines: [Protocol Buffers style guide](https://developers.google.com/protocol-buffers/docs/style) -and [Buf](https://buf.build/docs/style-guide), see more details in [ADR 023](../../build/architecture/adr-023-protobuf-naming.md) - -### How to update modules to protobuf encoding - -If modules do not contain any interfaces (e.g. `Account` or `Content`), then they -may simply migrate any existing types that -are encoded and persisted via their concrete Amino codec to Protobuf (see 1. for further guidelines) and accept a `Marshaler` as the codec which is implemented via the `ProtoCodec` -without any further customization. - -However, if a module type composes an interface, it must wrap it in the `sdk.Any` (from `/types` package) type. To do that, a module-level .proto file must use [`google.protobuf.Any`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto) for respective message type interface types. - -For example, in the `x/evidence` module defines an `Evidence` interface, which is used by the `MsgSubmitEvidence`. The structure definition must use `sdk.Any` to wrap the evidence file. In the proto file we define it as follows: - -```protobuf -// proto/cosmos/evidence/v1beta1/tx.proto - -message MsgSubmitEvidence { - string submitter = 1; - google.protobuf.Any evidence = 2 [(cosmos_proto.accepts_interface) = "cosmos.evidence.v1beta1.Evidence"]; -} -``` - -The Cosmos SDK `codec.Codec` interface provides support methods `MarshalInterface` and `UnmarshalInterface` to easy encoding of state to `Any`. - -Module should register interfaces using `InterfaceRegistry` which provides a mechanism for registering interfaces: `RegisterInterface(protoName string, iface interface{}, impls ...proto.Message)` and implementations: `RegisterImplementations(iface interface{}, impls ...proto.Message)` that can be safely unpacked from Any, similarly to type registration with Amino: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/codec/types/interface_registry.go#L28-L75 -``` - -In addition, an `UnpackInterfaces` phase should be introduced to deserialization to unpack interfaces before they're needed. Protobuf types that contain a protobuf `Any` either directly or via one of their members should implement the `UnpackInterfacesMessage` interface: - -```go -type UnpackInterfacesMessage interface { - UnpackInterfaces(InterfaceUnpacker) error -} -``` diff --git a/docs/learn/advanced/06-grpc_rest.md b/docs/learn/advanced/06-grpc_rest.md deleted file mode 100644 index f9ef118a3979..000000000000 --- a/docs/learn/advanced/06-grpc_rest.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -sidebar_position: 1 ---- - -# gRPC, REST, and CometBFT Endpoints - -:::note Synopsis -This document presents an overview of all the endpoints a node exposes: gRPC, REST as well as some other endpoints. -::: - -## An Overview of All Endpoints - -Each node exposes the following endpoints for users to interact with a node, each endpoint is served on a different port. Details on how to configure each endpoint is provided in the endpoint's own section. - -* the gRPC server (default port: `9090`), -* the REST server (default port: `1317`), -* the CometBFT RPC endpoint (default port: `26657`). - -:::tip -The node also exposes some other endpoints, such as the CometBFT P2P endpoint, or the [Prometheus endpoint](https://docs.cometbft.com/v0.37/core/metrics), which are not directly related to the Cosmos SDK. Please refer to the [CometBFT documentation](https://docs.cometbft.com/v0.37/core/configuration) for more information about these endpoints. -::: - -:::note -All endpoints are defaulted to localhost and must be modified to be exposed to the public internet. -::: - -## gRPC Server - -In the Cosmos SDK, Protobuf is the main [encoding](./encoding) library. This brings a wide range of Protobuf-based tools that can be plugged into the Cosmos SDK. One such tool is [gRPC](https://grpc.io), a modern open-source high performance RPC framework that has decent client support in several languages. - -Each module exposes a [Protobuf `Query` service](../../build/building-modules/02-messages-and-queries.md#queries) that defines state queries. The `Query` services and a transaction service used to broadcast transactions are hooked up to the gRPC server via the following function inside the application: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/types/app.go#L46-L48 -``` - -Note: It is not possible to expose any [Protobuf `Msg` service](../../build/building-modules/02-messages-and-queries.md#messages) endpoints via gRPC. Transactions must be generated and signed using the CLI or programmatically before they can be broadcasted using gRPC. See [Generating, Signing, and Broadcasting Transactions](../../user/run-node/03-txs.md) for more information. - -The `grpc.Server` is a concrete gRPC server, which spawns and serves all gRPC query requests and a broadcast transaction request. This server can be configured inside `~/.simapp/config/app.toml`: - -* `grpc.enable = true|false` field defines if the gRPC server should be enabled. Defaults to `true`. -* `grpc.address = {string}` field defines the `ip:port` the server should bind to. Defaults to `localhost:9090`. - -:::tip -`~/.simapp` is the directory where the node's configuration and databases are stored. By default, it's set to `~/.{app_name}`. -::: - -Once the gRPC server is started, you can send requests to it using a gRPC client. Some examples are given in our [Interact with the Node](../../user/run-node/02-interact-node.md#using-grpc) tutorial. - -An overview of all available gRPC endpoints shipped with the Cosmos SDK is [Protobuf documentation](https://buf.build/cosmos/cosmos-sdk). - -## REST Server - -Cosmos SDK supports REST routes via gRPC-gateway. - -All routes are configured under the following fields in `~/.simapp/config/app.toml`: - -* `api.enable = true|false` field defines if the REST server should be enabled. Defaults to `false`. -* `api.address = {string}` field defines the `ip:port` the server should bind to. Defaults to `tcp://localhost:1317`. -* some additional API configuration options are defined in `~/.simapp/config/app.toml`, along with comments, please refer to that file directly. - -### gRPC-gateway REST Routes - -If, for various reasons, you cannot use gRPC (for example, you are building a web application, and browsers don't support HTTP2 on which gRPC is built), then the Cosmos SDK offers REST routes via gRPC-gateway. - -[gRPC-gateway](https://grpc-ecosystem.github.io/grpc-gateway/) is a tool to expose gRPC endpoints as REST endpoints. For each gRPC endpoint defined in a Protobuf `Query` service, the Cosmos SDK offers a REST equivalent. For instance, querying a balance could be done via the `/cosmos.bank.v1beta1.QueryAllBalances` gRPC endpoint, or alternatively via the gRPC-gateway `"/cosmos/bank/v1beta1/balances/{address}"` REST endpoint: both will return the same result. For each RPC method defined in a Protobuf `Query` service, the corresponding REST endpoint is defined as an option: - -```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/bank/v1beta1/query.proto#L23-L30 -``` - -For application developers, gRPC-gateway REST routes needs to be wired up to the REST server, this is done by calling the `RegisterGRPCGatewayRoutes` function on the ModuleManager. - -### Swagger - -A [Swagger](https://swagger.io/) (or OpenAPIv2) specification file is exposed under the `/swagger` route on the API server. Swagger is an open specification describing the API endpoints a server serves, including description, input arguments, return types and much more about each endpoint. - -Enabling the `/swagger` endpoint is configurable inside `~/.simapp/config/app.toml` via the `api.swagger` field, which is set to false by default. - -For application developers, you may want to generate your own Swagger definitions based on your custom modules. -The Cosmos SDK's [Swagger generation script](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/scripts/protoc-swagger-gen.sh) is a good place to start. - -## CometBFT RPC - -Independently from the Cosmos SDK, CometBFT also exposes a RPC server. This RPC server can be configured by tuning parameters under the `rpc` table in the `~/.simapp/config/config.toml`, the default listening address is `tcp://localhost:26657`. An OpenAPI specification of all CometBFT RPC endpoints is available [here](https://docs.cometbft.com/main/rpc/). - -Some CometBFT RPC endpoints are directly related to the Cosmos SDK: - -* `/abci_query`: this endpoint will query the application for state. As the `path` parameter, you can send the following strings: - * any Protobuf fully-qualified service method, such as `/cosmos.bank.v1beta1.Query/AllBalances`. The `data` field should then include the method's request parameter(s) encoded as bytes using Protobuf. - * `/app/simulate`: this will simulate a transaction, and return some information such as gas used. - * `/app/version`: this will return the application's version. - * `/store/{storeName}/key`: this will directly query the named store for data associated with the key represented in the `data` parameter. - * `/store/{storeName}/subspace`: this will directly query the named store for key/value pairs in which the key has the value of the `data` parameter as a prefix. - * `/p2p/filter/addr/{port}`: this will return a filtered list of the node's P2P peers by address port. - * `/p2p/filter/id/{id}`: this will return a filtered list of the node's P2P peers by ID. -* `/broadcast_tx_{aync,async,commit}`: these 3 endpoint will broadcast a transaction to other peers. CLI, gRPC and REST expose [a way to broadcast transations](./01-transactions.md#broadcasting-the-transaction), but they all use these 3 CometBFT RPCs under the hood. - -## Comparison Table - -| Name | Advantages | Disadvantages | -| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -| gRPC | - can use code-generated stubs in various languages
- supports streaming and bidirectional communication (HTTP2)
- small wire binary sizes, faster transmission | - based on HTTP2, not available in browsers
- learning curve (mostly due to Protobuf) | -| REST | - ubiquitous
- client libraries in all languages, faster implementation
| - only supports unary request-response communication (HTTP1.1)
- bigger over-the-wire message sizes (JSON) | -| CometBFT RPC | - easy to use | - bigger over-the-wire message sizes (JSON) | diff --git a/docs/learn/advanced/07-cli.md b/docs/learn/advanced/07-cli.md deleted file mode 100644 index 802e89eeddec..000000000000 --- a/docs/learn/advanced/07-cli.md +++ /dev/null @@ -1,211 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Command-Line Interface - -:::note Synopsis -This document describes how command-line interface (CLI) works on a high-level, for an [**application**](../beginner/00-app-anatomy.md). A separate document for implementing a CLI for a Cosmos SDK [**module**](../../build/building-modules/00-intro.md) can be found [here](../../build/building-modules/09-module-interfaces.md#cli). -::: - -## Command-Line Interface - -### Example Command - -There is no set way to create a CLI, but Cosmos SDK modules typically use the [Cobra Library](https://github.com/spf13/cobra). Building a CLI with Cobra entails defining commands, arguments, and flags. [**Commands**](#root-command) understand the actions users wish to take, such as `tx` for creating a transaction and `query` for querying the application. Each command can also have nested subcommands, necessary for naming the specific transaction type. Users also supply **Arguments**, such as account numbers to send coins to, and [**Flags**](#flags) to modify various aspects of the commands, such as gas prices or which node to broadcast to. - -Here is an example of a command a user might enter to interact with the simapp CLI `simd` in order to send some tokens: - -```bash -simd tx bank send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000stake --gas auto --gas-prices -``` - -The first four strings specify the command: - -* The root command for the entire application `simd`. -* The subcommand `tx`, which contains all commands that let users create transactions. -* The subcommand `bank` to indicate which module to route the command to ([`x/bank`](../../build/modules/bank/README.md) module in this case). -* The type of transaction `send`. - -The next two strings are arguments: the `from_address` the user wishes to send from, the `to_address` of the recipient, and the `amount` they want to send. Finally, the last few strings of the command are optional flags to indicate how much the user is willing to pay in fees (calculated using the amount of gas used to execute the transaction and the gas prices provided by the user). - -The CLI interacts with a [node](./03-node.md) to handle this command. The interface itself is defined in a `main.go` file. - -### Building the CLI - -The `main.go` file needs to have a `main()` function that creates a root command, to which all the application commands will be added as subcommands. The root command additionally handles: - -* **setting configurations** by reading in configuration files (e.g. the Cosmos SDK config file). -* **adding any flags** to it, such as `--chain-id`. -* **instantiating the `codec`** by injecting the application codecs. The [`codec`](./05-encoding.md) is used to encode and decode data structures for the application - stores can only persist `[]byte`s so the developer must define a serialization format for their data structures or use the default, Protobuf. -* **adding subcommand** for all the possible user interactions, including [transaction commands](#transaction-commands) and [query commands](#query-commands). - -The `main()` function finally creates an executor and [execute](https://pkg.go.dev/github.com/spf13/cobra#Command.Execute) the root command. See an example of `main()` function from the `simapp` application: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/main.go#L12-L24 -``` - -The rest of the document will detail what needs to be implemented for each step and include smaller portions of code from the `simapp` CLI files. - -## Adding Commands to the CLI - -Every application CLI first constructs a root command, then adds functionality by aggregating subcommands (often with further nested subcommands) using `rootCmd.AddCommand()`. The bulk of an application's unique capabilities lies in its transaction and query commands, called `TxCmd` and `QueryCmd` respectively. - -### Root Command - -The root command (called `rootCmd`) is what the user first types into the command line to indicate which application they wish to interact with. The string used to invoke the command (the "Use" field) is typically the name of the application suffixed with `-d`, e.g. `simd` or `gaiad`. The root command typically includes the following commands to support basic functionality in the application. - -* **Status** command from the Cosmos SDK rpc client tools, which prints information about the status of the connected [`Node`](./03-node.md). The Status of a node includes `NodeInfo`,`SyncInfo` and `ValidatorInfo`. -* **Keys** [commands](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/keys) from the Cosmos SDK client tools, which includes a collection of subcommands for using the key functions in the Cosmos SDK crypto tools, including adding a new key and saving it to the keyring, listing all public keys stored in the keyring, and deleting a key. For example, users can type `simd keys add ` to add a new key and save an encrypted copy to the keyring, using the flag `--recover` to recover a private key from a seed phrase or the flag `--multisig` to group multiple keys together to create a multisig key. For full details on the `add` key command, see the code [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/keys/add.go). For more details about usage of `--keyring-backend` for storage of key credentials look at the [keyring docs](../../user/run-node/00-keyring.md). -* **Server** commands from the Cosmos SDK server package. These commands are responsible for providing the mechanisms necessary to start an ABCI CometBFT application and provides the CLI framework (based on [cobra](https://github.com/spf13/cobra)) necessary to fully bootstrap an application. The package exposes two core functions: `StartCmd` and `ExportCmd` which creates commands to start the application and export state respectively. -Learn more [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server). -* [**Transaction**](#transaction-commands) commands. -* [**Query**](#query-commands) commands. - -Next is an example `rootCmd` function from the `simapp` application. It instantiates the root command, adds a [*persistent* flag](#flags) and `PreRun` function to be run before every execution, and adds all of the necessary subcommands. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L47-L130 -``` - -:::tip -Use the `EnhanceRootCommand()` from the AutoCLI options to automatically add auto-generated commands from the modules to the root command. -Additionnally it adds all manually defined modules commands (`tx` and `query`) as well. -Read more about [AutoCLI](https://docs.cosmos.network/main/core/autocli) in its dedicated section. -::: - -`rootCmd` has a function called `initAppConfig()` which is useful for setting the application's custom configs. -By default app uses CometBFT app config template from Cosmos SDK, which can be over-written via `initAppConfig()`. -Here's an example code to override default `app.toml` template. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L144-L199 -``` - -The `initAppConfig()` also allows overriding the default Cosmos SDK's [server config](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/config/config.go#L235). One example is the `min-gas-prices` config, which defines the minimum gas prices a validator is willing to accept for processing a transaction. By default, the Cosmos SDK sets this parameter to `""` (empty string), which forces all validators to tweak their own `app.toml` and set a non-empty value, or else the node will halt on startup. This might not be the best UX for validators, so the chain developer can set a default `app.toml` value for validators inside this `initAppConfig()` function. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L164-L180 -``` - -The root-level `status` and `keys` subcommands are common across most applications and do not interact with application state. The bulk of an application's functionality - what users can actually *do* with it - is enabled by its `tx` and `query` commands. - -### Transaction Commands - -[Transactions](./01-transactions.md) are objects wrapping [`Msg`s](../../build/building-modules/02-messages-and-queries.md#messages) that trigger state changes. To enable the creation of transactions using the CLI interface, a function `txCommand` is generally added to the `rootCmd`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L222-L229 -``` - -This `txCommand` function adds all the transaction available to end-users for the application. This typically includes: - -* **Sign command** from the [`auth`](../../build/modules/auth/README.md) module that signs messages in a transaction. To enable multisig, add the `auth` module's `MultiSign` command. Since every transaction requires some sort of signature in order to be valid, the signing command is necessary for every application. -* **Broadcast command** from the Cosmos SDK client tools, to broadcast transactions. -* **All [module transaction commands](../../build/building-modules/09-module-interfaces.md#transaction-commands)** the application is dependent on, retrieved by using the [basic module manager's](../../build/building-modules/01-module-manager.md#basic-manager) `AddTxCommands()` function, or enhanced by [AutoCLI](https://docs.cosmos.network/main/core/autocli). - -Here is an example of a `txCommand` aggregating these subcommands from the `simapp` application: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L270-L292 -``` - -:::tip -When using AutoCLI to generate module transaction commands, `EnhanceRootCommand()` automatically adds the module `tx` command to the root command. -Read more about [AutoCLI](https://docs.cosmos.network/main/core/autocli) in its dedicated section. -::: - -### Query Commands - -[**Queries**](../../build/building-modules/02-messages-and-queries.md#queries) are objects that allow users to retrieve information about the application's state. To enable the creation of queries using the CLI interface, a function `queryCommand` is generally added to the `rootCmd`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L222-L229 -``` - -This `queryCommand` function adds all the queries available to end-users for the application. This typically includes: - -* **QueryTx** and/or other transaction query commands from the `auth` module which allow the user to search for a transaction by inputting its hash, a list of tags, or a block height. These queries allow users to see if transactions have been included in a block. -* **Account command** from the `auth` module, which displays the state (e.g. account balance) of an account given an address. -* **Validator command** from the Cosmos SDK rpc client tools, which displays the validator set of a given height. -* **Block command** from the Cosmos SDK RPC client tools, which displays the block data for a given height. -* **All [module query commands](../../build/building-modules/09-module-interfaces.md#query-commands)** the application is dependent on, retrieved by using the [basic module manager's](../../build/building-modules/01-module-manager.md#basic-manager) `AddQueryCommands()` function, or enhanced by [AutoCLI](https://docs.cosmos.network/main/core/autocli). - -Here is an example of a `queryCommand` aggregating subcommands from the `simapp` application: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L249-L268 -``` - -:::tip -When using AutoCLI to generate module query commands, `EnhanceRootCommand()` automatically adds the module `query` command to the root command. -Read more about [AutoCLI](https://docs.cosmos.network/main/core/autocli) in its dedicated section. -::: - -## Flags - -Flags are used to modify commands; developers can include them in a `flags.go` file with their CLI. Users can explicitly include them in commands or pre-configure them by inside their [`app.toml`](../../user/run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml). Commonly pre-configured flags include the `--node` to connect to and `--chain-id` of the blockchain the user wishes to interact with. - -A *persistent* flag (as opposed to a *local* flag) added to a command transcends all of its children: subcommands will inherit the configured values for these flags. Additionally, all flags have default values when they are added to commands; some toggle an option off but others are empty values that the user needs to override to create valid commands. A flag can be explicitly marked as *required* so that an error is automatically thrown if the user does not provide a value, but it is also acceptable to handle unexpected missing flags differently. - -Flags are added to commands directly (generally in the [module's CLI file](../../build/building-modules/09-module-interfaces.md#flags) where module commands are defined) and no flag except for the `rootCmd` persistent flags has to be added at application level. It is common to add a *persistent* flag for `--chain-id`, the unique identifier of the blockchain the application pertains to, to the root command. Adding this flag can be done in the `main()` function. Adding this flag makes sense as the chain ID should not be changing across commands in this application CLI. - -## Environment variables - -Each flag is bound to it's respecteve named environment variable. Then name of the environment variable consist of two parts - capital case `basename` followed by flag name of the flag. `-` must be substituted with `_`. For example flag `--home` for application with basename `GAIA` is bound to `GAIA_HOME`. It allows reducing the amount of flags typed for routine operations. For example instead of: - -```shell -gaia --home=./ --node= --chain-id="testchain-1" --keyring-backend=test tx ... --from= -``` - -this will be more convenient: - -```shell -# define env variables in .env, .envrc etc -GAIA_HOME= -GAIA_NODE= -GAIA_CHAIN_ID="testchain-1" -GAIA_KEYRING_BACKEND="test" - -# and later just use -gaia tx ... --from= -``` - -## Configurations - -It is vital that the root command of an application uses `PersistentPreRun()` cobra command property for executing the command, so all child commands have access to the server and client contexts. These contexts are set as their default values initially and maybe modified, scoped to the command, in their respective `PersistentPreRun()` functions. Note that the `client.Context` is typically pre-populated with "default" values that may be useful for all commands to inherit and override if necessary. - -Here is an example of an `PersistentPreRun()` function from `simapp`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L81-L120 -``` - -The `SetCmdClientContextHandler` call reads persistent flags via `ReadPersistentCommandFlags` which creates a `client.Context` and sets that on the root command's `Context`. - -The `InterceptConfigsPreRunHandler` call creates a viper literal, default `server.Context`, and a logger and sets that on the root command's `Context`. The `server.Context` will be modified and saved to disk. The internal `interceptConfigs` call reads or creates a CometBFT configuration based on the home path provided. In addition, `interceptConfigs` also reads and loads the application configuration, `app.toml`, and binds that to the `server.Context` viper literal. This is vital so the application can get access to not only the CLI flags, but also to the application configuration values provided by this file. - -:::tip -When willing to configure which logger is used, do not to use `InterceptConfigsPreRunHandler`, which sets the default SDK logger, but instead use `InterceptConfigsAndCreateContext` and set the server context and the logger manually: - -```diff --return server.InterceptConfigsPreRunHandler(cmd, customAppTemplate, customAppConfig, customCMTConfig) - -+serverCtx, err := server.InterceptConfigsAndCreateContext(cmd, customAppTemplate, customAppConfig, customCMTConfig) -+if err != nil { -+ return err -+} - -+// overwrite default server logger -+logger, err := server.CreateSDKLogger(serverCtx, cmd.OutOrStdout()) -+if err != nil { -+ return err -+} -+serverCtx.Logger = logger.With(log.ModuleKey, "server") - -+// set server context -+return server.SetCmdServerContext(cmd, serverCtx) -``` - -::: diff --git a/docs/learn/advanced/08-events.md b/docs/learn/advanced/08-events.md deleted file mode 100644 index 410e20ade09d..000000000000 --- a/docs/learn/advanced/08-events.md +++ /dev/null @@ -1,159 +0,0 @@ ---- -sidebar_position: 1 ---- -# Events - -:::note Synopsis -`Event`s are objects that contain information about the execution of the application. They are mainly used by service providers like block explorers and wallet to track the execution of various messages and index transactions. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) -* [CometBFT Documentation on Events](https://docs.cometbft.com/v0.37/spec/abci/abci++_basic_concepts#events) - -::: - -## Events - -Events are implemented in the Cosmos SDK as an alias of the ABCI `Event` type and -take the form of: `{eventType}.{attributeKey}={attributeValue}`. - -```protobuf reference -https://github.com/cometbft/cometbft/blob/v0.37.0/proto/tendermint/abci/types.proto#L334-L343 -``` - -An Event contains: - -* A `type` to categorize the Event at a high-level; for example, the Cosmos SDK uses the `"message"` type to filter Events by `Msg`s. -* A list of `attributes` are key-value pairs that give more information about the categorized Event. For example, for the `"message"` type, we can filter Events by key-value pairs using `message.action={some_action}`, `message.module={some_module}` or `message.sender={some_sender}`. -* A `msg_index` to identify which messages relate to the same transaction - -:::tip -To parse the attribute values as strings, make sure to add `'` (single quotes) around each attribute value. -::: - -_Typed Events_ are Protobuf-defined [messages](../../build/architecture/adr-032-typed-events.md) used by the Cosmos SDK -for emitting and querying Events. They are defined in a `event.proto` file, on a **per-module basis** and are read as `proto.Message`. -_Legacy Events_ are defined on a **per-module basis** in the module's `/types/events.go` file. -They are triggered from the module's Protobuf [`Msg` service](../../build/building-modules/03-msg-services.md) -by using the [`EventManager`](#eventmanager). - -In addition, each module documents its events under in the `Events` sections of its specs (x/{moduleName}/`README.md`). - -Lastly, Events are returned to the underlying consensus engine in the response of the following ABCI messages: - -* [`BeginBlock`](./00-baseapp.md#beginblock) -* [`EndBlock`](./00-baseapp.md#endblock) -* [`CheckTx`](./00-baseapp.md#checktx) -* [`Transaction Execution`](./00-baseapp.md#transactionexecution) - -### Examples - -The following examples show how to query Events using the Cosmos SDK. - -| Event | Description | -| ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `tx.height=23` | Query all transactions at height 23 | -| `message.action='/cosmos.bank.v1beta1.Msg/Send'` | Query all transactions containing a x/bank `Send` [Service `Msg`](../../build/building-modules/03-msg-services.md). Note the `'`s around the value. | -| `message.module='bank'` | Query all transactions containing messages from the x/bank module. Note the `'`s around the value. | -| `create_validator.validator='cosmosval1...'` | x/staking-specific Event, see [x/staking SPEC](../../build/modules/staking/README.md). | - -## EventManager - -In Cosmos SDK applications, Events are managed by an abstraction called the `EventManager`. -Internally, the `EventManager` tracks a list of Events for the entire execution flow of `FinalizeBlock` -(i.e. transaction execution, `BeginBlock`, `EndBlock`). - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/events.go#L19-L26 -``` - -The `EventManager` comes with a set of useful methods to manage Events. The method -that is used most by module and application developers is `EmitTypedEvent` or `EmitEvent` that tracks -an Event in the `EventManager`. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/events.go#L53-L62 -``` - -Module developers should handle Event emission via the `EventManager#EmitTypedEvent` or `EventManager#EmitEvent` in each message -`Handler` and in each `BeginBlock`/`EndBlock` handler. The `EventManager` is accessed via -the [`Context`](./02-context.md), where Event should be already registered, and emitted like this: - - -**Typed events:** - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/group/keeper/msg_server.go#L95-L97 -``` - -**Legacy events:** - -```go -ctx.EventManager().EmitEvent( - sdk.NewEvent(eventType, sdk.NewAttribute(attributeKey, attributeValue)), -) -``` - -Where the `EventManager` is accessed via the [`Context`](./02-context.md). - -See the [`Msg` services](../../build/building-modules/03-msg-services.md) concept doc for a more detailed -view on how to typically implement Events and use the `EventManager` in modules. - -## Subscribing to Events - -You can use CometBFT's [Websocket](https://docs.cometbft.com/v0.37/core/subscription) to subscribe to Events by calling the `subscribe` RPC method: - -```json -{ - "jsonrpc": "2.0", - "method": "subscribe", - "id": "0", - "params": { - "query": "tm.event='eventCategory' AND eventType.eventAttribute='attributeValue'" - } -} -``` - -The main `eventCategory` you can subscribe to are: - -* `NewBlock`: Contains Events triggered during `BeginBlock` and `EndBlock`. -* `Tx`: Contains Events triggered during `DeliverTx` (i.e. transaction processing). -* `ValidatorSetUpdates`: Contains validator set updates for the block. - -These Events are triggered from the `state` package after a block is committed. You can get the -full list of Event categories [on the CometBFT Go documentation](https://pkg.go.dev/github.com/cometbft/cometbft/types#pkg-constants). - -The `type` and `attribute` value of the `query` allow you to filter the specific Event you are looking for. For example, a `Mint` transaction triggers an Event of type `EventMint` and has an `Id` and an `Owner` as `attributes` (as defined in the [`events.proto` file of the `NFT` module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/nft/v1beta1/event.proto#L21-L31)). - -Subscribing to this Event would be done like so: - -```json -{ - "jsonrpc": "2.0", - "method": "subscribe", - "id": "0", - "params": { - "query": "tm.event='Tx' AND mint.owner='ownerAddress'" - } -} -``` - -where `ownerAddress` is an address following the [`AccAddress`](../beginner/03-accounts.md#addresses) format. - -The same way can be used to subscribe to [legacy events](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/types/events.go). - -## Default Events - -There are a few events that are automatically emitted for all messages, directly from `baseapp`. - -* `message.action`: The name of the message type. -* `message.sender`: The address of the message signer. -* `message.module`: The name of the module that emitted the message. - -:::tip -The module name is assumed by `baseapp` to be the second element of the message route: `"cosmos.bank.v1beta1.MsgSend" -> "bank"`. -In case a module does not follow the standard message path, (e.g. IBC), it is advised to keep emitting the module name event. -`Baseapp` only emits that event if the module have not already done so. -::: diff --git a/docs/learn/advanced/09-telemetry.md b/docs/learn/advanced/09-telemetry.md deleted file mode 100644 index c5916544f111..000000000000 --- a/docs/learn/advanced/09-telemetry.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Telemetry - -:::note Synopsis -Gather relevant insights about your application and modules with custom metrics and telemetry. -::: - -The Cosmos SDK enables operators and developers to gain insight into the performance and behavior of -their application through the use of the `telemetry` package. To enable telemetrics, set `telemetry.enabled = true` in the app.toml config file. - -The Cosmos SDK currently supports enabling in-memory and prometheus as telemetry sinks. In-memory sink is always attached (when the telemetry is enabled) with 10 second interval and 1 minute retention. This means that metrics will be aggregated over 10 seconds, and metrics will be kept alive for 1 minute. - -To query active metrics (see retention note above) you have to enable API server (`api.enabled = true` in the app.toml). Single API endpoint is exposed: `http://localhost:1317/metrics?format={text|prometheus}`, the default being `text`. - -## Emitting metrics - -If telemetry is enabled via configuration, a single global metrics collector is registered via the -[go-metrics](https://github.com/hashicorp/go-metrics) library. This allows emitting and collecting -metrics through simple [API](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/telemetry/wrapper.go). Example: - -```go -func EndBlocker(ctx sdk.Context, k keeper.Keeper) { - defer telemetry.ModuleMeasureSince(types.ModuleName, time.Now(), telemetry.MetricKeyEndBlocker) - - // ... -} -``` - -Developers may use the `telemetry` package directly, which provides wrappers around metric APIs -that include adding useful labels, or they must use the `go-metrics` library directly. It is preferable -to add as much context and adequate dimensionality to metrics as possible, so the `telemetry` package -is advised. Regardless of the package or method used, the Cosmos SDK supports the following metrics -types: - -* gauges -* summaries -* counters - -## Labels - -Certain components of modules will have their name automatically added as a label (e.g. `BeginBlock`). -Operators may also supply the application with a global set of labels that will be applied to all -metrics emitted using the `telemetry` package (e.g. chain-id). Global labels are supplied as a list -of [name, value] tuples. - -Example: - -```toml -global-labels = [ - ["chain_id", "chain-OfXo4V"], -] -``` - -## Cardinality - -Cardinality is key, specifically label and key cardinality. Cardinality is how many unique values of -something there are. So there is naturally a tradeoff between granularity and how much stress is put -on the telemetry sink in terms of indexing, scrape, and query performance. - -Developers should take care to support metrics with enough dimensionality and granularity to be -useful, but not increase the cardinality beyond the sink's limits. A general rule of thumb is to not -exceed a cardinality of 10. - -Consider the following examples with enough granularity and adequate cardinality: - -* begin/end blocker time -* tx gas used -* block gas used -* amount of tokens minted -* amount of accounts created - -The following examples expose too much cardinality and may not even prove to be useful: - -* transfers between accounts with amount -* voting/deposit amount from unique addresses - -## Supported Metrics - -| Metric | Description | Unit | Type | -|:--------------------------------|:------------------------------------------------------------------------------------------|:----------------|:--------| -| `tx_count` | Total number of txs processed via `DeliverTx` | tx | counter | -| `tx_successful` | Total number of successful txs processed via `DeliverTx` | tx | counter | -| `tx_failed` | Total number of failed txs processed via `DeliverTx` | tx | counter | -| `tx_gas_used` | The total amount of gas used by a tx | gas | gauge | -| `tx_gas_wanted` | The total amount of gas requested by a tx | gas | gauge | -| `tx_msg_send` | The total amount of tokens sent in a `MsgSend` (per denom) | token | gauge | -| `tx_msg_withdraw_reward` | The total amount of tokens withdrawn in a `MsgWithdrawDelegatorReward` (per denom) | token | gauge | -| `tx_msg_withdraw_commission` | The total amount of tokens withdrawn in a `MsgWithdrawValidatorCommission` (per denom) | token | gauge | -| `tx_msg_delegate` | The total amount of tokens delegated in a `MsgDelegate` | token | gauge | -| `tx_msg_begin_unbonding` | The total amount of tokens undelegated in a `MsgUndelegate` | token | gauge | -| `tx_msg_begin_begin_redelegate` | The total amount of tokens redelegated in a `MsgBeginRedelegate` | token | gauge | -| `tx_msg_ibc_transfer` | The total amount of tokens transferred via IBC in a `MsgTransfer` (source or sink chain) | token | gauge | -| `ibc_transfer_packet_receive` | The total amount of tokens received in a `FungibleTokenPacketData` (source or sink chain) | token | gauge | -| `new_account` | Total number of new accounts created | account | counter | -| `gov_proposal` | Total number of governance proposals | proposal | counter | -| `gov_vote` | Total number of governance votes for a proposal | vote | counter | -| `gov_deposit` | Total number of governance deposits for a proposal | deposit | counter | -| `staking_delegate` | Total number of delegations | delegation | counter | -| `staking_undelegate` | Total number of undelegations | undelegation | counter | -| `staking_redelegate` | Total number of redelegations | redelegation | counter | -| `ibc_transfer_send` | Total number of IBC transfers sent from a chain (source or sink) | transfer | counter | -| `ibc_transfer_receive` | Total number of IBC transfers received to a chain (source or sink) | transfer | counter | -| `ibc_client_create` | Total number of clients created | create | counter | -| `ibc_client_update` | Total number of client updates | update | counter | -| `ibc_client_upgrade` | Total number of client upgrades | upgrade | counter | -| `ibc_client_misbehaviour` | Total number of client misbehaviours | misbehaviour | counter | -| `ibc_connection_open-init` | Total number of connection `OpenInit` handshakes | handshake | counter | -| `ibc_connection_open-try` | Total number of connection `OpenTry` handshakes | handshake | counter | -| `ibc_connection_open-ack` | Total number of connection `OpenAck` handshakes | handshake | counter | -| `ibc_connection_open-confirm` | Total number of connection `OpenConfirm` handshakes | handshake | counter | -| `ibc_channel_open-init` | Total number of channel `OpenInit` handshakes | handshake | counter | -| `ibc_channel_open-try` | Total number of channel `OpenTry` handshakes | handshake | counter | -| `ibc_channel_open-ack` | Total number of channel `OpenAck` handshakes | handshake | counter | -| `ibc_channel_open-confirm` | Total number of channel `OpenConfirm` handshakes | handshake | counter | -| `ibc_channel_close-init` | Total number of channel `CloseInit` handshakes | handshake | counter | -| `ibc_channel_close-confirm` | Total number of channel `CloseConfirm` handshakes | handshake | counter | -| `tx_msg_ibc_recv_packet` | Total number of IBC packets received | packet | counter | -| `tx_msg_ibc_acknowledge_packet` | Total number of IBC packets acknowledged | acknowledgement | counter | -| `ibc_timeout_packet` | Total number of IBC timeout packets | timeout | counter | -| `store_iavl_get` | Duration of an IAVL `Store#Get` call | ms | summary | -| `store_iavl_set` | Duration of an IAVL `Store#Set` call | ms | summary | -| `store_iavl_has` | Duration of an IAVL `Store#Has` call | ms | summary | -| `store_iavl_delete` | Duration of an IAVL `Store#Delete` call | ms | summary | -| `store_iavl_commit` | Duration of an IAVL `Store#Commit` call | ms | summary | -| `store_iavl_query` | Duration of an IAVL `Store#Query` call | ms | summary | diff --git a/docs/learn/advanced/10-ocap.md b/docs/learn/advanced/10-ocap.md deleted file mode 100644 index c5a472b7f6c4..000000000000 --- a/docs/learn/advanced/10-ocap.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Object-Capability Model - -## Intro - -When thinking about security, it is good to start with a specific threat model. Our threat model is the following: - -> We assume that a thriving ecosystem of Cosmos SDK modules that are easy to compose into a blockchain application will contain faulty or malicious modules. - -The Cosmos SDK is designed to address this threat by being the -foundation of an object capability system. - -> The structural properties of object capability systems favor -> modularity in code design and ensure reliable encapsulation in -> code implementation. -> -> These structural properties facilitate the analysis of some -> security properties of an object-capability program or operating -> system. Some of these — in particular, information flow properties -> — can be analyzed at the level of object references and -> connectivity, independent of any knowledge or analysis of the code -> that determines the behavior of the objects. -> -> As a consequence, these security properties can be established -> and maintained in the presence of new objects that contain unknown -> and possibly malicious code. -> -> These structural properties stem from the two rules governing -> access to existing objects: -> -> 1. An object A can send a message to B only if object A holds a -> reference to B. -> 2. An object A can obtain a reference to C only -> if object A receives a message containing a reference to C. As a -> consequence of these two rules, an object can obtain a reference -> to another object only through a preexisting chain of references. -> In short, "Only connectivity begets connectivity." - -For an introduction to object-capabilities, see this [Wikipedia article](https://en.wikipedia.org/wiki/Object-capability_model). - -## Ocaps in practice - -The idea is to only reveal what is necessary to get the work done. - -For example, the following code snippet violates the object capabilities -principle: - -```go -type AppAccount struct {...} -account := &AppAccount{ - Address: pub.Address(), - Coins: sdk.Coins{sdk.NewInt64Coin("ATM", 100)}, -} -sumValue := externalModule.ComputeSumValue(account) -``` - -The method `ComputeSumValue` implies a pure function, yet the implied -capability of accepting a pointer value is the capability to modify that -value. The preferred method signature should take a copy instead. - -```go -sumValue := externalModule.ComputeSumValue(*account) -``` - -In the Cosmos SDK, you can see the application of this principle in simapp. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go -``` - -The following diagram shows the current dependencies between keepers. - -![Keeper dependencies](https://raw.githubusercontent.com/cosmos/cosmos-sdk/release/v0.46.x/docs/uml/svg/keeper_dependencies.svg) diff --git a/docs/learn/advanced/11-runtx_middleware.md b/docs/learn/advanced/11-runtx_middleware.md deleted file mode 100644 index f083a77888e5..000000000000 --- a/docs/learn/advanced/11-runtx_middleware.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -sidebar_position: 1 ---- - -# RunTx recovery middleware - -`BaseApp.runTx()` function handles Go panics that might occur during transactions execution, for example, keeper has faced an invalid state and paniced. -Depending on the panic type different handler is used, for instance the default one prints an error log message. -Recovery middleware is used to add custom panic recovery for Cosmos SDK application developers. - -More context can found in the corresponding [ADR-022](../../build/architecture/adr-022-custom-panic-handling.md) and the implementation in [recovery.go](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/recovery.go). - -## Interface - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/recovery.go#L14-L17 -``` - -`recoveryObj` is a return value for `recover()` function from the `buildin` Go package. - -**Contract:** - -* RecoveryHandler returns `nil` if `recoveryObj` wasn't handled and should be passed to the next recovery middleware; -* RecoveryHandler returns a non-nil `error` if `recoveryObj` was handled; - -## Custom RecoveryHandler register - -`BaseApp.AddRunTxRecoveryHandler(handlers ...RecoveryHandler)` - -BaseApp method adds recovery middleware to the default recovery chain. - -## Example - -Lets assume we want to emit the "Consensus failure" chain state if some particular error occurred. - -We have a module keeper that panics: - -```go -func (k FooKeeper) Do(obj interface{}) { - if obj == nil { - // that shouldn't happen, we need to crash the app - err := errorsmod.Wrap(fooTypes.InternalError, "obj is nil") - panic(err) - } -} -``` - -By default that panic would be recovered and an error message will be printed to log. To override that behaviour we should register a custom RecoveryHandler: - -```go -// Cosmos SDK application constructor -customHandler := func(recoveryObj interface{}) error { - err, ok := recoveryObj.(error) - if !ok { - return nil - } - - if fooTypes.InternalError.Is(err) { - panic(fmt.Errorf("FooKeeper did panic with error: %w", err)) - } - - return nil -} - -baseApp := baseapp.NewBaseApp(...) -baseApp.AddRunTxRecoveryHandler(customHandler) -``` diff --git a/docs/learn/advanced/12-simulation.md b/docs/learn/advanced/12-simulation.md deleted file mode 100644 index dfbcddd0d29f..000000000000 --- a/docs/learn/advanced/12-simulation.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Cosmos Blockchain Simulator - -The Cosmos SDK offers a full fledged simulation framework to fuzz test every -message defined by a module. - -On the Cosmos SDK, this functionality is provided by [`SimApp`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_v2.go), which is a -`Baseapp` application that is used for running the [`simulation`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/simulation) module. -This module defines all the simulation logic as well as the operations for -randomized parameters like accounts, balances etc. - -## Goals - -The blockchain simulator tests how the blockchain application would behave under -real life circumstances by generating and sending randomized messages. -The goal of this is to detect and debug failures that could halt a live chain, -by providing logs and statistics about the operations run by the simulator as -well as exporting the latest application state when a failure was found. - -Its main difference with integration testing is that the simulator app allows -you to pass parameters to customize the chain that's being simulated. -This comes in handy when trying to reproduce bugs that were generated in the -provided operations (randomized or not). - -## Simulation commands - -The simulation app has different commands, each of which tests a different -failure type: - -* `AppImportExport`: The simulator exports the initial app state and then it - creates a new app with the exported `genesis.json` as an input, checking for - inconsistencies between the stores. -* `AppSimulationAfterImport`: Queues two simulations together. The first one provides the app state (_i.e_ genesis) to the second. Useful to test software upgrades or hard-forks from a live chain. -* `AppStateDeterminism`: Checks that all the nodes return the same values, in the same order. -* `BenchmarkInvariants`: Analysis of the performance of running all modules' invariants (_i.e_ sequentially runs a [benchmark](https://pkg.go.dev/testing/#hdr-Benchmarks) test). An invariant checks for - differences between the values that are on the store and the passive tracker. Eg: total coins held by accounts vs total supply tracker. -* `FullAppSimulation`: General simulation mode. Runs the chain and the specified operations for a given number of blocks. Tests that there're no `panics` on the simulation. It does also run invariant checks on every `Period` but they are not benchmarked. - -Each simulation must receive a set of inputs (_i.e_ flags) such as the number of -blocks that the simulation is run, seed, block size, etc. -Check the full list of flags [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/simulation/client/cli/flags.go#L35-L59). - -## Simulator Modes - -In addition to the various inputs and commands, the simulator runs in three modes: - -1. Completely random where the initial state, module parameters and simulation - parameters are **pseudo-randomly generated**. -2. From a `genesis.json` file where the initial state and the module parameters are defined. - This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested. -3. From a `params.json` file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. - This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated. - The list of available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/simulation/client/cli/flags.go#L59-L78). - -:::tip -These modes are not mutually exclusive. So you can for example run a randomly -generated genesis state (`1`) with manually generated simulation params (`3`). -::: - -## Usage - -This is a general example of how simulations are run. For more specific examples -check the Cosmos SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/Makefile#L282-L318). - -```bash - $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \ - -run=TestApp \ - ... - -v -timeout 24h -``` - -## Debugging Tips - -Here are some suggestions when encountering a simulation failure: - -* Export the app state at the height where the failure was found. You can do this - by passing the `-ExportStatePath` flag to the simulator. -* Use `-Verbose` logs. They could give you a better hint on all the operations - involved. -* Reduce the simulation `-Period`. This will run the invariants checks more - frequently. -* Print all the failed invariants at once with `-PrintAllInvariants`. -* Try using another `-Seed`. If it can reproduce the same error and if it fails - sooner, you will spend less time running the simulations. -* Reduce the `-NumBlocks` . How's the app state at the height previous to the - failure? -* Run invariants on every operation with `-SimulateEveryOperation`. _Note_: this - will slow down your simulation **a lot**. -* Try adding logs to operations that are not logged. You will have to define a - [Logger](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keeper.go#L65-L68) on your `Keeper`. - -## Use simulation in your Cosmos SDK-based application - -Learn how you can build the simulation into your Cosmos SDK-based application: - -* Application Simulation Manager -* [Building modules: Simulator](../../build/building-modules/14-simulator.md) -* Simulator tests diff --git a/docs/learn/advanced/13-proto-docs.md b/docs/learn/advanced/13-proto-docs.md deleted file mode 100644 index 6c857446512f..000000000000 --- a/docs/learn/advanced/13-proto-docs.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Protobuf Documentation - -See [Cosmos SDK Buf Proto-docs](https://buf.build/cosmos/cosmos-sdk/docs/main) diff --git a/docs/learn/advanced/15-upgrade.md b/docs/learn/advanced/15-upgrade.md deleted file mode 100644 index 5d56f2b59605..000000000000 --- a/docs/learn/advanced/15-upgrade.md +++ /dev/null @@ -1,162 +0,0 @@ ---- -sidebar_position: 1 ---- - -# In-Place Store Migrations - -:::warning -Read and understand all the in-place store migration documentation before you run a migration on a live chain. -::: - -:::note Synopsis -Upgrade your app modules smoothly with custom in-place store migration logic. -::: - -The Cosmos SDK uses two methods to perform upgrades: - -* Exporting the entire application state to a JSON file using the `export` CLI command, making changes, and then starting a new binary with the changed JSON file as the genesis file. - -* Perform upgrades in place, which significantly decrease the upgrade time for chains with a larger state. Use the [Module Upgrade Guide](../../build/building-modules/13-upgrade.md) to set up your application modules to take advantage of in-place upgrades. - -This document provides steps to use the In-Place Store Migrations upgrade method. - -## Tracking Module Versions - -Each module gets assigned a consensus version by the module developer. The consensus version serves as the breaking change version of the module. The Cosmos SDK keeps track of all module consensus versions in the x/upgrade `VersionMap` store. During an upgrade, the difference between the old `VersionMap` stored in state and the new `VersionMap` is calculated by the Cosmos SDK. For each identified difference, the module-specific migrations are run and the respective consensus version of each upgraded module is incremented. - -### Consensus Version - -The consensus version is defined on each app module by the module developer and serves as the breaking change version of the module. The consensus version informs the Cosmos SDK on which modules need to be upgraded. For example, if the bank module was version 2 and an upgrade introduces bank module 3, the Cosmos SDK upgrades the bank module and runs the "version 2 to 3" migration script. - -### Version Map - -The version map is a mapping of module names to consensus versions. The map is persisted to x/upgrade's state for use during in-place migrations. When migrations finish, the updated version map is persisted in the state. - -## Upgrade Handlers - -Upgrades use an `UpgradeHandler` to facilitate migrations. The `UpgradeHandler` functions implemented by the app developer must conform to the following function signature. These functions retrieve the `VersionMap` from x/upgrade's state and return the new `VersionMap` to be stored in x/upgrade after the upgrade. The diff between the two `VersionMap`s determines which modules need upgrading. - -```go -type UpgradeHandler func(ctx sdk.Context, plan Plan, fromVM VersionMap) (VersionMap, error) -``` - -Inside these functions, you must perform any upgrade logic to include in the provided `plan`. All upgrade handler functions must end with the following line of code: - -```go - return app.mm.RunMigrations(ctx, cfg, fromVM) -``` - -## Running Migrations - -Migrations are run inside of an `UpgradeHandler` using `app.mm.RunMigrations(ctx, cfg, vm)`. The `UpgradeHandler` functions describe the functionality to occur during an upgrade. The `RunMigration` function loops through the `VersionMap` argument and runs the migration scripts for all versions that are less than the versions of the new binary app module. After the migrations are finished, a new `VersionMap` is returned to persist the upgraded module versions to state. - -```go -cfg := module.NewConfigurator(...) -app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { - - // ... - // additional upgrade logic - // ... - - // returns a VersionMap with the updated module ConsensusVersions - return app.mm.RunMigrations(ctx, fromVM) -}) -``` - -To learn more about configuring migration scripts for your modules, see the [Module Upgrade Guide](../../build/building-modules/13-upgrade.md). - -### Order Of Migrations - -By default, all migrations are run in module name alphabetical ascending order, except `x/auth` which is run last. The reason is state dependencies between x/auth and other modules (you can read more in [issue #10606](https://github.com/cosmos/cosmos-sdk/issues/10606)). - -If you want to change the order of migration, then you should call `app.mm.SetOrderMigrations(module1, module2, ...)` in your app.go file. The function will panic if you forget to include a module in the argument list. - -## Adding New Modules During Upgrades - -You can introduce entirely new modules to the application during an upgrade. New modules are recognized because they have not yet been registered in `x/upgrade`'s `VersionMap` store. In this case, `RunMigrations` calls the `InitGenesis` function from the corresponding module to set up its initial state. - -### Add StoreUpgrades for New Modules - -All chains preparing to run in-place store migrations will need to manually add store upgrades for new modules and then configure the store loader to apply those upgrades. This ensures that the new module's stores are added to the multistore before the migrations begin. - -```go -upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() -if err != nil { - panic(err) -} - -if upgradeInfo.Name == "my-plan" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { - storeUpgrades := storetypes.StoreUpgrades{ - // add store upgrades for new modules - // Example: - // Added: []string{"foo", "bar"}, - // ... - } - - // configure store loader that checks if version == upgradeHeight and applies store upgrades - app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) -} -``` - -## Genesis State - -When starting a new chain, the consensus version of each module MUST be saved to state during the application's genesis. To save the consensus version, add the following line to the `InitChainer` method in `app.go`: - -```diff -func (app *MyApp) InitChainer(ctx sdk.Context, req abci.RequestInitChain) abci.ResponseInitChain { - ... -+ app.UpgradeKeeper.SetModuleVersionMap(ctx, app.mm.GetVersionMap()) - ... -} -``` - -This information is used by the Cosmos SDK to detect when modules with newer versions are introduced to the app. - -For a new module `foo`, `InitGenesis` is called by `RunMigration` only when `foo` is registered in the module manager but it's not set in the `fromVM`. Therefore, if you want to skip `InitGenesis` when a new module is added to the app, then you should set its module version in `fromVM` to the module consensus version: - -```go -app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { - // ... - - // Set foo's version to the latest ConsensusVersion in the VersionMap. - // This will skip running InitGenesis on Foo - fromVM[foo.ModuleName] = foo.AppModule{}.ConsensusVersion() - - return app.mm.RunMigrations(ctx, fromVM) -}) -``` - -### Overwriting Genesis Functions - -The Cosmos SDK offers modules that the application developer can import in their app. These modules often have an `InitGenesis` function already defined. - -You can write your own `InitGenesis` function for an imported module. To do this, manually trigger your custom genesis function in the upgrade handler. - -:::warning -You MUST manually set the consensus version in the version map passed to the `UpgradeHandler` function. Without this, the SDK will run the Module's existing `InitGenesis` code even if you triggered your custom function in the `UpgradeHandler`. -::: - -```go -import foo "github.com/my/module/foo" - -app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { - - // Register the consensus version in the version map - // to avoid the SDK from triggering the default - // InitGenesis function. - fromVM["foo"] = foo.AppModule{}.ConsensusVersion() - - // Run custom InitGenesis for foo - app.mm["foo"].InitGenesis(ctx, app.appCodec, myCustomGenesisState) - - return app.mm.RunMigrations(ctx, cfg, fromVM) -}) -``` - -## Syncing a Full Node to an Upgraded Blockchain - -You can sync a full node to an existing blockchain which has been upgraded using Cosmovisor - -To successfully sync, you must start with the initial binary that the blockchain started with at genesis. If all Software Upgrade Plans contain binary instruction, then you can run Cosmovisor with auto-download option to automatically handle downloading and switching to the binaries associated with each sequential upgrade. Otherwise, you need to manually provide all binaries to Cosmovisor. - -To learn more about Cosmovisor, see the [Cosmovisor Quick Start](../../build/tooling/01-cosmovisor.md). diff --git a/docs/learn/advanced/16-config.md b/docs/learn/advanced/16-config.md deleted file mode 100644 index 03aa55a21ec9..000000000000 --- a/docs/learn/advanced/16-config.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Configuration - -This documentation refers to the app.toml, if you'd like to read about the config.toml please visit [CometBFT docs](https://docs.cometbft.com/v0.37/). - - -```python reference -https://github.com/cosmos/cosmos-sdk/blob/main/tools/confix/data/v0.47-app.toml -``` - -## inter-block-cache - -This feature will consume more ram than a normal node, if enabled. - -## iavl-cache-size - -Using this feature will increase ram consumption - -## iavl-lazy-loading - -This feature is to be used for archive nodes, allowing them to have a faster start up time. diff --git a/docs/learn/advanced/_category_.json b/docs/learn/advanced/_category_.json deleted file mode 100644 index 6677a95ae7e5..000000000000 --- a/docs/learn/advanced/_category_.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "label": "Advanced", - "position": 2, - "link": null -} \ No newline at end of file diff --git a/docs/learn/advanced/baseapp_state-begin_block.png b/docs/learn/advanced/baseapp_state-begin_block.png deleted file mode 100644 index 745d4a5a9712..000000000000 Binary files a/docs/learn/advanced/baseapp_state-begin_block.png and /dev/null differ diff --git a/docs/learn/advanced/baseapp_state-checktx.png b/docs/learn/advanced/baseapp_state-checktx.png deleted file mode 100644 index 38b217acdd04..000000000000 Binary files a/docs/learn/advanced/baseapp_state-checktx.png and /dev/null differ diff --git a/docs/learn/advanced/baseapp_state-commit.png b/docs/learn/advanced/baseapp_state-commit.png deleted file mode 100644 index b23c73125288..000000000000 Binary files a/docs/learn/advanced/baseapp_state-commit.png and /dev/null differ diff --git a/docs/learn/advanced/baseapp_state-deliver_tx.png b/docs/learn/advanced/baseapp_state-deliver_tx.png deleted file mode 100644 index f0a54b4ec34b..000000000000 Binary files a/docs/learn/advanced/baseapp_state-deliver_tx.png and /dev/null differ diff --git a/docs/learn/advanced/baseapp_state-initchain.png b/docs/learn/advanced/baseapp_state-initchain.png deleted file mode 100644 index 167b4fad9ed0..000000000000 Binary files a/docs/learn/advanced/baseapp_state-initchain.png and /dev/null differ diff --git a/docs/learn/advanced/baseapp_state-prepareproposal.png b/docs/learn/advanced/baseapp_state-prepareproposal.png deleted file mode 100644 index 146e804b09e8..000000000000 Binary files a/docs/learn/advanced/baseapp_state-prepareproposal.png and /dev/null differ diff --git a/docs/learn/advanced/baseapp_state-processproposal.png b/docs/learn/advanced/baseapp_state-processproposal.png deleted file mode 100644 index fb6012378dd5..000000000000 Binary files a/docs/learn/advanced/baseapp_state-processproposal.png and /dev/null differ diff --git a/docs/learn/advanced/baseapp_state.png b/docs/learn/advanced/baseapp_state.png deleted file mode 100644 index 5cf54fdb4afa..000000000000 Binary files a/docs/learn/advanced/baseapp_state.png and /dev/null differ diff --git a/docs/learn/beginner/00-app-anatomy.md b/docs/learn/beginner/00-app-anatomy.md deleted file mode 100644 index 0d9ae4036be8..000000000000 --- a/docs/learn/beginner/00-app-anatomy.md +++ /dev/null @@ -1,279 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Anatomy of a Cosmos SDK Application - -:::note Synopsis -This document describes the core parts of a Cosmos SDK application, represented throughout the document as a placeholder application named `app`. -::: - -## Node Client - -The Daemon, or [Full-Node Client](../advanced/03-node.md), is the core process of a Cosmos SDK-based blockchain. Participants in the network run this process to initialize their state-machine, connect with other full-nodes, and update their state-machine as new blocks come in. - -```text - ^ +-------------------------------+ ^ - | | | | - | | State-machine = Application | | - | | | | Built with Cosmos SDK - | | ^ + | | - | +----------- | ABCI | ----------+ v - | | + v | ^ - | | | | -Blockchain Node | | Consensus | | - | | | | - | +-------------------------------+ | CometBFT - | | | | - | | Networking | | - | | | | - v +-------------------------------+ v -``` - -The blockchain full-node presents itself as a binary, generally suffixed by `-d` for "daemon" (e.g. `appd` for `app` or `gaiad` for `gaia`). This binary is built by running a simple [`main.go`](../advanced/03-node.md#main-function) function placed in `./cmd/appd/`. This operation usually happens through the [Makefile](#dependencies-and-makefile). - -Once the main binary is built, the node can be started by running the [`start` command](../advanced/03-node.md#start-command). This command function primarily does three things: - -1. Create an instance of the state-machine defined in [`app.go`](#core-application-file). -2. Initialize the state-machine with the latest known state, extracted from the `db` stored in the `~/.app/data` folder. At this point, the state-machine is at height `appBlockHeight`. -3. Create and start a new CometBFT instance. Among other things, the node performs a handshake with its peers. It gets the latest `blockHeight` from them and replays blocks to sync to this height if it is greater than the local `appBlockHeight`. The node starts from genesis and CometBFT sends an `InitChain` message via the ABCI to the `app`, which triggers the [`InitChainer`](#initchainer). - -:::note -When starting a CometBFT instance, the genesis file is the `0` height and the state within the genesis file is committed at block height `1`. When querying the state of the node, querying block height 0 will return an error. -::: - -## Core Application File - -In general, the core of the state-machine is defined in a file called `app.go`. This file mainly contains the **type definition of the application** and functions to **create and initialize it**. - -### Type Definition of the Application - -The first thing defined in `app.go` is the `type` of the application. It is generally comprised of the following parts: - -* **A reference to [`baseapp`](../advanced/00-baseapp.md).** The custom application defined in `app.go` is an extension of `baseapp`. When a transaction is relayed by CometBFT to the application, `app` uses `baseapp`'s methods to route them to the appropriate module. `baseapp` implements most of the core logic for the application, including all the [ABCI methods](https://docs.cometbft.com/v0.37/spec/abci/) and the [routing logic](../advanced/00-baseapp.md#routing). -* **A list of store keys**. The [store](../advanced/04-store.md), which contains the entire state, is implemented as a [`multistore`](../advanced/04-store.md#multistore) (i.e. a store of stores) in the Cosmos SDK. Each module uses one or multiple stores in the multistore to persist their part of the state. These stores can be accessed with specific keys that are declared in the `app` type. These keys, along with the `keepers`, are at the heart of the [object-capabilities model](../advanced/10-ocap.md) of the Cosmos SDK. -* **A list of module's `keeper`s.** Each module defines an abstraction called [`keeper`](../../build/building-modules/06-keeper.md), which handles reads and writes for this module's store(s). The `keeper`'s methods of one module can be called from other modules (if authorized), which is why they are declared in the application's type and exported as interfaces to other modules so that the latter can only access the authorized functions. -* **A reference to an [`appCodec`](../advanced/05-encoding.md).** The application's `appCodec` is used to serialize and deserialize data structures in order to store them, as stores can only persist `[]bytes`. The default codec is [Protocol Buffers](../advanced/05-encoding.md). -* **A reference to a [`legacyAmino`](../advanced/05-encoding.md) codec.** Some parts of the Cosmos SDK have not been migrated to use the `appCodec` above, and are still hardcoded to use Amino. Other parts explicitly use Amino for backwards compatibility. For these reasons, the application still holds a reference to the legacy Amino codec. Please note that the Amino codec will be removed from the SDK in the upcoming releases. -* **A reference to a [module manager](../../build/building-modules/01-module-manager.md#manager)** and a [basic module manager](../../build/building-modules/01-module-manager.md#basicmanager). The module manager is an object that contains a list of the application's modules. It facilitates operations related to these modules, like registering their [`Msg` service](../advanced/00-baseapp.md#msg-services) and [gRPC `Query` service](../advanced/00-baseapp.md#grpc-query-services), or setting the order of execution between modules for various functions like [`InitChainer`](#initchainer), [`PreBlocker`](#preblocker) and [`BeginBlocker` and `EndBlocker`](#beginblocker-and-endblocker). - -See an example of application type definition from `simapp`, the Cosmos SDK's own app used for demo and testing purposes: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L173-L212 -``` - -### Constructor Function - -Also defined in `app.go` is the constructor function, which constructs a new application of the type defined in the preceding section. The function must fulfill the `AppCreator` signature in order to be used in the [`start` command](../advanced/03-node.md#start-command) of the application's daemon command. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/types/app.go#L66-L68 -``` - -Here are the main actions performed by this function: - -* Instantiate a new [`codec`](../advanced/05-encoding.md) and initialize the `codec` of each of the application's modules using the [basic manager](../../build/building-modules/01-module-manager.md#basicmanager). -* Instantiate a new application with a reference to a `baseapp` instance, a codec, and all the appropriate store keys. -* Instantiate all the [`keeper`](#keeper) objects defined in the application's `type` using the `NewKeeper` function of each of the application's modules. Note that keepers must be instantiated in the correct order, as the `NewKeeper` of one module might require a reference to another module's `keeper`. -* Instantiate the application's [module manager](../../build/building-modules/01-module-manager.md#manager) with the [`AppModule`](#application-module-interface) object of each of the application's modules. -* With the module manager, initialize the application's [`Msg` services](../advanced/00-baseapp.md#msg-services), [gRPC `Query` services](../advanced/00-baseapp.md#grpc-query-services), [legacy `Msg` routes](../advanced/00-baseapp.md#routing), and [legacy query routes](../advanced/00-baseapp.md#query-routing). When a transaction is relayed to the application by CometBFT via the ABCI, it is routed to the appropriate module's [`Msg` service](#msg-services) using the routes defined here. Likewise, when a gRPC query request is received by the application, it is routed to the appropriate module's [`gRPC query service`](#grpc-query-services) using the gRPC routes defined here. The Cosmos SDK still supports legacy `Msg`s and legacy CometBFT queries, which are routed using the legacy `Msg` routes and the legacy query routes, respectively. -* With the module manager, register the [application's modules' invariants](../../build/building-modules/07-invariants.md). Invariants are variables (e.g. total supply of a token) that are evaluated at the end of each block. The process of checking invariants is done via a special module called the [`InvariantsRegistry`](../../build/building-modules/07-invariants.md#invariant-registry). The value of the invariant should be equal to a predicted value defined in the module. Should the value be different than the predicted one, special logic defined in the invariant registry is triggered (usually the chain is halted). This is useful to make sure that no critical bug goes unnoticed, producing long-lasting effects that are hard to fix. -* With the module manager, set the order of execution between the `InitGenesis`, `PreBlocker`, `BeginBlocker`, and `EndBlocker` functions of each of the [application's modules](#application-module-interface). Note that not all modules implement these functions. -* Set the remaining application parameters: - * [`InitChainer`](#initchainer): used to initialize the application when it is first started. - * [`PreBlocker`](#preblocker): called before BeginBlock. - * [`BeginBlocker`, `EndBlocker`](#beginblocker-and-endblocker): called at the beginning and at the end of every block. - * [`anteHandler`](../advanced/00-baseapp.md#antehandler): used to handle fees and signature verification. -* Mount the stores. -* Return the application. - -Note that the constructor function only creates an instance of the app, while the actual state is either carried over from the `~/.app/data` folder if the node is restarted, or generated from the genesis file if the node is started for the first time. - -See an example of application constructor from `simapp`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L223-L575 -``` - -### InitChainer - -The `InitChainer` is a function that initializes the state of the application from a genesis file (i.e. token balances of genesis accounts). It is called when the application receives the `InitChain` message from the CometBFT engine, which happens when the node is started at `appBlockHeight == 0` (i.e. on genesis). The application must set the `InitChainer` in its [constructor](#constructor-function) via the [`SetInitChainer`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetInitChainer) method. - -In general, the `InitChainer` is mostly composed of the [`InitGenesis`](../../build/building-modules/08-genesis.md#initgenesis) function of each of the application's modules. This is done by calling the `InitGenesis` function of the module manager, which in turn calls the `InitGenesis` function of each of the modules it contains. Note that the order in which the modules' `InitGenesis` functions must be called has to be set in the module manager using the [module manager's](../../build/building-modules/01-module-manager.md) `SetOrderInitGenesis` method. This is done in the [application's constructor](#application-constructor), and the `SetOrderInitGenesis` has to be called before the `SetInitChainer`. - -See an example of an `InitChainer` from `simapp`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L626-L634 -``` - -### PreBlocker - -There are two semantics around the new lifecycle method: - -* It runs before the `BeginBlocker` of all modules -* It can modify consensus parameters in storage, and signal the caller through the return value. - -When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the finalize context: - -```go -app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) -``` - -The new ctx must be passed to all the other lifecycle methods. - -### BeginBlocker and EndBlocker - -The Cosmos SDK offers developers the possibility to implement automatic execution of code as part of their application. This is implemented through two functions called `BeginBlocker` and `EndBlocker`. They are called when the application receives the `FinalizeBlock` messages from the CometBFT consensus engine, which happens respectively at the beginning and at the end of each block. The application must set the `BeginBlocker` and `EndBlocker` in its [constructor](#constructor-function) via the [`SetBeginBlocker`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetBeginBlocker) and [`SetEndBlocker`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetEndBlocker) methods. - -In general, the `BeginBlocker` and `EndBlocker` functions are mostly composed of the [`BeginBlock` and `EndBlock`](../../build/building-modules/06-beginblock-endblock.md) functions of each of the application's modules. This is done by calling the `BeginBlock` and `EndBlock` functions of the module manager, which in turn calls the `BeginBlock` and `EndBlock` functions of each of the modules it contains. Note that the order in which the modules' `BeginBlock` and `EndBlock` functions must be called has to be set in the module manager using the `SetOrderBeginBlockers` and `SetOrderEndBlockers` methods, respectively. This is done via the [module manager](../../build/building-modules/01-module-manager.md) in the [application's constructor](#application-constructor), and the `SetOrderBeginBlockers` and `SetOrderEndBlockers` methods have to be called before the `SetBeginBlocker` and `SetEndBlocker` functions. - -As a sidenote, it is important to remember that application-specific blockchains are deterministic. Developers must be careful not to introduce non-determinism in `BeginBlocker` or `EndBlocker`, and must also be careful not to make them too computationally expensive, as [gas](./04-gas-fees.md) does not constrain the cost of `BeginBlocker` and `EndBlocker` execution. - -See an example of `BeginBlocker` and `EndBlocker` functions from `simapp` - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L613-L620 -``` - -### Register Codec - -The `EncodingConfig` structure is the last important part of the `app.go` file. The goal of this structure is to define the codecs that will be used throughout the app. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/params/encoding.go#L9-L16 -``` - -Here are descriptions of what each of the four fields means: - -* `InterfaceRegistry`: The `InterfaceRegistry` is used by the Protobuf codec to handle interfaces that are encoded and decoded (we also say "unpacked") using [`google.protobuf.Any`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto). `Any` could be thought as a struct that contains a `type_url` (name of a concrete type implementing the interface) and a `value` (its encoded bytes). `InterfaceRegistry` provides a mechanism for registering interfaces and implementations that can be safely unpacked from `Any`. Each application module implements the `RegisterInterfaces` method that can be used to register the module's own interfaces and implementations. - * You can read more about `Any` in [ADR-019](../../build/architecture/adr-019-protobuf-state-encoding.md). - * To go more into details, the Cosmos SDK uses an implementation of the Protobuf specification called [`gogoprotobuf`](https://github.com/cosmos/gogoproto). By default, the [gogo protobuf implementation of `Any`](https://pkg.go.dev/github.com/cosmos/gogoproto/types) uses [global type registration](https://github.com/cosmos/gogoproto/blob/master/proto/properties.go#L540) to decode values packed in `Any` into concrete Go types. This introduces a vulnerability where any malicious module in the dependency tree could register a type with the global protobuf registry and cause it to be loaded and unmarshaled by a transaction that referenced it in the `type_url` field. For more information, please refer to [ADR-019](../../build/architecture/adr-019-protobuf-state-encoding.md). -* `Codec`: The default codec used throughout the Cosmos SDK. It is composed of a `BinaryCodec` used to encode and decode state, and a `JSONCodec` used to output data to the users (for example, in the [CLI](#cli)). By default, the SDK uses Protobuf as `Codec`. -* `TxConfig`: `TxConfig` defines an interface a client can utilize to generate an application-defined concrete transaction type. Currently, the SDK handles two transaction types: `SIGN_MODE_DIRECT` (which uses Protobuf binary as over-the-wire encoding) and `SIGN_MODE_LEGACY_AMINO_JSON` (which depends on Amino). Read more about transactions [here](../advanced/01-transactions.md). -* `Amino`: Some legacy parts of the Cosmos SDK still use Amino for backwards-compatibility. Each module exposes a `RegisterLegacyAmino` method to register the module's specific types within Amino. This `Amino` codec should not be used by app developers anymore, and will be removed in future releases. - -An application should create its own encoding config. -See an example of a `simappparams.EncodingConfig` from `simapp`: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/params/encoding.go#L11-L16 -``` - -## Modules - -[Modules](../../build/building-modules/00-intro.md) are the heart and soul of Cosmos SDK applications. They can be considered as state-machines nested within the state-machine. When a transaction is relayed from the underlying CometBFT engine via the ABCI to the application, it is routed by [`baseapp`](../advanced/00-baseapp.md) to the appropriate module in order to be processed. This paradigm enables developers to easily build complex state-machines, as most of the modules they need often already exist. **For developers, most of the work involved in building a Cosmos SDK application revolves around building custom modules required by their application that do not exist yet, and integrating them with modules that do already exist into one coherent application**. In the application directory, the standard practice is to store modules in the `x/` folder (not to be confused with the Cosmos SDK's `x/` folder, which contains already-built modules). - -### Application Module Interface - -Modules must implement [interfaces](../../build/building-modules/01-module-manager.md#application-module-interfaces) defined in the Cosmos SDK, [`AppModuleBasic`](../../build/building-modules/01-module-manager.md#appmodulebasic) and [`AppModule`](../../build/building-modules/01-module-manager.md#appmodule). The former implements basic non-dependent elements of the module, such as the `codec`, while the latter handles the bulk of the module methods (including methods that require references to other modules' `keeper`s). Both the `AppModule` and `AppModuleBasic` types are, by convention, defined in a file called `module.go`. - -`AppModule` exposes a collection of useful methods on the module that facilitates the composition of modules into a coherent application. These methods are called from the [`module manager`](../../build/building-modules/01-module-manager.md#manager), which manages the application's collection of modules. - -### `Msg` Services - -Each application module defines two [Protobuf services](https://developers.google.com/protocol-buffers/docs/proto#services): one `Msg` service to handle messages, and one gRPC `Query` service to handle queries. If we consider the module as a state-machine, then a `Msg` service is a set of state transition RPC methods. -Each Protobuf `Msg` service method is 1:1 related to a Protobuf request type, which must implement `sdk.Msg` interface. -Note that `sdk.Msg`s are bundled in [transactions](../advanced/01-transactions.md), and each transaction contains one or multiple messages. - -When a valid block of transactions is received by the full-node, CometBFT relays each one to the application via [`DeliverTx`](https://docs.cometbft.com/v0.37/spec/abci/abci++_app_requirements#specifics-of-responsedelivertx). Then, the application handles the transaction: - -1. Upon receiving the transaction, the application first unmarshalls it from `[]byte`. -2. Then, it verifies a few things about the transaction like [fee payment and signatures](./04-gas-fees.md#antehandler) before extracting the `Msg`(s) contained in the transaction. -3. `sdk.Msg`s are encoded using Protobuf [`Any`s](#register-codec). By analyzing each `Any`'s `type_url`, baseapp's `msgServiceRouter` routes the `sdk.Msg` to the corresponding module's `Msg` service. -4. If the message is successfully processed, the state is updated. - -For more details, see [transaction lifecycle](./01-tx-lifecycle.md). - -Module developers create custom `Msg` services when they build their own module. The general practice is to define the `Msg` Protobuf service in a `tx.proto` file. For example, the `x/bank` module defines a service with two methods to transfer tokens: - -```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/bank/v1beta1/tx.proto#L13-L36 -``` - -Service methods use `keeper` in order to update the module state. - -Each module should also implement the `RegisterServices` method as part of the [`AppModule` interface](#application-module-interface). This method should call the `RegisterMsgServer` function provided by the generated Protobuf code. - -### gRPC `Query` Services - -gRPC `Query` services allow users to query the state using [gRPC](https://grpc.io). They are enabled by default, and can be configured under the `grpc.enable` and `grpc.address` fields inside [`app.toml`](../../user/run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml). - -gRPC `Query` services are defined in the module's Protobuf definition files, specifically inside `query.proto`. The `query.proto` definition file exposes a single `Query` [Protobuf service](https://developers.google.com/protocol-buffers/docs/proto#services). Each gRPC query endpoint corresponds to a service method, starting with the `rpc` keyword, inside the `Query` service. - -Protobuf generates a `QueryServer` interface for each module, containing all the service methods. A module's [`keeper`](#keeper) then needs to implement this `QueryServer` interface, by providing the concrete implementation of each service method. This concrete implementation is the handler of the corresponding gRPC query endpoint. - -Finally, each module should also implement the `RegisterServices` method as part of the [`AppModule` interface](#application-module-interface). This method should call the `RegisterQueryServer` function provided by the generated Protobuf code. - -### Keeper - -[`Keepers`](../../build/building-modules/06-keeper.md) are the gatekeepers of their module's store(s). To read or write in a module's store, it is mandatory to go through one of its `keeper`'s methods. This is ensured by the [object-capabilities](../advanced/10-ocap.md) model of the Cosmos SDK. Only objects that hold the key to a store can access it, and only the module's `keeper` should hold the key(s) to the module's store(s). - -`Keepers` are generally defined in a file called `keeper.go`. It contains the `keeper`'s type definition and methods. - -The `keeper` type definition generally consists of the following: - -* **Key(s)** to the module's store(s) in the multistore. -* Reference to **other module's `keepers`**. Only needed if the `keeper` needs to access other module's store(s) (either to read or write from them). -* A reference to the application's **codec**. The `keeper` needs it to marshal structs before storing them, or to unmarshal them when it retrieves them, because stores only accept `[]bytes` as value. - -Along with the type definition, the next important component of the `keeper.go` file is the `keeper`'s constructor function, `NewKeeper`. This function instantiates a new `keeper` of the type defined above with a `codec`, stores `keys` and potentially references other modules' `keeper`s as parameters. The `NewKeeper` function is called from the [application's constructor](#constructor-function). The rest of the file defines the `keeper`'s methods, which are primarily getters and setters. - -### Command-Line, gRPC Services and REST Interfaces - -Each module defines command-line commands, gRPC services, and REST routes to be exposed to the end-user via the [application's interfaces](#application-interfaces). This enables end-users to create messages of the types defined in the module, or to query the subset of the state managed by the module. - -#### CLI - -Generally, the [commands related to a module](../../build/building-modules/09-module-interfaces.md#cli) are defined in a folder called `client/cli` in the module's folder. The CLI divides commands into two categories, transactions and queries, defined in `client/cli/tx.go` and `client/cli/query.go`, respectively. Both commands are built on top of the [Cobra Library](https://github.com/spf13/cobra): - -* Transactions commands let users generate new transactions so that they can be included in a block and eventually update the state. One command should be created for each [message type](#message-types) defined in the module. The command calls the constructor of the message with the parameters provided by the end-user, and wraps it into a transaction. The Cosmos SDK handles signing and the addition of other transaction metadata. -* Queries let users query the subset of the state defined by the module. Query commands forward queries to the [application's query router](../advanced/00-baseapp.md#query-routing), which routes them to the appropriate [querier](#querier) the `queryRoute` parameter supplied. - -#### gRPC - -[gRPC](https://grpc.io) is a modern open-source high performance RPC framework that has support in multiple languages. It is the recommended way for external clients (such as wallets, browsers and other backend services) to interact with a node. - -Each module can expose gRPC endpoints called [service methods](https://grpc.io/docs/what-is-grpc/core-concepts/#service-definition), which are defined in the [module's Protobuf `query.proto` file](#grpc-query-services). A service method is defined by its name, input arguments, and output response. The module then needs to perform the following actions: - -* Define a `RegisterGRPCGatewayRoutes` method on `AppModuleBasic` to wire the client gRPC requests to the correct handler inside the module. -* For each service method, define a corresponding handler. The handler implements the core logic necessary to serve the gRPC request, and is located in the `keeper/grpc_query.go` file. - -#### gRPC-gateway REST Endpoints - -Some external clients may not wish to use gRPC. In this case, the Cosmos SDK provides a gRPC gateway service, which exposes each gRPC service as a corresponding REST endpoint. Please refer to the [grpc-gateway](https://grpc-ecosystem.github.io/grpc-gateway/) documentation to learn more. - -The REST endpoints are defined in the Protobuf files, along with the gRPC services, using Protobuf annotations. Modules that want to expose REST queries should add `google.api.http` annotations to their `rpc` methods. By default, all REST endpoints defined in the SDK have a URL starting with the `/cosmos/` prefix. - -The Cosmos SDK also provides a development endpoint to generate [Swagger](https://swagger.io/) definition files for these REST endpoints. This endpoint can be enabled inside the [`app.toml`](../../user/run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml) config file, under the `api.swagger` key. - -## Application Interface - -[Interfaces](#command-line-grpc-services-and-rest-interfaces) let end-users interact with full-node clients. This means querying data from the full-node or creating and sending new transactions to be relayed by the full-node and eventually included in a block. - -The main interface is the [Command-Line Interface](../advanced/07-cli.md). The CLI of a Cosmos SDK application is built by aggregating [CLI commands](#cli) defined in each of the modules used by the application. The CLI of an application is the same as the daemon (e.g. `appd`), and is defined in a file called `appd/main.go`. The file contains the following: - -* **A `main()` function**, which is executed to build the `appd` interface client. This function prepares each command and adds them to the `rootCmd` before building them. At the root of `appd`, the function adds generic commands like `status`, `keys`, and `config`, query commands, tx commands, and `rest-server`. -* **Query commands**, which are added by calling the `queryCmd` function. This function returns a Cobra command that contains the query commands defined in each of the application's modules (passed as an array of `sdk.ModuleClients` from the `main()` function), as well as some other lower level query commands such as block or validator queries. Query command are called by using the command `appd query [query]` of the CLI. -* **Transaction commands**, which are added by calling the `txCmd` function. Similar to `queryCmd`, the function returns a Cobra command that contains the tx commands defined in each of the application's modules, as well as lower level tx commands like transaction signing or broadcasting. Tx commands are called by using the command `appd tx [tx]` of the CLI. - -See an example of an application's main command-line file from the [Cosmos Hub](https://github.com/cosmos/gaia). - -```go reference -https://github.com/cosmos/gaia/blob/26ae7c2/cmd/gaiad/cmd/root.go#L39-L80 -``` - -## Dependencies and Makefile - -This section is optional, as developers are free to choose their dependency manager and project building method. That said, the current most used framework for versioning control is [`go.mod`](https://github.com/golang/go/wiki/Modules). It ensures each of the libraries used throughout the application are imported with the correct version. - -The following is the `go.mod` of the [Cosmos Hub](https://github.com/cosmos/gaia), provided as an example. - -```go reference -https://github.com/cosmos/gaia/blob/26ae7c2/go.mod#L1-L28 -``` - -For building the application, a [Makefile](https://en.wikipedia.org/wiki/Makefile) is generally used. The Makefile primarily ensures that the `go.mod` is run before building the two entrypoints to the application, [`appd`](#node-client) and [`appd`](#application-interface). - -Here is an example of the [Cosmos Hub Makefile](https://github.com/cosmos/gaia/blob/main/Makefile). diff --git a/docs/learn/beginner/01-tx-lifecycle.md b/docs/learn/beginner/01-tx-lifecycle.md deleted file mode 100644 index e3831ae3b45c..000000000000 --- a/docs/learn/beginner/01-tx-lifecycle.md +++ /dev/null @@ -1,264 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Transaction Lifecycle - -:::note Synopsis -This document describes the lifecycle of a transaction from creation to committed state changes. Transaction definition is described in a [different doc](../advanced/01-transactions.md). The transaction is referred to as `Tx`. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK Application](./00-app-anatomy.md) -::: - -## Creation - -### Transaction Creation - -One of the main application interfaces is the command-line interface. The transaction `Tx` can be created by the user inputting a command in the following format from the [command-line](../advanced/07-cli.md), providing the type of transaction in `[command]`, arguments in `[args]`, and configurations such as gas prices in `[flags]`: - -```bash -[appname] tx [command] [args] [flags] -``` - -This command automatically **creates** the transaction, **signs** it using the account's private key, and **broadcasts** it to the specified peer node. - -There are several required and optional flags for transaction creation. The `--from` flag specifies which [account](./03-accounts.md) the transaction is originating from. For example, if the transaction is sending coins, the funds are drawn from the specified `from` address. - -#### Gas and Fees - -Additionally, there are several [flags](../advanced/07-cli.md) users can use to indicate how much they are willing to pay in [fees](./04-gas-fees.md): - -* `--gas` refers to how much [gas](./04-gas-fees.md), which represents computational resources, `Tx` consumes. Gas is dependent on the transaction and is not precisely calculated until execution, but can be estimated by providing `auto` as the value for `--gas`. -* `--gas-adjustment` (optional) can be used to scale `gas` up in order to avoid underestimating. For example, users can specify their gas adjustment as 1.5 to use 1.5 times the estimated gas. -* `--gas-prices` specifies how much the user is willing to pay per unit of gas, which can be one or multiple denominations of tokens. For example, `--gas-prices=0.025uatom, 0.025upho` means the user is willing to pay 0.025uatom AND 0.025upho per unit of gas. -* `--fees` specifies how much in fees the user is willing to pay in total. -* `--timeout-height` specifies a block timeout height to prevent the tx from being committed past a certain height. - -The ultimate value of the fees paid is equal to the gas multiplied by the gas prices. In other words, `fees = ceil(gas * gasPrices)`. Thus, since fees can be calculated using gas prices and vice versa, the users specify only one of the two. - -Later, validators decide whether or not to include the transaction in their block by comparing the given or calculated `gas-prices` to their local `min-gas-prices`. `Tx` is rejected if its `gas-prices` is not high enough, so users are incentivized to pay more. - -#### CLI Example - -Users of the application `app` can enter the following command into their CLI to generate a transaction to send 1000uatom from a `senderAddress` to a `recipientAddress`. The command specifies how much gas they are willing to pay: an automatic estimate scaled up by 1.5 times, with a gas price of 0.025uatom per unit gas. - -```bash -appd tx send 1000uatom --from --gas auto --gas-adjustment 1.5 --gas-prices 0.025uatom -``` - -#### Other Transaction Creation Methods - -The command-line is an easy way to interact with an application, but `Tx` can also be created using a [gRPC or REST interface](../advanced/06-grpc_rest.md) or some other entry point defined by the application developer. From the user's perspective, the interaction depends on the web interface or wallet they are using (e.g. creating `Tx` using [Lunie.io](https://lunie.io/#/) and signing it with a Ledger Nano S). - -## Addition to Mempool - -Each full-node (running CometBFT) that receives a `Tx` sends an [ABCI message](https://docs.cometbft.com/v0.37/spec/p2p/messages/), -`CheckTx`, to the application layer to check for validity, and receives an `abci.ResponseCheckTx`. If the `Tx` passes the checks, it is held in the node's -[**Mempool**](https://docs.cometbft.com/v0.37/spec/p2p/messages/mempool/), an in-memory pool of transactions unique to each node, pending inclusion in a block - honest nodes discard a `Tx` if it is found to be invalid. Prior to consensus, nodes continuously check incoming transactions and gossip them to their peers. - -### Types of Checks - -The full-nodes perform stateless, then stateful checks on `Tx` during `CheckTx`, with the goal to -identify and reject an invalid transaction as early on as possible to avoid wasted computation. - -**_Stateless_** checks do not require nodes to access state - light clients or offline nodes can do -them - and are thus less computationally expensive. Stateless checks include making sure addresses -are not empty, enforcing nonnegative numbers, and other logic specified in the definitions. - -**_Stateful_** checks validate transactions and messages based on a committed state. Examples -include checking that the relevant values exist and can be transacted with, the address -has sufficient funds, and the sender is authorized or has the correct ownership to transact. -At any given moment, full-nodes typically have [multiple versions](../advanced/00-baseapp.md#state-updates) -of the application's internal state for different purposes. For example, nodes execute state -changes while in the process of verifying transactions, but still need a copy of the last committed -state in order to answer queries - they should not respond using state with uncommitted changes. - -In order to verify a `Tx`, full-nodes call `CheckTx`, which includes both _stateless_ and _stateful_ -checks. Further validation happens later in the [`DeliverTx`](#delivertx) stage. `CheckTx` goes -through several steps, beginning with decoding `Tx`. - -### Decoding - -When `Tx` is received by the application from the underlying consensus engine (e.g. CometBFT ), it is still in its [encoded](../advanced/05-encoding.md) `[]byte` form and needs to be unmarshaled in order to be processed. Then, the [`runTx`](../advanced/00-baseapp.md#runtx-antehandler-runmsgs-posthandler) function is called to run in `runTxModeCheck` mode, meaning the function runs all checks but exits before executing messages and writing state changes. - -### ValidateBasic (deprecated) - -Messages ([`sdk.Msg`](../advanced/01-transactions.md#messages)) are extracted from transactions (`Tx`). The `ValidateBasic` method of the `sdk.Msg` interface implemented by the module developer is run for each transaction. -To discard obviously invalid messages, the `BaseApp` type calls the `ValidateBasic` method very early in the processing of the message in the [`CheckTx`](../advanced/00-baseapp.md#checktx) and [`DeliverTx`](../advanced/00-baseapp.md#delivertx) transactions. -`ValidateBasic` can include only **stateless** checks (the checks that do not require access to the state). - -:::warning -The `ValidateBasic` method on messages has been deprecated in favor of validating messages directly in their respective [`Msg` services](../../build/building-modules/03-msg-services.md#Validation). - -Read [RFC 001](https://docs.cosmos.network/main/rfc/rfc-001-tx-validation) for more details. -::: - -:::note -`BaseApp` still calls `ValidateBasic` on messages that implements that method for backwards compatibility. -::: - -#### Guideline - -`ValidateBasic` should not be used anymore. Message validation should be performed in the `Msg` service when [handling a message](../../build/building-modules/msg-services#Validation) in a module Msg Server. - -### AnteHandler - -`AnteHandler`s even though optional, are in practice very often used to perform signature verification, gas calculation, fee deduction, and other core operations related to blockchain transactions. - -A copy of the cached context is provided to the `AnteHandler`, which performs limited checks specified for the transaction type. Using a copy allows the `AnteHandler` to do stateful checks for `Tx` without modifying the last committed state, and revert back to the original if the execution fails. - -For example, the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/main/x/auth/spec) module `AnteHandler` checks and increments sequence numbers, checks signatures and account numbers, and deducts fees from the first signer of the transaction - all state changes are made using the `checkState`. - -### Gas - -The [`Context`](../advanced/02-context.md), which keeps a `GasMeter` that tracks how much gas is used during the execution of `Tx`, is initialized. The user-provided amount of gas for `Tx` is known as `GasWanted`. If `GasConsumed`, the amount of gas consumed during execution, ever exceeds `GasWanted`, the execution stops and the changes made to the cached copy of the state are not committed. Otherwise, `CheckTx` sets `GasUsed` equal to `GasConsumed` and returns it in the result. After calculating the gas and fee values, validator-nodes check that the user-specified `gas-prices` is greater than their locally defined `min-gas-prices`. - -### Discard or Addition to Mempool - -If at any point during `CheckTx` the `Tx` fails, it is discarded and the transaction lifecycle ends -there. Otherwise, if it passes `CheckTx` successfully, the default protocol is to relay it to peer -nodes and add it to the Mempool so that the `Tx` becomes a candidate to be included in the next block. - -The **mempool** serves the purpose of keeping track of transactions seen by all full-nodes. -Full-nodes keep a **mempool cache** of the last `mempool.cache_size` transactions they have seen, as a first line of -defense to prevent replay attacks. Ideally, `mempool.cache_size` is large enough to encompass all -of the transactions in the full mempool. If the mempool cache is too small to keep track of all -the transactions, `CheckTx` is responsible for identifying and rejecting replayed transactions. - -Currently existing preventative measures include fees and a `sequence` (nonce) counter to distinguish -replayed transactions from identical but valid ones. If an attacker tries to spam nodes with many -copies of a `Tx`, full-nodes keeping a mempool cache reject all identical copies instead of running -`CheckTx` on them. Even if the copies have incremented `sequence` numbers, attackers are -disincentivized by the need to pay fees. - -Validator nodes keep a mempool to prevent replay attacks, just as full-nodes do, but also use it as -a pool of unconfirmed transactions in preparation of block inclusion. Note that even if a `Tx` -passes all checks at this stage, it is still possible to be found invalid later on, because -`CheckTx` does not fully validate the transaction (that is, it does not actually execute the messages). - -## Inclusion in a Block - -Consensus, the process through which validator nodes come to agreement on which transactions to -accept, happens in **rounds**. Each round begins with a proposer creating a block of the most -recent transactions and ends with **validators**, special full-nodes with voting power responsible -for consensus, agreeing to accept the block or go with a `nil` block instead. Validator nodes -execute the consensus algorithm, such as [CometBFT](https://docs.cometbft.com/v0.37/spec/consensus/), -confirming the transactions using ABCI requests to the application, in order to come to this agreement. - -The first step of consensus is the **block proposal**. One proposer amongst the validators is chosen -by the consensus algorithm to create and propose a block - in order for a `Tx` to be included, it -must be in this proposer's mempool. - -## State Changes - -The next step of consensus is to execute the transactions to fully validate them. All full-nodes -that receive a block proposal from the correct proposer execute the transactions by calling the ABCI function `FinalizeBlock`. -As mentioned throughout the documentation `BeginBlock`, `ExecuteTx` and `EndBlock` are called within FinalizeBlock. -Although every full-node operates individually and locally, the outcome is always consistent and unequivocal. This is because the state changes brought about by the messages are predictable, and the transactions are specifically sequenced in the proposed block. - -```text - ----------------------- - |Receive Block Proposal| - ----------------------- - | - v - ------------------------- - | FinalizeBlock | - | - v - ------------------- - | BeginBlock | - ------------------- - | - v - -------------------- - | ExecuteTx(tx0) | - | ExecuteTx(tx1) | - | ExecuteTx(tx2) | - | ExecuteTx(tx3) | - | . | - | . | - | . | - ------------------- - | - v - -------------------- - | EndBlock | - -------------------- - ------------------------- - | - v - ----------------------- - | Consensus | - ----------------------- - | - v - ----------------------- - | Commit | - ----------------------- -``` - -### Transaction Execution - -The `FinalizeBlock` ABCI function defined in [`BaseApp`](../advanced/00-baseapp.md) does the bulk of the -state transitions: it is run for each transaction in the block in sequential order as committed -to during consensus. Under the hood, transaction execution is almost identical to `CheckTx` but calls the -[`runTx`](../advanced/00-baseapp.md#runtx) function in deliver mode instead of check mode. -Instead of using their `checkState`, full-nodes use `finalizeblock`: - -* **Decoding:** Since `FinalizeBlock` is an ABCI call, `Tx` is received in the encoded `[]byte` form. - Nodes first unmarshal the transaction, using the [`TxConfig`](./app-anatomy#register-codec) defined in the app, then call `runTx` in `execModeFinalize`, which is very similar to `CheckTx` but also executes and writes state changes. - -* **Checks and `AnteHandler`:** Full-nodes call `validateBasicMsgs` and `AnteHandler` again. This second check - happens because they may not have seen the same transactions during the addition to Mempool stage - and a malicious proposer may have included invalid ones. One difference here is that the - `AnteHandler` does not compare `gas-prices` to the node's `min-gas-prices` since that value is local - to each node - differing values across nodes yield nondeterministic results. - -* **`MsgServiceRouter`:** After `CheckTx` exits, `FinalizeBlock` continues to run - [`runMsgs`](../advanced/00-baseapp.md#runtx-antehandler-runmsgs-posthandler) to fully execute each `Msg` within the transaction. - Since the transaction may have messages from different modules, `BaseApp` needs to know which module - to find the appropriate handler. This is achieved using `BaseApp`'s `MsgServiceRouter` so that it can be processed by the module's Protobuf [`Msg` service](../../build/building-modules/03-msg-services.md). - For `LegacyMsg` routing, the `Route` function is called via the [module manager](../../build/building-modules/01-module-manager.md) to retrieve the route name and find the legacy [`Handler`](../../build/building-modules/03-msg-services.md#handler-type) within the module. - -* **`Msg` service:** Protobuf `Msg` service is responsible for executing each message in the `Tx` and causes state transitions to persist in `finalizeBlockState`. - -* **PostHandlers:** [`PostHandler`](../advanced/00-baseapp.md#posthandler)s run after the execution of the message. If they fail, the state change of `runMsgs`, as well of `PostHandlers`, are both reverted. - -* **Gas:** While a `Tx` is being delivered, a `GasMeter` is used to keep track of how much - gas is being used; if execution completes, `GasUsed` is set and returned in the - `abci.ExecTxResult`. If execution halts because `BlockGasMeter` or `GasMeter` has run out or something else goes - wrong, a deferred function at the end appropriately errors or panics. - -If there are any failed state changes resulting from a `Tx` being invalid or `GasMeter` running out, -the transaction processing terminates and any state changes are reverted. Invalid transactions in a -block proposal cause validator nodes to reject the block and vote for a `nil` block instead. - -### Commit - -The final step is for nodes to commit the block and state changes. Validator nodes -perform the previous step of executing state transitions in order to validate the transactions, -then sign the block to confirm it. Full nodes that are not validators do not -participate in consensus - i.e. they cannot vote - but listen for votes to understand whether or -not they should commit the state changes. - -When they receive enough validator votes (2/3+ _precommits_ weighted by voting power), full nodes commit to a new block to be added to the blockchain and -finalize the state transitions in the application layer. A new state root is generated to serve as -a merkle proof for the state transitions. Applications use the [`Commit`](../advanced/00-baseapp.md#commit) -ABCI method inherited from [Baseapp](../advanced/00-baseapp.md); it syncs all the state transitions by -writing the `deliverState` into the application's internal state. As soon as the state changes are -committed, `checkState` starts afresh from the most recently committed state and `deliverState` -resets to `nil` in order to be consistent and reflect the changes. - -Note that not all blocks have the same number of transactions and it is possible for consensus to -result in a `nil` block or one with none at all. In a public blockchain network, it is also possible -for validators to be **byzantine**, or malicious, which may prevent a `Tx` from being committed in -the blockchain. Possible malicious behaviors include the proposer deciding to censor a `Tx` by -excluding it from the block or a validator voting against the block. - -At this point, the transaction lifecycle of a `Tx` is over: nodes have verified its validity, -delivered it by executing its state changes, and committed those changes. The `Tx` itself, -in `[]byte` form, is stored in a block and appended to the blockchain. diff --git a/docs/learn/beginner/02-query-lifecycle.md b/docs/learn/beginner/02-query-lifecycle.md deleted file mode 100644 index 04e4fc9ed644..000000000000 --- a/docs/learn/beginner/02-query-lifecycle.md +++ /dev/null @@ -1,147 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Query Lifecycle - -:::note Synopsis -This document describes the lifecycle of a query in a Cosmos SDK application, from the user interface to application stores and back. The query is referred to as `MyQuery`. -::: - -:::note Pre-requisite Readings - -* [Transaction Lifecycle](./01-tx-lifecycle.md) -::: - -## Query Creation - -A [**query**](../../build/building-modules/02-messages-and-queries.md#queries) is a request for information made by end-users of applications through an interface and processed by a full-node. Users can query information about the network, the application itself, and application state directly from the application's stores or modules. Note that queries are different from [transactions](../advanced/01-transactions.md) (view the lifecycle [here](./01-tx-lifecycle.md)), particularly in that they do not require consensus to be processed (as they do not trigger state-transitions); they can be fully handled by one full-node. - -For the purpose of explaining the query lifecycle, let's say the query, `MyQuery`, is requesting a list of delegations made by a certain delegator address in the application called `simapp`. As is to be expected, the [`staking`](../../build/modules/staking/README.md) module handles this query. But first, there are a few ways `MyQuery` can be created by users. - -### CLI - -The main interface for an application is the command-line interface. Users connect to a full-node and run the CLI directly from their machines - the CLI interacts directly with the full-node. To create `MyQuery` from their terminal, users type the following command: - -```bash -simd query staking delegations -``` - -This query command was defined by the [`staking`](../../build/modules/staking/README.md) module developer and added to the list of subcommands by the application developer when creating the CLI. - -Note that the general format is as follows: - -```bash -simd query [moduleName] [command] --flag -``` - -To provide values such as `--node` (the full-node the CLI connects to), the user can use the [`app.toml`](../../user/run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml) config file to set them or provide them as flags. - -The CLI understands a specific set of commands, defined in a hierarchical structure by the application developer: from the [root command](../advanced/07-cli.md#root-command) (`simd`), the type of command (`Myquery`), the module that contains the command (`staking`), and command itself (`delegations`). Thus, the CLI knows exactly which module handles this command and directly passes the call there. - -### gRPC - -Another interface through which users can make queries is [gRPC](https://grpc.io) requests to a [gRPC server](../advanced/06-grpc_rest.md#grpc-server). The endpoints are defined as [Protocol Buffers](https://developers.google.com/protocol-buffers) service methods inside `.proto` files, written in Protobuf's own language-agnostic interface definition language (IDL). The Protobuf ecosystem developed tools for code-generation from `*.proto` files into various languages. These tools allow to build gRPC clients easily. - -One such tool is [grpcurl](https://github.com/fullstorydev/grpcurl), and a gRPC request for `MyQuery` using this client looks like: - -```bash -grpcurl \ - -plaintext # We want results in plain test - -import-path ./proto \ # Import these .proto files - -proto ./proto/cosmos/staking/v1beta1/query.proto \ # Look into this .proto file for the Query protobuf service - -d '{"address":"$MY_DELEGATOR"}' \ # Query arguments - localhost:9090 \ # gRPC server endpoint - cosmos.staking.v1beta1.Query/Delegations # Fully-qualified service method name -``` - -### REST - -Another interface through which users can make queries is through HTTP Requests to a [REST server](../advanced/06-grpc_rest.md#rest-server). The REST server is fully auto-generated from Protobuf services, using [gRPC-gateway](https://github.com/grpc-ecosystem/grpc-gateway). - -An example HTTP request for `MyQuery` looks like: - -```bash -GET http://localhost:1317/cosmos/staking/v1beta1/delegators/{delegatorAddr}/delegations -``` - -## How Queries are Handled by the CLI - -The preceding examples show how an external user can interact with a node by querying its state. To understand in more detail the exact lifecycle of a query, let's dig into how the CLI prepares the query, and how the node handles it. The interactions from the users' perspective are a bit different, but the underlying functions are almost identical because they are implementations of the same command defined by the module developer. This step of processing happens within the CLI, gRPC, or REST server, and heavily involves a `client.Context`. - -### Context - -The first thing that is created in the execution of a CLI command is a `client.Context`. A `client.Context` is an object that stores all the data needed to process a request on the user side. In particular, a `client.Context` stores the following: - -* **Codec**: The [encoder/decoder](../advanced/05-encoding.md) used by the application, used to marshal the parameters and query before making the CometBFT RPC request and unmarshal the returned response into a JSON object. The default codec used by the CLI is Protobuf. -* **Account Decoder**: The account decoder from the [`auth`](../../build/modules/auth/README.md) module, which translates `[]byte`s into accounts. -* **RPC Client**: The CometBFT RPC Client, or node, to which requests are relayed. -* **Keyring**: A [Key Manager]../beginner/03-accounts.md#keyring) used to sign transactions and handle other operations with keys. -* **Output Writer**: A [Writer](https://pkg.go.dev/io/#Writer) used to output the response. -* **Configurations**: The flags configured by the user for this command, including `--height`, specifying the height of the blockchain to query, and `--indent`, which indicates to add an indent to the JSON response. - -The `client.Context` also contains various functions such as `Query()`, which retrieves the RPC Client and makes an ABCI call to relay a query to a full-node. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/context.go#L25-L68 -``` - -The `client.Context`'s primary role is to store data used during interactions with the end-user and provide methods to interact with this data - it is used before and after the query is processed by the full-node. Specifically, in handling `MyQuery`, the `client.Context` is utilized to encode the query parameters, retrieve the full-node, and write the output. Prior to being relayed to a full-node, the query needs to be encoded into a `[]byte` form, as full-nodes are application-agnostic and do not understand specific types. The full-node (RPC Client) itself is retrieved using the `client.Context`, which knows which node the user CLI is connected to. The query is relayed to this full-node to be processed. Finally, the `client.Context` contains a `Writer` to write output when the response is returned. These steps are further described in later sections. - -### Arguments and Route Creation - -At this point in the lifecycle, the user has created a CLI command with all of the data they wish to include in their query. A `client.Context` exists to assist in the rest of the `MyQuery`'s journey. Now, the next step is to parse the command or request, extract the arguments, and encode everything. These steps all happen on the user side within the interface they are interacting with. - -#### Encoding - -In our case (querying an address's delegations), `MyQuery` contains an [address](./03-accounts.md#addresses) `delegatorAddress` as its only argument. However, the request can only contain `[]byte`s, as it is ultimately relayed to a consensus engine (e.g. CometBFT) of a full-node that has no inherent knowledge of the application types. Thus, the `codec` of `client.Context` is used to marshal the address. - -Here is what the code looks like for the CLI command: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/client/cli/query.go#L315-L318 -``` - -#### gRPC Query Client Creation - -The Cosmos SDK leverages code generated from Protobuf services to make queries. The `staking` module's `MyQuery` service generates a `queryClient`, which the CLI uses to make queries. Here is the relevant code: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/client/cli/query.go#L308-L343 -``` - -Under the hood, the `client.Context` has a `Query()` function used to retrieve the pre-configured node and relay a query to it; the function takes the query fully-qualified service method name as path (in our case: `/cosmos.staking.v1beta1.Query/Delegations`), and arguments as parameters. It first retrieves the RPC Client (called the [**node**](../advanced/03-node.md)) configured by the user to relay this query to, and creates the `ABCIQueryOptions` (parameters formatted for the ABCI call). The node is then used to make the ABCI call, `ABCIQueryWithOptions()`. - -Here is what the code looks like: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/query.go#L79-L113 -``` - -## RPC - -With a call to `ABCIQueryWithOptions()`, `MyQuery` is received by a [full-node](../advanced/05-encoding.md) which then processes the request. Note that, while the RPC is made to the consensus engine (e.g. CometBFT) of a full-node, queries are not part of consensus and so are not broadcasted to the rest of the network, as they do not require anything the network needs to agree upon. - -Read more about ABCI Clients and CometBFT RPC in the [CometBFT documentation](https://docs.cometbft.com/v0.37/spec/rpc/). - -## Application Query Handling - -When a query is received by the full-node after it has been relayed from the underlying consensus engine, it is at that point being handled within an environment that understands application-specific types and has a copy of the state. [`baseapp`](../advanced/00-baseapp.md) implements the ABCI [`Query()`](../advanced/00-baseapp.md#query) function and handles gRPC queries. The query route is parsed, and it matches the fully-qualified service method name of an existing service method (most likely in one of the modules), then `baseapp` relays the request to the relevant module. - -Since `MyQuery` has a Protobuf fully-qualified service method name from the `staking` module (recall `/cosmos.staking.v1beta1.Query/Delegations`), `baseapp` first parses the path, then uses its own internal `GRPCQueryRouter` to retrieve the corresponding gRPC handler, and routes the query to the module. The gRPC handler is responsible for recognizing this query, retrieving the appropriate values from the application's stores, and returning a response. Read more about query services [here](../../build/building-modules/04-query-services.md). - -Once a result is received from the querier, `baseapp` begins the process of returning a response to the user. - -## Response - -Since `Query()` is an ABCI function, `baseapp` returns the response as an [`abci.ResponseQuery`](https://docs.cometbft.com/master/spec/abci/abci.html#query-2) type. The `client.Context` `Query()` routine receives the response and. - -### CLI Response - -The application [`codec`](../advanced/05-encoding.md) is used to unmarshal the response to a JSON and the `client.Context` prints the output to the command line, applying any configurations such as the output type (text, JSON or YAML). - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/context.go#L341-L349 -``` - -And that's a wrap! The result of the query is outputted to the console by the CLI. diff --git a/docs/learn/beginner/03-accounts.md b/docs/learn/beginner/03-accounts.md deleted file mode 100644 index 7280108ac744..000000000000 --- a/docs/learn/beginner/03-accounts.md +++ /dev/null @@ -1,281 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Accounts - -:::note Synopsis -This document describes the in-built account and public key system of the Cosmos SDK. -::: - -:::note Pre-requisite Readings - - -* [Anatomy of a Cosmos SDK Application](./00-app-anatomy.md) - -::: - -## Account Definition - -In the Cosmos SDK, an _account_ designates a pair of _public key_ `PubKey` and _private key_ `PrivKey`. The `PubKey` can be derived to generate various `Addresses`, which are used to identify users (among other parties) in the application. `Addresses` are also associated with [`message`s](../../build/building-modules/02-messages-and-queries.md#messages) to identify the sender of the `message`. The `PrivKey` is used to generate [digital signatures](#signatures) to prove that an `Address` associated with the `PrivKey` approved of a given `message`. - -For HD key derivation the Cosmos SDK uses a standard called [BIP32](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki). The BIP32 allows users to create an HD wallet (as specified in [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)) - a set of accounts derived from an initial secret seed. A seed is usually created from a 12- or 24-word mnemonic. A single seed can derive any number of `PrivKey`s using a one-way cryptographic function. Then, a `PubKey` can be derived from the `PrivKey`. Naturally, the mnemonic is the most sensitive information, as private keys can always be re-generated if the mnemonic is preserved. - -```text - Account 0 Account 1 Account 2 - -+------------------+ +------------------+ +------------------+ -| | | | | | -| Address 0 | | Address 1 | | Address 2 | -| ^ | | ^ | | ^ | -| | | | | | | | | -| | | | | | | | | -| | | | | | | | | -| + | | + | | + | -| Public key 0 | | Public key 1 | | Public key 2 | -| ^ | | ^ | | ^ | -| | | | | | | | | -| | | | | | | | | -| | | | | | | | | -| + | | + | | + | -| Private key 0 | | Private key 1 | | Private key 2 | -| ^ | | ^ | | ^ | -+------------------+ +------------------+ +------------------+ - | | | - | | | - | | | - +--------------------------------------------------------------------+ - | - | - +---------+---------+ - | | - | Master PrivKey | - | | - +-------------------+ - | - | - +---------+---------+ - | | - | Mnemonic (Seed) | - | | - +-------------------+ -``` - -In the Cosmos SDK, keys are stored and managed by using an object called a [`Keyring`](#keyring). - -## Keys, accounts, addresses, and signatures - -The principal way of authenticating a user is done using [digital signatures](https://en.wikipedia.org/wiki/Digital_signature). Users sign transactions using their own private key. Signature verification is done with the associated public key. For on-chain signature verification purposes, we store the public key in an `Account` object (alongside other data required for a proper transaction validation). - -In the node, all data is stored using Protocol Buffers serialization. - -The Cosmos SDK supports the following digital key schemes for creating digital signatures: - -* `secp256k1`, as implemented in the [Cosmos SDK's `crypto/keys/secp256k1` package](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/keys/secp256k1/secp256k1.go). -* `secp256r1`, as implemented in the [Cosmos SDK's `crypto/keys/secp256r1` package](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/keys/secp256r1/pubkey.go), -* `tm-ed25519`, as implemented in the [Cosmos SDK `crypto/keys/ed25519` package](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/keys/ed25519/ed25519.go). This scheme is supported only for the consensus validation. - -| | Address length in bytes | Public key length in bytes | Used for transaction authentication | Used for consensus (cometbft) | -| :----------: | :---------------------: | :------------------------: | :---------------------------------: | :-----------------------------: | -| `secp256k1` | 20 | 33 | yes | no | -| `secp256r1` | 32 | 33 | yes | no | -| `tm-ed25519` | -- not used -- | 32 | no | yes | - -## Addresses - -`Addresses` and `PubKey`s are both public information that identifies actors in the application. `Account` is used to store authentication information. The basic account implementation is provided by a `BaseAccount` object. - -Each account is identified using `Address` which is a sequence of bytes derived from a public key. In the Cosmos SDK, we define 3 types of addresses that specify a context where an account is used: - -* `AccAddress` identifies users (the sender of a `message`). -* `ValAddress` identifies validator operators. -* `ConsAddress` identifies validator nodes that are participating in consensus. Validator nodes are derived using the **`ed25519`** curve. - -These types implement the `Address` interface: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/address.go#L126-L134 -``` - -Address construction algorithm is defined in [ADR-28](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-028-public-key-addresses.md). -Here is the standard way to obtain an account address from a `pub` public key: - -```go -sdk.AccAddress(pub.Address().Bytes()) -``` - -Of note, the `Marshal()` and `Bytes()` method both return the same raw `[]byte` form of the address. `Marshal()` is required for Protobuf compatibility. - -For user interaction, addresses are formatted using [Bech32](https://en.bitcoin.it/wiki/Bech32) and implemented by the `String` method. The Bech32 method is the only supported format to use when interacting with a blockchain. The Bech32 human-readable part (Bech32 prefix) is used to denote an address type. Example: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/address.go#L299-L316 -``` - -| | Address Bech32 Prefix | -| ------------------ | --------------------- | -| Accounts | cosmos | -| Validator Operator | cosmosvaloper | -| Consensus Nodes | cosmosvalcons | - -### Public Keys - -Public keys in Cosmos SDK are defined by `cryptotypes.PubKey` interface. Since public keys are saved in a store, `cryptotypes.PubKey` extends the `proto.Message` interface: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/types/types.go#L8-L17 -``` - -A compressed format is used for `secp256k1` and `secp256r1` serialization. - -* The first byte is a `0x02` byte if the `y`-coordinate is the lexicographically largest of the two associated with the `x`-coordinate. -* Otherwise the first byte is a `0x03`. - -This prefix is followed by the `x`-coordinate. - -Public Keys are not used to reference accounts (or users) and in general are not used when composing transaction messages (with few exceptions: `MsgCreateValidator`, `Validator` and `Multisig` messages). -For user interactions, `PubKey` is formatted using Protobufs JSON ([ProtoMarshalJSON](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/codec/json.go#L14-L34) function). Example: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/keys/output.go#L23-L39 -``` - -## Keyring - -A `Keyring` is an object that stores and manages accounts. In the Cosmos SDK, a `Keyring` implementation follows the `Keyring` interface: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/keyring/keyring.go#L57-L105 -``` - -The default implementation of `Keyring` comes from the third-party [`99designs/keyring`](https://github.com/99designs/keyring) library. - -A few notes on the `Keyring` methods: - -* `Sign(uid string, msg []byte) ([]byte, types.PubKey, error)` strictly deals with the signature of the `msg` bytes. You must prepare and encode the transaction into a canonical `[]byte` form. Because protobuf is not deterministic, it has been decided in [ADR-020](../../build/architecture/adr-020-protobuf-transaction-encoding.md) that the canonical `payload` to sign is the `SignDoc` struct, deterministically encoded using [ADR-027](../../build/architecture/adr-027-deterministic-protobuf-serialization.md). Note that signature verification is not implemented in the Cosmos SDK by default, it is deferred to the [`anteHandler`](../advanced/00-baseapp.md#antehandler). - -```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L50-L66 -``` - -* `NewAccount(uid, mnemonic, bip39Passphrase, hdPath string, algo SignatureAlgo) (*Record, error)` creates a new account based on the [`bip44 path`](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) and persists it on disk. The `PrivKey` is **never stored unencrypted**, instead it is [encrypted with a passphrase](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/armor.go) before being persisted. In the context of this method, the key type and sequence number refer to the segment of the BIP44 derivation path (for example, `0`, `1`, `2`, ...) that is used to derive a private and a public key from the mnemonic. Using the same mnemonic and derivation path, the same `PrivKey`, `PubKey` and `Address` is generated. The following keys are supported by the keyring: - -* `secp256k1` -* `ed25519` - -* `ExportPrivKeyArmor(uid, encryptPassphrase string) (armor string, err error)` exports a private key in ASCII-armored encrypted format using the given passphrase. You can then either import the private key again into the keyring using the `ImportPrivKey(uid, armor, passphrase string)` function or decrypt it into a raw private key using the `UnarmorDecryptPrivKey(armorStr string, passphrase string)` function. - -### Create New Key Type - -To create a new key type for using in keyring, `keyring.SignatureAlgo` interface must be fulfilled. - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/keyring/signing_algorithms.go#L10-L15 -``` - -The interface consists in three methods where `Name()` returns the name of the algorithm as a `hd.PubKeyType` and `Derive()` and `Generate()` must return the following functions respectively: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/hd/algo.go#L28-L31 -``` - -Once the `keyring.SignatureAlgo` has been implemented it must be added to the [list of supported algos](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/keyring/keyring.go#L217) of the keyring. - -For simplicity the implementation of a new key type should be done inside the `crypto/hd` package. -There is an example of a working `secp256k1` implementation in [algo.go](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/hd/algo.go#L38). - - -#### Implementing secp256r1 algo - -Here is an example of how secp256r1 could be implemented. - -First a new function to create a private key from a secret number is needed in the secp256r1 package. This function could look like this: - -```go -// cosmos-sdk/crypto/keys/secp256r1/privkey.go - -// NewPrivKeyFromSecret creates a private key derived for the secret number -// represented in big-endian. The `secret` must be a valid ECDSA field element. -func NewPrivKeyFromSecret(secret []byte) (*PrivKey, error) { - var d = new(big.Int).SetBytes(secret) - if d.Cmp(secp256r1.Params().N) >= 1 { - return nil, errorsmod.Wrap(errors.ErrInvalidRequest, "secret not in the curve base field") - } - sk := new(ecdsa.PrivKey) - return &PrivKey{&ecdsaSK{*sk}}, nil -} -``` - -After that `secp256r1Algo` can be implemented. - -```go -// cosmos-sdk/crypto/hd/secp256r1Algo.go - -package hd - -import ( - "github.com/cosmos/go-bip39" - - "github.com/cosmos/cosmos-sdk/crypto/keys/secp256r1" - "github.com/cosmos/cosmos-sdk/crypto/types" -) - -// Secp256r1Type uses the secp256r1 ECDSA parameters. -const Secp256r1Type = PubKeyType("secp256r1") - -var Secp256r1 = secp256r1Algo{} - -type secp256r1Algo struct{} - -func (s secp256r1Algo) Name() PubKeyType { - return Secp256r1Type -} - -// Derive derives and returns the secp256r1 private key for the given seed and HD path. -func (s secp256r1Algo) Derive() DeriveFn { - return func(mnemonic string, bip39Passphrase, hdPath string) ([]byte, error) { - seed, err := bip39.NewSeedWithErrorChecking(mnemonic, bip39Passphrase) - if err != nil { - return nil, err - } - - masterPriv, ch := ComputeMastersFromSeed(seed) - if len(hdPath) == 0 { - return masterPriv[:], nil - } - derivedKey, err := DerivePrivateKeyForPath(masterPriv, ch, hdPath) - - return derivedKey, err - } -} - -// Generate generates a secp256r1 private key from the given bytes. -func (s secp256r1Algo) Generate() GenerateFn { - return func(bz []byte) types.PrivKey { - key, err := secp256r1.NewPrivKeyFromSecret(bz) - if err != nil { - panic(err) - } - return key - } -} -``` - -Finally, the algo must be added to the list of [supported algos](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/crypto/keyring/keyring.go#L217) by the keyring. - -```go -// cosmos-sdk/crypto/keyring/keyring.go - -func newKeystore(kr keyring.Keyring, cdc codec.Codec, backend string, opts ...Option) keystore { - // Default options for keybase, these can be overwritten using the - // Option function - options := Options{ - SupportedAlgos: SigningAlgoList{hd.Secp256k1, hd.Secp256r1}, // added here - SupportedAlgosLedger: SigningAlgoList{hd.Secp256k1}, - } -... -``` - -Hereafter to create new keys using your algo, you must specify it with the flag `--algo` : - -`simd keys add myKey --algo secp256r1` diff --git a/docs/learn/beginner/04-gas-fees.md b/docs/learn/beginner/04-gas-fees.md deleted file mode 100644 index 2e04af462e89..000000000000 --- a/docs/learn/beginner/04-gas-fees.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Gas and Fees - -:::note Synopsis -This document describes the default strategies to handle gas and fees within a Cosmos SDK application. -::: - -:::note Pre-requisite Readings - -* [Anatomy of a Cosmos SDK Application](./00-app-anatomy.md) - -::: - -## Introduction to `Gas` and `Fees` - -In the Cosmos SDK, `gas` is a special unit that is used to track the consumption of resources during execution. `gas` is typically consumed whenever read and writes are made to the store, but it can also be consumed if expensive computation needs to be done. It serves two main purposes: - -* Make sure blocks are not consuming too many resources and are finalized. This is implemented by default in the Cosmos SDK via the [block gas meter](#block-gas-meter). -* Prevent spam and abuse from end-user. To this end, `gas` consumed during [`message`](../../build/building-modules/02-messages-and-queries.md#messages) execution is typically priced, resulting in a `fee` (`fees = gas * gas-prices`). `fees` generally have to be paid by the sender of the `message`. Note that the Cosmos SDK does not enforce `gas` pricing by default, as there may be other ways to prevent spam (e.g. bandwidth schemes). Still, most applications implement `fee` mechanisms to prevent spam by using the [`AnteHandler`](#antehandler). - -## Gas Meter - -In the Cosmos SDK, `gas` is a simple alias for `uint64`, and is managed by an object called a _gas meter_. Gas meters implement the `GasMeter` interface - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/gas.go#L40-L51 -``` - -where: - -* `GasConsumed()` returns the amount of gas that was consumed by the gas meter instance. -* `GasConsumedToLimit()` returns the amount of gas that was consumed by gas meter instance, or the limit if it is reached. -* `GasRemaining()` returns the gas left in the GasMeter. -* `Limit()` returns the limit of the gas meter instance. `0` if the gas meter is infinite. -* `ConsumeGas(amount Gas, descriptor string)` consumes the amount of `gas` provided. If the `gas` overflows, it panics with the `descriptor` message. If the gas meter is not infinite, it panics if `gas` consumed goes above the limit. -* `RefundGas()` deducts the given amount from the gas consumed. This functionality enables refunding gas to the transaction or block gas pools so that EVM-compatible chains can fully support the go-ethereum StateDB interface. -* `IsPastLimit()` returns `true` if the amount of gas consumed by the gas meter instance is strictly above the limit, `false` otherwise. -* `IsOutOfGas()` returns `true` if the amount of gas consumed by the gas meter instance is above or equal to the limit, `false` otherwise. - -The gas meter is generally held in [`ctx`](../advanced/02-context.md), and consuming gas is done with the following pattern: - -```go -ctx.GasMeter().ConsumeGas(amount, "description") -``` - -By default, the Cosmos SDK makes use of two different gas meters, the [main gas meter](#main-gas-metter) and the [block gas meter](#block-gas-meter). - -### Main Gas Meter - -`ctx.GasMeter()` is the main gas meter of the application. The main gas meter is initialized in `FinalizeBlock` via `setFinalizeBlockState`, and then tracks gas consumption during execution sequences that lead to state-transitions, i.e. those originally triggered by [`FinalizeBlock`](../advanced/00-baseapp.md#finalizeblock). At the beginning of each transaction execution, the main gas meter **must be set to 0** in the [`AnteHandler`](#antehandler), so that it can track gas consumption per-transaction. - -Gas consumption can be done manually, generally by the module developer in the [`BeginBlocker`, `EndBlocker`](../../build/building-modules/06-beginblock-endblock.md) or [`Msg` service](../../build/building-modules/03-msg-services.md), but most of the time it is done automatically whenever there is a read or write to the store. This automatic gas consumption logic is implemented in a special store called [`GasKv`](../advanced/04-store.md#gaskv-store). - -### Block Gas Meter - -`ctx.BlockGasMeter()` is the gas meter used to track gas consumption per block and make sure it does not go above a certain limit. A new instance of the `BlockGasMeter` is created each time [`FinalizeBlock`](../advanced/00-baseapp.md#finalizeblock) is called. The `BlockGasMeter` is finite, and the limit of gas per block is defined in the application's consensus parameters. By default, Cosmos SDK applications use the default consensus parameters provided by CometBFT: - -```go reference -https://github.com/cometbft/cometbft/blob/v0.37.0/types/params.go#L66-L105 -``` - -When a new [transaction](../advanced/01-transactions.md) is being processed via `FinalizeBlock`, the current value of `BlockGasMeter` is checked to see if it is above the limit. If it is, the transaction fails and returned to the consensus engine as a failed transaction. This can happen even with the first transaction in a block, as `FinalizeBlock` itself can consume gas. If not, the transaction is processed normally. At the end of `FinalizeBlock`, the gas tracked by `ctx.BlockGasMeter()` is increased by the amount consumed to process the transaction: - -```go -ctx.BlockGasMeter().ConsumeGas( - ctx.GasMeter().GasConsumedToLimit(), - "block gas meter", -) -``` - -## AnteHandler - -The `AnteHandler` is run for every transaction during `CheckTx` and `FinalizeBlock`, before a Protobuf `Msg` service method for each `sdk.Msg` in the transaction. - -The anteHandler is not implemented in the core Cosmos SDK but in a module. That said, most applications today use the default implementation defined in the [`auth` module](https://github.com/cosmos/cosmos-sdk/tree/main/x/auth). Here is what the `anteHandler` is intended to do in a normal Cosmos SDK application: - -* Verify that the transactions are of the correct type. Transaction types are defined in the module that implements the `anteHandler`, and they follow the transaction interface: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/tx_msg.go#L51-L56 -``` - - This enables developers to play with various types for the transaction of their application. In the default `auth` module, the default transaction type is `Tx`: - -```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L14-L27 -``` - -* Verify signatures for each [`message`](../../build/building-modules/02-messages-and-queries.md#messages) contained in the transaction. Each `message` should be signed by one or multiple sender(s), and these signatures must be verified in the `anteHandler`. -* During `CheckTx`, verify that the gas prices provided with the transaction is greater than the local `min-gas-prices` (as a reminder, gas-prices can be deducted from the following equation: `fees = gas * gas-prices`). `min-gas-prices` is a parameter local to each full-node and used during `CheckTx` to discard transactions that do not provide a minimum amount of fees. This ensures that the mempool cannot be spammed with garbage transactions. -* Verify that the sender of the transaction has enough funds to cover for the `fees`. When the end-user generates a transaction, they must indicate 2 of the 3 following parameters (the third one being implicit): `fees`, `gas` and `gas-prices`. This signals how much they are willing to pay for nodes to execute their transaction. The provided `gas` value is stored in a parameter called `GasWanted` for later use. -* Set `newCtx.GasMeter` to 0, with a limit of `GasWanted`. **This step is crucial**, as it not only makes sure the transaction cannot consume infinite gas, but also that `ctx.GasMeter` is reset in-between each transaction (`ctx` is set to `newCtx` after `anteHandler` is run, and the `anteHandler` is run each time a transactions executes). - -As explained above, the `anteHandler` returns a maximum limit of `gas` the transaction can consume during execution called `GasWanted`. The actual amount consumed in the end is denominated `GasUsed`, and we must therefore have `GasUsed =< GasWanted`. Both `GasWanted` and `GasUsed` are relayed to the underlying consensus engine when [`FinalizeBlock`](../advanced/00-baseapp.md#finalizeblock) returns. diff --git a/docs/learn/beginner/_category_.json b/docs/learn/beginner/_category_.json deleted file mode 100644 index 558d052b8b5f..000000000000 --- a/docs/learn/beginner/_category_.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "label": "Beginner", - "position": 1, - "link": null -} \ No newline at end of file diff --git a/docs/learn/intro/01-why-app-specific.md b/docs/learn/intro/01-why-app-specific.md deleted file mode 100644 index 5830530ae706..000000000000 --- a/docs/learn/intro/01-why-app-specific.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Application-Specific Blockchains - -:::note Synopsis -This document explains what application-specific blockchains are, and why developers would want to build one as opposed to writing Smart Contracts. -::: - -## What are application-specific blockchains - -Application-specific blockchains are blockchains customized to operate a single application. Instead of building a decentralized application on top of an underlying blockchain like Ethereum, developers build their own blockchain from the ground up. This means building a full-node client, a light-client, and all the necessary interfaces (CLI, REST, ...) to interact with the nodes. - -```text - ^ +-------------------------------+ ^ - | | | | Built with Cosmos SDK - | | State-machine = Application | | - | | | v - | +-------------------------------+ - | | | ^ -Blockchain node | | Consensus | | - | | | | - | +-------------------------------+ | CometBFT - | | | | - | | Networking | | - | | | | - v +-------------------------------+ v -``` - -## What are the shortcomings of Smart Contracts - -Virtual-machine blockchains like Ethereum addressed the demand for more programmability back in 2014. At the time, the options available for building decentralized applications were quite limited. Most developers would build on top of the complex and limited Bitcoin scripting language, or fork the Bitcoin codebase which was hard to work with and customize. - -Virtual-machine blockchains came in with a new value proposition. Their state-machine incorporates a virtual-machine that is able to interpret turing-complete programs called Smart Contracts. These Smart Contracts are very good for use cases like one-time events (e.g. ICOs), but they can fall short for building complex decentralized platforms. Here is why: - -* Smart Contracts are generally developed with specific programming languages that can be interpreted by the underlying virtual-machine. These programming languages are often immature and inherently limited by the constraints of the virtual-machine itself. For example, the Ethereum Virtual Machine does not allow developers to implement automatic execution of code. Developers are also limited to the account-based system of the EVM, and they can only choose from a limited set of functions for their cryptographic operations. These are examples, but they hint at the lack of **flexibility** that a smart contract environment often entails. -* Smart Contracts are all run by the same virtual machine. This means that they compete for resources, which can severely restrain **performance**. And even if the state-machine were to be split in multiple subsets (e.g. via sharding), Smart Contracts would still need to be interpreted by a virtual machine, which would limit performance compared to a native application implemented at state-machine level (our benchmarks show an improvement on the order of 10x in performance when the virtual-machine is removed). -* Another issue with the fact that Smart Contracts share the same underlying environment is the resulting limitation in **sovereignty**. A decentralized application is an ecosystem that involves multiple players. If the application is built on a general-purpose virtual-machine blockchain, stakeholders have very limited sovereignty over their application, and are ultimately superseded by the governance of the underlying blockchain. If there is a bug in the application, very little can be done about it. - -Application-Specific Blockchains are designed to address these shortcomings. - -## Application-Specific Blockchains Benefits - -### Flexibility - -Application-specific blockchains give maximum flexibility to developers: - -* In Cosmos blockchains, the state-machine is typically connected to the underlying consensus engine via an interface called the [ABCI](https://docs.cometbft.com/v0.37/spec/abci/). This interface can be wrapped in any programming language, meaning developers can build their state-machine in the programming language of their choice. - -* Developers can choose among multiple frameworks to build their state-machine. The most widely used today is the Cosmos SDK, but others exist (e.g. [Lotion](https://github.com/nomic-io/lotion), [Weave](https://github.com/iov-one/weave), ...). Typically the choice will be made based on the programming language they want to use (Cosmos SDK and Weave are in Golang, Lotion is in Javascript, ...). -* The ABCI also allows developers to swap the consensus engine of their application-specific blockchain. Today, only CometBFT is production-ready, but in the future other consensus engines are expected to emerge. -* Even when they settle for a framework and consensus engine, developers still have the freedom to tweak them if they don't perfectly match their requirements in their pristine forms. -* Developers are free to explore the full spectrum of tradeoffs (e.g. number of validators vs transaction throughput, safety vs availability in asynchrony, ...) and design choices (DB or IAVL tree for storage, UTXO or account model, ...). -* Developers can implement automatic execution of code. In the Cosmos SDK, logic can be automatically triggered at the beginning and the end of each block. They are also free to choose the cryptographic library used in their application, as opposed to being constrained by what is made available by the underlying environment in the case of virtual-machine blockchains. - -The list above contains a few examples that show how much flexibility application-specific blockchains give to developers. The goal of Cosmos and the Cosmos SDK is to make developer tooling as generic and composable as possible, so that each part of the stack can be forked, tweaked and improved without losing compatibility. As the community grows, more alternatives for each of the core building blocks will emerge, giving more options to developers. - -### Performance - -decentralized applications built with Smart Contracts are inherently capped in performance by the underlying environment. For a decentralized application to optimise performance, it needs to be built as an application-specific blockchain. Next are some of the benefits an application-specific blockchain brings in terms of performance: - -* Developers of application-specific blockchains can choose to operate with a novel consensus engine such as CometBFT BFT. Compared to Proof-of-Work (used by most virtual-machine blockchains today), it offers significant gains in throughput. -* An application-specific blockchain only operates a single application, so that the application does not compete with others for computation and storage. This is the opposite of most non-sharded virtual-machine blockchains today, where smart contracts all compete for computation and storage. -* Even if a virtual-machine blockchain offered application-based sharding coupled with an efficient consensus algorithm, performance would still be limited by the virtual-machine itself. The real throughput bottleneck is the state-machine, and requiring transactions to be interpreted by a virtual-machine significantly increases the computational complexity of processing them. - -### Security - -Security is hard to quantify, and greatly varies from platform to platform. That said here are some important benefits an application-specific blockchain can bring in terms of security: - -* Developers can choose proven programming languages like Go when building their application-specific blockchains, as opposed to smart contract programming languages that are often more immature. -* Developers are not constrained by the cryptographic functions made available by the underlying virtual-machines. They can use their own custom cryptography, and rely on well-audited crypto libraries. -* Developers do not have to worry about potential bugs or exploitable mechanisms in the underlying virtual-machine, making it easier to reason about the security of the application. - -### Sovereignty - -One of the major benefits of application-specific blockchains is sovereignty. A decentralized application is an ecosystem that involves many actors: users, developers, third-party services, and more. When developers build on virtual-machine blockchain where many decentralized applications coexist, the community of the application is different than the community of the underlying blockchain, and the latter supersedes the former in the governance process. If there is a bug or if a new feature is needed, stakeholders of the application have very little leeway to upgrade the code. If the community of the underlying blockchain refuses to act, nothing can happen. - -The fundamental issue here is that the governance of the application and the governance of the network are not aligned. This issue is solved by application-specific blockchains. Because application-specific blockchains specialize to operate a single application, stakeholders of the application have full control over the entire chain. This ensures that the community will not be stuck if a bug is discovered, and that it has the freedom to choose how it is going to evolve. diff --git a/docs/learn/intro/02-sdk-app-architecture.md b/docs/learn/intro/02-sdk-app-architecture.md deleted file mode 100644 index 475326903ea7..000000000000 --- a/docs/learn/intro/02-sdk-app-architecture.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Blockchain Architecture - -## State machine - -At its core, a blockchain is a [replicated deterministic state machine](https://en.wikipedia.org/wiki/State_machine_replication). - -A state machine is a computer science concept whereby a machine can have multiple states, but only one at any given time. There is a `state`, which describes the current state of the system, and `transactions`, that trigger state transitions. - -Given a state S and a transaction T, the state machine will return a new state S'. - -```text -+--------+ +--------+ -| | | | -| S +---------------->+ S' | -| | apply(T) | | -+--------+ +--------+ -``` - -In practice, the transactions are bundled in blocks to make the process more efficient. Given a state S and a block of transactions B, the state machine will return a new state S'. - -```text -+--------+ +--------+ -| | | | -| S +----------------------------> | S' | -| | For each T in B: apply(T) | | -+--------+ +--------+ -``` - -In a blockchain context, the state machine is deterministic. This means that if a node is started at a given state and replays the same sequence of transactions, it will always end up with the same final state. - -The Cosmos SDK gives developers maximum flexibility to define the state of their application, transaction types and state transition functions. The process of building state-machines with the Cosmos SDK will be described more in depth in the following sections. But first, let us see how the state-machine is replicated using **CometBFT**. - -## CometBFT - -Thanks to the Cosmos SDK, developers just have to define the state machine, and [*CometBFT*](https://docs.cometbft.com/v0.37/introduction/what-is-cometbft) will handle replication over the network for them. - -```text - ^ +-------------------------------+ ^ - | | | | Built with Cosmos SDK - | | State-machine = Application | | - | | | v - | +-------------------------------+ - | | | ^ -Blockchain node | | Consensus | | - | | | | - | +-------------------------------+ | CometBFT - | | | | - | | Networking | | - | | | | - v +-------------------------------+ v -``` - -[CometBFT](https://docs.cometbft.com/v0.37/introduction/what-is-cometbft) is an application-agnostic engine that is responsible for handling the *networking* and *consensus* layers of a blockchain. In practice, this means that CometBFT is responsible for propagating and ordering transaction bytes. CometBFT relies on an eponymous Byzantine-Fault-Tolerant (BFT) algorithm to reach consensus on the order of transactions. - -The CometBFT [consensus algorithm](https://docs.cometbft.com/v0.37/introduction/what-is-cometbft#consensus-overview) works with a set of special nodes called *Validators*. Validators are responsible for adding blocks of transactions to the blockchain. At any given block, there is a validator set V. A validator in V is chosen by the algorithm to be the proposer of the next block. This block is considered valid if more than two thirds of V signed a `prevote` and a `precommit` on it, and if all the transactions that it contains are valid. The validator set can be changed by rules written in the state-machine. - -## ABCI - -CometBFT passes transactions to the application through an interface called the [ABCI](https://docs.cometbft.com/v0.37/spec/abci/), which the application must implement. - -```text - +---------------------+ - | | - | Application | - | | - +--------+---+--------+ - ^ | - | | ABCI - | v - +--------+---+--------+ - | | - | | - | CometBFT | - | | - | | - +---------------------+ -``` - -Note that **CometBFT only handles transaction bytes**. It has no knowledge of what these bytes mean. All CometBFT does is order these transaction bytes deterministically. CometBFT passes the bytes to the application via the ABCI, and expects a return code to inform it if the messages contained in the transactions were successfully processed or not. - -Here are the most important messages of the ABCI: - -* `CheckTx`: When a transaction is received by CometBFT, it is passed to the application to check if a few basic requirements are met. `CheckTx` is used to protect the mempool of full-nodes against spam transactions. . A special handler called the [`AnteHandler`]../beginner/04-gas-fees.md#antehandler) is used to execute a series of validation steps such as checking for sufficient fees and validating the signatures. If the checks are valid, the transaction is added to the [mempool](https://docs.cometbft.com/v0.37/spec/p2p/messages/mempool) and relayed to peer nodes. Note that transactions are not processed (i.e. no modification of the state occurs) with `CheckTx` since they have not been included in a block yet. -* `DeliverTx`: When a [valid block](https://docs.cometbft.com/v0.37/spec/core/data_structures#block) is received by CometBFT, each transaction in the block is passed to the application via `DeliverTx` in order to be processed. It is during this stage that the state transitions occur. The `AnteHandler` executes again, along with the actual [`Msg` service](../../build/building-modules/03-msg-services.md) RPC for each message in the transaction. -* `BeginBlock`/`EndBlock`: These messages are executed at the beginning and the end of each block, whether the block contains transactions or not. It is useful to trigger automatic execution of logic. Proceed with caution though, as computationally expensive loops could slow down your blockchain, or even freeze it if the loop is infinite. - -Find a more detailed view of the ABCI methods from the [CometBFT docs](https://docs.cometbft.com/v0.37/spec/abci/). - -Any application built on CometBFT needs to implement the ABCI interface in order to communicate with the underlying local CometBFT engine. Fortunately, you do not have to implement the ABCI interface. The Cosmos SDK provides a boilerplate implementation of it in the form of [baseapp](./03-sdk-design.md#baseapp). diff --git a/docs/learn/intro/03-sdk-design.md b/docs/learn/intro/03-sdk-design.md deleted file mode 100644 index 78afb12989cc..000000000000 --- a/docs/learn/intro/03-sdk-design.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Main Components of the Cosmos SDK - -The Cosmos SDK is a framework that facilitates the development of secure state-machines on top of CometBFT. At its core, the Cosmos SDK is a boilerplate implementation of the [ABCI](./02-sdk-app-architecture.md#abci) in Golang. It comes with a [`multistore`](../advanced/04-store.md#multistore) to persist data and a [`router`](../advanced/00-baseapp.md#routing) to handle transactions. - -Here is a simplified view of how transactions are handled by an application built on top of the Cosmos SDK when transferred from CometBFT via `DeliverTx`: - -1. Decode `transactions` received from the CometBFT consensus engine (remember that CometBFT only deals with `[]bytes`). -2. Extract `messages` from `transactions` and do basic sanity checks. -3. Route each message to the appropriate module so that it can be processed. -4. Commit state changes. - -## `baseapp` - -`baseapp` is the boilerplate implementation of a Cosmos SDK application. It comes with an implementation of the ABCI to handle the connection with the underlying consensus engine. Typically, a Cosmos SDK application extends `baseapp` by embedding it in [`app.go`]../beginner/00-app-anatomy.md#core-application-file). - -Here is an example of this from `simapp`, the Cosmos SDK demonstration app: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L170-L212 -``` - -The goal of `baseapp` is to provide a secure interface between the store and the extensible state machine while defining as little about the state machine as possible (staying true to the ABCI). - -For more on `baseapp`, please click [here](../advanced/00-baseapp.md). - -## Multistore - -The Cosmos SDK provides a [`multistore`](../advanced/04-store.md#multistore) for persisting state. The multistore allows developers to declare any number of [`KVStores`](../advanced/04-store.md#base-layer-kvstores). These `KVStores` only accept the `[]byte` type as value and therefore any custom structure needs to be marshalled using [a codec](../advanced/05-encoding.md) before being stored. - -The multistore abstraction is used to divide the state in distinct compartments, each managed by its own module. For more on the multistore, click [here](../advanced/04-store.md#multistore) - -## Modules - -The power of the Cosmos SDK lies in its modularity. Cosmos SDK applications are built by aggregating a collection of interoperable modules. Each module defines a subset of the state and contains its own message/transaction processor, while the Cosmos SDK is responsible for routing each message to its respective module. - -Here is a simplified view of how a transaction is processed by the application of each full-node when it is received in a valid block: - -```text - + - | - | Transaction relayed from the full-node's - | CometBFT engine to the node's application - | via DeliverTx - | - | - +---------------------v--------------------------+ - | APPLICATION | - | | - | Using baseapp's methods: Decode the Tx, | - | extract and route the message(s) | - | | - +---------------------+--------------------------+ - | - | - | - +---------------------------+ - | - | - | Message routed to - | the correct module - | to be processed - | - | -+----------------+ +---------------+ +----------------+ +------v----------+ -| | | | | | | | -| AUTH MODULE | | BANK MODULE | | STAKING MODULE | | GOV MODULE | -| | | | | | | | -| | | | | | | Handles message,| -| | | | | | | Updates state | -| | | | | | | | -+----------------+ +---------------+ +----------------+ +------+----------+ - | - | - | - | - +--------------------------+ - | - | Return result to CometBFT - | (0=Ok, 1=Err) - v -``` - -Each module can be seen as a little state-machine. Developers need to define the subset of the state handled by the module, as well as custom message types that modify the state (*Note:* `messages` are extracted from `transactions` by `baseapp`). In general, each module declares its own `KVStore` in the `multistore` to persist the subset of the state it defines. Most developers will need to access other 3rd party modules when building their own modules. Given that the Cosmos SDK is an open framework, some of the modules may be malicious, which means there is a need for security principles to reason about inter-module interactions. These principles are based on [object-capabilities](../advanced/10-ocap.md). In practice, this means that instead of having each module keep an access control list for other modules, each module implements special objects called `keepers` that can be passed to other modules to grant a pre-defined set of capabilities. - -Cosmos SDK modules are defined in the `x/` folder of the Cosmos SDK. Some core modules include: - -* `x/auth`: Used to manage accounts and signatures. -* `x/bank`: Used to enable tokens and token transfers. -* `x/staking` + `x/slashing`: Used to build Proof-Of-Stake blockchains. - -In addition to the already existing modules in `x/`, that anyone can use in their app, the Cosmos SDK lets you build your own custom modules. You can check an [example of that in the tutorial](https://tutorials.cosmos.network/). diff --git a/docs/learn/intro/_category_.json b/docs/learn/intro/_category_.json deleted file mode 100644 index b218fe9be4aa..000000000000 --- a/docs/learn/intro/_category_.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "label": "Introduction", - "position": 0, - "link": null -} \ No newline at end of file diff --git a/docs/post.sh b/docs/post.sh index 2f786302fc10..20dd8c914ce9 100755 --- a/docs/post.sh +++ b/docs/post.sh @@ -1,6 +1,5 @@ #!/usr/bin/env bash -<<<<<<< HEAD find docs/build/modules ! -name '_category_.json' -type f -exec rm -rf {} + rm -rf docs/build/tooling/01-cosmovisor.md rm -rf docs/build/tooling/02-confix.md @@ -8,27 +7,11 @@ rm -rf docs/build/tooling/03-hubl.md rm -rf docs/build/packages/01-depinject.md rm -rf docs/build/packages/02-collections.md rm -rf docs/build/packages/03-orm.md -rm -rf docs/develop/advaced-concepts/17-autocli.md +rm -rf docs/learn/advaced-concepts/17-autocli.md rm -rf docs/user/run-node/04-rosetta.md rm -rf docs/build/architecture rm -rf docs/build/spec rm -rf docs/build/rfc -rm -rf docs/develop/advanced/17-autocli.md +rm -rf docs/learn/advanced/17-autocli.md rm -rf docs/build/migrations/02-upgrading.md rm -rf versioned_docs versioned_sidebars versions.json -======= -find build/modules ! -name '_category_.json' -type f -exec rm -rf {} + -rm -rf build/tooling/01-cosmovisor.md -rm -rf build/tooling/02-confix.md -rm -rf build/tooling/03-hubl.md -rm -rf build/packages/01-depinject.md -rm -rf build/packages/02-collections.md -rm -rf build/packages/03-orm.md -rm -rf learn/advaced-concepts/17-autocli.md -rm -rf user/run-node/04-rosetta.md -rm -rf build/architecture -rm -rf build/spec -rm -rf build/rfc -rm -rf learn/advanced/17-autocli.md -rm -rf build/migrations/02-upgrading.md ->>>>>>> 2efafee65 (chore: rename develop to learn (#17821)) diff --git a/docs/pre.sh b/docs/pre.sh index 906b70fb116e..ed9dbb7b78aa 100755 --- a/docs/pre.sh +++ b/docs/pre.sh @@ -3,7 +3,7 @@ ## Create modules pages for D in ../x/*; do if [ -d "${D}" ]; then - MODDOC=build/modules/$(echo $D | awk -F/ '{print $NF}') + MODDOC=docs/build/modules/$(echo $D | awk -F/ '{print $NF}') rm -rf $MODDOC mkdir -p $MODDOC && cp -r $D/README.md "$_" fi @@ -11,7 +11,6 @@ done ## Vesting is a submodule of auth, but we still want to display it in docs ## TODO to be removed in https://github.com/cosmos/cosmos-sdk/issues/9958 -<<<<<<< HEAD cp ../x/auth/vesting/README.md ./docs/build/modules/auth/1-vesting.md cp ../x/auth/tx/README.md ./docs/build/modules/auth/2-tx.md @@ -29,26 +28,6 @@ cp ../client/v2/README.md ./docs/develop/advanced/17-autocli.md cp ../depinject/README.md ./docs/build/packages/01-depinject.md cp ../collections/README.md ./docs/build/packages/02-collections.md cp ../orm/README.md ./docs/build/packages/03-orm.md -======= -cp -r ../x/auth/vesting/README.md ./build/modules/auth/1-vesting.md -cp -r ../x/auth/tx/README.md ./build/modules/auth/2-tx.md - -## Add modules page list -cat ../x/README.md | sed 's/\.\.\/\/build\/building-modules\/README\.md/\/building-modules\/intro\.html/g' > ./build/modules/README.md - -## Add tooling documentation -cp ../tools/cosmovisor/README.md ./build/tooling/01-cosmovisor.md -cp ../tools/confix/README.md ./build/tooling/02-confix.md -cp ../tools/hubl/README.md ./build/tooling/03-hubl.md - -wget -x -O ./user/run-node/04-rosetta.md https://raw.githubusercontent.com/cosmos/rosetta/main/README.md - -## Add package documentation -cp ../client/v2/README.md ./learn/advanced/17-autocli.md -cp ../depinject/README.md ./build/packages/01-depinject.md -cp ../collections/README.md ./build/packages/02-collections.md -cp ../orm/README.md ./build/packages/03-orm.md ->>>>>>> 2efafee65 (chore: rename develop to learn (#17821)) ## Add architecture documentation cp -r ./architecture ./docs/build @@ -60,4 +39,4 @@ cp -r ./spec ./docs/build cp -r ./rfc ./docs/build/rfc ## Add SDK migration documentation -cp -r ../UPGRADING.md ./docs/build/migrations/02-upgrading.md \ No newline at end of file +cp -r ../UPGRADING.md ./docs/build/migrations/02-upgrading.md