Skip to content

Commit

Permalink
Updating the node operator guide (#231)
Browse files Browse the repository at this point in the history
* adding full node to the node operator guide

* replacing observer node with light client

* adding description for full node; adjusting format

* copying over files to version-stable to show up on vercel

* removing the ethereum vs flow doc for now

* adding full node to the node operator guide

* replacing observer node with light client

* adding description for full node; adjusting format

* copying over files to version-stable to show up on vercel

* removing the ethereum vs flow doc for now

* reverting changes to versioned_docs

* updating the readme directly under nodes

* fixing paths

* reverting a change to nodes/index.md

* reverting changes to index.md under nodes

* Update docs/building-on-flow/nodes/node-operation/index.md

Co-authored-by: Alexander Hentschel <[email protected]>

* Update docs/building-on-flow/nodes/node-operation/index.md

Co-authored-by: Peter Argue <[email protected]>

* Update docs/building-on-flow/nodes/node-operation/index.md

Co-authored-by: Peter Argue <[email protected]>

* Update docs/building-on-flow/nodes/node-operation/index.md

Co-authored-by: Alexander Hentschel <[email protected]>

* Update docs/building-on-flow/nodes/node-operation/index.md

Co-authored-by: Peter Argue <[email protected]>

* Update docs/building-on-flow/nodes/node-operation/observer-node.mdx

Co-authored-by: Alexander Hentschel <[email protected]>

* Update docs/building-on-flow/nodes/node-operation/observer-node.mdx

Co-authored-by: Alexander Hentschel <[email protected]>

* incorporating review comments and renaming light client to light node

* misssed some rename of the observer to light node

* adding diagram for full nodes

* updating the flow nodes diagram

---------

Co-authored-by: Tom Haile <[email protected]>
Co-authored-by: Alexander Hentschel <[email protected]>
Co-authored-by: Peter Argue <[email protected]>
  • Loading branch information
4 people authored Sep 4, 2023
1 parent 4dac0e0 commit 6a00297
Show file tree
Hide file tree
Showing 3 changed files with 67 additions and 38 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
43 changes: 36 additions & 7 deletions docs/building-on-flow/nodes/node-operation/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,54 @@ Flow multirole architecture makes it more scalable and provides several node typ

The different types of nodes are described [here](./node-roles.mdx). As node operator, you can choose to run any of the different types of node that best fits your needs.

### Observer node
The observer node is one of the easiest node to spin up and can be run by Dapp developers who need the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height.
The nodes are classified as follows

![Flownodesdiagram.jpg](Flownodesdiagram.jpg)

## Light node a.k.a. Observer node

The light node is one of the easiest nodes to spin up and can be run by Dapp developers who need the latest block data available locally, e.g. a wallet application that needs to track the latest block ID and height.
In addition to supporting dapps, an observer node can also be run by access node operators who want to scale their access nodes' endpoints. Access node operators can spin up geographically dispersed observer nodes which can talk to their staked access nodes and to each other.

The observer node is not staked but still provides the same API as the access node.

To run an observer node, follow this [guide](./observer-node.mdx).
To run a light node, follow this [guide](./observer-node.mdx).

## Archive node

The Archive node provides a scalable and efficient way to access the history of Flow protocol and the execution state for the current spork. Like the observer node, it too can be run by anyone without being staked or added to the approved list of nodes.
The Archive node follows the chain, stores and indexes both protocol and execution state, and allows retrieval of blocks, collections, transactions and events from the genesis of the current spork.

It also allows script execution and other read-only queries that require the execution state to be read. It can answer any queries from past data e.g. “what was the Flow account balance at height X?”, where X is several thousand blocks in the past.

### Access node
The archive node is currently in beta and will be available as a GA release in H2 2023 (see [here](https://flow.com/post/flow-blockchain-node-operation-archive-node) for more)

## Full node

In a nutshell, Full Nodes are staked network participants that drive network progress, e.g. by creating and executing new blocks. They are the primary contributors to network safety (all of them validate the correctness of the consensus process and secure the network additionally through their role-specific tasks). In comparison, Light Nodes don't contribute to the networks progress. Though, they help to secure the network by also validating the integrity of the consensus process.
- The Access node is a full node that serves as an RPC node and acts as a gateway node for the network.
- The Validator node (Collection, Consensus, Verification and Execution) is a full node that plays a role in block generation.


## Access node
If you want local access to the protocol state data (blocks, collections, transactions) and do not want to use one of the community access nodes you can run an access node.
Dapp developers, chain explorers, chain analytics etc. who want exclusive access to chain data and not be subject to the rate-limits on the community access node can choose to run an access node.

An access node is staked but since it does not participate in the core Flow protocol, it does not receive any staking rewards.
An access node is minimally staked for network security.
The central goal for Access Nodes is to provide RPC functionality to its node operator.
In comparison, contributing to protocol progress (e.g. routing transactions to collector clusters, relaying blocks to the unstaked peer-to-peer network, etc.) should only take up a marginal fraction an Access Node's computational resources.
Furthermore, Access Node operators can freely rate-limit the amount of resources their Access Node dedicates to supporting the broader ecosystem. Therefore, Access Nodes do not receive staking rewards.
To run an access node, see the [Running a staked node](#running-a-staked-node) section below.

Alternately, instead of running an access node, you can use the [Flow community](../access-api.mdx) access nodes or the ones run by any of the other node operators.

### Collection, Consensus, Verification and Execution node
If you want your node to participate in the nitty-gritty of Flow protocol and help in block or collection creation, transaction execution, result verification or block verification then you should run one of these four node roles.
## Validator node

You can also be a core participant in running the Flow network and contribute to securing it. Depending on your preference, you could run one or any combination of the following node roles:
- Collection Nodes collaboratively create batches of transactions (in Flow terminology collections).
- Consensus Nodes create blocks, schedule them for asynchronous execution, and commit execution results once they are verified (so called sealing). In addition, they orchestrate the Flow protocol and enforce protocol compliance.
- Execution Nodes asynchronously execute blocks. They are the power-houses in the protocol, providing the vast computational resources available to Flow transactions.
- Verification Nodes check the execution results in a distributed manner.

Nodes with these roles are staked and also receive staking rewards.

Expand Down
62 changes: 31 additions & 31 deletions docs/building-on-flow/nodes/node-operation/observer-node.mdx
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
---
title: Observer Node
sidebar_label: Observer Node
title: Light Node a.k.a Observer Node
sidebar_label: Light Node
---

An observer node is similar to an access node and provides a locally accessible, continuously updated, verified copy of the block data. It serves the [gRPC Access API](../access-api.mdx) but unlike an access node, an observer node does not need to be staked, and **anyone** can run it without being added to the approved list of nodes.
A light node also known as the observer node is similar to an access node and provides a locally accessible, continuously updated, verified copy of the block data. It serves the [gRPC Access API](../access-api.mdx) but unlike an access node, an light node does not need to be staked, and **anyone** can run it without being added to the approved list of nodes.

The observer node bootstraps by connecting to an access node and becoming part of the public network comprised of access nodes and other observer nodes. It then continuously receives blocks that are being added to the chain either directly from the access node or from other observer nodes that are part of the public network. However, it makes no trust assumption of the upstream access node or the observer node which is providing the block and locally verifies that the blocks that are received are the correct extension of the chain e.g. after receiving valid blocks A, B and C when it receives block D, it verifies that block D is indeed signed by the consensus nodes and is a valid next block. The received block data is indexed and made available via the Access API. For Collection, Transactions and Account queries, it delegates those requests to the upstream access node. Similarly, transactions and scripts sent to an observer node are also forwarded to the upstream access node. Future versions of the observer node will be able to serve this data locally as well.
The light node bootstraps by connecting to an access node and becoming part of the public network comprised of access nodes and other light nodes. It then continuously receives blocks, which the consensus nodes are adding to the chain, either directly from access nodes or from other light nodes that are part of the public network. However, it makes no trust assumption of the upstream access node or the light node which is providing the block and locally verifies that the blocks that are received are the correct extension of the chain e.g. after receiving valid blocks A, B and C when it receives block D, it verifies that block D is indeed signed by the consensus nodes and is a valid next block. The received block data is indexed and made available via the Access API. For Collection, Transactions and Account queries, it delegates those requests to the upstream access node. Similarly, transactions and scripts sent to a light node are also forwarded to the upstream access node. Future versions of the light node will be able to serve this data locally as well.

Since the observer node is not staked, it does not produce or execute blocks but instead serves as an _unstaked access node_ that can be easily run on any consumer-grade computer which has enough disk space.
Since the light node is not staked, it does not produce or execute blocks but instead serves as an _unstaked access node_ that can be easily run on any consumer-grade computer which has enough disk space.

![Observer nodes](observer.png)

## Who should run an observer node?
## Who should run a light node?

The observer node provides an alternative to running an access node. Hence, it is ideal for Dapps that need access to the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height. In addition to that, access node operators who want to scale their access node endpoints geographically can spin up geographically dispersed observer nodes which can talk to their staked access node and to each other.
The light node provides an alternative to running an access node. Hence, it is ideal for Dapps that need access to the latest block data locally on a machine they control. Examples include a wallet application that needs to track the latest block ID and height. Alternatively, access node operators that want to scale their access node endpoints geographically can spin up light nodes in different regions, which can talk to their staked access node and to each other.

## Running an observer node
## Running an light node

### Hardware

In general, any consumer-grade computer with a decent network connection and sufficient disk space should be able to run an observer node.
In general, any consumer-grade computer with a decent network connection and sufficient disk space should be able to run a light node.

Minimum requirements

Expand All @@ -28,20 +28,20 @@ Minimum requirements
- 300 GB SSD disk
- 10Mbps network connection

### Steps to run an observer node
### Steps to run a light node

> [Here](https://www.loom.com/share/990a725531754106b91d8ccec6244219) is video walk-though of these 👇 steps.
#### Step 1 - Generate the node directory structure
The observer node requires the following directory structure,
The light node requires the following directory structure,
```shell
$ tree flow_observer
flow_observer/
├── bootstrap
│ ├── network.key (file containing the node private network key)
│ └── public-root-information
│ └── root-protocol-state-snapshot.json (the genesis data of the current spork)
└── data (directory used by the observer node to store block data)
└── data (directory used by the light node to store block data)
```

Create the parent and the sub-directories
Expand All @@ -53,8 +53,8 @@ mkdir flow_observer/data

#### Step 2 - Generate the network key

Like any other Flow node, the observer also needs a networking [ECDSA key](https://github.com/onflow/flow-go/blob/master/cmd/bootstrap/utils/key_generation.go#L52-L54) to talk to the network.
Download the Bootstrapping kit, and generate the observer networking key.
Like any other Flow node, the light node also needs a networking [ECDSA key](https://github.com/onflow/flow-go/blob/master/cmd/bootstrap/utils/key_generation.go#L52-L54) to talk to the network.
Download the Bootstrapping kit, and generate the networking key.

```shell
curl -sL -O storage.googleapis.com/flow-genesis-bootstrap/boot-tools.tar
Expand Down Expand Up @@ -128,7 +128,7 @@ flow snapshot save ./flow_observer/bootstrap/public-root-information/root-protoc

#### Step 4 - Start the node

The observer node can be run as a docker container
The light node can be run as a docker container

##### Observer for Flow Mainnet

Expand Down Expand Up @@ -178,15 +178,15 @@ docker run --rm \
--observer-networking-key-path=/bootstrap/network.key
```

The observer node acts as a DHT client and bootstraps from upstream access nodes which run the DHT server.
The light node acts as a DHT client and bootstraps from upstream access nodes which run the DHT server.
The upstream bootstrap server is specified using the `bootstrap-node-addresses` which is the comma-separated list of hostnames of the access nodes.
The `bootstrap-node-public-keys` is the list of the corresponding networking public key of those nodes.

The observer node delegates many of the API calls to the upstream access nodes.
The `upstream-node-addresses` is the list of access node hostnames to which this observer node can delegate to. The list can be different from the bootstrap node list.
The light node delegates many of the API calls to the upstream access nodes.
The `upstream-node-addresses` is the list of access node hostnames to which this light node can delegate to. The list can be different from the bootstrap node list.
The `bootstrap-node-public-key` is the list of the corresponding networking public key of those nodes.

> In the above docker commands, the Flow community access nodes are being used as the upstream access nodes. However, any other Flow access node that supports an observer node can be used
> In the above docker commands, the Flow community access nodes are being used as the upstream access nodes. However, any other Flow access node that supports a light node can be used
_All parameters and their explanation can be found [here](https://github.com/onflow/flow-go/blob/82da35141ff095fbf75ce2c950efec240ad38565/cmd/access/node_builder/access_node_builder.go#L523-L558)_

Expand All @@ -201,37 +201,37 @@ flow blocks get latest --host localhost:9000

> REST API is currently not support (see [issue](https://github.com/onflow/flow-go/issues/3138))
The observer node, like the other type of Flow nodes, also produces Prometheus metrics that can be used to monitor node health. More on that [here](./node-setup.mdx#monitoring-and-metrics)
The light node, like the other type of Flow nodes, also produces Prometheus metrics that can be used to monitor node health. More on that [here](./node-setup.mdx#monitoring-and-metrics)


## FAQs

### Does the observer node need to be staked?
### Does the light node need to be staked?

No, the observer node is not a staked node.
No, the light node is not a staked node.

### Can any access be used to bootstrap an observer node?
### Can any access node be used to bootstrap a light node?

No, only Access nodes which have explicitly turned ON support for observer nodes can be used to bootstrap an observer node.
No, only Access nodes which have explicitly turned ON support for light nodes can be used to bootstrap a light node.

The public access nodes that support observer nodes are listed below. Apart from these, other public access nodes run by node operators other than DapperLabs may choose to support observer nodes.
The public access nodes that support light nodes are listed below. Apart from these, other public access nodes run by node operators other than the Flow foundation team may choose to support light nodes.

### How can an access node turn ON support for observer node?
### How can an access node turn ON support for light node?

An access node can support an observer node by passing in the following two parameters when starting the access node
An access node can support a light node by passing in the following two parameters when starting the access node
```shell
--supports-observer=true --public-network-address=0.0.0.0:3570
```
`public-network-address` is the address the observer nodes will connect to.
`public-network-address` is the address the light nodes will connect to.

### Are observer nodes subject to rate limits?
### Are light nodes subject to rate limits?

The observer serves all the [Block related queries](https://github.com/onflow/flow/blob/master/protobuf/flow/access/access.proto#L24-L42) from is local database. These are not subjected to any rate limits.
The light node serves all the [Block related queries](https://github.com/onflow/flow/blob/master/protobuf/flow/access/access.proto#L24-L42) from is local database. These are not subjected to any rate limits.

However, it proxies all the other requests to the access node and those will be rate limited as per the rate limits defined on that access node.


### Flow community access nodes that support connections from observer nodes
### Flow community access nodes that support connections from light nodes

#### For mainnet

Expand Down

1 comment on commit 6a00297

@vercel
Copy link

@vercel vercel bot commented on 6a00297 Sep 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.