Skip to content

Commit

Permalink
[ENG-370] Rename relayer to sentinel (#95)
Browse files Browse the repository at this point in the history
* Consolidate mev metrics

* Fix linter

* Mev metric names

* relayer to sentinel for logs

* relayer to sentinel for code

* Rename relayer to sentinel

* Add warnings

* print warning only if relayer config is set and sentinel config is not set

Co-authored-by: Jacob Gadikian <[email protected]>
Co-authored-by: bpiv400 <[email protected]>
  • Loading branch information
3 people committed Dec 31, 2022
1 parent e7bfc7a commit 851797b
Show file tree
Hide file tree
Showing 11 changed files with 195 additions and 146 deletions.
32 changes: 16 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,29 +50,29 @@ mev-tendermint introduces a new section of config in `config.toml` called `[side

# **EXAMPLE** below (please use the correct values)
[sidecar]
relayer_peer_string = "d1463b730c6e0dcea59db726836aeaff13a8119f@uni-5-sentinel.skip.money:26656"
relayer_rpc_string = "uni-5-gateway.skip.money"
sentinel_peer_string = "d1463b730c6e0dcea59db726836aeaff13a8119f@uni-5-sentinel.skip.money:26656"
sentinel_rpc_string = "uni-5-gateway.skip.money"
api_key = "2314ajinashg2389jfjap"
personal_peer_ids = "557611c7a7307ce023a7d13486b570282521296d,5740acbf39a9ae59953801fe4997421b6736e091"
```

Here’s an explanation of what these are:

### `relayer_peer_string`
### `sentinel_peer_string`

- The `p2p@ip:port` for the Skip Sentinel that is used to establish a secret, authenticated handshake between your node and the Skip sentinel
- For nodes that should not communicate with the sentinel directly (e.g. validator nodes that have sentries), this does not need to be set.
- **⭐  Find the `relayer_peer_string` here:** [⚙️ Skip Configurations By Chain](https://www.notion.so/Skip-Configurations-By-Chain-a6076cfa743f4ab38194096403e62f3c)
- **⭐  Find the `sentinel_peer_string` here:** [⚙️ Skip Configurations By Chain](https://www.notion.so/Skip-Configurations-By-Chain-a6076cfa743f4ab38194096403e62f3c)

### `relayer_rpc_string`
### `sentinel_rpc_string`

- The `api` for the Skip Sentinel that is used to register your node
- For nodes that should not communicate with the sentinel directly (e.g. validator nodes that have sentries), this does not need to be set.
- **⭐  Find the `relayer_rpc_string` here:** [⚙️ Skip Configurations By Chain](https://www.notion.so/Skip-Configurations-By-Chain-a6076cfa743f4ab38194096403e62f3c)
- **⭐  Find the `sentinel_rpc_string` here:** [⚙️ Skip Configurations By Chain](https://www.notion.so/Skip-Configurations-By-Chain-a6076cfa743f4ab38194096403e62f3c)

### `api_key`

- This is the unique string key Skip uses to ensure a node establishing a connection with our relay actually belongs to your validator.
- This is the unique string key Skip uses to ensure a node establishing a connection with our Sentinel actually belongs to your validator.
- If you don't have one, please request one from the Skip team on our **[discord](https://discord.gg/amAgf9Z39w)**

### `personal_peer_ids`
Expand Down Expand Up @@ -102,7 +102,7 @@ After you have completed the steps above, you can check you connectivity either
- Check if you are peered with the sentinel by calling `curl http://localhost:26657/status`

```jsx
is_peered_with_relayer: true
is_peered_with_sentinel: true
```

- Check if you are running `mev-tendermint` by running either:
Expand All @@ -115,7 +115,7 @@ After you have completed the steps above, you can check you connectivity either
junod version --long | grep mev
```

- Via the new prometheus metrics exposed on mev-tendermint, in particular `sidecar_relay_connected`
- Via the new prometheus metrics exposed on mev-tendermint, in particular `mev_sentinel_connected`

---

Expand Down Expand Up @@ -221,10 +221,10 @@ The design goals of MEV-Tendermint is to allow & preserve:

| Name | Type | Description |
| --- | --- | --- |
| sidecar_size_bytes | Histogram | Histogram of sidecar mev transaction sizes, in bytes |
| sidecar_size | Gauge | Size of the sidecar |
| sidecar_num_bundles_total | Counter | Number of MEV bundles received by the sidecar in total |
| sidecar_num_bundles_last_block | Gauge | Number of mev bundles received during the last block |
| sidecar_num_mev_txs_total | Counter | Number of mev transactions added in total |
| sidecar_num_mev_txs_last_block | Gauge | Number of mev transactions received by sidecar in the last block |
| p2p_relay_connected | Gauge | Whether or not a node is connected to the relay, 1 if connected, 0 if not. |
| mev_tx_size_bytes | Histogram | Histogram of mev transaction sizes, in bytes |
| mev_bundle_mempool_size | Gauge | Size of the MEV bundle mempool |
| mev_num_bundles_total | Counter | Number of MEV bundles received in total |
| mev_num_bundles_last_block | Gauge | Number of mev bundles received during the last block |
| mev_num_txs_total | Counter | Number of mev transactions added in total |
| mev_num_txs_last_block | Gauge | Number of mev transactions received in the last block |
| mev_sentinel_connected | Gauge | Whether or not a node is connected to the sentinel, 1 if connected, 0 if not. |
21 changes: 14 additions & 7 deletions cmd/tendermint/commands/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,19 +53,26 @@ func ParseConfig(cmd *cobra.Command) (*cfg.Config, error) {
if err := conf.ValidateBasic(); err != nil {
return nil, fmt.Errorf("error in config file: %v", err)
}
// Add auction relayer to config if not present and set
if len(conf.Sidecar.RelayerPeerString) > 0 {
relayerID := strings.Split(conf.Sidecar.RelayerPeerString, "@")[0]
if !strings.Contains(conf.P2P.PrivatePeerIDs, relayerID) {
// Add auction sentinel to config if not present and set
// Temporarily support both SentinelPeerString and RelayerPeerString
sentinelPeerString := conf.Sidecar.SentinelPeerString
if len(sentinelPeerString) == 0 {
logger.Info(`[mev-tendermint]: WARNING: sentinel_peer_string not found in config.toml.
relayer_peer_string is being deprecated for sentinel_peer_string`)
sentinelPeerString = conf.Sidecar.RelayerPeerString
}
if len(sentinelPeerString) > 0 {
sentinelID := strings.Split(sentinelPeerString, "@")[0]
if !strings.Contains(conf.P2P.PrivatePeerIDs, sentinelID) {
// safety check to not blow away existing private peers if any
if len(conf.P2P.PrivatePeerIDs) > 0 {
if conf.P2P.PrivatePeerIDs[len(conf.P2P.PrivatePeerIDs)-1:] == "," {
conf.P2P.PrivatePeerIDs = conf.P2P.PrivatePeerIDs + relayerID
conf.P2P.PrivatePeerIDs = conf.P2P.PrivatePeerIDs + sentinelID
} else {
conf.P2P.PrivatePeerIDs = conf.P2P.PrivatePeerIDs + "," + relayerID
conf.P2P.PrivatePeerIDs = conf.P2P.PrivatePeerIDs + "," + sentinelID
}
} else {
conf.P2P.PrivatePeerIDs = relayerID
conf.P2P.PrivatePeerIDs = sentinelID
}
}
}
Expand Down
14 changes: 9 additions & 5 deletions config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -1084,9 +1084,13 @@ func (cfg *ConsensusConfig) ValidateBasic() error {
// SidecarConfig

// Sidecar defines configuration for gossiping the private sidecar
// mempool among the relayer and the nodes that belong to a particular proposer
// mempool among the sentinel and the nodes that belong to a particular proposer
type SidecarConfig struct {
RootDir string `mapstructure:"home"`
RootDir string `mapstructure:"home"`
// Temporarily support both Sentinel and Relayer names
SentinelRPCString string `mapstructure:"sentinel_rpc_string"`
SentinelPeerString string `mapstructure:"sentinel_peer_string"`

RelayerRPCString string `mapstructure:"relayer_rpc_string"`
RelayerPeerString string `mapstructure:"relayer_peer_string"`

Expand All @@ -1100,9 +1104,9 @@ func DefaultSidecarConfig() *SidecarConfig {

func TestSidecarConfig() *SidecarConfig {
return &SidecarConfig{
RelayerRPCString: "test-api.skip.money",
RelayerPeerString: "[email protected]:26656",
APIKey: "api-key",
SentinelRPCString: "test-api.skip.money",
SentinelPeerString: "[email protected]:26656",
APIKey: "api-key",
}
}

Expand Down
4 changes: 2 additions & 2 deletions config/toml.go
Original file line number Diff line number Diff line change
Expand Up @@ -543,8 +543,8 @@ namespace = "{{ .Instrumentation.Namespace }}"
# nodes in your network receive auction-winning
# txs when when your validator is the proposer)
personal_peer_ids = "{{ .Sidecar.PersonalPeerIDs }}"
relayer_peer_string = "{{ .Sidecar.RelayerPeerString }}"
relayer_rpc_string = "{{ .Sidecar.RelayerRPCString }}"
sentinel_peer_string = "{{ .Sidecar.SentinelPeerString }}"
sentinel_rpc_string = "{{ .Sidecar.SentinelRPCString }}"
api_key = "{{ .Sidecar.APIKey }}"
`

Expand Down
4 changes: 2 additions & 2 deletions config/toml_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ func checkConfig(configFile string) bool {
"max",
"genesis",
"skip",
"relayer_rpc_string",
"relayer_peer_string",
"sentinel_rpc_string",
"sentinel_peer_string",
"personal_peer_ids",
}
for _, e := range elems {
Expand Down
12 changes: 6 additions & 6 deletions mev/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ const (

// Metrics contains metrics exposed by this package.
type Metrics struct {
// Whether or not a node is connected to the relay.
RelayConnected metrics.Gauge
// Whether or not a node is connected to the sentinel.
SentinelConnected metrics.Gauge

// SIDECAR METRICS
// Size of the sidecar.
Expand Down Expand Up @@ -44,11 +44,11 @@ func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
RelayConnected: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
SentinelConnected: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "relay_connected",
Help: "Whether or not a node is connected to the mev relay / sentinel. 1 if yes, 0 if no.",
Name: "sentinel_connected",
Help: "Whether or not a node is connected to the mev sentinel. 1 if yes, 0 if no.",
}, labels).With(labelsAndValues...),
// SIDECAR METRICS
MevBundleMempoolSize: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Expand Down Expand Up @@ -93,7 +93,7 @@ func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
// NopMetrics returns no-op Metrics.
func NopMetrics() *Metrics {
return &Metrics{
RelayConnected: discard.NewGauge(),
SentinelConnected: discard.NewGauge(),
// SIDECAR METRICS
MevBundleMempoolSize: discard.NewGauge(),
MevTxSizeBytes: discard.NewHistogram(),
Expand Down
88 changes: 61 additions & 27 deletions node/node.go
Original file line number Diff line number Diff line change
Expand Up @@ -594,11 +594,19 @@ func createSwitch(config *cfg.Config,
p2pLogger log.Logger) *p2p.Switch {
sidecarPeers := splitAndTrimEmpty(config.Sidecar.PersonalPeerIDs, ",", " ")

// Temporarily support both SentinelPeerString and RelayerPeerString
sentinelPeerString := config.Sidecar.SentinelPeerString
if len(sentinelPeerString) == 0 {
p2pLogger.Info(`[mev-tendermint]: WARNING: sentinel_peer_string not found in config.toml.
relayer_peer_string is being deprecated for sentinel_peer_string`)
sentinelPeerString = config.Sidecar.RelayerPeerString
}

sw := p2p.NewSwitch(
config.P2P,
sidecarPeers,
transport,
config.Sidecar.RelayerPeerString,
sentinelPeerString,
p2p.WithMetrics(p2pMetrics),
p2p.WithMevMetrics(mevMetrics),
p2p.SwitchPeerFilters(peerFilters...),
Expand Down Expand Up @@ -885,26 +893,35 @@ func NewNode(config *cfg.Config,
return nil, fmt.Errorf("could not add peers from persistent_peers field: %w", err)
}

if config.Sidecar.RelayerPeerString != "" {
err = sw.SetRelayerPeer(config.Sidecar.RelayerPeerString)
// Temporarily support both SentinelPeerString and RelayerPeerString
sentinelPeerString := config.Sidecar.SentinelPeerString
if len(sentinelPeerString) == 0 && len(config.Sidecar.RelayerPeerString) > 0 {
logger.Info(`[mev-tendermint]: WARNING: sentinel_peer_string not found in config.toml.
relayer_peer_string is being deprecated for sentinel_peer_string. Please update your
config.toml to use sentinel_peer_string.`)
sentinelPeerString = config.Sidecar.RelayerPeerString
}

if sentinelPeerString != "" {
err = sw.SetSentinelPeer(sentinelPeerString)
if err != nil {
return nil, fmt.Errorf("could not add relayer from relayer_conn_string field: %w", err)
return nil, fmt.Errorf("could not add sentinel from sentinel_peer_string field: %w", err)
}
} else {
logger.Info("[node startup]: No relayer_conn_string specified, not adding relayer as peer")
logger.Info("[node startup]: No sentinel_peer_string specified, not adding sentinel as peer")
}

unconditionalPeerIDs := splitAndTrimEmpty(config.P2P.UnconditionalPeerIDs, ",", " ")
if config.Sidecar.RelayerPeerString != "" {
splitStr := strings.Split(config.Sidecar.RelayerPeerString, "@")
if sentinelPeerString != "" {
splitStr := strings.Split(sentinelPeerString, "@")
if len(splitStr) > 0 {
relayerID := splitStr[0]
logger.Info("[node startup]: Adding relayer as an unconditional peer", relayerID)
unconditionalPeerIDs = append(unconditionalPeerIDs, relayerID)
sentinelID := splitStr[0]
logger.Info("[node startup]: Adding sentinel as an unconditional peer", sentinelID)
unconditionalPeerIDs = append(unconditionalPeerIDs, sentinelID)
} else {
fmt.Println("[node startup]: ERR: Could not parse relayer peer string",
fmt.Println("[node startup]: ERR: Could not parse sentinel peer string",
" to add as unconditional peer, is it correctly configured?",
config.Sidecar.RelayerPeerString)
sentinelPeerString)
}
}
err = sw.AddUnconditionalPeerIDs(unconditionalPeerIDs)
Expand Down Expand Up @@ -994,27 +1011,44 @@ func (n *Node) OnStart() error {
time.Sleep(genTime.Sub(now))
}

// Temporarily support both SentinelRPCString and RelayerRPCString
sentinelRPCString := n.config.Sidecar.SentinelRPCString
if len(sentinelRPCString) == 0 && len(n.config.Sidecar.RelayerRPCString) > 0 {
n.Logger.Info(`[mev-tendermint]: WARNING: sentinel_rpc_string not found in config.toml.
relayer_rpc_string is being deprecated for sentinel_rpc_string. Please update your
config.toml to use sentinel_rpc_string.`)
sentinelRPCString = n.config.Sidecar.RelayerRPCString
}

// If all required info is set in config, register with sentinel
if n.config.Sidecar.APIKey != "" && n.config.Sidecar.RelayerRPCString != "" {
if n.config.Sidecar.APIKey != "" && sentinelRPCString != "" {
p2p.RegisterWithSentinel(n.Logger, n.config.Sidecar.APIKey,
string(n.nodeInfo.ID()), n.config.Sidecar.RelayerRPCString)
string(n.nodeInfo.ID()), sentinelRPCString)
} else {
n.Logger.Info("[node startup]: Not registering with relayer, config has API Key:", n.config.Sidecar.APIKey,
"relayer rpc string:", n.config.Sidecar.RelayerRPCString)
n.Logger.Info("[node startup]: Not registering with sentinel, config has API Key:", n.config.Sidecar.APIKey,
"sentinel rpc string:", sentinelRPCString)
}

// Temporarily support both SentinelPeerString and RelayerPeerString
sentinelPeerString := n.config.Sidecar.SentinelPeerString
if len(sentinelPeerString) == 0 {
n.Logger.Info(`[mev-tendermint]: WARNING: sentinel_peer_string not found in config.toml.
relayer_peer_string is being deprecated for sentinel_peer_string`)
sentinelPeerString = n.config.Sidecar.RelayerPeerString
}

// Add private IDs to addrbook to block those peers being added
privateIDs := splitAndTrimEmpty(n.config.P2P.PrivatePeerIDs, ",", " ")
if n.config.Sidecar.RelayerPeerString != "" {
splitStr := strings.Split(n.config.Sidecar.RelayerPeerString, "@")
if sentinelPeerString != "" {
splitStr := strings.Split(sentinelPeerString, "@")
if len(splitStr) > 1 {
relayerID := splitStr[0]
n.Logger.Info("[node startup]: Adding relayer as a private peer", relayerID)
privateIDs = append(privateIDs, relayerID)
sentinelID := splitStr[0]
n.Logger.Info("[node startup]: Adding sentinel as a private peer", sentinelID)
privateIDs = append(privateIDs, sentinelID)
} else {
n.Logger.Info("[node startup]: ERR Could not parse relayer peer string ",
n.Logger.Info("[node startup]: ERR Could not parse sentinel peer string ",
"to add as private peer, is it correctly configured?",
n.config.Sidecar.RelayerPeerString)
sentinelPeerString)
}
}
n.addrBook.AddPrivateIDs(privateIDs)
Expand Down Expand Up @@ -1053,16 +1087,16 @@ func (n *Node) OnStart() error {

// Always connect to persistent peers
peersToDialOnStartup := splitAndTrimEmpty(n.config.P2P.PersistentPeers, ",", " ")
if n.config.Sidecar.RelayerPeerString != "" {
peersToDialOnStartup = append(peersToDialOnStartup, n.config.Sidecar.RelayerPeerString)
if sentinelPeerString != "" {
peersToDialOnStartup = append(peersToDialOnStartup, sentinelPeerString)
} else {
n.Logger.Info("[node startup]: No relayer_conn_string specified, not dialing relayer on startup")
n.Logger.Info("[node startup]: No sentinel_peer_string specified, not dialing sentinel on startup")
}
err = n.sw.DialPeersAsync(peersToDialOnStartup)
if err != nil {
return fmt.Errorf("could not dial peers from persistent_peers field: %w", err)
}
n.sw.StartRelayerConnectionCheckRoutine()
n.sw.StartSentinelConnectionCheckRoutine()

// Run state sync
if n.stateSync {
Expand Down
Loading

0 comments on commit 851797b

Please sign in to comment.