ENR Size limit

[fryorcraken]

Problem

31/WAKU2-ENR conforms to the EIP-778 size limit of 300 bytes.

• multiaddrs field was added to enable storage of information for secure websocket connections: FQDN + WSS PORT.

• Circuit relay information also needs to be stored, currently using multiaddrs field: To investigate: adding circuit relay information to multiaddr waku-org/nwaku#1491

• rs relay sharding field is being added: update 31/WAKU2-ENR: add static sharding fields #178

• WSS + Circuit Relay information already takes the ENR over 300 bytes (especially when trying to store 2 circuit relay multiaddr)

• WSS + relay shard information takes the ENR over 300 bytes in some instances

Solutions

1. The preliminary idea is limiting (or not using) multiaddrs in ENRs that have the rs attribute.
2. Add fqdn and wss_port fields to the ENR and prefer it over multiaddrs to suport secure websocket information. Also do not encode wss_port if value is 443. 31/WAKU2-ENR: Move websocket connection info from multiadrrs to ws_port + fqdn rfc#578
3. We encode information in several ENRs. Meaning that one node can produce 2 ENRs: one contains rs, other contains multiaddrs for wss address, (and a 3rd one could contains rendez vous multiaddrs)
4. We replace ENR with something else that does not have a size limit (libp2p's signed envelops?)
5. Introduce a simple req/resp protocol that allows retrieving additional info from a node (maybe extending libp2p's identify protocol).

Evaluation of solutions

1. Is fine short term but js-waku (status-web) does need access to the rs field even if only using store/filter/light push
2. Is the current preferred solution
3. Not a solution we want to push
4. waku2 info is covered as part of identify so it could be removed from ENR?

Acceptance criteria

• Define a way to convey all necessary information in ENR

Notes

a. For efficiency reason, the rs information must be present in the ENR and cannot be transmitted via req/resp protocol (4)
b. Connection information (ws/tcp/ip/fqdn details) must be present in the ENR otherwise the req/resp protocol cannot happen
c. waku2 information is already covered as part of identify so could be removed?
d. A node that provide circuit relay info would not also provide ws info: if it needs to provide circuit relay info to be connected to, then it means it does not expose a direct port to the internet and hence ws connection is not possible @richard-ramos to confirm this point.

[fryorcraken]

Regarding circuit relay.

Alice --> Bob (Relay) <-- Carole (behind NAT)

Talking about Carole's ENR.

If Carole cannot be reached from the outside, then she needs to connect to a Relay node (Bob) and provide circuit relay information in her ENR.

Carole's ip4, tcp4, wss connection details are useless. Indeed, these ports/ip are not reachable from the outside.

Hence, she could encode Bob's ip4, tcp4 and even wss connections details in her ENR. The only missing piece of info is Bob's peer id.
We could then add a new cr_secp256k1 field to the ENR to store Bob (relay) peer id.

When decoding an ENR, if cr_secp256k field is present, then we know to reconstruct Carole's multiaddr, we need to build a circuit relay multiaddr and that tcp4 etc fields are used for Bob's part of the multiaddr.

The only remaining question is: How does Carole determine that she is not reachable from the Internet and that she:

1. Needs to connect to a (circuit) relay peer
2. Needs to encode the (circuit) relay peer's (Bob's) details in her ENR instead of her (local) connection details.

Ref: waku-org/nwaku#1493

[jm-clius]

Regarding circuit relay.

Indeed. Circuit relay is already used quite extensively by go-waku and nwaku supports the relay process as server. I think following the process as described here would allow Carol to encode information in her ENR as you described.

In short:

How does Carole determine that she is not reachable from the Internet

She uses AutoNAT, the first step of the circuit relay process (already supported as server by nwaku): https://docs.libp2p.io/concepts/nat/autonat/

1. Needs to connect to a (circuit) relay peer
2. Needs to encode the (circuit) relay peer's (Bob's) details in her ENR instead of her (local) connection details.

Carole knows that she has successfully reserved a relay slot in the circuit-relay server and hence knows the address.

That said, if Carole is behind a restrictive NAT, how does she advertise her ENR as her discv5 UDP port is also likely to be unreachable? For this reason I think a multiaddrs discovery scheme, such as libp2p rendezvous, needs to be introduced regardless, as this will allow Carole to actively register her circuit-relay address at a rendezvous point.