From 68577df2e85c6ac762857bc1a96960db9824948a Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 16 Mar 2021 10:56:29 +0100 Subject: [PATCH 1/2] propose: js active conn mgr --- proposals/js-connection-manager.md | 125 +++++++++++++++++++++++++++++ 1 file changed, 125 insertions(+) create mode 100644 proposals/js-connection-manager.md diff --git a/proposals/js-connection-manager.md b/proposals/js-connection-manager.md new file mode 100644 index 00000000..1d148322 --- /dev/null +++ b/proposals/js-connection-manager.md @@ -0,0 +1,125 @@ +# js active connection management + +Authors: @vasco-santos + +Initial PR: https://github.com/protocol/web3-dev-team/pull/81 + + + + + +## Purpose & impact +#### Background & intent +_Describe the desired state of the world after this project? Why does that matter?_ + + +With the existing protocols in libp2p, as well as IPFS subsystems built on top of libp2p, such as Pubsub and the DHT, the need for a connection manager overhaul becomes an import work stream, so that these protocols operate as expected by the users, i.e. out of the box solution. + +Currently, js nodes have a reactive connection manager that can be decoupled into two parts: establishing new connections and trimming connections. The former relies on an `autoDial` approach, where each time a new node is discovered it will try to establish a connection with that peer. The later consists on blindly trimming connections when reaching a configurable threshold. + +From the current solution, there is a lot of space for improvement with tremendous value for the users. Either evolving the current connection manager to the state of the go-implementation or implementing a fully fledged [Connection Manager v2](https://github.com/libp2p/specs/pull/161) (+ [more notes](https://github.com/libp2p/notes/issues/13)). + +With the connection manager overhaul in JS we aim to: +- Find our closest peers on the network, and attempt to stay connected to n to them +- Finding, connecting to and protecting our gossipsub peers (same topics search) +- Finding and binding to relays with AutoRelay +- Finding and binding to application protocol peers (as needed via MulticodecTopology) -- We should clarify what libp2p will handle intrinsically and what users need to do + +All the above scenarios need proper setups from our users at the moment, which means they need to manually find the nodes they want to connect and connect to them, as libp2p connection manager will not be actively looking at meaningful connection to establish. + +An active connection manager will enable out of the box creation of meaningful overlay networks, as well as enable browser nodes to bind to AutoRelay nodes, so that they can be dialed from other nodes in the network. Moreover, JS nodes keep the discovered peers stored in the PeerStore. On subsequent node starts, these nodes should prioritize their previous established topology over the bootstrap nodes if possible. + +#### Assumptions & hypotheses +_What must be true for this project to matter?_ + + +- Web3 developers want to have a reliable pubsub topology out of the box without relying on star servers +- Web3 developers want to find and connect to other peers in a given scope +- Browser developers want to have their nodes reachable via more transports than `webrtcSTAR` out of the box +- PL wants to decrease the load on bootstrap nodes + +#### User workflow example +_How would a developer or user use this new capability?_ + + +Out of the box usage with sane defaults. However, the developer should be enabled to configure the proactive dial strategies (power users) + +#### Impact +_How would this directly contribute to web3 dev stack product-market fit?_ + + + +This solves (together with browsers being able to dial go nodes) most of the common issues of our users when trying to use features like IPNS and Pubsub and hitting problems. + +#### Leverage +_How much would nailing this project improve our knowledge and ability to execute future projects?_ + + + +#### Confidence +_How sure are we that this impact would be realized? Label from [this scale](https://medium.com/@nimay/inside-product-introduction-to-feature-priority-using-ice-impact-confidence-ease-and-gist-5180434e5b15)_. + + + + +## Project definition +#### Brief plan of attack + + +There is a proposal already for a [full connection manager overhaul](https://github.com/libp2p/js-libp2p/issues/744). There are a few items that should not be considered into this scope. Connection Gating, Retries and Disconnect are good candidates for a follow up proposal. Decaying tags might have their value, but depend on how far we aim to reach as a first step. Proactive dial, Trim connections, Keep Alive and being able to protect important connections are essential for the scope of this proposal. + +#### What does done look like? +_What specific deliverables should completed to consider this project done?_ + +#### What does success look like? +_Success means impact. How will we know we did the right thing?_ + + + +#### Counterpoints & pre-mortem +_Why might this project be lower impact than expected? How could this project fail to complete, or fail to be successful?_ + +#### Alternatives +_How might this project’s intent be realized in other ways (other than this project proposal)? What other potential solutions can address the same need?_ + +#### Dependencies/prerequisites + + +#### Future opportunities + + +## Required resources + +#### Effort estimate + + +#### Roles / skills needed + From fa120b1e82f911cec2dd0734cc0aa6ac6160b5de Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 30 Mar 2021 11:10:02 +0200 Subject: [PATCH 2/2] chore: apply suggestions from code review Co-authored-by: Jacob Heun Co-authored-by: Max Inden --- proposals/js-connection-manager.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/js-connection-manager.md b/proposals/js-connection-manager.md index 1d148322..7ea1a991 100644 --- a/proposals/js-connection-manager.md +++ b/proposals/js-connection-manager.md @@ -29,12 +29,12 @@ Outline the status quo, including any relevant context on the problem you’re s With the existing protocols in libp2p, as well as IPFS subsystems built on top of libp2p, such as Pubsub and the DHT, the need for a connection manager overhaul becomes an import work stream, so that these protocols operate as expected by the users, i.e. out of the box solution. -Currently, js nodes have a reactive connection manager that can be decoupled into two parts: establishing new connections and trimming connections. The former relies on an `autoDial` approach, where each time a new node is discovered it will try to establish a connection with that peer. The later consists on blindly trimming connections when reaching a configurable threshold. +Currently, js nodes have a reactive connection manager that can be decoupled into two parts: establishing new connections and trimming connections. The former relies on an `autoDial` approach, where each time a new node is discovered it will try to establish a connection with that peer, if it has less connections than its desired minimum. The latter consists on blindly trimming connections when reaching a configurable threshold. From the current solution, there is a lot of space for improvement with tremendous value for the users. Either evolving the current connection manager to the state of the go-implementation or implementing a fully fledged [Connection Manager v2](https://github.com/libp2p/specs/pull/161) (+ [more notes](https://github.com/libp2p/notes/issues/13)). With the connection manager overhaul in JS we aim to: -- Find our closest peers on the network, and attempt to stay connected to n to them +- Find our closest peers on the network, and attempt to stay connected to n of them - Finding, connecting to and protecting our gossipsub peers (same topics search) - Finding and binding to relays with AutoRelay - Finding and binding to application protocol peers (as needed via MulticodecTopology) -- We should clarify what libp2p will handle intrinsically and what users need to do @@ -49,8 +49,7 @@ _What must be true for this project to matter?_ - Web3 developers want to have a reliable pubsub topology out of the box without relying on star servers - Web3 developers want to find and connect to other peers in a given scope -- Browser developers want to have their nodes reachable via more transports than `webrtcSTAR` out of the box -- PL wants to decrease the load on bootstrap nodes +- Reduce the reliance nodes have on Bootstrap nodes outside of joining the network, to reduce the load on those servers #### User workflow example _How would a developer or user use this new capability?_ @@ -108,6 +107,7 @@ _How might this project’s intent be realized in other ways (other than this pr #### Future opportunities +- Propose a spec for JS, that can also be implemented in Go ## Required resources