-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Networking] FLIP: Message Forensic (MF) System #195
base: main
Are you sure you want to change the base?
Conversation
router’s signature verification procedure at the engine level. Furthermore, it meshes well with the Flow protocol’s existing state. | ||
|
||
## Review Guide | ||
This FLIP is presented as a Pull Request (PR) in the `flow-go` repository. We welcome reviewers to express their opinions and share feedback directly on the PR page, aiming for a structured and productive discussion. To aid this, please adhere to one of the following response frameworks: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The referenced PR in flow-go
is closed. Should this be updated to ask for feedback on the current PR (195) instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I favor the "Enforced Flow-level Signing Policy For All Messages" (I have suggested to rename this to FLS) and here are my thoughts:
My biggest concern in adopting FLS is the backward compatibility issues this will cause as it will be a major breaking change. But from a design and maintainability perspective, I like this appraach over GMF. |
Co-authored-by: Misha <[email protected]>
Co-authored-by: Misha <[email protected]>
Co-authored-by: Misha <[email protected]>
Co-authored-by: Misha <[email protected]>
Co-authored-by: Misha <[email protected]>
Co-authored-by: Misha <[email protected]>
I support the "GossipSub Message Forensic (GMF)" approach, articulating my views as follows:
I raised before (on discord) about moving all communication on GossipSub ( unicast messages too ), but turned out there are some messages assumed to be sent on a direct 1:1 connection. I think solving that is also important. ( I don't have much information about current topology, but I think it maybe possible to have direct peering between components that require 1:1 connections ) Eventually I think objective is to allow anyone to join network (let's say any AN), making it over GossipSub security and scalability guarantees and would benefit in the long run. |
@bluesign, it is imperative in GMF for an engine to retain the full integrity of the GossipSub envelope; the function hinges fundamentally on the wholeness of the data present. This is not merely a procedural stipulation but a requisite grounded in the necessity of verifying the signature, which encapsulates the entire envelope, thereby making it infeasible to relay the envelope devoid of its data contents.
This is not about entrusting the networking layer blindly; it is about endorsing a methodologically sound practice that employs cryptographic primitives to furnish verifiable proofs, fostering a trust environment among Flow nodes that should not build trust but rather on cryptographically verifiable primitives. Otherwise, we may not even need a forensic mechanism in the first place. By reducing the envelope to a fragmentary state, the engine's self-sufficient capacity for verification is impeded and essentially undermines the core objective of establishing irrefutable proofs. It is important to not overlook the cardinal principles that govern the GMF, which pivot on the complete and unaltered state of the data encompassed in the envelope. Thus, the GMF solution must consider the integral role of the intact envelope in sustaining the very foundation of the system we are discussing, ensuring the accurate verification of both the event’s association with the envelope and the envelope's signature. |
What I was proposing is something like this: Some struct like message.Message as the Flow message. ( enriched with seqno, topic. signature, key from pb.Message, and decoded event as interface{} ) passed to engine. So technically engine will have everything to prove (with little overhead). But as gossipSub is defending against impersonation etc, I don't think engine needs to do a signature check here. In case of conflict, this new struct is self sufficient to raise a claim. Then who is responsible checking for this violation can get the this struct from the node, reconstruct pb.Message and do the signature verification. if signature verification succeed, then it can punish the offender ( if not can punish the claimer ) |
@bluesign yes, technically the engine doesn't have to check the signature itself and can rely on the data from its networking layer. But, the core idea behind the FLIP is to make sure that any proof it gives when reporting a rule-breaking move is fully self-standing. This means that if node A is saying that node B did something wrong, using evidence E, then anyone else should be able to see that node B was indeed in the wrong just by looking at E. For this to work, node A needs to share the original message as it was, signature and all. Even a tiny change to the message stops others from being able to confirm the signature is real. The signature covers everything in the entire envelope, not just parts of it. Below is a copy of the entire type Message struct {
From []byte `protobuf:"bytes,1,opt,name=from" json:"from,omitempty"`
Data []byte `protobuf:"bytes,2,opt,name=data" json:"data,omitempty"`
Seqno []byte `protobuf:"bytes,3,opt,name=seqno" json:"seqno,omitempty"`
Topic *string `protobuf:"bytes,4,opt,name=topic" json:"topic,omitempty"`
Signature []byte `protobuf:"bytes,5,opt,name=signature" json:"signature,omitempty"`
Key []byte `protobuf:"bytes,6,opt,name=key" json:"key,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
} In conclusion, when we talk about “reducing overhead,” creating a new structure with only some details from the
It appears you are advocating for the alteration of the current one-step process into a more interactive two-step procedure, though the advantages of this aren't entirely clear. Let's take a closer look:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have concerns with both options.
I agree that that GossipSub specific option isn't universal enough to cover all of our usecases, but I worry about the overhead of adding an extra layer of signatures to every message. I wonder if there is a hybrid approach we could take that avoids generating and verifying 2 signatures for every/most messages, plus the additional network data.
At its core, the requirement is to collect cryptographic evidence that can be used by a 3rd party to prove that a protocol violation occurred, and that a specific node sent the message. The evidence itself is raw bytes of data making up the original message and the signature signed by the misbehaving node (and probably a block id/height to get the node's public keys from protocol state).
What if instead of adding a new signature to all messages, we use a usecase specific method, and encapsulate it in a shared interface to reduce complexity.
something like:
- Have a high level envelope interface with methods to get the underlying event and forensic data.
- Implementations hold references to the decoded event plus source data/signature needed for attribution
- Engines are updated to accept this new interface
- If a misbehavior is detected, the envelope is passed along to a slashing system
Then, there could be different implementations for each of the message types.
- Gossibsub would include the
pb.Message
. - Unicast would include the new mechanism.
- Future optimizations could use the actual event itself as the evidence if it already contained a signature from the sender.
- The implementation could produce some consistent format that the slashing system used. I assume that would be something like
NodeID
plus 2[]byte
, one for the raw data and one for the signature.
I think this addresses most of the disadvantages for both options (except for being a breaking change), while also avoiding additional signatures.
This is going to be a bit of a longer reply, sorry 😅. I think there is quite a few nuances and applications patterns that need to be considered. Central design goals
Thoughts on the metrics to rank designs by
Analyzing the proposalsRegarding proposal-2: Flow-level Signing Policy (FSP)
I would argue that with the peer ranking system, we already have a a good foundational defense. But it leaves that gap of attributable but not provable protocol violations. FSP makes this gap smaller, but FSP doesn't close it entirely. Therefore, I questioning whether FSP is impactful enough compared to the required engineering time and the runtime cost -- especially since I believe there are ways to close this gap entirely.
Regarding proposal-1: GossipSub Message Forensic (GMF)Generally I think this is the direction to go. Lets look at our ranking criteria:
My thoughts on complexity
Suggestions:
My take on the Disadvantages
|
I support the "GossipSub Message Forensic (GMF)" approach, with some of the extensions suggested by Peter & Alex. Interface ChangesTo accommodate a Process(channel channels.Channel, originID flow.Identifier, event interface{}, envelope *pb.Message) error I agree with Peter's suggestion of instead including the event itself and forensic data within a single higher-level structure:
There are benefits to passing a
Since we need to change the interface for this proposal, we should change it in a way that opens up some of these options in the future without further API changes. Process(channel channels.Channel, originID flow.Identifier, message flow.Message) error
type Message interface {
Event() any
ForensicsCtx() ForensicsContext
// ...
} Side Note: As the Adding some colour to the comparison of runtime impactTo explicitly state what Alex touched on in his comment: To enable linking messages to their corresponding network signature and envelope, we need to retain a reference to already-allocated memory for (slightly) longer. We do not need to allocate additional memory for each message. What is the memory impact of retaining
|
I support the "GossipSub Message Forensic (GMF)" approach, articulating my views as follows.
|
I do not support the "Enforced Flow-level Signing Policy For All Messages", I support the idea of "GossipSub Message Forensic (GMF)" but some changes may be necessary. First, I wanted to clarify two concepts or cryptographic services, since they will be used twice in my comment:
These definitions may have social/legal aspects so we can assume for simplicity that we are not attributing messages to Bob as a party, but we are attributing messages to a party controlling a private key corresponding to some public key shared with the public. I do not support "Enforced Flow-level Signing Policy For All Messages" because:
While I support "GossipSub Message Forensic (GMF)", I wanted to clarify a few points, in particular about our current implementation using libp2p:
I didn't look at PubSub in details, it seems that the networking key (not libp2p static key) is used to sign original payloads (which is great news) but we still need to confirm it. If we want to implement GMF with libp2p we would need to make some changes to libp2p and its use of Noise for 1-1 (maybe through a fork). I will stop my reply here and we can get into ideas on how to update libp2p later. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice write-up 👏🏼
Within the Flow protocol, nodes converse through a networking layer, a medium which undertakes the dispatch and receipt of messages among nodes | ||
via different communication methods: unicast, multicast, and publish. Unicast messages are transmitted to a single recipient over a direct connection. | ||
The multicast and publish, on the other hand, utilize a pub-sub network constructed with the LibP2P GossipSub protocol for message | ||
dissemination. However, this protocol encounters challenges in message attribution, particularly in determining the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The context here is great. It would be also clearer if we recall the Flow protocol state assumptions (for each node, the protocol state tracks 2 public keys, a staking and a networking key)
thanks @tarakby for your detailed comment. Lets try to use the established terminology of authentication and non-repudiation going forward. Thanks Tarak for clearly explaining the concepts. |
Hi @yhassanzadeh13 - this FLIP is not reflected on FLIP project tracker. Did you follow the process outlined in https://github.com/onflow/flips? Specifically please remember to do the following without which the FLIP won't get visibility on the project tracker- Create an issue by using one of the FLIP issue templates based on the type of the FLIP - application, governance, cadence or protocol. The title of the issue should be the title of your FLIP, e.g., "Dynamic Inclusion fees". Submit the issue. Note the issue number that gets assigned. Thank you! |
Hi @yhassanzadeh13 - following up on my message above. This FLIP is not reflected on FLIP project tracker. Can you pls follow the process outlined in https://github.com/onflow/flips? Specifically the following without which the FLIP won't get visibility on the project tracker-
|
@KshitijChaudhary666 Yahya has moved on to a new professional opportunity. |
Co-authored-by: Yahya Hassanzadeh, Ph.D. <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering what state we should be targeting for this PR. There is a lot of important and valuable discussion here in the PR, which might still lead to changes in the proposed FLIP.
hey @AlexHentschel - we will not close the PR. Just to be clear, |
Co-authored-by: Tarak Ben Youssef <[email protected]>
#259
Summary
This FLIP discusses and compares two potential solutions for the Message Forensic (MF) system in the Flow protocol — a system that identifies and attributes protocol violations to the original malicious sender. The two solutions under consideration are: (1) GossipSub Message Forensic (GMF), and (2) Enforced Flow-level Signing Policy For All Messages. We delve into both, listing their pros and cons, to determine which would be more feasible given the considerations of ease of implementation, performance efficiency, and security guarantees.
Our analysis finds the "Enforced Flow-level Signing Policy For All Messages" to be the more promising option, offering a generalized solution that doesn’t hinge on the protocol utilized to send the message, steering clear of the complexities tied to maintaining GossipSub envelopes and dodging the necessity of duplicating GossipSub router’s signature verification procedure at the engine level. Furthermore, it meshes well with the Flow protocol’s existing state.
Review Guide
This FLIP is presented as a Pull Request (PR) in the
flow-go
repository. We welcome reviewers to express their opinions and share feedback directly on the PR page, aiming for a structured and productive discussion. To aid this, please adhere to one of the following response frameworks: