Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose pending snark work query and proof submission in CLI/GraphQL #16214

Open
deepthiskumar opened this issue Oct 11, 2024 · 6 comments
Open
Assignees

Comments

@deepthiskumar
Copy link
Member

deepthiskumar commented Oct 11, 2024

PR:

@svv232
Copy link
Member

svv232 commented Oct 14, 2024

This should already be possible via the "pendingSnarkWork" graphql end point

https://github.com/MinaProtocol/mina/blob/develop/src/lib/mina_graphql/mina_graphql.ml#L2201

@svv232
Copy link
Member

svv232 commented Oct 14, 2024

I think the original idea for this was to query for specific ranges of work.

@deepthiskumar deepthiskumar changed the title Expose pending snark work in CLI/GraphQL Expose pending snark work query and proof submission in CLI/GraphQL Nov 12, 2024
@deepthiskumar
Copy link
Member Author

The changes are intended to decouple daemon and snark worker process to allow for custom setups.

Snark worker:

  • Stand alone snark worker process that can be spawned when required
  • Takes statement, witness, prover fees and prover public key as arguments
  • Connects to a mina daemon node to submit generated proof

Snark coordinator/Daemon:

  • Graphql command to retrieve pending snark work (includes proofs in the pool with fee, in the case prover wants to ignore or outbid)
  • GraphQL command to accept generated proofs

External service/script (To be implemented by node operators)

  • Queries pending work from a node
  • Determines the order of execution
  • Spawns stand alone snark work by passing the required arguments
  • Maintain progress state to re-request proof in case of failures

@jrwashburn
Copy link

Will the current coordinator, built into the daemon, and the current worker (mina internal snark-worker) still operate as-is?

I like that this would allow for third party coordinators to be built, I don't like that this sounds like it will be required to have any distribution of work. I also don't like the closeness implied by the proposal (coordinator service spawns a snark worker.)

I think the persistent workers that can register with / get work from a coordinator is a better way to manage distributed work, vs. spawning a worker for a specific job. There should be a published/supported protocol from worker to coordinator that can be implemented by anyone.

The value of a coordinator is to coordinate work across many independent, distributed machines - we should not lose that capability, or need to invent a new protocol and add a shim on the remote end to spawn workers one-by-one. I don't think we want to deal with connection setup delays or add extra work to the third party coordinator to also have to deal with creating their own protocol and method for distributing work.

I think there should be a clearly defined protocol from daemon to coordinator and from coordinator to worker. It should be supported by both ocaml and rust nodes, and can be independently implemented in private or open source coordinators alongside the "base" implementation that daemon developers ship.

@deepthiskumar
Copy link
Member Author

@jrwashburn the current coordinator-worker setup will remain as is. This is to expose necessary tools/endpoints for a custom setup

@deepthiskumar
Copy link
Member Author

There should be a published/supported protocol from worker to coordinator that can be implemented by anyone.

I think this solution is trying to achieve this to some extent? a new graphql endpoint that returns a sequence of work that is required by BPs in the same order that required to add transactions, another endpoint that accepts snark work and then standalone snark worker process that consumes the spec and produces snarks. The main dependency on coordinator is to get the sequence of work that BPs will need and this can be requested using the proposed graphql endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

4 participants