Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: First very dirty prototype of SP1 distributed proof #283

Closed
wants to merge 23 commits into from

Conversation

Champii
Copy link

@Champii Champii commented Jun 6, 2024

Still a WIP

@petarvujovic98
Copy link
Contributor

@Champii Aside from updating this branch with main, I've left some comments concering some code quality.

As far as I see from the code, this makes every Raiko instance a potential distributed sp1 worker. Meaning every instance can handle a shard of execution. Maybe it would make more sense to have a service outside of Raiko handle just the sp1 distributed proving part so that it is less coupled with the host. This would allow us to change and/or scale the distributed part of sp1 proofs independently of the host which should not care about how a single block is proven (whereas now it distinguishes between a full proof and a shard proof).

@Champii
Copy link
Author

Champii commented Jun 7, 2024

@Champii Aside from updating this branch with main, I've left some comments concering some code quality.

As far as I see from the code, this makes every Raiko instance a potential distributed sp1 worker. Meaning every instance can handle a shard of execution. Maybe it would make more sense to have a service outside of Raiko handle just the sp1 distributed proving part so that it is less coupled with the host. This would allow us to change and/or scale the distributed part of sp1 proofs independently of the host which should not care about how a single block is proven (whereas now it distinguishes between a full proof and a shard proof).

Awesome, thanks for looking into it :)

I can probably create a new CLI command/argument that starts a Raiko node as a distributed worker specifically. I'll look into it.

As for the code quality, this is only a first draft of a POC, so I will definitely improve it before making a final PR proposal.

@Brechtpd
Copy link
Contributor

Brechtpd commented Jun 9, 2024

Not sure how to best handle this. Having a separate service also seems to be not so ideal because a worker ideally can still be used to process different kinds of things (not only SP1 shards, but also risc zero segments, and even full proofs of smaller programs). If different binaries/endpoints need to be used to start this work on the workers then maybe it makes things more complicated then just having every type of work pass through a single interface within a single binary. Once we create a general prover interface that allows anybody to create their own custom prover backend, it will also be harder to differentiate between the multiple prover backend types, so keeping the host interface general may also pay off for this.

I guess we can wait and see because not a lot of code and easy to move stuff around, at least for now.

@Champii
Copy link
Author

Champii commented Jul 2, 2024

Closing in favor of #302

@Champii Champii closed this Jul 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants