diff --git a/README-branching.md b/README-branching.md index 13e624cd5b6..d8dee224e7f 100644 --- a/README-branching.md +++ b/README-branching.md @@ -1,78 +1,75 @@ -# Branch management: `compatible` vs `develop` - -The first section describes the reasoning behind the current state of branching, -and the second section gives you a few tips on how to handle this on a day to -day basis. - -## Rationale - -Instead of a single `main` or `master` branch, there are two active branches -`compatible` and `develop` at all times, where you might want to make changes. - -- `compatible` contains all changes which are literally backwards compatible - with what people currently run on mainnet. Any nodes running a version of - mina based off of compatible should connect to current mainnet just fine. -- `develop` contains changes which break backwards compatibility, or changes - that depend on past compatibility-breaking changes. “Not backwards - compatible” means that a daemon running this version of mina will not connect - to mainnet. - -Major changes to the daemon, protocol, or crypto sometimes will sometimes cause -backwards-compatibility breaking changes, and of course such changes need to be -done with deliberation and are not to be taken lightly. Changes to -infrastructure, auxiliary develop scripts, tests, CI, are usually not be -backwards compatibility breaking, and thereby should go into compatible (unless -you are doing something very special and you know what you’re doing). - -On a semi-regular basis, `compatible` gets manually merged into `develop` so -that — generally speaking — `develop` contains all changes in `compatible.` As -such, `develop` is a superset of `compatible` i.e. `develop` contains everything -that `compatible` contains, and more. - -### Hard fork - -Whenever a hard fork happens, the code in `develop` is released. When this -happens, the current `compatible` is entirely discarded and a new `compatible` -gets created based off of `develop` - -![Illustration of the branching strategy](docs/res/branching_flow.png) - -### Releases - -`release/1.X.X` branches are made off of `compatible` and tagged with alpha and -beta tags until the code is deemed stable, then the `release/1.X.X` branch is -merged into `master` and given a stable tag. Whenever code is tagged, if -anything is missing in in the upstream branches (compatible, develop) then the -tagged branch is also merged back for consistency. - -`release/2.0.0`is the branch where Berkeley QA Net releases are being cut, -between `compatible`and `develop` in the tree. So far nothing has been tagged -there but there will be `2.0.0alphaX` tags once the code is more stable and we -are closer to the Incentivized testnet. - -Unless it is an emergency, code should flow from feature branches -into `compatible`then in batches into the release branch for tagging and testing - -## Day to day - -When developing a feature, if it’s not something that breaks compatibility, then -you should be developing a feature branch, called `foo` for example, based off -of `compatible`. If you’re not sure whether or not your changes are breaking, -they probably are not and should build upon compatible. - -There is a CI job called “merges cleanly to develop” which runs whenever you -have a PR off of `compatible`. If that CI job passes, then you can simply merge -`foo` into `compatible`. If it does not pass, then when you’re done with your -changes to `foo` and the PR is all approved, then make a new branch+PR based off -of your original PR called `foo_DEVELOP` (for example), and then merge -`develop` into `foo_DEVELOP`. Fix any merge errors that result, then once -`foo_DEVELOP` is approved, you can merge it into `develop`. Once that’s done, -the “merges cleanly to develop” CI job in your original `foo` PR should -automagically start passing when you manually re-run it in CI, in which case you -can merge. - -If, after making `foo_DEVELOP`, you need to make changes to `foo`, then make -sure to merge `foo` into `foo_DEVELOP`. In order for the git magic to work, -`foo_DEVELOP` needs to be a superset of the commits from `foo`, and it also -needs to merge first. You can make further changes post-merge in `foo_DEVELOP` -as needed to ensure correctness. +# Mina Branching Policy + +Mina's current public release is "mainnet", version 1.X. The next hardfork release is "berkeley" 2.0, and the one planned after is "izmir" 3.0. + +The development branches in progress in `mina` are as follows: +- `master`: current stable release, currently mainnet 1.X. + - It is frozen from the crypto side, does not depend on `proof-systems`. + - Never commit to it directly, except to introduce a hotfix. +- `compatible`: scheduled to be softwork released. + - The staging branch for mainnet soft fork releases. + - It contains all the changes which are literally backwards compatible with the current mainnet deployment. Any nodes running a version of mina based off of compatible should connect to the current mainnet. + - It serves as the preparation ground for the next mainnet soft fork release. +- `rampup`: what is deployed on the testnet + - The public incentivized network where an early version of the 2.0 hardfork is deployed for community testing. + - `rampup` is a temporary branch maintained until public testnets requiring compatibility are running. + - Never make PRs to `rampup` unless you're explicitly fixing a testnet bug. +- `berkeley`/`izmir`: next hardfork branch / release candidate. + - Contains all the new features that are scheduled for the release (berkeley or izmir). + - `berkeley` is a 2.0 temporary branch maintained until the hard fork after which compatible will include all berkeley changes. + - The "devnet" testnet is running from `master`, sometimes `compatible`, and features only the current release (not cutting edge/berkeley). +- `develop`: 2.0 compatible changes not scoped for the 2.0 hard fork upgrade. + - In other words, `develop` is next non-harmful release (after `berkeley`). + - Is not *the most cutting edge: might not contain protocol features that scheduled for the subsequent (3.0) release. + - Contains changes which break backwards compatibility, or changes that depend on past compatibility-breaking changes. “Not backwards compatible” means that a daemon running this version of mina will not connect to mainnet. + - Major changes to the daemon, protocol, or crypto will sometimes cause backwards-compatibility breaking changes, and of course such changes need to be done with deliberation and are not to be taken lightly. Changes to infrastructure, auxiliary develop scripts, tests, CI, are usually not be backwards compatibility breaking, and thereby should go into compatible (unless you are doing something very special and you know what you’re doing). + - The difference between `develop` and `berkeley` is that `berkeley` will be the actual hardfork release, while `develop` is subsequent softfork release candidate, softfork after `berkeley`. `develop` is just not tested as rigorously, but it's softfork compatible with `berkeley`. So if `berkeley` can be thought of as 2.0, then `develop` is 2.01. +- `o1js-main`: compatible with testnet, but has latest `proof-systems` features so that they can be used in `o1js` + - Contains mina changes from `rampup` + - But `proof-systems/develop` which by default is used by `mina/develop`. + - Uses `o1js/main` and `o1js-bindings/main` as explained [here](https://github.com/o1-labs/o1js/blob/main/README-dev.md#branch-compatibility?). + - When `proof-systems/develop` is too cutting-edge and the adaptations of its changes haven't been landed in mina, `o1js` will use the `proof-systems/o1js-main` branch which is lagging behind `proof-systems/develop` a bit. + + +The relationship between the branches is as presented: `master ⊆ compatible ⊆ rampup ⊆ berkeley ⊆ develop`. +- This means `compatible` includes all the changes in `master`, `rampup` all the changes in `compatible` and so on. So `develop` contains all the changes from *all* the stable branches, but also contains features that do not exist in any of the "subsets". +- The back-merging direction is thus left-to-right: whenever a feature lands in this chain, it has to be periodically "back-merged" to the right. +- The branches are merged in the other direction (upstream) only when released. +- When merely a new feature is introduced, it should aim at the exact target branch. This place depends on the feature, e.g. `compatible` for softfork features, `develop` for more experimental/next release, etc. And then the merged feature is back-propagated downstream (to the right). + + +![Illustration of the branching strategy](docs/res/branching_flow_david_wong.png) + + + +### Hard forks / releases: + +Whenever a hard fork happens, the code in the corresponding release branch, e.g. `berkeley`, is released to become the new `master`. +- The intention is then to again have `compatible` as a next soft-fork branch. + - The transition will be gradual: right after HF, `berkeley` will be copied into both `master` *and* `compatible`, and `develop` will remain as is for a while. PRs from `develop` will be gradually picked based on release scope and included in `compatible` for subsequent soft-fork releases. + - The pre-Berkeley `compatible` is entirely discarded. The pre-Berkeley branch `berkeley` is completely removed from both `mina` and `proof-systems`. +- `release/1.X.X` branches are made off of `compatible` and tagged with alpha and beta tags until the code is deemed stable, then the `release/1.X.X` branch is merged into `master` and given a stable tag. Whenever code is tagged, if anything is missing in in the downstream branches (compatible, develop) then the tagged branch is also merged back for consistency. + +## Day to day: which branch should I use? + +When developing a feature, use the general description of the branches above to decide. Here's a quick rule: +- If a feature/enhancement/bug fix is not feature breaking, and scoped for a mainnet then base it off of `compatible`. If you’re not sure whether or not your changes are breaking, they probably are not and should build upon `compatible`. +- If the feature is scoped for hardfork and is not compatible against a running public testnet, then base it off of the hardfork branch (for example, `berkeley`). +- If it is a bug fix required for a public testnet testing upcoming hardfork then base it off of `rampup`. + +### Handling back-merging conflicts + +We have CI jobs named `check-merges-cleanly-into-BRANCH` that fail if a PR introduces changes conflicting with changes in a downstream branch `BRANCH`. E.g. `check-merges-cleanly-into-develop` will check that a PR aimed at `compatible` is easily back-mergable downstream up to `develop`. PR authors must create new PRs against those branches to resolve conflicts before merging the original PR. + +If that CI job passes, then you can proceed and no further action is needed. + +PRs resolving merge conflicts (merge-PRs) should only be merged after the original PR is approved, and all changes from the original PR are incorporated into the merge-PRs. Consider a PR which is made from `mybranch` branch against `rampup`, and causes conflicts in `berkeley` and `develop`. In this case the workflow is as follows: +- Review and approve the original PR against `rampup` (PR-rampup). CI passes except for `check-merges-cleanly-into-*` jobs. +- Incorporate all changes from PR-rampup into a new PR against `berkeley` (PR-berkeley) and resolve conflicts. Concretely, make a new branch+PR based off of `mybranch` called `mybranch-berkeley` (for example), and then merge `berkeley` into `mybranch-berkeley`. Fix any merge errors that result. + - Keeping branches in sync: If after making e.g. `mybranch-berkeley`, you need to make changes to `mybranch`, then do so, but make sure to merge the newly updated `mybranch` into `mybranch-berkeley`. In order for the git magic to work, `mybranch-berkeley` needs to be a superset of the commits from `mybranch`, and it also needs to be merged first. +- Similarly, incorporate all changes from PR-rampup into a new PR against `develop` (PR-develop) and resolve conflicts. +- Review, approve, and merge PR-berkeley and PR-develop. They can be done in parallel. +- Rerun failing `check-merges-cleanly-into-*` jobs against the original PR-rampup and merge PR-rampup after CI is green. + + +The protocol team at o1labs will conduct weekly synchronization of all branches for all non-conflicting changes to ensure a smooth experience for everyone involved in the Mina repository. The protocol team will reach out to respective teams if there are any conflicting changes (due to force-merges performed mistakenly) and/or failing tests caused by code changes in the upstream branches. diff --git a/docs/res/branching_flow_david_wong.png b/docs/res/branching_flow_david_wong.png new file mode 100644 index 00000000000..063d258e8da Binary files /dev/null and b/docs/res/branching_flow_david_wong.png differ diff --git a/graphql_schema.json b/graphql_schema.json index 0bd86aa2767..f6c1c580175 100644 --- a/graphql_schema.json +++ b/graphql_schema.json @@ -14527,7 +14527,25 @@ "name": "fork_config", "description": "The runtime configuration for a blockchain fork intended to be a continuation of the current one.", - "args": [], + "args": [ + { + "name": "height", + "description": + "The height of the desired block in the best chain", + "type": { "kind": "SCALAR", "name": "Int", "ofType": null }, + "defaultValue": null + }, + { + "name": "stateHash", + "description": "The state hash of the desired block", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], "type": { "kind": "NON_NULL", "name": null, diff --git a/src/app/ledger_export_bench/ledger_export_benchmark.ml b/src/app/ledger_export_bench/ledger_export_benchmark.ml index ba2102d8578..cd5846345a2 100644 --- a/src/app/ledger_export_bench/ledger_export_benchmark.ml +++ b/src/app/ledger_export_bench/ledger_export_benchmark.ml @@ -11,8 +11,15 @@ let load_daemon_cfg filename () = let serialize cfg () = Runtime_config.to_yojson cfg |> Yojson.Safe.to_string +let map_results ~f = + List.fold ~init:(Ok []) ~f:(fun acc x -> + let open Result.Let_syntax in + let%bind accum = acc in + let%map y = f x in + y :: accum ) + let convert accounts () = - Runtime_config.(map_results ~f:Accounts.Single.of_account accounts) + map_results ~f:Runtime_config.Accounts.Single.of_account accounts let () = let runtime_config = Sys.getenv_exn "RUNTIME_CONFIG" in diff --git a/src/lib/mina_graphql/mina_graphql.ml b/src/lib/mina_graphql/mina_graphql.ml index 3bec9a01c24..b82e04836c9 100644 --- a/src/lib/mina_graphql/mina_graphql.ml +++ b/src/lib/mina_graphql/mina_graphql.ml @@ -2118,25 +2118,13 @@ module Queries = struct ~resolve:(fun { ctx = mina; _ } () (state_hash_base58_opt : string option) (height_opt : int option) -> let open Result.Let_syntax in - let get_transition_frontier () = - let transition_frontier_pipe = Mina_lib.transition_frontier mina in - Pipe_lib.Broadcast_pipe.Reader.peek transition_frontier_pipe - |> Result.of_option ~error:"Could not obtain transition frontier" - in let block_from_state_hash state_hash_base58 = let%bind state_hash = State_hash.of_base58_check state_hash_base58 |> Result.map_error ~f:Error.to_string_hum in - let%bind transition_frontier = get_transition_frontier () in let%map breadcrumb = - Transition_frontier.find transition_frontier state_hash - |> Result.of_option - ~error: - (sprintf - "Block with state hash %s not found in transition \ - frontier" - state_hash_base58 ) + Mina_lib.best_chain_block_by_state_hash mina state_hash in block_of_breadcrumb mina breadcrumb in @@ -2149,29 +2137,10 @@ module Queries = struct *) Unsigned.UInt32.of_int height in - let%bind transition_frontier = get_transition_frontier () in - let best_chain_breadcrumbs = - Transition_frontier.best_tip_path transition_frontier - in - let%map desired_breadcrumb = - List.find best_chain_breadcrumbs ~f:(fun bc -> - let validated_transition = - Transition_frontier.Breadcrumb.validated_transition bc - in - let block_height = - Mina_block.( - blockchain_length @@ With_hash.data - @@ Validated.forget validated_transition) - in - Unsigned.UInt32.equal block_height height_uint32 ) - |> Result.of_option - ~error: - (sprintf - "Could not find block in transition frontier with height \ - %d" - height ) + let%map breadcrumb = + Mina_lib.best_chain_block_by_height mina height_uint32 in - block_of_breadcrumb mina desired_breadcrumb + block_of_breadcrumb mina breadcrumb in match (state_hash_base58_opt, height_opt) with | Some state_hash_base58, None -> @@ -2309,85 +2278,108 @@ module Queries = struct "The runtime configuration for a blockchain fork intended to be a \ continuation of the current one." ~typ:(non_null Types.json) - ~args:Arg.[] - ~resolve:(fun { ctx = mina; _ } () -> - match Mina_lib.best_tip mina with - | `Bootstrapping -> - Deferred.Result.fail "Daemon is bootstrapping" - | `Active best_tip -> - let open Deferred.Result.Let_syntax in - let block = Transition_frontier.Breadcrumb.(block best_tip) in - let blockchain_length = Mina_block.blockchain_length block in - let global_slot = - Mina_block.consensus_state block - |> Consensus.Data.Consensus_state.curr_global_slot - in - let staged_ledger = - Transition_frontier.Breadcrumb.staged_ledger best_tip - |> Staged_ledger.ledger - in - let protocol_state = - Transition_frontier.Breadcrumb.protocol_state best_tip - in - let consensus = - Mina_state.Protocol_state.consensus_state protocol_state - in - let staking_epoch = - Consensus.Proof_of_stake.Data.Consensus_state.staking_epoch_data - consensus - in - let next_epoch = - Consensus.Proof_of_stake.Data.Consensus_state.next_epoch_data - consensus - in - let staking_epoch_seed = - Mina_base.Epoch_seed.to_base58_check - staking_epoch.Mina_base.Epoch_data.Poly.seed - in - let next_epoch_seed = - Mina_base.Epoch_seed.to_base58_check - next_epoch.Mina_base.Epoch_data.Poly.seed - in - let runtime_config = Mina_lib.runtime_config mina in - let%bind staking_ledger = - match Mina_lib.staking_ledger mina with - | None -> - Deferred.Result.fail "Staking ledger is not initialized." - | Some (Genesis_epoch_ledger l) -> - return (Ledger.Any_ledger.cast (module Ledger) l) - | Some (Ledger_db l) -> - return (Ledger.Any_ledger.cast (module Ledger.Db) l) - in + ~args: + Arg. + [ arg "stateHash" ~doc:"The state hash of the desired block" + ~typ:string + ; arg "height" + ~doc:"The height of the desired block in the best chain" ~typ:int + ] + ~resolve:(fun { ctx = mina; _ } () state_hash_opt block_height_opt -> + let open Deferred.Result.Let_syntax in + let%bind breadcrumb = + match (state_hash_opt, block_height_opt) with + | None, None -> ( + match Mina_lib.best_tip mina with + | `Bootstrapping -> + Deferred.Result.fail "Daemon is bootstrapping" + | `Active breadcrumb -> + return breadcrumb ) + | Some state_hash_base58, None -> + let open Result.Monad_infix in + State_hash.of_base58_check state_hash_base58 + |> Result.map_error ~f:Error.to_string_hum + >>= Mina_lib.best_chain_block_by_state_hash mina + |> Deferred.return + | None, Some block_height -> + Mina_lib.best_chain_block_by_height mina + (Unsigned.UInt32.of_int block_height) + |> Deferred.return + | Some _, Some _ -> + Deferred.Result.fail "Cannot specify both state hash and height" + in + let block = Transition_frontier.Breadcrumb.block breadcrumb in + let blockchain_length = Mina_block.blockchain_length block in + let global_slot = + Mina_block.consensus_state block + |> Consensus.Data.Consensus_state.curr_global_slot + in + let staged_ledger = + Transition_frontier.Breadcrumb.staged_ledger breadcrumb + |> Staged_ledger.ledger + in + let protocol_state = + Transition_frontier.Breadcrumb.protocol_state breadcrumb + in + let consensus = + Mina_state.Protocol_state.consensus_state protocol_state + in + let staking_epoch = + Consensus.Proof_of_stake.Data.Consensus_state.staking_epoch_data + consensus + in + let next_epoch = + Consensus.Proof_of_stake.Data.Consensus_state.next_epoch_data + consensus + in + let staking_epoch_seed = + Mina_base.Epoch_seed.to_base58_check + staking_epoch.Mina_base.Epoch_data.Poly.seed + in + let next_epoch_seed = + Mina_base.Epoch_seed.to_base58_check + next_epoch.Mina_base.Epoch_data.Poly.seed + in + let runtime_config = Mina_lib.runtime_config mina in + let%bind staking_ledger = + match Mina_lib.staking_ledger mina with + | None -> + Deferred.Result.fail "Staking ledger is not initialized." + | Some (Genesis_epoch_ledger l) -> + return (Ledger.Any_ledger.cast (module Ledger) l) + | Some (Ledger_db l) -> + return (Ledger.Any_ledger.cast (module Ledger.Db) l) + in + assert ( + Mina_base.Ledger_hash.equal + (Ledger.Any_ledger.M.merkle_root staking_ledger) + staking_epoch.ledger.hash ) ; + let%bind next_epoch_ledger = + match Mina_lib.next_epoch_ledger mina with + | None -> + Deferred.Result.fail "Next epoch ledger is not initialized." + | Some `Notfinalized -> + return None + | Some (`Finalized (Genesis_epoch_ledger l)) -> + return (Some (Ledger.Any_ledger.cast (module Ledger) l)) + | Some (`Finalized (Ledger_db l)) -> + return (Some (Ledger.Any_ledger.cast (module Ledger.Db) l)) + in + Option.iter next_epoch_ledger ~f:(fun ledger -> assert ( Mina_base.Ledger_hash.equal - (Ledger.Any_ledger.M.merkle_root staking_ledger) - staking_epoch.ledger.hash ) ; - let%bind next_epoch_ledger = - match Mina_lib.next_epoch_ledger mina with - | None -> - Deferred.Result.fail "Next epoch ledger is not initialized." - | Some `Notfinalized -> - return None - | Some (`Finalized (Genesis_epoch_ledger l)) -> - return (Some (Ledger.Any_ledger.cast (module Ledger) l)) - | Some (`Finalized (Ledger_db l)) -> - return (Some (Ledger.Any_ledger.cast (module Ledger.Db) l)) - in - let protocol_state_hash = - Transition_frontier.Breadcrumb.state_hash best_tip - in - Option.iter next_epoch_ledger ~f:(fun ledger -> - assert ( - Mina_base.Ledger_hash.equal - (Ledger.Any_ledger.M.merkle_root ledger) - next_epoch.ledger.hash ) ) ; - let%map new_config = - Runtime_config.make_fork_config ~staged_ledger ~global_slot - ~staking_ledger ~staking_epoch_seed ~next_epoch_ledger - ~next_epoch_seed ~blockchain_length ~protocol_state_hash - runtime_config - in - Runtime_config.to_yojson new_config |> Yojson.Safe.to_basic ) + (Ledger.Any_ledger.M.merkle_root ledger) + next_epoch.ledger.hash ) ) ; + let%bind new_config = + Runtime_config.make_fork_config ~staged_ledger ~global_slot + ~staking_ledger ~staking_epoch_seed ~next_epoch_ledger + ~next_epoch_seed ~blockchain_length ~protocol_state runtime_config + in + let%map () = + let open Async.Deferred.Infix in + Async_unix.Scheduler.yield () >>| Result.return + in + Runtime_config.to_yojson new_config |> Yojson.Safe.to_basic ) let thread_graph = field "threadGraph" diff --git a/src/lib/mina_lib/mina_lib.ml b/src/lib/mina_lib/mina_lib.ml index 1f5cf552629..b680ee61676 100644 --- a/src/lib/mina_lib/mina_lib.ml +++ b/src/lib/mina_lib/mina_lib.ml @@ -2339,3 +2339,35 @@ let verifier { processes = { verifier; _ }; _ } = verifier let vrf_evaluator { processes = { vrf_evaluator; _ }; _ } = vrf_evaluator let genesis_ledger t = Genesis_proof.genesis_ledger t.config.precomputed_values + +let get_transition_frontier (t : t) = + transition_frontier t |> Pipe_lib.Broadcast_pipe.Reader.peek + |> Result.of_option ~error:"Could not obtain transition frontier" + +let best_chain_block_by_height (t : t) height = + let open Result.Let_syntax in + let%bind transition_frontier = get_transition_frontier t in + Transition_frontier.best_tip_path transition_frontier + |> List.find ~f:(fun bc -> + let validated_transition = + Transition_frontier.Breadcrumb.validated_transition bc + in + let block_height = + Mina_block.( + blockchain_length @@ With_hash.data + @@ Validated.forget validated_transition) + in + Unsigned.UInt32.equal block_height height ) + |> Result.of_option + ~error: + (sprintf "Could not find block in transition frontier with height %s" + (Unsigned.UInt32.to_string height) ) + +let best_chain_block_by_state_hash (t : t) hash = + let open Result.Let_syntax in + let%bind transition_frontier = get_transition_frontier t in + Transition_frontier.find transition_frontier hash + |> Result.of_option + ~error: + (sprintf "Block with state hash %s not found in transition frontier" + (State_hash.to_base58_check hash) ) diff --git a/src/lib/mina_lib/mina_lib.mli b/src/lib/mina_lib/mina_lib.mli index 2d4e4e2efa4..761559a7e80 100644 --- a/src/lib/mina_lib/mina_lib.mli +++ b/src/lib/mina_lib/mina_lib.mli @@ -230,3 +230,9 @@ val vrf_evaluator : t -> Vrf_evaluator.t val genesis_ledger : t -> Mina_ledger.Ledger.t Lazy.t val vrf_evaluation_state : t -> Block_producer.Vrf_evaluation_state.t + +val best_chain_block_by_height : + t -> Unsigned.UInt32.t -> (Transition_frontier.Breadcrumb.t, string) Result.t + +val best_chain_block_by_state_hash : + t -> State_hash.t -> (Transition_frontier.Breadcrumb.t, string) Result.t diff --git a/src/lib/runtime_config/dune b/src/lib/runtime_config/dune index fd1635d338b..e899b708035 100644 --- a/src/lib/runtime_config/dune +++ b/src/lib/runtime_config/dune @@ -5,6 +5,7 @@ ;; opam libraries async async_kernel + async_unix core_kernel bin_prot.shape base.caml @@ -14,7 +15,9 @@ result sexplib0 ;; local libraries + block_time currency + genesis_constants data_hash_lib kimchi_backend.pasta kimchi_backend.pasta.basic @@ -23,6 +26,7 @@ mina_base.import mina_numbers ppx_dhall_type + mina_state snark_params unsigned_extended pickles diff --git a/src/lib/runtime_config/runtime_config.ml b/src/lib/runtime_config/runtime_config.ml index 39adfaf7140..99768a36bc4 100644 --- a/src/lib/runtime_config/runtime_config.ml +++ b/src/lib/runtime_config/runtime_config.ml @@ -58,6 +58,14 @@ let of_yojson_generic ~fields of_yojson json = dump_on_error json @@ of_yojson @@ yojson_strip_fields ~keep_fields:fields json +let rec deferred_list_fold ~init ~f = function + | [] -> + Async.Deferred.Result.return init + | h :: t -> + let open Async.Deferred.Result.Let_syntax in + let%bind init = f init h in + deferred_list_fold ~init ~f t + module Json_layout = struct module Accounts = struct module Single = struct @@ -1312,9 +1320,22 @@ let gen = } let ledger_accounts (ledger : Mina_ledger.Ledger.Any_ledger.witness) = - Mina_ledger.Ledger.Any_ledger.M.to_list ledger - |> Async.Deferred.map - ~f:(Mina_stdlib.Result.List.map ~f:Accounts.Single.of_account) + let open Async.Deferred.Result.Let_syntax in + let yield = Async_unix.Scheduler.yield_every ~n:100 |> Staged.unstage in + let%bind accounts = + Mina_ledger.Ledger.Any_ledger.M.to_list ledger + |> Async.Deferred.map ~f:Result.return + in + let%map accounts = + deferred_list_fold ~init:[] + ~f:(fun acc el -> + let open Async.Deferred.Infix in + let%bind () = yield () >>| Result.return in + let%map elt = Accounts.Single.of_account el |> Async.Deferred.return in + elt :: acc ) + accounts + in + List.rev accounts let ledger_of_accounts accounts = Ledger. @@ -1327,13 +1348,18 @@ let ledger_of_accounts accounts = } let make_fork_config ~staged_ledger ~global_slot ~blockchain_length - ~protocol_state_hash ~staking_ledger ~staking_epoch_seed ~next_epoch_ledger + ~protocol_state ~staking_ledger ~staking_epoch_seed ~next_epoch_ledger ~next_epoch_seed (runtime_config : t) = let open Async.Deferred.Result.Let_syntax in let global_slot = Mina_numbers.Global_slot_since_hard_fork.to_int global_slot in let blockchain_length = Unsigned.UInt32.to_int blockchain_length in + let yield () = + let open Async.Deferred.Infix in + Async_unix.Scheduler.yield () >>| Result.return + in + let%bind () = yield () in let%bind accounts = Mina_ledger.Ledger.Any_ledger.cast (module Mina_ledger.Ledger) staged_ledger |> ledger_accounts @@ -1345,16 +1371,35 @@ let make_fork_config ~staged_ledger ~global_slot ~blockchain_length let%map fork = proof.fork in fork.previous_length + blockchain_length in + let protocol_constants = Mina_state.Protocol_state.constants protocol_state in + let genesis = + { Genesis.k = + Some + (Unsigned.UInt32.to_int + protocol_constants.Genesis_constants.Protocol.Poly.k ) + ; delta = Some (Unsigned.UInt32.to_int protocol_constants.delta) + ; slots_per_epoch = + Some (Unsigned.UInt32.to_int protocol_constants.slots_per_epoch) + ; slots_per_sub_window = + Some (Unsigned.UInt32.to_int protocol_constants.slots_per_sub_window) + ; genesis_state_timestamp = + Some + (Block_time.to_string_exn protocol_constants.genesis_state_timestamp) + } + in let fork = Fork_config. { previous_state_hash = - Mina_base.State_hash.to_base58_check protocol_state_hash + Mina_base.State_hash.to_base58_check + protocol_state.Mina_state.Protocol_state.Poly.previous_state_hash ; previous_length = Option.value ~default:blockchain_length previous_length ; previous_global_slot = global_slot } in + let%bind () = yield () in let%bind staking_ledger_accounts = ledger_accounts staking_ledger in + let%bind () = yield () in let%map next_epoch_ledger_accounts = match next_epoch_ledger with | None -> @@ -1383,7 +1428,7 @@ let make_fork_config ~staged_ledger ~global_slot ~blockchain_length artificially add it. In fact, it wouldn't work at all, because the new node would try to create this account at startup, even though it already exists, leading to an error.*) - ~epoch_data + ~epoch_data ~genesis ~ledger: { ledger with base = Accounts accounts