Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A0-4231: Extract chain bootstrap to a separate crate #1699

Merged
merged 30 commits into from
May 10, 2024

Conversation

Marcin-Radecki
Copy link
Contributor

@Marcin-Radecki Marcin-Radecki commented Apr 23, 2024

Description

This final PR achieves the goal of removing the aleph-runtime dependency from aleph-node. A new crate chain-bootstraper is made from the current chain bootstrap logic in aleph-node.

The dependencies in aleph-node should be clearer now:

  • aleph-node does not compile native runtime anymore via default features build.
  • chain-bootstrapper does compile both native and wasm runtime during build.
    The amount of packages that node compiles is about 150 less than it was before. Depending on the machine, It's up to 40% compilation speedup for aaleph-node binary. Here's local build, clean repo, on my laptop:

After change - aleph-node and chain-bootstrapper builds in the similar time.

time ./scripts/run_nodes.sh -v 6 
2024-04-22 13:37:54:233 Starting ./scripts/run_nodes.sh
2024-04-22 13:37:54:234 Creating base path ./run-nodes-local if it does not exist
2024-04-22 13:37:54:236 Stopping all current node processes
2024-04-22 13:37:54:241 No aleph-node processes found.
2024-04-22 13:37:54:242 Building testing aleph-node binary (short session) and chainspec-generator binary.
   Compiling proc-macro2 v1.0.81
[...]
    Finished release [optimized] target(s) in 9m 02s
[...]
    Finished release [optimized] target(s) in 9m 31s
[...]

________________________________________________________
Executed in   18.59 mins    fish           external
   usr time  119.30 mins  874.00 micros  119.30 mins
   sys time    5.67 mins    0.00 micros    5.67 mins

Before change - note though the overall time is faster than before:

marcin@marcin-Latitude-3520 ~/g/aleph-node-3 (main)> time ./scripts/run_nodes.sh -v 6                                            (base) 
[...]

    Finished release [optimized] target(s) in 16m 44s

[...]
Executed in   16.77 mins    fish           external
   usr time  107.24 mins    0.00 micros  107.24 mins
   sys time    5.32 mins  791.00 micros    5.32 mins

This is because of polkadot-sdk crates architecture - we still need to compile most of the crates in both node and bootstrapper. There's small optimization in this PR to not build three polkadot-sdk crates with default features, ie to disable rocks-db build when chain-bootstrapper is built.

In Ci though, compilation should be faster as we can parallelize aleph-node and chain-bootstrapper build, which this PR achieves so. Most of the changes are workflows anyway.

Type of change

Please delete options that are not relevant.

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

@Marcin-Radecki Marcin-Radecki changed the title A0 4231 move chainspec generation A0-4231: Extract chain bootstrap to a separate crate Apr 23, 2024
@Marcin-Radecki Marcin-Radecki marked this pull request as ready for review April 25, 2024 08:00
Copy link
Contributor

@ggawryal ggawryal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rust code looks good, workflows as far as I'm understanding too. Good job!

bin/node/Cargo.toml Show resolved Hide resolved
scripts/run_nodes.sh Outdated Show resolved Hide resolved
.github/workflows/_build-test-node.yml Show resolved Hide resolved
@Marcin-Radecki Marcin-Radecki added this pull request to the merge queue May 10, 2024
Merged via the queue into main with commit ee6414a May 10, 2024
15 checks passed
@Marcin-Radecki Marcin-Radecki deleted the A0-4231-move-chainspec-generation branch May 10, 2024 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants