Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy two hosts for benchmarking nimbus-eth1 import #194

Open
jakubgs opened this issue Aug 20, 2024 · 3 comments
Open

Deploy two hosts for benchmarking nimbus-eth1 import #194

jakubgs opened this issue Aug 20, 2024 · 3 comments
Assignees

Comments

@jakubgs
Copy link
Member

jakubgs commented Aug 20, 2024

The development of nimbus-eth1 is ramping up, and for that reason we will need to perform benchmarking of the process of importing the network state data with validation from ERA files. Currently this process is not optimized, as it's in it's early stages of development, which means a full sync of mainnet would probably take more than a week. Despite that we need to start measuring results in order to figure out progress in import process optimization.

This benchmarking will require two kind of tests on two hosts:

  • A long running test that lasts a week.
  • A "short" running test that lasts 24 hours.

Both of those will not finish, so they will have to be aborted, but the amount of blocks they are able to sync will be the measure of performance. These performance reports will have to be archived in some way, simplest way would be to commit them to a dedicated repository. In addition to the reports gained this way the import process will make available a /metrics endpoint which we can scrape with Prometheus.

The two hosts can be purchased from Hetzner as the hosts will not be using external connections. The storage required will need to be at least 2x the size of Mainnet ERA and ERA1 files, which is currently ~1 2B, so a 2 TB additional NVMe would suffice. Aside from that more than 16 GB of RAM and 4 cores is enough.

@siddarthkay
Copy link
Contributor

How urgently do we need the 2 hosts?
I could either get a cheaper host from auction via Hetzner

Screenshot 2024-08-22 at 5 27 30 PM

OR

I could get a dedicated host which would be comparatively more expensive
example : https://www.hetzner.com/dedicated-rootserver/ax52/

Screenshot 2024-08-22 at 5 29 36 PM

My assumption is that the host from Auction might take longer to get compared to the dedicated one.

@siddarthkay
Copy link
Contributor

After discussing with @jakubgs, I finally went ahead with the following

2 x Dedicated Server AX42
* Location: Finland, HEL1
* For Finland, HEL1, support is only available in English.
* Rescue system (English)
* 1 x Primary IPv4
* 1 x 2 TB NVMe SSD
* 8 Core CPU
* 64 GB DDR5 ECC RAM

Order Details :

Screenshot_2024-08-22_at_5 55 10_PM

Possible wait times :

Screenshot_2024-08-22_at_5 54 55_PM

@siddarthkay
Copy link
Contributor

These 2 AX42 hosts have been activated by Hetzner and currently boot into rescue system.
I'll bootstrap these hosts and add them to our inventory.

siddarthkay added a commit that referenced this issue Sep 4, 2024
This commit adds 2 hetzner AX42 hosts for eth1 benchmarking to our network.

related issue: #194
siddarthkay added a commit that referenced this issue Sep 5, 2024
This commit adds 2 hetzner AX42 hosts for eth1 benchmarking to our network.

related issue: #194
siddarthkay added a commit that referenced this issue Sep 5, 2024
This commit adds 2 hetzner AX42 hosts for eth1 benchmarking to our network.

related issue: #194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants