[EVM GW] Remote registers and state re-execution

[sideninja]

Re-executing state locally requires for us to listen to all new transactions (ingestion engine handles that) and re-execute all the transactions locally. In order to re-execute a transaction we need to have a state upon which the transaction is applied and we need to have a EVM which the transaction uses to be executed.

We are reusing the flow-go EVM Emulator for re-execution of the transaction, we just need to make sure we can provide the state. Emulator uses a atree.Ledger to fetch register values. We can plug-in the remote register API to fetch the registers upon a get function is called. This will simulate registers being present locally and will avoid the complexity of actually keeping the state locally.

The biggest concern with this approach is scalability. Most importantly will the register API requests be fast enough during the re-execution, so that the re-execution won't be delayed behind the on-chain state. If the latency of those requests is too high this approach won't work and we will have to move toward local register index #322

We should do stress-testing to figure out the scalability possibilities with this approach, with a local AN and EVM GW setup (so the latency of the network is minimized).

Tasks

Beta Give feedback

1. Implement remote register API ledger #575
   +0
   
2. Re-execute the state for calls and transactions #576
   +0
   
3. Allow queries of the state using the remote registers #577
   +0
   
4. Simulate cadence arch with the replayer #578
   +0
   
5. Fallback to remote execution in case of cadence arch calls #579
   +0
6. Implement metrics for meassuring the latency on remote register API and re-execution #580
   +0
7. Deploy on a testnet with local AN and monitor metrics #581
   +0
8. Stress test the node with load of transactions #582
   +0
9. Implement custom metrics re-execution #583
   +0
   
10. Validate and protect against register API index delays #585
    +0
11. Validate re-executed state

[j1010001]

no longer needed, moving to dry-run solution (onflow/flow-go#6539)

[m-Peter]

Alternative implemented by #635