[Enhancement]: add pagination logic for syncing block headers #123
Labels
enhancement
New feature or request
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Use case and current behavior
when launching a goerli testnet retriever from scratch, i got this error
ERROR[12-14|04:12:16.803|github.com/Layr-Labs/eigenda/indexer/indexer.go:140] Error pulling new headers err="413 Request Entity Too Large: <html>\r\n<head><title>413 Request Entity Too Large</title></head>\r\n<body>\r\n<center><h1>413 Request Entity Too Large</h1></center>\r\n<hr><center>cloudflare</center>\r\n</body>\r\n</html>\r\n" caller=indexer.go:140this is because indexer trying batch get headers from block number 0 to latest block number
https://github.com/Layr-Labs/eigenda/blob/master/indexer/eth/header_service.go#L45
there is no pagination logic, and the batch size is too large
Enhancement
https://github.com/Layr-Labs/eigenda/blob/master/indexer/eth/header_service.go#L43
should check the count, if count > X, then let count = X
Solution proposal
No response
Additional Information
No response
The text was updated successfully, but these errors were encountered: