Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjusts log levels for ErrExceedsMaxMempoolSize #692

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

diegoximenes
Copy link

@diegoximenes diegoximenes commented Oct 3, 2024

Adjusts log levels for ErrExceedsMaxMempoolSize.

With this PR logs related to ErrExceedsMaxMempoolSize are initially logged as warnings, and only if they persist for a while that will be logged as errors.

Similarly to what was done with StopWaiter, EphemeralErrorHandler was copied from https://github.com/OffchainLabs/nitro.

Resolves NIT-2785

@diegoximenes diegoximenes changed the title Adjust log levels Adjusts log levels for ErrExceedsMaxMempoolSize Oct 3, 2024
Copy link

codecov bot commented Oct 3, 2024

Codecov Report

Attention: Patch coverage is 32.50000% with 54 lines in your changes missing coverage. Please review.

Project coverage is 59.67%. Comparing base (162553c) to head (c822d29).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #692      +/-   ##
==========================================
- Coverage   59.68%   59.67%   -0.02%     
==========================================
  Files          51       53       +2     
  Lines        9057     9264     +207     
==========================================
+ Hits         5406     5528     +122     
- Misses       3645     3730      +85     
  Partials        6        6              

@diegoximenes diegoximenes marked this pull request as ready for review October 3, 2024 14:05
@diegoximenes diegoximenes marked this pull request as draft October 3, 2024 17:04
@diegoximenes diegoximenes marked this pull request as ready for review October 3, 2024 19:19
Copy link

@ganeshvanahalli ganeshvanahalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
(with a suggestion)

Comment on lines +91 to +97
backoffLogLevel *= 2
logLevel := log.Error
if backoffLogLevel > time.Minute {
backoffLogLevel = time.Minute
} else {
logLevel = log.Warn
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seeing the number of times this code block is being repeated, do you think its worth it to incorporate this logic in one place?

(thinking out loud here) maybe something simpler like having a backoff duration set to ephemeral handler itself and define a new method on it LogLevelAtCurrentBackOff( time.Duration ) that would return what initial loglevel we want to begin with?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this comment.

Furthermore, I really feel like this whole feature should be encapsulated in a customer Logger https://pkg.go.dev/log/slog#Logger or handler https://pkg.go.dev/log/slog#Handler and we should be able to just plug it into the go-ethereum/log library directly. It seems wrong that the callers have to do this level of configuration each time they want to say, "treat this message I'm about to log specially."

Sorry to drop a comment which basically reads like, "go back to the drawing board" so long after you requested the review and after you've already gotten OffchainLabs/nitro#2719 merged.

I'm willing to approve this one and merge it in for now, but we should open a Linear ticket for improving our logging code to be more decoupled from our logic. Let me know what you think.

"github.com/ethereum/go-ethereum/log"
)

// EphemeralErrorHandler handles errors that are ephemeral in nature i.h these are errors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean "i.e." instead of "i.h"?

Comment on lines +91 to +97
backoffLogLevel *= 2
logLevel := log.Error
if backoffLogLevel > time.Minute {
backoffLogLevel = time.Minute
} else {
logLevel = log.Warn
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this comment.

Furthermore, I really feel like this whole feature should be encapsulated in a customer Logger https://pkg.go.dev/log/slog#Logger or handler https://pkg.go.dev/log/slog#Handler and we should be able to just plug it into the go-ethereum/log library directly. It seems wrong that the callers have to do this level of configuration each time they want to say, "treat this message I'm about to log specially."

Sorry to drop a comment which basically reads like, "go back to the drawing board" so long after you requested the review and after you've already gotten OffchainLabs/nitro#2719 merged.

I'm willing to approve this one and merge it in for now, but we should open a Linear ticket for improving our logging code to be more decoupled from our logic. Let me know what you think.

}

func (h *EphemeralErrorHandler) Reset() {
*h.FirstOccurrence = time.Time{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not h.FirstOccurrence = nil?

}

if *h.FirstOccurrence == (time.Time{}) {
*h.FirstOccurrence = time.Now()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One trick I've learned, and that I always try to use when I'm writing library code that uses time, is that if you'll make the time library stubbable or mockable, you can avoid having to call sleep in your test code which really speeds up the test execution time and doesn't really compromise the integrity of your qualification. Plus, if you really want to make sure it works with the "real" implementation of time, you can usually write a simple "happy-path" test that uses the real time library and do all your complicated edge-case checking logic using the stubbed implementation.

I should probably write a blog post about this, it comes up frequently enough.

@@ -23,6 +23,7 @@ go_library(
"//runtime",
"//solgen/go/rollupgen",
"//util/stopwaiter",
"//util/log",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actively allergic to directories or packages called util
https://www.jeromethibaud.com/en/blog/utils-considered-harmful/

Also, I very strongly try to avoid having to override golang package imports. Maybe the package could live in logs/ephemeral and the main type could be Logger or Handler then client code calls it like ephemeral.NewHandler()?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants