Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute KafkaProducer#send calls directly and not on the blocking thread pool #555

Merged
merged 1 commit into from
Nov 22, 2022

Conversation

svroonland
Copy link
Collaborator

See #531

@svroonland svroonland marked this pull request as ready for review November 22, 2022 20:40
@iravid
Copy link
Member

iravid commented Nov 22, 2022

I think this is generally fine but just want to say that in catastrophic cases where the broker is down (for example) and the producer buffer fills up, calls to send most definitely will block. I think this is documented in the Java API.

@svroonland
Copy link
Collaborator Author

Right, found it:

When the buffer space is exhausted additional send calls will block. The threshold for time to block is determined by <code>max.block.ms</code> after which it throws a TimeoutException.

Thanks!

@svroonland svroonland merged commit d4ef8f2 into master Nov 22, 2022
@svroonland svroonland deleted the producer-send-no-blocking branch November 22, 2022 21:12
@guizmaii
Copy link
Member

Can't we bypass this mechanism from the Java client by implementing our own buffer and using a non-blocking "blocking mechanism"? Might be a crazy idea 🤔😅

@svroonland
Copy link
Collaborator Author

I suppose we could yes, but it's quite some effort for an exceptional case. Feel free to do so of course! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants