Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds partition-size-factor to rebalance #289

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jamiealquiza
Copy link
Collaborator

Adds --partition-size-factor to the Rebalance command to calibrate the bin-packer for future partition sizes. Closes #287.

Copy link
Contributor

@brouberol brouberol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 * size factor

@jamiealquiza
Copy link
Collaborator Author

A problem with this: the partition size is multiplied by some factor when in the placement stage, but partitions at rest are not part of the calibration.

For instance, if we have topic_a and intend to double retention, we may rebalance with --partition-size-factor 2. Partition 20 may be destined for broker 1005, even accounting for a 2x partition size increase. If broker 1005 is also holding partition 15 for topic_a, what happens if both partitions 15 and 20 were to 2x in size; would the broker be pushed out of bounds?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[topicmappr] add partition size factor to rebalance
3 participants