Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tune cleanup_old_versions request rate to S3 rate limits #3291

Open
wjones127 opened this issue Dec 24, 2024 · 0 comments
Open

Tune cleanup_old_versions request rate to S3 rate limits #3291

wjones127 opened this issue Dec 24, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@wjones127
Copy link
Contributor

A user reported that they easily hit rate limits deleting a lot of versions.

https://discord.com/channels/1030247538198061086/1321118496247250954

Given their situation, I'm guessing they had 57,000 versions to delete. Each with a data file, manifest, and transaction file. So that's 171,000 objects to delete.

We should see if we can keep within rate limits if we have tens of thousands of objects we need to delete, and tune our rate limits appropriately.

@wjones127 wjones127 added the enhancement New feature or request label Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant