gcp: bulk request for gcp object storage deletion

[tilman-aiven]

About this change - What it does

Batch multiple delete google API calls in a single HTTP request.
A single batch request is limited to 1000 calls.
When we request more calls that that, we execute multiple batch requests.

Error handling

• if the a batch request with N calls fails, the execution of batch request is retried including all calls
• we cannot assume, which call was successful in that batch

References

• https://googleapis.github.io/google-api-python-client/docs/batch.html
• https://googleapis.github.io/google-api-python-client/docs/epy/index.html, see BatchHttpRequest
• https://googleapis.github.io/google-api-python-client/docs/thread_safety.html

Discussion

• delete_keys vs batch_delete_keys, s3 already implemented a batch operation for key deletion using interface delete_keys
• thread safety (reference https://googleapis.github.io/google-api-python-client/docs/thread_safety.html)
• batch exactly 1000 items vs less

[sjamgade]

Does the call fail, if some object is already deleted ?