perf: conditional put for default log store (e.g. azure, gcs, minio, cloudflare) #2813
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ion-elgreco
requested review from
wjones127,
roeap,
rtyler and
hntd187
as code owners
ion-elgreco
changed the title
ion-elgreco
changed the title
ion-elgreco
force-pushed
the
feat/conditional_put
branch
from
a9738fd
to
b786513
Compare
ion-elgreco
changed the title
ion-elgreco
force-pushed
the
feat/conditional_put
branch
from
4c6fa7f
to
2a190fc
Compare
ion-elgreco
changed the title
ion-elgreco
force-pushed
the
feat/conditional_put
branch
from
158d09e
to
b818f8b
Compare
ion-elgreco
changed the title
ion-elgreco
force-pushed
the
feat/conditional_put
branch
from
a7d5e2f
to
2ddfc6a
Compare
rtyler
approved these changes
Aug 23, 2024
|
@rtyler what are you worried about in the interaction between Spark and Rust here? |
|
@ion-elgreco we don't have good round-trip tests with Spark writers and Rust writers collaborating (or not) with our log store implementation. |
|
@rtyler I see, but in this case I am not touching the S3DynamoDBLogStore, only S3 flavours that use copyIfNotExists or PutIfAbsent |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
When it's a default log store, we pass bytes around, for other log stores we keep the default tmp_commit logic in place (for stores like
S3DynamoDBLogStoreor the newS3LogStore)Related Issue(s)