Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(python): reading/writing transaction identifiers in python #2942

Merged
merged 2 commits into from
Oct 16, 2024

Conversation

PeterKeDer
Copy link
Contributor

@PeterKeDer PeterKeDer commented Oct 14, 2024

Description

We'd like to read/write the delta transaction identifiers for stream processing. It seems that this is currently available to through rust but not python. This PR adds the following to the python API:

  • app_transactions field in CommitProperties, which is a list of the Transaction class (PyTransaction in rust)
  • DeltaTable.transaction_versions for accessing the latest transaction of a delta table, which returns a dictionary of app id -> Transaction

Would like some feedback on the naming/API. I'm not sure which is the correct terminology to use - "app transaction" vs "transaction version" vs "transaction identifier"? The delta protocol refers to it as transaction identifiers.

@github-actions github-actions bot added the binding/python Issues for the Python package label Oct 14, 2024
Copy link

codecov bot commented Oct 15, 2024

Codecov Report

Attention: Patch coverage is 0% with 44 lines in your changes missing coverage. Please review.

Project coverage is 72.36%. Comparing base (7a3b03f) to head (eb57bf6).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
python/src/lib.rs 0.00% 44 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2942      +/-   ##
==========================================
- Coverage   72.41%   72.36%   -0.06%     
==========================================
  Files         131      131              
  Lines       40542    40586      +44     
  Branches    40542    40586      +44     
==========================================
+ Hits        29358    29369      +11     
- Misses       9290     9332      +42     
+ Partials     1894     1885       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@houqp
Copy link
Member

houqp commented Oct 15, 2024

transaction identifier would have been a better name if we are to start from scratch, but to keep it consistent with the rust core, it's better to keep using app transaction for now.

houqp
houqp previously approved these changes Oct 15, 2024
python/src/lib.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@ion-elgreco ion-elgreco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@ion-elgreco ion-elgreco added this pull request to the merge queue Oct 16, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 16, 2024
@ion-elgreco ion-elgreco added this pull request to the merge queue Oct 16, 2024
Merged via the queue into delta-io:main with commit a846879 Oct 16, 2024
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/python Issues for the Python package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants