Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Calculate hashes in memory #846

Merged
merged 1 commit into from
Oct 15, 2024
Merged

Conversation

tnatt
Copy link
Collaborator

@tnatt tnatt commented Oct 11, 2024

PR that adds relevant functionality and changes from calculating MD5 hashes from file stream to memory stream when populating the checksum_md5 field if possible.

For objects that do not support writing to memory streams they will still be written to temporary file before computing MD5, for dataio supported datatypes this applies to the xtgeo.Cube on SEGY format.

Improves performance by dropping the need for doing a temporary export the object to disk in order to get the MD5 sum.

Closes #552

Copy link
Collaborator

@mferrera mferrera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice

@tnatt tnatt force-pushed the hashes_in_memory branch 6 times, most recently from 95c2cb8 to d206c09 Compare October 14, 2024 13:55
@tnatt tnatt merged commit 5d939ba into equinor:main Oct 15, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Calculate hashes in-memory
2 participants