Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate creating a low-level crate so that users can easily customize zip-crate. #204

Open
xMAC94x opened this issue Jun 30, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@xMAC94x
Copy link

xMAC94x commented Jun 30, 2024

Is your feature request related to a problem? Please describe.
So this story for this issue originates from the veloren project, where we have a 300MB zip file, network hosted for the game and want to find a way to update client files without re-downloading the whole zip and unzipping it. We found out that due to the zip structure, we could use HTTP RANGE requests and just analyze the CentralDirectoryHeaders and check CRC mismatches locally. Somehow inject this in the zip-extracting process.

Describe the solution you'd like
The first thing I found out is that this is currently not possible with zip, and maybe also not the intended use-case of zip-crate.
However, it would be nice to have a low-level crate that shares structs with zip so that users of the zip crate maybe can interact with it, by constructing/reading the low-level stuff manually and passing it to zip.

Additionally, I looked at other crates rc-zip and zip-structs, all have the issue that they have to redefine the APPNOTE.TXT someway or the other. Making each crate incompatible with others.

Where network crates have http or even url in the world of zip there is no crate that reexports the simplest stuff for people to work with.

IMO such a crate is only worth it if it finds adoption. To find adoption it must be somewhat helpful for zip developers, on the other hand not restrictive to future development. It should include as less dependencies as necessary. Should not force anyone to use sync or async.

I looked at the APPNOTE.TXT and tried to implement a minimum example low-level crate. Spend a few days for: https://crates.io/crates/zip-core/0.0.2
Nothing extraordinary, defining 1:1 mappings of the spec (Central Directory, Local File Header, a few helpers for checking if signatures are correct, optional parsing (via a crate feature if needed), Structs that contain all valid CompressionMethods, all without dependencies by default and no-std.

I wanted to start a discussion, does it make sense to create such a low-level crate, that focuses on structs of the APPNOTE.TXT, optionally some parsing helpers and that's it. While the zip crate focuses on the hard stuff. figuring out which version works together with which field. Use the correct Encryption Algorithm, find the position of the EndOfCentralDirectoryRecord. Reading data from std::io::Read, etc...

Feel free to comment on this. Even if it's just a declination, the idea and crate is at an early stage and still can be shaped easily :)

Describe alternatives you've considered
The simplest alternative is to ignore this approach, say such a low-level crate is unhelpful, maybe not necessary at all and keep continuing like usual. However, making it harder for people to interact in the zip environment.

Best regards,
Marcel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant