Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming issue "No space left on device" in lrzip 0.651 #233

Open
stevenshiau opened this issue Aug 21, 2022 · 1 comment
Open

Streaming issue "No space left on device" in lrzip 0.651 #233

stevenshiau opened this issue Aug 21, 2022 · 1 comment

Comments

@stevenshiau
Copy link

stevenshiau commented Aug 21, 2022

We encountered an issue when using lrzip to compress the stdin data and output to a file. It gave an error "No space left on device". However, we are very sure the disk space is enough.
It's on Debian Sid live system. If we switched to lrzip 0.641, this issue was not reproduced.
More information is provided in the following.
If you need more info, please let me know. Thanks.

root@debian:~# dpkg -l lrzip
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-======================================================
ii lrzip 0.651-2 amd64 compression program with a very high compression ratio

root@debian:~# dd if=/dev/nvme0n1p3 | lrzip -v - > /home/partimag/nvme0n1p3.dd.lrz
The following options are in effect for this COMPRESSION.
Threading is ENABLED. Number of CPUs detected: 4
Detected 4119420928 bytes ram
Compression level 7
Nice Value: 19
Show Progress
Verbose
Temporary Directory set as: ./
Compression mode is: LZMA. LZ4 Compressibility testing enabled
Heuristically Computed Compression Window: 6 = 600MB
Outputting to stdout.
Minimising number of threads to 3 to limit memory usage
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Unable to compress entirely in ram, will use physical files
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Write of length 25336877 failed - No space left on device
Failed to write_buf s_buf in compthread 0
No space left on device
Fatal error - exiting

After downgraded to 0.641, no such issue.
root@debian:~# dpkg -l lrzip
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-===============-============-======================================================
ii lrzip 0.641-1+deb11u1 amd64 compression program with a very high compression ratio

root@debian:~# dd if=/dev/nvme0n1p3 | lrzip -v - > /home/partimag/nvme0n1p3.dd.lrz
The following options are in effect for this COMPRESSION.
Threading is ENABLED. Number of CPUs detected: 4
Detected 4119420928 bytes ram
Compression level 7
Nice Value: 19
Show Progress
Verbose
Temporary Directory set as: ./
Compression mode is: LZMA. LZ4 Compressibility testing enabled
Heuristically Computed Compression Window: 6 = 600MB
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
124370944+0 records in
124370944+0 records out
63677923328 bytes (64 GB, 59 GiB) copied, 3372.52 s, 18.9 MB/s
Compression Ratio: 0.000. Average Compression Speed: 17.930MB/s.
Total time: 00:56:26.90

root@debian:~# free -h
total used free shared buff/cache available
Mem: 3.8Gi 242Mi 2.8Gi 287Mi 788Mi 3.1Gi
Swap: 0B 0B 0B

root@debian:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 393M 2.1M 391M 1% /run
/dev/sr0 365M 365M 0 100% /run/live/medium
/dev/loop0 305M 305M 0 100% /run/live/rootfs/filesystem.squashfs
tmpfs 2.0G 286M 1.7G 15% /run/live/overlay
overlay 2.0G 286M 1.7G 15% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 8.0K 5.0M 1% /run/lock
tmpfs 2.0G 48K 2.0G 1% /tmp
tmpfs 393M 0 393M 0% /run/user/1000
/dev/nvme0n3p1 1000G 66G 935G 7% /home/partimag

@pete4abw
Copy link
Contributor

pete4abw commented Sep 6, 2022

This commit may be the culprit.

Repository owner deleted a comment Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants