-
Notifications
You must be signed in to change notification settings - Fork 10
/
WHATS-NEW
836 lines (662 loc) · 30.2 KB
/
WHATS-NEW
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
lrzip-next: June 25, 2024 v 0.13.0
Update to lzma SDK 24.07.
Adds RISC-V BCJ filter.
lrzip-next: March 28, 2024 v 0.12.7
Add configuration item to lrzip.conf.
PROCESSORS = ## which will always override found processors.
Useful in rare cases where another program may call lrzip-next.
lrzip-next: March 18, 2024 v 0.12.6
Add shortcut for lrzip-next, lrzn.
Update manpage.
Remove .PHONY directive in Makefile.am to make doc directory.
lrzip-next: January 27, 2024 v 0.12.6
BUGFIX: Correct two MacOS related errors.
1. write_fd() must limit buffer size to < 2G. Issue #88.
2. fake_mremap() function failed with fatal() macro.
Reverted to using fprintf() and exit. Issue #151.
Thanks to Nick Chan for pointing this out.
lrzip-next: January 7, 2024 v 0.12.5
Expand threshhold testing buffer to 100MB from 10MB.
Handle zstd errors properly. Fixes bug 148.
lrzip-next: November 18, 2023 v 0.12.4
Update file headers with SPDX license identifiers.
Fix double free bug when reading user configuration file.
Obsolete changelog file.
lrzip-next: October 24, 2023 v 0.12.3
Update lzo backend compressor to use
lzo1x_999_compress() function for compression level 9.
lrzip-next: August 25, 2023 v 0.12.2
BUGFIX. Revert the fix for #132. Again...
lrzip-next: August 15, 2023 v 0.12.1
Add --nobemt option. No Backend Multi Threading.
This option will inhibit lzma from using multiple
threads above that selected by `-p` option or default.
lrzip-next: July 18, 2023 v 0.12.0
Remove bzip3 code and depend on library.
Update configure.ac to include error checks for include
files.
lrzip-next: July 3, 2023 v 0.12.0
LZMA SDK 23.01.
Add ARM64 BCJ Filter.
Update Delta filter algorithm.
lrzip-next: June 29, 2023 v 0.11.3
REVERT ZPAQ block size. Commit 2f84b91 caused error when
incompressible block sent to backend.
Set STDIN device when required.
lrzip-next: May 14, 2023 v 0.11.2
Fix ZPAQ so large blocks can be sent to backend.
Use compressBlock function to do this with StringBuffer
class.
Update bzip3 library to 1.3.0 fixing compiler warnings
and some security and memory use fixes.
lrzip-next: April 2023 v0.11.1
Fix ZPAQ Decompression code so all levels can be decoded.
lrzip-next: April 2023 v0.11.0
Store zstd compression info.
Update magic header.
lrzip-next: April 2023 v0.10.6
Permit independent setting of zstd compression level (1-22).
Zstd strategy will be selected according to defaults.
Zstd level will not be stored at present. Would require
magic header change. This is future task.
lrzip-next: March 2023 v0.10.4
zstd compression/decompression backend
lrzip-next: March 2023 v0.10.3
Add -Q | --very-quiet option from Upstream.
Suppress ALL output. Will need to check $? variable.
lrzip-next: February 2023 v0.10.2
Permit maximum use of block/dictionary sizes while reducing
threads to accoodate. See commit message.
lrzip-next: January 2023 v0.10.1
New bzip3 block size settings. Use 0-8 for compression levels
1-9. 32MB-511MB
lrzip-next: January 2023 v0.10.0
Merge to main branch
lrzip-next: December 2022
bzip3 integration and doc updates
lrzip-next: November 2022 v0.9.3
Add --enable-debug switch to configure.
lrzip-next: November 2022 v0.9.3
Remove printf in code. Only use print macros in lrzip_private.h
lrzip-next: October 2022 v0.9.3
Add bzip3 support with -B option - Kamila Szewczyk
lrzip-next: August 2022 v0.9.2
lzma SDK 22.01
/tmp is now default TMP directory
Address error when lrzip-next run from write-protected dir
lrzip-next: July 2022 v0.9.1
BUGFIX: Fix magic read from STDIN
Add BSD support
lrzip-next: July 2022 v0.9.0
Beta is over. Now default branch
lrzip-next: June 2022 v 0.9.0beta
Add byte to store compression levels
Add comment facility
Minor lzma SDK 22.00 updates
lrzip-next: May 2022 v0.8.14
BUGFIX: fix yasm compiling (Issue #75).
Remove incorrect reference to 7zCrc.h which caused make dist to fail.
Remove doxygen documentation requirement (used only for liblrzip
which has been removed).
lrzip-next: May 2022 v0.8.13
Show error if trying to decrypt any earlier file <0.6x
Doc updates
Upstream bugfix
Don't try and show compression stats if using STDOUT
whats-next: April 2022
lrzip-next v0.8.12
Remove recursive (-r) option. No need anymore with `tar`.
Remove gzip compatibility options.
Remove lrzn link to lrzip-next.
Remove pod2man dependency and .pod files.
Add check for incompatible encrypted archive.
whats-next: March 2022
lrzip-next v0.8.11
Decouple from lrzip. All filenames unique. All man pages unique.
lrz --> lrzn
lrzcat --> lrzncat
lrztar --> lrzntar
lrzuntar --> lrznuntar
lrzunzip --> lrznunzip
LOCALE can be overridden in lrzip.conf
A user complained that using localized redix and separators
made confusing output. Now, a specific country locale can
be used or not.
LOCALE = DEFAULT (not necessary but will default to current locale)
LOCALE = NONE (ignore any localized settings)
LOCALE = locale (e.g. en_US, it_IT, en_AU)
Remove support for 32 bit architecture.
Update autofoo to replace obsolete macros.
Cleanup configure.ac. Remove rzip large file test from v 0.24
Remove support for lrzip archives < 0.6
lrzip-next v0.8.10
Upstream fixes, merge in whats-next.
whats-next: lrzip-next v0.8.9
Remove convoluted MD5 and HASH check defines.
!NO_HASH becomes HAS_HASH
Add multiple hash and encryption options
Use -H# option to set hash to one of
enum hashcodes {CRC=0, MD5, RIPEMD, SHA256, SHA384, SHA512, SHA3_256, SHA3_512,
SHAKE128_16, SHAKE128_32, SHAKE128_64, SHAKE256_16, SHAKE256_32, SHAKE256_64};
The SHAKE options are XOF functions using a variable output. These constants
use 16, 32, or 64 bytes.
and use -E# to set encryption to one of
enum enccodes {NONE=0, AES128, AES256};
lrzip-next v0.8.8
Add option zpaqbs which will allow user to override default
ZPAQ block size computation.
Some fixups to stream.c:open_stream_out to properly compute
zpaq overhead size.
lrzip-next v0.8.7
LZMA SDK 21.06
BUGFIX release, LzmaEnc.c:RangeEnc_Flush(), GetOptimum()
Beta removed.
lrzip-next v0.8.6
LZMA SDK 21.04beta
Minor updates plus addition of file LzFindOpt.c which will be
used when no Assembler is used. Contains the function
GetMatchesSpecN_2().
Merge in stream.c changes from stream_buffsize_fix branch.
lrzip-next v0.8.5
ZPAQ compression fixed. Imposed compression block size limits.
Set Type to pass to backend based on Redundancy estimate.
Type 0 = binary, type 1 = text when return from lz4 is 25% or less.
No type 3 since e8e9 is not needed since there is an --x86 etc filter.
lrzip-next v0.8.4
Remove lzma CRC code. Use libgcrypt for CRC hashing.
Allow Apple to compute MD5 hash. Revert 2011 restriction.
Update autogen.sh and util/gitdesc.sh to use /bin/env bash
to improve compatibility to MacOS and others.
Remove ASM compatility check for 32 bit x86 systems. 64 bit only.
Update and cleanups to configure.ac
lrzip-next v0.8.3
Use ` printf flag to print thousands separators.
Improve info format. Align columns.
Replace printf %lld, llum ld, lu with PRId64,d32,u32
Enhance compatibility of gitdesc.sh for MacOS and small systems.
Thanks Lucas Rademaker
lrzip-next v0.8.2
Store ZPAQ compression level and block size in magic header.
See write_magic() function and magic.header.txt for details.
lrzip-next 0.8.0
SCRYPT Key Derivation. Breaks ability to read older encrypted files.
lzma finder ASM module (x86_64 only)
LZMA 21.03beta SDK
All new Magic Header. 18 bytes vs 24.
lrzip-next 0.7.60
Get file info from encrypted files using lrzip-next -i.
lrzip-next 0.7.55
Use libgcrypt for mdd5, sha4, aes.
Update how encryption command-line options are interpreted
lrzip-next 0.7.50
New method for adjusting LZMA Dictionary sizes. Follow the
LZMA2 paradigm, See open_stream_out() function. See Wiki page
Update on LZMA Compression Memory Usage for details.
lrzip-next 0.7.43
Update util/gitdesc.sh so that it can compile even if a tarball
is downloaded. Utilize a VERSION file to provide Major, Minor, and
Micro versions to configure.ac.
Thanks to Jim Garlick's scrub application for inspiration.
Optimize open_stream_out() function to expand compression buffer
computation. Use simplified method of reducing dictionary size
when necessary. If dictSize too large, reduce as follows:
if dictSize == 2^n, reduce to 3^(n-1);
if dictSize == 3^n, reduce to 2^n;
This is the lzma2 method.
lrzip-next 0.7.42
We do know file size even when coming from STDIN or tar -I
Static Binaries available: https://peterhyman.com/download/lrzip-next.
lrzip-0.700
New configure process. Version taken from git-describe.
LZMA SDK 19.00
Add x86, BCJ, ARM, ARMT, PPC, SPARC, IA64 and Delta Filters.
Update Magic Header. Need one extra byte for Filtering.
Filters will work with all compression modes.
Add code to test for lrzip versions < 0.700.
Incudes code for lzma and lzma2 direct from SDK. Multi Threading still
dependent on p7zip code with regression to SDK 16.02.
Multi Threading for decompression is not yet implemented.
Update decompression output to display every 5 seconds or so.
Memory and block size computations now favor larger blocks
and larger dictionary sizes for better pre-processing and to give
compression backends more to work with.
Removal of scaling for lzma levels.
Larger blocks in lrzip files may result in faster compression and
decompression times and smaller output files.
Add Dictionary size setting --dictsize.
Add Dictionary size setting to lrzip.conf.
Dictionary size now ranging from 2^12 (16384) to 2^30 (1G).
Dictionary will be set automatically for all levels if not specified.
Dictionary size will be reduced in some cases where ram overhead
for LZMA exceeds allotted memory.
Update ZPAQ to 7.15.
Add minilzo to source tree.
Updated help listing and options summary messages to make more
logical.
Add Assembler Decompression code. Thanks to Igor Pavlov and
Conor McCarthy for their help in debugging. 30%+ speed increase.
lrzip-0.631
Assembler code is back and works with x86_64
lrzip-0.621
Substantial speed ups for the rzip stage in both regular and unlimited modes.
Lrzip now supports long command line options.
Proper support for the various forms of TMPDIR environment variables.
More unix portability fixes.
OSX fixes.
Fixed order of lrzip.conf search.
Addressed all warnings created with pedantic compiler settings and clang
Fixes for some stderr messages being swallowed up.
Fixed being unable to decompress to STDOUT when in a non-writable directory.
Changed broken liblrzip callback function API to match lrzip proper.
lrzip-0.620
Fixes display output of lrzip -i for large files greater than one chunk.
Fixes for various failure to allocate memory conditions when dealing with
large files and STDIO.
Fixes for more unix portability.
Fixes for failure to decompress to STDOUT.
lrzip-0.616
Fixes for various issues with -O not working with trailing slashes and
outputting to directories that already exist.
lrzip-0.615
Fixed -O not working on lrztar.
Made it less likely to run out of ram when working with STDIN/OUT.
Fixed running out of ram when using -U on huge files.
Fixed corrupt archives being generated from incompressible data.
Fixed corrupt archives being generated from very small files.
Fixed endianness on various platforms for MD5 calculation to work.
Fixed rare corruption when compressing with lzma from STDIN.
Fixed all blank data being generated when compressing from STDIN on OSX.
Performance micro-optimisations.
Fixed corrupt archive being generated when all the same non-zero bytes exist on
large files.
lrzip-0.614
Fixed lrztar not working.
lrzip-0.613
Fixed the bug where massive files would show an incorrect md5 value on
decompression - this was a bug from the md5 code upstream.
Compressing ultra-small files to corrupt archives was fixed.
Compilation on various other platforms was fixed.
A crash with using -S was fixed.
lrzip-0.612
Updated to a new zpaq library back end which is faster and now supports three
different compression levels, which will be activated at lrzip levels -L 1+, 4+
and 8+. This significantly increases the maximum compression available by lrzip
with -L 9.
The include file Lrzip.h used by liblrzip is now properly installed into
$prefix/include.
lrzip-0.611
lrzcat and lrzuntar have been fixed.
The update counter will continue to update even when there is nothing being
matched (like a file full of zeroes).
Numerous optimisations in the rzip stage speeds up the faster compression modes
noticeably.
Checksumming is done in a separate thread during rzip compression for more
compression speed improvements.
lrzip-0.610
The new liblrzip library allows you to add lrzip compression and decompression
to other applications with either simple lrzip_compress and lrzip_decompress
functions or fine control over all the options with low level functions.
Faster rzip stage when files are large enough to require the sliding mmap
feature (usually >1/3 of ram) and in unlimited mode.
A bug where multiple files being compressed or decompressed from the one
command line could have gotten corrupted was fixed.
Modification date of the decompressed file is now set to that of the lrzip
archive (support for storing the original file's date would require modifying
the archive format again).
Compilation warning fixes.
Make lrztar work with directories with spaces in their names.
lrzip-0.608
Faster rzip stage through use of a selective get_sb function.
The bash completion script is no longer installed by default to not conflict
with distribution bash completion packages.
More compilation fixes for non-linux platforms.
lrzip-0.607
A rare case of not being able to decompress archives was fixed.
The lzma library was updated to version 920.
A bash completion script for lrzip was added.
More debugging info was added in maximum verbose mode.
Less messages occur without verbose mode.
FreeBSD and posix compilation fixes were committed.
lrzip-0.606
lrzuntar, which broke last version leaving behind an untarred .tar file, is
working properly again.
lrzip-0.605
Addition of lrzcat - automatically decompresses .lrz files to stdout.
lrzip and lrunzip will no longer automatically output to stdout due to
addition of lrzcat executable, and to be consistent with gzip.
lrzip progress output will no longer spam the output unless the percentage
has changed.
lrzip now has no lower limit on file sizes it will happily compress and is
able to work with zero byte sized files.
The percentage counter when getting file info on small files will not show
%nan.
The executable bit will not be enabled when compressing via a means that
can't preserve the original permissions (e.g. from STDIN).
lrzip-0.604
lrzip will no longer fail with a "resource temporarily unavailable" error
when compressing files over 100GB that require hundreds of threads to
complete.
lrzip-0.603
lrzip now supports stdout without requiring the '-o -' option. It detects when
output is being redirected without a filename and will automatically output to
stdout so you can do:
lrunzip patch-2.6.38.4.lrz | patch -p1
Apple builds will not have errors on compressing files >2GB in size which
broke with 0.600.
lrztar will properly support -o, -O and -S.
lrzip.conf file now supports encryption.
lrzip will now warn if it's inappropriately passed a directory as an argument
directly.
lrzip-0.602
Fixed wrong symlinks which broke some package generation.
Imposed limits for 32bit machines with way too much ram for their own good.
Disable md5 generation on Apple for now since it's faulty.
Displays full version with -V.
Checks for podman on ./configure
Now builds on Cygwin.
File permissions are better carried over instead of being only 0600.
lrzip-0.601
lrzuntar, lrunzip symlinks and the pod-based manpages are installed again.
Configuration clearly shows now that ASM isn't supported on 64bit.
lrzip-0.600
Compressing/decompressing to/from STDIN/STDOUT now works without generating
any temporary files. Very large files compressed in this way will be less
efficiently compressed than if the whole solid file is presented to lrzip,
but it is guaranteed not to generate temporary files on compression.
Decompressing files on a machine with the same amount of ram will also not
generate temporary files, but if a file was generated on a larger ram machine,
lrzip might employ temporary files, but they will not be the full size of the
final file.
Decompression should now be faster as the rzip reconstruction stage is mostly
performed in ram before being written to disk, and testing much faster.
Final file sizes should be slightly smaller as block headers are now also
compressed.
Heavy grade encryption is now provided with the -e option. A combination of
a time scaled multiply hashed sha512 password with random salt followed by
aes128 block encryption of all data, including the data headers, provides for
extremely secure encryption. Passwords up to 500 characters in length are
supported, and the same file encrypted with the same password is virtually
guaranteed to never produce the same data twice. All data beyond the basic
lrzip opening header is completely obscured. Don't lose your password!
Lrzip will not try to malloc a negative amount of ram on smaller ram machines,
preferring to decrease the number of threads used when compressing, and then
aborting to a nominal minimum.
A new build configuration system which should be more robust and provides
neater output during compilation.
lrzip should work again on big endian hardware.
lrztar / lrzuntar will no longer use temporary files.
lrzip-0.571
Avoid spurious errors on failing to mmap a file.
Fee space will now be checked to ensure there is enough room for the
compressed or decompressed file and lrzip will abort unless the -f option is
passed to it.
The extra little chunk at the end of every large file should now be fixed.
The file lzma.txt now has unix end-of-lines.
There will be a more accurate summary of what compression window will be used
when lrzip is invoked with STDIN/STDOUT.
STDIN will now be able to show estimated time to completion and percentage
complete once lrzip knows how much file is left.
Temporary files are much less likely to be left lying around.
Less temporary file space will be used when decompressing to stdout.
File checking will not be attempted when it's meaningless (like to stdout).
Times displayed should avoid the nonsense thousands of seconds bug.
lrzip-0.570
Multi-threaded performance has been improved with a significant speed-up on
both compression and decompression. New benchmark results have been added to
the README.benchmarks file.
Visual output has been further improved, with an updated help menu and no
unrelated system errors on failure.
lrzip.conf supports the newer options available.
TMP environment is now respected when using temporary files and TMPDIR can be
set in lrzip.conf.
LRZIP=NOCONFIG environment variable setting can be used to bypass lrzip.conf.
The -M option has been removed as the -U option achieves more and has
understandable semantics.
Memory usage should be very tightly controlled on compression now by default,
using the most possible without running out of ram.
Temporary files generated when doing -t from stdin will no longer be left lying
around.
lrzip will no longer stupidly sit waiting to read from stdin/stdout when called
from a terminal without other arguments.
Executable size will be slightly smaller due to stripping symbols by default
now.
The -T option no longer takes an argument. It simply denotes that lzo testing
should be disabled.
Verbose added to -i now prints a lot more information about an lrzip archive.
lrzip-0.560
Implemented OSX multi-threading by converting all semaphores to pthread_mutexes.
Converted the integrity checking to also use md5 hash checking. As a bonus it
is still backwardly compatible by still storing the crc value, and yet is
faster on large files than the old one. On decompression it detects whether
the md5 value has been stored and chooses what integrity checking to use.
Implemented the -H feature which shows the md5 hash value on compression and
decompression. It is also shown in max verbose mode.
Added information about what integrity testing will be used in verbose mode,
and with the -i option.
Added the -c option which will perform a hash check on the file generated on
disk on decompression, comparing it to that from the archive to validate the
decompressed file.
Modified lrzip to delete broken or damaged files when lrzip is interrupted or
the file generated fails an integrity test.
Added the -k keep option to keep broken or damaged files.
Case reports of corruption have been confirmed to NOT BE DUE TO LRZIP.
lrzip-0.552
Fixed a potential silent corruption bug on decompression.
Fixed compilation on freebsd.
Fixed failures on incompressible blocks with bzip2 or gzip.
Fixed osx failing to work. It does not support threaded compression or
decompression but should work again.
lrzip-0.551
Compressing from stdin should be unbroken again.
Compression values returned at the end of stdin work.
lzma failing to compress a block will not cause a failure.
lrzip-0.550
Speed up compression on large files that take more than one pass by overlapping
work on successive streams, thus using multiple CPUs better.
Fix for failures to decompress large files. Decompression will be slightly
slower but more reliable.
Faster lzma compression by default, less prone to memory failures, but at slight
compression cost.
Recover from multithreaded failures by serialising work that there isn't enough
ram to do in parallel.
Revert the "smooth out spacing" change in 0.544 as it slowed things down instead
of speeding them up.
Larger compression windows are back for 32 bits now that memory usage is kept
under better control.
Fixed some memory allocation issues which may have been causing subtle bugs.
lrzip-0.544
Hopefully a fix for corrupt decompression on large files with multiple stream 0
entries.
Fix for use under uclibc.
Fix for memory allocation errors on large files on 32 bits.
Smooth out spacing of compression threads making better use of CPU on compress
and decompress.
Fix for using -U on ultra-small files.
Use bzip2 on blocks that lzma fails to compress to make sure they are still
compressed.
lrzip-0.543
A fix for when large files being decompressed fail with multithreaded
decompression.
Slight speedup on multithreaded workloads by decreasing the nice value of the
main process compared to the back end threads as it tends to be the rate
limiting component.
Fixed lzma compression windows being set way too small by default.
lrzip-0.542
Lrzip will now try to select sane defaults for memory usage in cases where the
virtual memory heavily overcommits (eg. Linux) as this seriously slows down
compression.
For compression windows larger than 2/3 ram, lrzip will now use a sliding mmap
buffer for better performance.
The progress output is more informative in max verbose mode, and will no longer
do more passes than it estimates.
32 bit machines should be able to use slightly larger windows.
The sliding mmap not working on 2nd pass onwards has been fixed which should
speed up the slowdown of death.
lrzip-0.540
MASSIVE MULTITHREADING on the decompression phase. Provided there are enough
chunks of data in the archived file, lrzip will use as many threads as there
are CPUs for the backend decompression. Much like the multithreading on the
compression side, it makes the slower compression algorithms speed up the most.
Fixed output from being scrambled and consuming a lot of CPU time on threaded
zpaq compression.
Further fixes to ensure window sizes work on 32 bit machines.
Be more careful about testing for how much ram lrzip can use.
Minor build warning fixes.
Minor tweaks to screen output.
Updated benchmarks.
lrzip-0.530
MASSIVE MULTITHREADING on the compression phase. Lrzip will now use as many
threads as you have CPU cores for the back end compression, and even continue
doing the rzip preprocessing stage as long as it can which the other threads
continue. This makes the slower compression algorithms (lzma and zpaq) much
faster on multicore machines, to the point of making zpaq compression almost
as fast as single threaded lzma compression.
-p option added to allow you to specify number of processors to override the
built-in test, or if you wish to disable threading.
-P option to not set permissions has now been removed since failing to set
permissions is only a warning now and not a failure.
Further improvements to the progress output.
Updated benchmarks and docs.
lrzip-0.520
Just changed version numbering back to 2 point.
lrzip-0.5.2
Fixed the Darwin build again.
Fix the corner case of big ram usage on 32 bit zpaq failing due to the
compression window not being limited by limiting zpaq to 600MB windows on 32
bits as well.
Some previous failures now only induce warnings.
Improved progress output.
lrzip-0.5.1
Fixed the build on Darwin.
Rewrote the rzip compression phase to make it possible to use unlimited sized
windows now, not limited by ram. Unfortunately it gets progressively slower in
this mode the bigger the file gets but you can compress a file of any size as
one big compression window with it using the new -U option. Suggest you try
the new improved -M mode first or in combination. See the docs for more
information.
Changed the memory selection system to simply find the largest reasonable sized
window and use that by default instead of guessing the window size.
Setting -M now only affects the window size, trying to find the largest
unreasonably sized window that will still work.
The default compression level is now 9 and affects the rzip compression stage
as well as the backend compression.
Fixed some potential failures during compression.
Improved screen output with more reporting in verbose mode, and chunk size
percentage update.
Fixed file size reporting on compressed files generated from stdin.
Changed to 3 point releases in case we get more than 9 subversions ;)
lrzip-0.50
Rewrote the file format to be up to 5% more compact and slightly faster.
Made the memory initialisation much more robust, with attempted fallback
to still work even when initial settings fail.
Updated a lot of the stdin code.
The most common scenario of compression from stdin now works without
temporary files.
Lots more meaningful warnings if failure occurs.
May be able to decompress files on 32 bit machines that were compressed on 64
bit machines with >2GB windows now if there is enough ram.
lrzip-0.46
Added lrzuntar which works the same as lrztar -d.
Con Kolivas
May 2010
lrzip-0.45
Added docs for lrztar and lrunzip.
Added distclean and maintainer-clean make targets.
Created git repo: http://github.com/ckolivas/lrzip
Con Kolivas
March 2010
lrzip-0.44
Added an lrztar wrapper to compress / decompress whole directories (finally).
Added -i option to give information about a compressed file.
lrzip-0.43
Darwin support updated. Should build on OSX v10.5+
Finally, stdin/stdout support.
Test archive integrity support.
ZPAQ support in config files.
lrzip-0.42
ZPAQ compression update now shows which rzip stream it's currently compressing
making the update more useful. It also doesn't update unnecessarily with every
byte compressed which was slowing it down a LOT.
lrzip-0.41
ZPAQ compression backend! ZPAQ is from the family of "paq" compressors that
have some of the best compression ratios around, but at the cost of extremely
long compression and equally long decompression times. This can be enabled
with the -z option and makes lrzip archives made with this not backwardly
compatible.
lrzip-0.40
Compression windows should be limited by available ram now on 64bit. The limit
on 32bit is still 2GB.
The compression advantages on large files on 64bit machines with large ram
should be substantially better.
The file format is no longer compatible with earlier versions of lrzip.
Support for decompressing older formats is present, but all new files will
be generated in the new format.
Minor speedups.
Decompression should no longer stall at 4GB boundaries for extended periods
making decompression much faster on files >4GB in size.
Documentation and benchmark updates galore.
lrzip-0.31
The window size limit is now 2GB on both 32bit and 64bit. While it appears to be
smaller than the old windows, only 900MB was being used on .30 even though it
claimed to use more. This can cause huge improvements in the compression of very
large files. Flushing of data to disk between compression windows was
implemented to minimise disk thrashing of read vs write.
Con Kolivas
November 2009
lrzip-0.30
-P option to not set permissions on output files allowing you to write to
braindead filesystems (eg fat32).
Probably other weird and wonderful bugs have been introduced.
Con Kolivas
November 2009
lrzip-0.24 has updated functionality
FEATURE ENHANCEMENTS
lrzip.conf file may be used to set default parameters.
Omit conf using environment: LRZIP=NOCONFIG lrzip.....
LRZIP environment variable may be used in the future
to store certain types of parameters.
LZMA SDK has been upgraded to version 4.63. This
version fixes some problems certain users observed,
and is much simpler using a C-only wrapper
interface.
lrzip now is able to compute an ETA for completion.
In order to do this, the file must be larger than
one compression window in size. That is, is the
compression window is 500MB, and the file is 1GB,
then after the first pass, an ETA will be computed.
If the file is smaller, then no estimate can be made.
lrzip is now able to compute MB/s transfer speeds
for both compression and decompression.
CLEANUPS
Some file cleanups have been done.
Peter Hyman
January 2009
lrzip-0.22 update
FEATURE ENHANCEMENTS
-g option. Now supports gzip compression. Very fast!
Expanded dictionary buffer size in lzma compressor.
Variable, expanded dictionary size buffer in both lzma
compressor and decompressor.
Improved output during compression when using -vv.
Multi-threading supprt when using multiple processors
or dual core processors when using lzma compression.
This results in a nearly 2x speed improvement.
Assembler module support to speed up CRC checking.
Improvements in autotools usage, system detection
and Makefile enhancements.
Lrzip now has a timer that will print total time
at the end of a compression or decompression if
-q command line option is not used.
BUG FIX!!!
Even though lrzip uses a compression threshold to
prevent the lzma compressor from getting data that
may not be compressible, there was still a possibility
that lrzip could hang. This was because a data chunk
could contain an uncompressible segment and if the
lzma compressor got it, it would hang.
THANKS TO LASSE COLLIN for uncovering the error in
the lzma wrapper code that was causing the hangup.
January 2008
Peter Hyman