Last active
January 27, 2020 04:16
-
-
Save sloppycoder/f20f9c030df2a8fcb506964f87c465cd to your computer and use it in GitHub Desktop.
large file parallel compression test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| centos6 is a directory of a VirtualBox VM, size 8689088000 | |
| command for compression | |
| time tar --use-compress-program=pbzip2|lzop -cf centos6_oracle11r2.tar.xxx centos6 | |
| time lzrtar centos6 | |
| command for decompresison | |
| time pbzip2|bzip2|lzop -cd centos6_oracle11r2.tar.xxx | tar tvf - | |
| time lrzip -o - -d centos6.tar.lrz | tar tvf - | |
| Size Compress Decompress | |
| 2930396334 Sep 11 23:37 centos6_oracle11r2.zip 302 58 <-- this is the reference | |
| 3708523626 Sep 11 23:56 centos6_oracle11r2.tar.lzo 36 21 | |
| 2710537316 Sep 11 23:42 centos6_oracle11r2.tar.bz2 213 73 (bzip2: 325) | |
| 1842342646 Sep 12 00:42 centos6.tar.lrz 519 750!! | |
| hardware: | |
| Macbook Pro Retina 2012, quad core 2.0G / 16G, original 256G Samsung SSD | |
| versions: | |
| macOS 10.12.6 | |
| homebrew 1.3.2, updated 2017-09-12 | |
| This is Zip 3.0 (July 5th 2008), by Info-ZIP. | |
| lrzip version 0.631 | |
| pbzip2 Parallel BZIP2 v1.1.13 [Dec 18, 2015] | |
| bzip2, a block-sorting file compressor. Version 1.0.6, 6-Sept-2010. | |
| lzop 1.04 LZO library 2.10 | |
| related post: | |
| https://askubuntu.com/questions/258202/multi-core-compression-tools | |
| Update 2020-01-27 | |
| MSI GF63 RCX quad core i5-9300H, 2.4G, 24G ram, 512G SSD | |
| xz (XZ Utils) 5.2.2 | |
| liblzma 5.2.2 | |
| l2win.vdi 26,027,753,472 bytes | |
| time ( tar -cf - l2win | xz --threads=4 > l2win_20200127_vbox6.0.16.tar.xz ) | |
| openssl md5 l2win_20200127_vbox6.0.16.tar.xz | |
| tar.gz is 7,810,053,656 bytes, 22 mins. | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment