Did you know that you can navigate the posts by swiping left and right?
xz compression program, run with defaults, can be so slow that you might wonder why anybody uses it. But on a modern machine,
xz -0 --threads=0 will run with less aggressive compression goals and use all available threads on a multi-core CPU, exceeding gzip performance by a good measure. One nice bonus with the xz format is that individual archives can be concatenated.
cat file*.xz > all-files.xz without decompressing first, and then unarchive in one step - nifty! You can even null-pad the space between concatenated files if you need to block-align the archive for random access on slow media. You might consider setting your XZ_DEFAULTS environment variable to use these higher performance options.
Let’s look at compressing this Postscript file:
$ ls -lh certificate5896072-3.ps -rw-r--r-- 1 user user 30M Nov 20 2012 certificate5896072-3.ps $ time gzip -9 -k certificate5896072-3.ps real 0m1.828s user 0m1.697s sys 0m0.029s $ ls -lh certificate5896072-3.ps.gz -rw-r--r-- 1 user user 28M Nov 20 2012 certificate5896072-3.ps.gz $ time xz -0 --threads=0 certificate5896072-3.ps real 0m1.250s user 0m8.781s sys 0m0.077s $ ls -lh certificate5896072-3.ps.xz -rw-r--r-- 1 user user 27M Nov 20 2012 certificate5896072-3.ps.xz $ time gzip -1 -k certificate5896072-3.ps real 0m1.365s user 0m1.330s sys 0m0.031s $ ls -lh certificate5896072-3.ps.gz -rw-r--r-- 1 user user 28M Nov 20 2012 certificate5896072-3.ps.gz $ time xz certificate5896072-3.ps real 0m1.974s user 0m9.843s sys 0m0.122s $ ls -lh certificate5896072-3.ps.xz -rw-r--r-- 1 user user 26M Nov 20 2012 certificate5896072-3.ps.xz $ time xz -9 --threads=0 certificate5896072-3.ps real 0m13.182s user 0m12.980s sys 0m0.190s $ ls -lh certificate5896072-3.ps.xz -rw-r--r-- 1 user user 23M Nov 20 2012 certificate5896072-3.ps.xz $ time xz -6 --threads=1 certificate5896072-3.ps real 0m13.615s user 0m13.494s sys 0m0.081s $ ls -lh certificate5896072-3.ps.xz -rw-r--r-- 1 user user 24M Nov 20 2012 certificate5896072-3.ps.xz
As you can see, the fastest xz exceeds the compression ratio of the slowest gzip and the speed of the fastest gzip, by using multiple cores (test machine is an 8-thread/4-core portable i7). Allowing slightly more time than gzip gives slighly better compression, and allowing significantly more time than gzip yields marginally better compression still. But note the last test - with the xz defaults, the time is the worst and the compression isn’t the best. In that case, compression is more than 10% better but the time is 10 times worse. These defaults will only hinder the adoption of xz, which is a shame because faster defaults would get more users and already provide better results than gzip on typical workloads on modern CPU’s.