LZ4 - Extremely fast compression

LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It also features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems. A high compression derivative, called LZ4_HC, is also provided. It trades CPU time for compression ratio.

Branch	Status
master
dev

Branch Policy:

• The "master" branch is considered stable, at all times.
• The "dev" branch is the one where all contributions must be merged before being promoted to master.
  • If you plan to propose a patch, please commit into the "dev" branch. Direct commit to "master" are not permitted.
• Feature branches can also exist, for dedicated testing of larger modifications before merge into "dev" branch.

Benchmarks

The benchmark uses the Open-Source Benchmark program by m^2 (v0.14.2) compiled with GCC v4.6.1 on Linux Ubuntu 64-bits v11.10, The reference system uses a Core i5-3340M @2.7GHz. Benchmark evaluates the compression of reference Silesia Corpus in single-thread mode.

Compressor	Ratio	Compression	Decompression
LZ4 (r101)	2.084	422 MB/s	1820 MB/s
LZO 2.06	2.106	414 MB/s	600 MB/s
QuickLZ 1.5.1b6	2.237	373 MB/s	420 MB/s
Snappy 1.1.0	2.091	323 MB/s	1070 MB/s
LZF	2.077	270 MB/s	570 MB/s
zlib 1.2.8 -1	2.730	65 MB/s	280 MB/s
LZ4 HC (r101)	2.720	25 MB/s	2080 MB/s
zlib 1.2.8 -6	3.099	21 MB/s	300 MB/s

The LZ4 block compression format is detailed within lz4_block_format.txt.

For streaming unknown amount of data and compress files of any size, a frame format has been published, and can be consulted within the file LZ4_Frame_Format.html .