[BBLISA] Compression Algorithms

[Edward Ned Harvey]

I realized I screwed up my lzop test.  Here is an update:

 

------------------

   LZO (lzop) compression level 1

78M   0m1.820s

------------------

   LZO (lzop) compression level 5

77M   0m1.913s

------------------

   LZO (lzop) compression level 9

71M   1m21.792s

 

 

 

 

 

From: bblisa-bounces at bblisa.org [mailto:bblisa-bounces at bblisa.org] On Behalf
Of Edward Ned Harvey
Sent: Tuesday, June 29, 2010 9:16 AM
To: bblisa at bblisa.org
Subject: [BBLISA] Compression Algorithms

 

I decided to benchmark some compression algorithms on our actual data.
Results are attached.

 

Summary:

.         LZO is the fastest.

.         7-zip (p7zip) produces the best compression ratio.  (Overall, it's
the most impressive.)  Doesn't seem possible to use it as an in-line filter.

.         gzip --fast is the most widely available.  Its speed is same order
of magnitude with LZO, and its compression is same order of magnitude as
7-zip, but it's unquestionably beat by those others, in their natural
habitat.  So for general purpose, gzip --fast is the most likely to be used
in general.

.         bzip2 is soundly destroyed by 7-zip, if only 7-zip is made
available as an in-line filter.  Presently, in-line filtering is the only
reason to ever use bzip2 instead of 7-zip.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.bblisa.org/pipermail/bblisa/attachments/20100629/90e0f33f/attachment.htm