Lossless Photo Compression Benchmark

updated on 27-Sep-2012
(c) Alex Rhatushnyak

The results presented in this benchmark are free for scientific, commercial or private use, under the sole restriction of mentioning the source (this page).
All images are 24-bit RGB, pages for 8-bit, 16, 32 and 48-bit images may be added later.
Notes:
Click on compressed sizes for details.
Bold value means that no compressor with higher quality is faster.


Compressed Compression Decompression Command line size, bytes, % time, seconds time, seconds (program and options) 971986179 100.00 1734.48 1897.09 Gralic c %1.gra111 %1.ppm 995740780 102.44 11724.74 3051.34 bmf -s -q9 %1.ppm 996947712 102.56 3678.03 3038.85 bmf -s %1.ppm 1012198342 104.13 14338.27 14436.08 zpaq -mbmp_j4 c %1 %1.bmp 1024282375 105.38 34390.37 34125.79 paq8im -7 %1.ppm 1038434818 106.83 209.85 231.50 Flic c %1.flic21 %1.ppm 1078284880 110.93 ~1410 ~1455 (StuffIt Deluxe 2010) console_stuffEN.exe /c -o --quiet %1.bmp 1089342602 112.07 3853.10 2057.10 rkim cx %1.tga %1.rkx 1131735795 116.43 1756.70 1861.66 uhic m %1.bmp %1.uhm 1139392552 117.22 417.91 269.95 bmf %1.ppm Top 10 entries - more details, percentage 1141690596 117.45 1390.06 1635.29 rkim c %1.tga %1.rki 1152493991 118.57 2502.29 566.23 (Pyramid Workshop) cbtpc -lv 8 -p 5 -c 65 %1.ppm %1.py5 1154592106 118.78 884.11 966.50 uhic e %1.bmp %1.uhi 1162946359 119.64 2198.68 578.54 nk /s %1.bmp 1179957421 121.39 832.30 167.29 bcif %1.bmp -c %1.bci 1224419506 125.97 1771.30 499.69 (lossy) dli c -q 100 -qc 80 -ac+ %1.ppm %1.dli 1232773556 126.83 ~7525 109.91 cWebP -lossless -m 4 %1.png -o %1.wp4 1236414764 127.20 ~6127 110.76 cWebP -lossless -m 0 %1.png -o %1.wp0 1239884576 127.56 173.31 230.86 lsp %1.ppm 1245914178 128.18 176.91 66.93 (lossy JPEG) NConvert -out jpeg -q 100 -subsampling 2 -opthuff %1.ppm 1247343688 128.32 381.56 373.81 (LuraWave JPEG-2000) NConvert -out lwfjpc %1.ppm 1247354751 128.33 327.75 276.01 (Kakadu) kdu_compress -i %1.ppm -o %1.kdu Creversible=yes -quiet 1247389133 128.33 605.57 554.72 JasPer --input %1.ppm --input-format pnm --output %1.jpr --output-format jpc 1247396798 128.33 1849.07 1674.81 (OpenJPEG) image_to_j2k -i %1.ppm -o %1.j2k 1275559788 131.23 1269.50 1140.48 NanoZip a -m1g -cc %1.nz1 %1.ppm 1290281728 132.74 448.06 332.06 PGF -e %1.ppm %1.pgf 1293857955 133.11 188.51 205.06 (JPEG-LS) locoe %1.ppm -o%1.loc 1293858918 133.11 107.02 92.53 (JPEG-LS) CharLS -encodepnm %1.ppm %1.chs 1299728137 133.71 138.23 170.08 libra8 c %1.ppm %1.lr8 0 1301083155 133.85 131.05 179.92 lsp -f %1.ppm 1308370225 134.60 700.03 419.72 (Pyramid Workshop) cbtpc %1.ppm %1.pyr 1320702831 135.87 173.20 235.26 lsp -g %1.ppm 1327774201 136.60 3870.06 3456.27 uharc a -mx -md32768 %1.uha-mx %1.ppm 1328708769 136.70 3700.69 3313.41 uharc a -mx -md- %1.uha-mx- %1.ppm 1331109154 136.94 1664.23 333.05 capt %1.ppm %1.apt 100 1338137564 137.67 1067.99 1135.89 packPNM_10e %1.ppm 1339149175 137.77 4275.11 501.06 uharc a -m3 -md32768 %1.uha-m3 %1.ppm 1341192869 137.98 4110.98 501.49 uharc a -m2 -md32768 %1.uha-m2 %1.ppm 1342696887 138.13 3758.73 501.86 uharc a -m1 -md32768 %1.uha-m1 %1.ppm 1348855983 138.77 140.09 67.53 (lossy JPEG) NConvert -out jpeg -q 100 -subsampling 2 %1.ppm 1349164676 138.80 ~696 ~768 GeoExpress 8 1354406543 139.34 979.37 513.97 uharc a -m3 -md- %1.uha-m3- %1.ppm 1354418303 139.34 977.99 513.83 uharc a -m2 -md- %1.uha-m2- %1.ppm 1354534276 139.35 872.79 513.88 uharc a -m1 -md- %1.uha-m1- %1.ppm 1403325039 144.37 330.28 206.08 NanoZip a -m1g -cD %1.nz2 %1.ppm 1415606383 145.64 446.11 369.23 (JPEG-XR reference software draft) JPEGXR -c %1.ppm -o %1.jxr 1422849993 146.38 371.13 335.65 (JPEG-XR) WMPEncApp -i %1.ppm -o %1.wdp 1432319902 147.36 27819.14 68.47 (optimized PNG) pngout /c2 /f5 %1.png 1449033110 149.07 131.96 160.61 arc a -mmm:3*8+grzip:m4:l %1.arc-grz %1.ppm 1459489679 150.15 429.20 487.58 uharc a -mz -md32768 %1.uha-mz %1.bmp 1471885647 151.43 844.37 72.62 (regular PNG) NConvert -out png %1.ppm 1495870625 153.89 266.29 86.48 (Lossless JPEG) lj-enc %1.ppm %1.ljp 1554200521 159.89 555.41 67.58 rar a -m5 %1.rar5 %1.ppm 1556783126 160.16 518.56 67.55 rar a -m3 %1.rar3 %1.ppm 1567733233 161.29 534.91 90.86 rar a -m5 -mcc+ %1.rar5p %1.ppm 1583937825 162.95 560.65 67.94 rar a -m5 %1.rar5b %1.bmp 1597320420 164.33 534.52 429.35 (LuraWave Format) NConvert -out lwf %1.ppm 1737745564 178.78 29.86 28.63 arc a -mmm:3*8+1xb %1.arc1xb %1.ppm 1742414838 179.26 1932.40 207.70 arc a -mx %1.arcx %1.ppm 1755332419 180.59 508.90 232.26 bzip2 -k -9 %1.ppm 1796117884 184.78 619.42 102.58 arc a %1.arc_ %1.ppm 1801111030 185.30 489.99 229.56 bzip2 -k -5 %1.ppm 1808959044 186.10 1426.18 138.43 7z a -mx %1.7mx %1.ppm 1813113393 186.53 1300.40 137.47 7z a %1.7z %1.ppm 1854900746 190.83 263.21 188.27 QBIT.exe QBIT %1.bmp 1940273041 199.61 277.22 76.29 arc a -m3 %1.arc3 %1.ppm 1950146903 200.63 476.15 218.26 bzip2 -k -1 %1.ppm 2008608845 206.64 547.34 82.69 rar a -m5 -mcc- %1.rar5- %1.ppm 2026386735 208.47 465.92 49.05 rar a -m3 -mcc- %1.rar3- %1.ppm 2254090100 231.90 106.67 43.11 arc a -m2 %1.arc2 %1.ppm 2366002376 243.41 186.50 50.02 rar a -m1 %1.rar1 %1.ppm 2635901832 271.18 1000+ ~165 MediaZip 3144733204 323.53 27.94 23.09 (Adobe Photoshop) NConvert -out psd %1.ppm 3145652200 323.63 39.24 73.97 (Explore (TDI) & Maya) NConvert -out tdi %1.ppm 3462571880 356.23 1.25 1.25 copy/b %1.ppm nul
About the test images. Six sets with 107 images totalling 3.46 Gb: 62 images (2254 Mb) from http://photographyblog.com and 45 images (1208 Mb) from http://photojournal.jpl.nasa.gov. More details on test data. History, comments and links 11 Apr 2011 - launched the benchmark, with 12 programs tested: GraLIC 1.9.demo, BMF 2.01, PAQ8im, FLIC 1.4.demo, BCIF 1.0 beta, Kakadu 6.4.1, JasPer 1.900.1, LOCO-I/JPEG-LS from HP Labs, packPNM 1.0d, regular PNG (created by NConvert 5.80), Lossless JPEG (Cornell University), RAR 4.00. Conversion to PPM and BMP formats was done by NConvert. JasPer failed to compress this large 139 Mb image (in PPM format), the file created by Kakadu was taken instead. JasPer and Lossless JPEG were compiled with GCC (version and options as in readme.txt in the FLIC package). All programs were tested on an Intel Core i7-930 CPU, 2.8 GHz, 3 Gb RAM, Windows XP. 12 Apr 2011 - added PGF Console Version 6.09.44 and PNGOUT Apr-10-2011 (hopefully no more that slow programs), compiled page Top 10 entries - more details. 13 Apr 2011 - added two programs that were competing in 1999 and 2000: RKIM 1.06 and UHIC 2.0. UHIC dealt with images in BMP format (and short 8.3 names), while for RKIM they were converted to TGA. UHIC fails to fully unpack olympu09.bmp compressed with command e. For measuring decompression time, olympu11.uhi was decompressed instead of olympu09.uhi. 14 Apr 2011 - added LSP, OpenJPEG 1.4, RAR -m5 -mcc+, -m3, -m1. 15 Apr 2011 - added Pyramid Workshop (ver. 21.03.2011 compiled by Niels Fröhling), RAR -m5 -mcc-, RAR -m3 -mcc-. PW with -lv 8 -p 5 -c 65 fails to compress two images, they were compressed with -lv 8 -p 5. With no options it fails to losslessly decompress two other images, but only three bytes differ. 16 Apr 2011 - added FreeArc 0.67 (March 18 2011), PAQ8im decompression time, compiled page Top 10 entries - percentage. 17 Apr 2011 - added Libra8 and ZPAQ 2.05 with bmp_j4 model. ZPAQ was recompiled (to turn on the optimization "o" feature for faster compression and decompression with config files) with this command: g++ -O3 -DNDEBUG -DOPT="\"g++ -O3 -DNDEBUG -march=pentiumpro -s -fomit-frame-pointer zpaq.cpp libzpaq.cpp zpaqopt.cpp -o zpaqopt.exe\"" zpaq.cpp libzpaq.cpp libzpaqo.cpp -o zpaq After you run it with command okcbmp_j4.cfg, it creates zpaqopt.exe, and then you run zpaqopt.exe instead of zpaq.exe. Output is bit-exact, but speed is much higher. 18 Apr 2011 - added 7-zip 9.20, BZIP2 1.0.6, HD Photo DPK 1.0. 7-zip exits with "Internal Error #1" if archive name is nul.ext, with "Unknown Error" if it's nul.exte, it says "Error: I won't write compressed data to a terminal" if you try 7z a -so archive image, and finally "System error: Not implemented" if you try 7z a -so archive image >redirect. Bzip2 was compiled with GCC. HD Photo DPK was compiled in Visual Studio 2008 - Release, no project settings were modified. If the extension of compressed image file is not .wdp, decompressor refuses to unpack the image, although the format name has changed twice: Windows Media Photo - HD Photo - JPEG XR. 19 Apr 2011 - added APT 1.0 and StuffIt 2010 ver. 14.0.0.18 (Compression Technology version 14.0.0.15). Compression time will be added later. 20 Apr 2011 - added NK2 and compression time for APT and StuffIt. Started testing DLI, first with -q 100 -qc 80 -ac. 21 Apr 2011 - DLI -or is extremely slow. Looks like it will take more than 333 hours to compress all 109 images. Started testing WinRK 3.1.2. 29 Apr 2011 - added WinRK 3.1.2. The only non command line program. Time is "Elapsed time" reported by WinRK itself. It crashes compressing the set of all 109 images with profile Best asymmetric. Does not crash if five sets are compressed separately. Decompression was bit-exact for all images, both tested profiles. DLI -or -q 100 -qc 80 -ac will finish compressing on Monday or Tuesday. 04 May 2011 - added WinRK with profile HIGH and CharLS v.64769 ( \trunk\charls\test\test.vcproj ). CharLS was compiled in Visual Studio 2008 - Release, no project settings were modified. WinRK with profile HIGH fails to decompress 41st image (canon_eos_600d_14) and all remaining images. 11 May 2011 - added FreeArc -mmm:3*8+1xb, FreeArc -mmm:3*8+grzip:m4:l, and two lossy algorithms: DLI 2.1 and JPEG, both at best possible quality. 13 May 2011 - UHARC 0.6b was tested, C- and D-time will be added later. 16 Jun 2011 - Added UHARC's compression time and decompression time. With -mz it was compressing images in BMP format, because on PPM files compressed size was 39% bigger. 21 Jun 2011 - NConvert 5.91 was released, it doesn't fail on raw images from Fujifilm FinePix X100 and Canon EOS 1100D, so everything* will soon be re-tested with the updated test set. When the most recent version 2.4 of Detector for ex-JPEGs is applied, 65 of 110 images from photojournal.jpl.nasa.gov are detected as ex-JPEGs (and discarded), but none of the 62 images from photographyblog.com. *Everything except WinRK, DLI in slow mode, Libra8 with options 1 and 2. 03 Jul 2011 - Everything was re-tested with the updated data set. Previous version of LPCB main page is here. This time more efforts were made to minimize the influence of HDD speed. New versions of programs this time: WinRAR 4.01, PGF 6.11.22, DLI 2.2, NConvert 5.91. Trial version of StuffIt Deluxe has expired, therefore only compression quality was tested (on a different computer), therefore times are approximate. Page More details on test data was updated. 13 Jul 2011 - Added FLIC 1.5.demo. 15 Jul 2011 - Added GraLIC 1.10.demo. 17 Jul 2011 - Compiled pages Top 10 entries - more details and Top 10 entries - percentage. 18 Jul 2011 - Added MediaZip 1.0. Because there is no command-line executable, and because in the GUI version you have to press OK after each file, timings are very approximate. If it compresses images in BMP format, compression quality is not as good as with PPMs, e.g. 309976499 => 310621406 bytes (PPM=>BMP) on the Canon set. Flag "Use maximum compression" was set in the options. If it is not, compressed size is ~4.5% bigger (the Canon set). 20 Jul 2011 - Updated GraLIC 1.10.demo and pages Top 10 entries/Percentage. 30 Jul 2011 - Added GraLIC 1.11.demo. 22 Aug 2011 - Added Qbit 1.0.0.10. It fails to losslessly decompress one file: canon_eos_1100d_11.qbt. 25 Aug 2011 - Added FLIC 1.6.demo. 03 Sep 2011 - Added FLIC 1.7.demo. 12 Sep 2011 - Added PGF 6.11.29, updated FLIC 1.7.demo. If you want to see the scripts used for testing software, please send an e-mail. 14 Sep 2011 - Added PNGOUT Jul 2 2011. 25 Sep 2011 - Added FLIC 1.8.demo. 03 Oct 2011 - Added JPEG-XR reference software draft (from Thomas Richter) 05 Oct 2011 - Added LuraWave Format, LuraWave JPEG-2000 (NConvert+plugins), PSD and TDI (just NConvert, no plugins). 11 Oct 2011 - Added FLIC 1.9.demo. 14 Oct 2011 - Updated FLIC 1.9.demo. 31 Oct 2011 - Added FLIC 2.0.demo. 01 Nov 2011 - Added GeoExpress 8. Timings are based on outfile timestamps. Bit-exactness after decompression could not be tested because all images were "watermarked with small white "X"'s indicating that the imagery was created with the evaluation version". 09 Nov 2011 - Added NanoZip 0.09 alpha and g+1 button. 10 Nov 2011 - Now you can download test images from CompressionRatings.Com in GraLIC 1.11 format or in ZIP. 15 Nov 2011 - Added ZPAQ 4.00. Comparing to version 2.05, compressed size is the same, compression time is ~14% less, decompression is ~7% faster. 16 Nov 2011 - Added FLIC 2.1.demo. 20 Nov 2011 - Added PackPNM 1.0e. It was compiled with MinGW 4.5.2 and switches from the native makefile: g++ -O3 -s -pedantic -funroll-loops -ffast-math -fsched-spec-load -fomit-frame-pointer packPNM\*.cpp 23 Nov 2011 - Updated FLIC 2.1.demo and pages Top 10 entries/Percentage. 27 Sep 2012 - Added WebP 0.2.0. Compression time is approximate. With "-lossless -m 6" it crashes on image PIA13894.
Please send your comments to this group or to Alex Rhatushnyak: pqr (alpha) rogers d_o_t com. Discussions on the most active data compression forum: FLIC, Lossless image compressors.