Lossless Photo Compression Benchmark

updated on 4-Feb-2018
(c) Alex Rhatushnyak

The results presented in this benchmark are free for scientific, commercial or private use, under the sole restriction of mentioning the source (this page).
All images are 24-bit RGB, pages for 8-bit, 16, 32 and 48-bit images may be added later.
About test data. Six sets with 107 images totalling 3.5 gigabytes:
62 images (2254 Mb) from http://photographyblog.com and
45 images (1208 Mb) from http://photojournal.jpl.nasa.gov. More details on test data.

Notes:
Click on compressed sizes for details. Red bold value indicates that no compressor with higher quality is faster.


Compressed Compression Decompression Command line size, bytes, % time, seconds time, seconds (program and options) 938042000 96.75 8152355 8199660 (tested by the author of cmix) cmix -c $1.ppm $1.cmix 969556533 100.00 131147 128112 (64-bit Linux) wineconsole --backend=user paq8px_v127.exe -8 $1.ppm 971986179 100.25 1734.48 1897.09 Gralic c %1.gra111 %1.ppm 995740780 102.70 11724.74 3051.34 bmf -s -q9 %1.ppm 996947712 102.83 3678.03 3038.85 bmf -s %1.ppm 1012198342 104.40 14338.27 14436.08 zpaq -mbmp_j4 c %1 %1.bmp 1038434818 107.10 209.85 231.50 Flic c %1.flic21 %1.ppm 1042883071 107.56 ~3400 ~3400 (StuffIt Deluxe) console_stuffEN.exe /c -o --quiet --recompression-level=2 %1.bmp 1046636212 107.95 ~2670 ~2730 (StuffIt Deluxe) console_stuffEN.exe /c -o --quiet --recompression-level=1 %1.bmp 1078284862 111.21 ~1346 ~1347 (StuffIt Deluxe) console_stuffEN.exe /c -o --quiet --recompression-level=0 %1.bmp 1089342602 112.35 3853.10 2057.10 rkim cx %1.tga %1.rkx 1126954659 116.23 13390 716 (64-bit Linux) flif -e -N --effort=100 $1.ppm -o $1.flif-E100 1128702803 116.41 7221 706 (64-bit Linux) flif -e -N $1.ppm -o $1.flif-N 1131735795 116.73 1756.70 1861.66 uhic m %1.bmp %1.uhm 1139392552 117.52 417.91 269.95 bmf %1.ppm 1141690596 117.75 1390.06 1635.29 rkim c %1.tga %1.rki 1141833624 117.77 369.13 356.59 bim c %1.ppm %1.bm3 1149113301 118.52 761.39 611.03 zcm a -m7 %1.zcm %1.ppm 1151144436 118.73 40.31 34.08 Qlic2 c %1.qlic2 %1.ppm 1152493991 118.87 2502.29 566.23 (Pyramid Workshop) cbtpc -lv 8 -p 5 -c 65 %1.ppm %1.py5 1154592106 119.08 884.11 966.50 uhic e %1.bmp %1.uhi 1162946359 119.95 2198.68 578.54 nk /s %1.bmp 1179957421 121.70 832.30 167.29 bcif %1.bmp -c %1.bci 1194936668 123.25 8015 39.54 (64-bit Linux) cwebp -lossless -q 100 -m 6 $1.png -o $1.cwebp 1204711066 124.25 465.47 525.43 wic c %1.bmp 1210043117 124.80 314.55 267.99 JpegXT_gpl -ls 0 -cls %1.ppm %1.jxt0 1222290070 126.07 1653 35.78 (64-bit Linux) cwebp -lossless -q 100 $1.png -o $1.cwbp 1224419506 126.29 1771.30 499.69 (lossy) dli c -q 100 -qc 80 -ac+ %1.ppm %1.dli 1230795584 126.94 38.39 43.20 Qlic c %1.qli %1.ppm 1239884576 127.88 173.31 230.86 lsp %1.ppm 1245914178 128.50 176.91 66.93 (lossy JPEG) NConvert -out jpeg -q 100 -subsampling 2 -opthuff %1.ppm 1247343688 128.65 381.56 373.81 (LuraWave JPEG-2000) NConvert -out lwfjpc %1.ppm 1247354537 128.65 312.77 273.03 (Kakadu) kdu_compress -i %1.ppm -o %1.kdu Creversible=yes -quiet 1247389133 128.66 605.57 554.72 JasPer --input %1.ppm --input-format pnm --output %1.jpr --output-format jpc 1247396798 128.66 1849.07 1674.81 (OpenJPEG) image_to_j2k -i %1.ppm -o %1.j2k 1275559788 131.56 1269.50 1140.48 NanoZip a -m1g -cc %1.nz1 %1.ppm 1283324851 132.36 491 342 (64-bit Linux) flif -e -N --effort=0 $1.ppm -o $1.flif-E0 1285421855 132.58 22.10 19.74 Qic c %1.qic %1.ppm 1290281728 133.08 448.06 332.06 PGF -e %1.ppm %1.pgf 1293857955 133.45 188.51 205.06 (JPEG-LS) locoe %1.ppm -o%1.loc 1293858918 133.45 106.40 92.36 (JPEG-LS) CharLS -encodepnm %1.ppm %1.chs 1299728137 134.05 138.23 170.08 libra8 c %1.ppm %1.lr8 0 1301083155 134.19 131.05 179.92 lsp -f %1.ppm 1308370225 134.95 700.03 419.72 (Pyramid Workshop) cbtpc %1.ppm %1.pyr 1320702831 136.22 173.20 235.26 lsp -g %1.ppm 1327774201 136.95 3870.06 3456.27 uharc a -mx -md32768 %1.uha-mx %1.ppm 1328708769 137.04 3700.69 3313.41 uharc a -mx -md- %1.uha-mx- %1.ppm 1331109154 137.29 1664.23 333.05 capt %1.ppm %1.apt 100 1338137564 138.02 1067.99 1135.89 packPNM_10e %1.ppm 1339149175 138.12 4275.11 501.06 uharc a -m3 -md32768 %1.uha-m3 %1.ppm 1341192869 138.33 4110.98 501.49 uharc a -m2 -md32768 %1.uha-m2 %1.ppm 1342696887 138.49 3758.73 501.86 uharc a -m1 -md32768 %1.uha-m1 %1.ppm 1348855983 139.12 140.09 67.53 (lossy JPEG) NConvert -out jpeg -q 100 -subsampling 2 %1.ppm 1349164676 139.15 ~696 ~768 GeoExpress 8 1354406543 139.69 979.37 513.97 uharc a -m3 -md- %1.uha-m3- %1.ppm 1354418303 139.69 977.99 513.83 uharc a -m2 -md- %1.uha-m2- %1.ppm 1354534276 139.71 872.79 513.88 uharc a -m1 -md- %1.uha-m1- %1.ppm 1403325039 144.74 330.28 206.08 NanoZip a -m1g -cD %1.nz2 %1.ppm 1415606383 146.01 446.11 369.23 (JPEG-XR reference software draft) JPEGXR -c %1.ppm -o %1.jxr 1415614410 146.01 128.06 93.69 (JPEG-XR) JXREncApp -i %1.ppm -o %1.jxr 1422849993 146.75 371.13 335.65 (JPEG-XR) WMPEncApp -i %1.ppm -o %1.wdp 1429846064 147.47 22.92 21.19 (ImageZero) iz c %1.ppm %1.iz 1432319902 147.73 27819.14 68.47 (optimized PNG) pngout /c2 /f5 %1.png 1449033110 149.45 131.96 160.61 arc a -mmm:3*8+grzip:m4:l %1.arc-grz %1.ppm 1459489679 150.53 429.20 487.58 uharc a -mz -md32768 %1.uha-mz %1.bmp 1462885000 150.88 1041.99 424.64 (IJG JPEG) cjpeg -rgb1 -block 1 -arithmetic -outfile %1.ijg %1.ppm 1471885647 151.81 844.37 72.62 (regular PNG) NConvert -out png %1.ppm 1495870625 154.28 266.29 86.48 (Lossless JPEG) lj-enc %1.ppm %1.ljp 1554944858 160.38 627.80 59.77 rar a -m5 %1.ra5 %1.ppm 1555436675 160.43 584.70 59.81 rar a -m4 %1.ra4 %1.ppm 1556568765 160.54 531.40 59.86 rar a -m3 %1.ra3 %1.ppm 1556653187 160.55 1801.95 317.91 BPGenc -lossless -e x265 -m9 -o %1.bpg9 %1.png 1557660259 160.66 1028.07 318.17 BPGenc -lossless -e x265 -m1 -o %1.bpg1 %1.png 1597320420 164.75 534.52 429.35 (LuraWave Format) NConvert -out lwf %1.ppm 1737745564 179.23 29.86 28.63 arc a -mmm:3*8+1xb %1.arc1xb %1.ppm 1742414838 179.71 1932.40 207.70 arc a -mx %1.arcx %1.ppm 1755332419 181.04 508.90 232.26 bzip2 -k -9 %1.ppm 1796117884 185.25 619.42 102.58 arc a %1.arc_ %1.ppm 1801111030 185.77 489.99 229.56 bzip2 -k -5 %1.ppm 1808959044 186.58 1426.18 138.43 7z a -mx %1.7mx %1.ppm 1813113393 187.00 1300.40 137.47 7z a %1.7z %1.ppm 1854900746 191.31 263.21 188.27 QBIT.exe QBIT %1.bmp 1940273041 200.12 277.22 76.29 arc a -m3 %1.arc3 %1.ppm 1950146903 201.14 476.15 218.26 bzip2 -k -1 %1.ppm 2254090100 232.49 106.67 43.11 arc a -m2 %1.arc2 %1.ppm 2635901832 271.87 1000+ ~165 MediaZip 3144733204 324.35 27.94 23.09 (Adobe Photoshop) NConvert -out psd %1.ppm 3145652200 324.44 39.24 73.97 (Explore (TDI) & Maya) NConvert -out tdi %1.ppm 3462571880 357.13 1.25 1.25 copy/b %1.ppm nul
History, comments and links 11 Apr 2011 - launched the benchmark, with 12 programs tested: GraLIC 1.9.demo, BMF 2.01, PAQ8im, FLIC 1.4.demo, BCIF 1.0 beta, Kakadu 6.4.1, JasPer 1.900.1, LOCO-I/JPEG-LS from HP Labs, packPNM 1.0d, regular PNG (created by NConvert 5.80), Lossless JPEG (Cornell University), RAR 4.00. Conversion to PPM and BMP formats was done by NConvert. JasPer failed to compress this large 139 Mb image (in PPM format), the file created by Kakadu was taken instead. JasPer and Lossless JPEG were compiled with GCC (version and options as in readme.txt in the FLIC package). All programs were tested on an Intel Core i7-930 CPU, 2.8 GHz, 3 Gb RAM, Windows XP. 12 Apr 2011 - added PGF Console Version 6.09.44 and PNGOUT Apr-10-2011 (hopefully no more that slow programs), compiled page Top 10 entries - more details. 13 Apr 2011 - added two programs that were competing in 1999 and 2000: RKIM 1.06 and UHIC 2.0. UHIC dealt with images in BMP format (and short 8.3 names), while for RKIM they were converted to TGA. UHIC fails to fully unpack olympu09.bmp compressed with command e. For measuring decompression time, olympu11.uhi was decompressed instead of olympu09.uhi. 14 Apr 2011 - added LSP, OpenJPEG 1.4, RAR -m5 -mcc+, -m3, -m1. 15 Apr 2011 - added Pyramid Workshop (ver. 21.03.2011 compiled by Niels Fröhling), RAR -m5 -mcc-, RAR -m3 -mcc-. PW with -lv 8 -p 5 -c 65 fails to compress two images, they were compressed with -lv 8 -p 5. With no options it fails to losslessly decompress two other images, but only three bytes differ. 16 Apr 2011 - added FreeArc 0.67 (March 18 2011), PAQ8im decompression time, compiled page Top 10 entries - percentage. 17 Apr 2011 - added Libra8 and ZPAQ 2.05 with bmp_j4 model. ZPAQ was recompiled (to turn on the optimization "o" feature for faster compression and decompression with config files) with this command: g++ -O3 -DNDEBUG -DOPT="\"g++ -O3 -DNDEBUG -march=pentiumpro -s -fomit-frame-pointer zpaq.cpp libzpaq.cpp zpaqopt.cpp -o zpaqopt.exe\"" zpaq.cpp libzpaq.cpp libzpaqo.cpp -o zpaq After you run it with command okcbmp_j4.cfg, it creates zpaqopt.exe, and then you run zpaqopt.exe instead of zpaq.exe. Output is bit-exact, but speed is much higher. 18 Apr 2011 - added 7-zip 9.20, BZIP2 1.0.6, HD Photo DPK 1.0. 7-zip exits with "Internal Error #1" if archive name is nul.ext, with "Unknown Error" if it's nul.exte, it says "Error: I won't write compressed data to a terminal" if you try 7z a -so archive image, and finally "System error: Not implemented" if you try 7z a -so archive image >redirect. Bzip2 was compiled with GCC. HD Photo DPK was compiled in Visual Studio 2008 - Release, no project settings were modified. If the extension of compressed image file is not .wdp, decompressor refuses to unpack the image, although the format name has changed twice: Windows Media Photo - HD Photo - JPEG XR. 19 Apr 2011 - added APT 1.0 and StuffIt 2010 ver. 14.0.0.18 (Compression Technology version 14.0.0.15). Compression time will be added later. 20 Apr 2011 - added NK2 and compression time for APT and StuffIt. Started testing DLI, first with -q 100 -qc 80 -ac. 21 Apr 2011 - DLI -or is extremely slow. Looks like it will take more than 333 hours to compress all 109 images. Started testing WinRK 3.1.2. 29 Apr 2011 - added WinRK 3.1.2. The only non command line program. Time is "Elapsed time" reported by WinRK itself. It crashes compressing the set of all 109 images with profile Best asymmetric. Does not crash if five sets are compressed separately. Decompression was bit-exact for all images, both tested profiles. DLI -or -q 100 -qc 80 -ac will finish compressing on Monday or Tuesday. 04 May 2011 - added WinRK with profile HIGH and CharLS v.64769 ( \trunk\charls\test\test.vcproj ). CharLS was compiled in Visual Studio 2008 - Release, no project settings were modified. WinRK with profile HIGH fails to decompress 41st image (canon_eos_600d_14) and all remaining images. 11 May 2011 - added FreeArc -mmm:3*8+1xb, FreeArc -mmm:3*8+grzip:m4:l, and two lossy algorithms: DLI 2.1 and JPEG, both at best possible quality. 13 May 2011 - UHARC 0.6b was tested, C- and D-time will be added later. 16 Jun 2011 - Added UHARC's compression time and decompression time. With -mz it was compressing images in BMP format, because on PPM files compressed size was 39% bigger. 21 Jun 2011 - NConvert 5.91 was released, it doesn't fail on raw images from Fujifilm FinePix X100 and Canon EOS 1100D, so everything* will soon be re-tested with the updated test set. When the most recent version 2.4 of Detector for ex-JPEGs is applied, 65 of 110 images from photojournal.jpl.nasa.gov are detected as ex-JPEGs (and discarded), but none of the 62 images from photographyblog.com. *Everything except WinRK, DLI in slow mode, Libra8 with options 1 and 2. 03 Jul 2011 - Everything was re-tested with the updated data set. Previous version of LPCB main page is here. This time more efforts were made to minimize the influence of HDD speed. New versions of programs this time: WinRAR 4.01, PGF 6.11.22, DLI 2.2, NConvert 5.91. Trial version of StuffIt Deluxe has expired, therefore only compression quality was tested (on a different computer), therefore times are approximate. Page More details on test data was updated. 13 Jul 2011 - Added FLIC 1.5.demo. 15 Jul 2011 - Added GraLIC 1.10.demo. 17 Jul 2011 - Compiled pages Top 10 entries - more details and Top 10 entries - percentage. 18 Jul 2011 - Added MediaZip 1.0. Because there is no command-line executable, and because in the GUI version you have to press OK after each file, timings are very approximate. If it compresses images in BMP format, compression quality is not as good as with PPMs, e.g. 309976499 => 310621406 bytes (PPM=>BMP) on the Canon set. Flag "Use maximum compression" was set in the options. If it is not, compressed size is ~4.5% bigger (the Canon set). 20 Jul 2011 - Updated GraLIC 1.10.demo and pages Top 10 entries/Percentage. 30 Jul 2011 - Added GraLIC 1.11.demo. 22 Aug 2011 - Added Qbit 1.0.0.10. It fails to losslessly decompress one file: canon_eos_1100d_11.qbt. 25 Aug 2011 - Added FLIC 1.6.demo. 03 Sep 2011 - Added FLIC 1.7.demo. 12 Sep 2011 - Added PGF 6.11.29, updated FLIC 1.7.demo. If you want to see the scripts used for testing software, please send an e-mail. 14 Sep 2011 - Added PNGOUT Jul 2 2011. 25 Sep 2011 - Added FLIC 1.8.demo. 03 Oct 2011 - Added JPEG-XR reference software draft (from Thomas Richter) 05 Oct 2011 - Added LuraWave Format, LuraWave JPEG-2000 (NConvert+plugins), PSD and TDI (just NConvert, no plugins). 11 Oct 2011 - Added FLIC 1.9.demo. 14 Oct 2011 - Updated FLIC 1.9.demo. 31 Oct 2011 - Added FLIC 2.0.demo. 01 Nov 2011 - Added GeoExpress 8. Timings are based on outfile timestamps. Bit-exactness after decompression could not be tested because all images were "watermarked with small white "X"'s indicating that the imagery was created with the evaluation version". 09 Nov 2011 - Added NanoZip 0.09 alpha. 10 Nov 2011 - Now you can download test images from CompressionRatings.Com in GraLIC 1.11 format or in ZIP. 15 Nov 2011 - Added ZPAQ 4.00. Comparing to version 2.05, compressed size is the same, compression time is ~14% less, decompression is ~7% faster. 16 Nov 2011 - Added FLIC 2.1.demo. 20 Nov 2011 - Added PackPNM 1.0e. It was compiled with MinGW 4.5.2 and switches from the native makefile: g++ -O3 -s -pedantic -funroll-loops -ffast-math -fsched-spec-load -fomit-frame-pointer packPNM\*.cpp 23 Nov 2011 - Updated FLIC 2.1.demo and pages Top 10 entries/Percentage. 27 Sep 2012 - Added WebP 0.2.0. Compression time is approximate. With "-lossless -m 6" it crashes on image PIA13894. 03 Oct 2012 - Added ImageZero 0.1 (discussion) and RAR 4.20. Previous version of LPCB main page is here. 09 Nov 2012 - Added version 9 of the Independent JPEG Group software. The source code was compiled with MinGW 4.5.2: gcc -O3 jpeg-9\*.c 04 Dec 2012 - Added Quo Vadis JPEG released on 3-Dec-2012. 19 Dec 2012 - Added CharLS v.79962 It was compiled in Visual Studio 2008 - Release, no project settings were modified. 10 Jan 2013 - Added QLIC 1.demo, another image compressor written by Alex Rhatushnyak. 19 Jan 2013 - Replaced links to the latest NConvert with links to NConvert 5.91. 20 Jan 2013 - Added QIC 1.dem, a twice faster and smaller edition of QLIC. 02 Mar 2013 - Added BIM 0.02. 20 Apr 2013 - Added Kakadu 7.2. 19 Jul 2013 - Added WIC released on 16-Jul-2013. 26 Jul 2013 - Added JXRlib 1.1. It was compiled in Visual Studio 2012 - Win32 Release, no project settings were modified except PlatformToolset = v110_xp. It looks like lossy quality seetings 2 and 3 are meaningless: Compressed Quality size setting 1415614410 -q 1 (lossless) 1663032190 -q 2 1421662746 -q 3 1257297614 -q 4 1073834445 -q 5 973161174 -q 6 891111613 -q 7 823388957 -q 8 29 Sep 2013 - Added BIM 0.03. 04 Oct 2015 - Moved to new home: qlic.altervista.org. Not sure for how long imagecompression.info will be accessible. 11 Oct 2015 - Added FLIF v0.1-alpha. Win32 executable was provided by Bulat Ziganshin here. With -n --r=10 it crashes on STA13843.ppm, with -n --r=2 and -n --r=5 compressed size is bigger, and decompression was not tested. 17 Oct 2015 - Added ZCM 0.93 and JPEG XT Demo Codec released on 12 Oct 2015 replacing earlier version (Quo Vadis JPEG). As before, with -ls N when N>0 all three of (Csize, Ctime, Dtime) are bigger. 18 Oct 2015 - Added BPG 0.9.5 and WebP 0.4.3. BPG seems to run many threads, and there is no switch to ask it to run in single-threaded mode. 24 Oct 2015 - Added StuffIt 14.0.0.16. As before, Ctime and Dtime were measured on a different computer (because "Your StuffIt Deluxe trial period has expired") and are very approximate. The "Top 10" pages will be updated later. 25 Oct 2015 - Added QLIC 2.demo. 28 Dec 2017 - Added paq8px_v125. Time was measured on a 64-bit desktop with Intel Core i7-4770 CPU @ 3.40 GHz (running Windows 8.1 Enterprise), then it was multiplied by 1734.48/1261.5 to estimate runtime on the original LPCB desktop. Moved the About test data section to the top, and updated page More details on test data. 29 Dec 2017 - Recompiled Detector for ex-JPEGs, there are now four executables: 32-bit and 64-bit, Linux and Windows. 30 Dec 2017 - Added WebP 0.6.1 and FLIF 0.3. Time was measured on the original LPCB desktop, after 64-bit Linux was installed on it. 02 Jan 2018 - Added paq8px_v127. Time was measured on the original LPCB desktop running 64-bit Linux. Temporarily removed pages LPCB-top and LPCB-top-percentage. 04 Feb 2018 - Added cmix, version from 24-Dec-2017. It was tested by the author (Byron Knoll) on Google Compute Engine instances.
Please send your comments to this group or to Alex Rhatushnyak: gralic1 (alpha) g mail d_o_t com. Discussions on the most active data compression forum: LPCB, Lossless image compressors.