Used by FolderShare to compress peer to peer transfers between OSX, Linux, and Win32 machines of files up to 4GB. Compression was automatically disabled for media that was already compressed (photos, music, etc).

If you are ever going to support compression in your product, look at the interfaces to zlib when you are first working on your networking and buffer management code. Unless you design for it from the start, adding compression support later may result in a design that copies your data unnecessarily.

Compression can consume lots of CPU. For two computers on a fast network, it may be faster to send data uncompressed.

If you are thinking of using this to implement Content-Encoding: gzip for a webserver be aware that some older browsers (as recent as MSIE6 without SP1) do weird things like choke on CSS and Javascript sent with gzip or cache even when you send Cache-Control: no-cache or just plain choke on the transfer. (This is more of an HTTP gotcha, but also note that the ETag for Content-Encoding: gzip content had better be different than ETag for non-gzip content or it will end in tears for anyone doing range requests.)

When designing your buffer scheme, keep in mind that a small amount of compressed data can expand dramatically when uncompressed (a test case that consists of several hundred megabytes of zeroes is highly recommended). This can be mitigated by flushing the stream on a regular basis during compression, but that does reduce the efficiency somewhat.

The library contains a set of functions (such as gzopen) that can be used for creating a stream that can be decompressed with the standard command line tool gunzip/gzip. However, there isn't a wide char version gzopen available, and I haven't been able to get the gzdopen (which takes a file descriptor) to work. So, I just converted the path to a single byte string.

libbzip2

zlib FAQ