To obtain the SZIP source code:
ftp://ftp.ncsa.uiuc.edu/HDF/szip/

Szip Compression in HDF5

Szip is an implementation of the extended-Rice lossless compression algorithm. The Consultative Committee on Space Data Systems (CCSDS) has adopted the extended-Rice algorithm for international standards for space applications[1,6,7]. Szip is reported to provide fast and effective compression, specifically for the EOS data generated by the NASA Earth Observatory System (EOS)[1]. It was originally developed at University of New Mexico (UNM) and integrated with HDF4 by UNM researchers and developers.

As the following graphs illustrate, the primary gain with Szip compression is in speed of processing. Szip also provides some advantage in compression ratio over other compression methods shown here. These results, including the data presented in the graphs below, are from tests conducted by Pen-Shu Yeh, et al. [1], with the HDF4 Szip integration.

Size of compressed output [1]

Speed of compression operation [1]

Using Szip compression in HDF5
Szip is a stand-alone library that is configured as an optional filter in HDF5. To use Szip, the Szip binary library must be downloaded. The HDF5 library must be configured and compiled with Szip support and with the Szip library.

Applications use Szip by setting Szip as an optional filter when a dataset is created. Once enabled, data is automatically compressed and decompressed with Szip during I/O.

See example of Szip usage in HDF5 for a sample HDF5 program using Szip.

Details of the required and optional parameters are provided in the H5Pset_szip entry in the HDF5 Reference Manual.

Software distribution
Starting with Release 1.6, HDF5 will be distributed with Szip enabled, making it easier to use Szip compression:

• Szip will be enabled and pre-configured in pre-compiled HDF5 binaries.
• Szip source code and Szip binaries will be distributed by NCSA.

Licensing terms
The version of Szip distributed with HDF products is free for non-commercial use, which may occur in two sets of circumstances:

• Non-commercial users may use the Szip software integrated with HDF5 to both encode (compress) and decode (uncompress) data. This applies to educational and research applications.
• Commercial users may use the software to decode any data. Further, they may use the software in internal activities that do not involve or result in the development of an Szip-based software product.

Commercial licenses are available for commercial users who wish to distribute an Szip-based software product or engage in commercial uses that are not allowed above. For further licensing information or to view a copy of the szip copyright statement, see Commercial use terms and the copyright and license notice pertaining to Szip in HDF5

Further information
See the following materials for further information:

• The H5Pset_szip entry in the Release 1.6 HDF5 Reference Manual for Szip usage information
• The SZIP source code at ftp://ftp.ncsa.uiuc.edu/HDF/szip/
• An example of Szip usage in HDF5
• Commercial use terms and the copyright and license notice pertaining to Szip in HDF5
• The HDF5/Szip collaborative agreement (PDF)
• References to publications discussing Szip or its use.