To obtain the SZIP source code:

Szip Compression in HDF5

Szip compression software, providing lossless compression of scientific data, is provided with HDF5 software as of HDF5 Release 1.6. It is scheduled for inclusion with the next release of HDF4, planned for the summer of 2003.

Szip is an implementation of the extended-Rice lossless compression algorithm. The Consultative Committee on Space Data Systems (CCSDS) has adopted the extended-Rice algorithm for international standards for space applications[1,6,7]. Szip is reported to provide fast and effective compression, specifically for the EOS data generated by the NASA Earth Observatory System (EOS)[1]. It was originally developed at University of New Mexico (UNM) and integrated with HDF4 by UNM researchers and developers.

As the following graphs illustrate, the primary gain with Szip compression is in speed of processing. Szip also provides some advantage in compression ratio over other compression methods shown here. These results, including the data presented in the graphs below, are from tests conducted by Pen-Shu Yeh, et al. [1], with the HDF4 Szip integration.

Size of compressed output [1]

Speed of compression operation [1]

Using Szip compression in HDF5
Szip is a stand-alone library that is configured as an optional filter in HDF5. To use Szip, the Szip binary library must be downloaded. The HDF5 library must be configured and compiled with Szip support and with the Szip library.

Applications use Szip by setting Szip as an optional filter when a dataset is created. Once enabled, data is automatically compressed and decompressed with Szip during I/O.

See example of Szip usage in HDF5 for a sample HDF5 program using Szip.

Details of the required and optional parameters are provided in the H5Pset_szip entry in the HDF5 Reference Manual.

Software distribution
Starting with Release 1.6, HDF5 will be distributed with Szip enabled, making it easier to use Szip compression:

Licensing terms
The version of Szip distributed with HDF products is free for non-commercial use, which may occur in two sets of circumstances:

Commercial licenses are available for commercial users who wish to distribute an Szip-based software product or engage in commercial uses that are not allowed above. For further licensing information or to view a copy of the szip copyright statement, see Commercial use terms and the copyright and license notice pertaining to Szip in HDF5

Further information
See the following materials for further information:

HDF Help Desk
Describes HDF5 Release 1.6.0, July 2003
Last modified: 15 July 2003