HDF4 Mapping Prototype: Schema and Tools


This project was completed and The HDF Group is no longer maintaining the software. You are free to send questions or software patches to help@hdfgroup.org, but The HDF Group cannot guarantee that we will respond to the questions or incorporate the patches.

Please use the contact form at https://support.hdfgroup.org/about/contact.html if you would like to discuss contracting with The HDF Group to provide support, maintenance, or further software development.

A desire to read HDF4 files without relying on the HDF4 library prompted the work described in this document-the construction of text-based "maps" of binary HDF4 files.

An XML-based prototype schema for HDF4 mapping files (XML documents) was created. For a given binary HDF4 file, an associated mapping file contains structural and application metadata for the HDF4 file, as well as the locations of the object data (array element values) in the HDF4 file. A tool was written to generate mapping files. Other tools were developed that use the mapping files to read HDF4 files without calling the HDF4 library, confirming the approach is viable.

While the focus of this effort was NASA EOSDIS data stored in HDF4 files, the general methodology is also relevant to other cases where the long-term accessibility of data stored in binary files is of concern. In addition, this work demonstrates how binary HDF files can be used to efficiently store large volumes of scientific data that is referenced by text-based XML documents (the mapping files).