HDF5 Tutorial: Examining JPSS NPP Files - Use Case 1

Examining a File Containing 1 Granule and Lat/Lon Information

The file used in this Use Case is:

Topics covered using this file are:


Determining the File Contents

The first thing you will probably want to do with the file is determine the file contents. You can use the command line tools or HDFView to do this:

Use command-line tools (h5dump, h5ls)

The command-line tools tutorial mentioned that you could view a list of file contents as follows:

To display the file contents using h5dump -n type:

Similarly, the -l and -r options of h5ls will display the file contents (and dataspace for each dataset):

You can also see in the h5ls output that the datasets are all unlimited in size (by the "Inf").

Note that there are two groups in this specific file (and in all NPP files):

Group Description
/All_Data Contains the raw data and optional geo-location information.
The geo-location information can be found in datasets Latitude and Longitude.
/Data_Products Contains a *_Aggr file with references to objects in the /All_Data group.
Contains *_Gran_# datasets with references to selected regions in datasets under /All_Data.

The *_Gran_# datasets are "granules". In this example, there is just one granule, but NPP files may contain numerous granules.

Use HDFView

TBD

Viewing Attributes (and suppressing them)

Attributes contain metadata describing the object they are attached to. Attributes are displayed by default with h5dump. In files that have many attributes (like JPSS files), it can be useful to suppress the display of attributes. With h5dump, this can be done by specifying -A 0, as shown below.

Examining Datasets

We will first examine the one granule in the file with h5dump, by specifying:

    h5dump -A 0 -d "/Data_Products/ATMS-SDR-GEO/ATMS-SDR-GEO_Gran_0" 
          GATMO_npp_d20100906_t0610367_e0611081_b00004_c20111024161800869170_noaa_ops.h5

where,
    The -A 0 option suppresses the display of attributes.
    The -d option is the name of the granule (dataset).

The above command displays:

As you can see in the output above, the datatype for this dataset is:

   DATATYPE  H5T_REFERENCE { H5T_STD_REF_DSETREG }

This indicates that it is a dataset specifically for storing references to regions (or subsets) in other datasets. The dataset contains 17 such references, and more can be added to it, as indicated by the dataspace (ie. it is unlimited):

   DATASPACE  SIMPLE { ( 17 ) / ( H5S_UNLIMITED ) }

In the above output we can see that SatelliteRange is the tenth reference in the "/Data_Products/ATMS-SDR-GEO/ATMS-SDR-GEO_Gran_0" reference dataset (if you count from the top).

How can we look at the SatelliteRange region values with h5dump?

The list of references shown above is a 0-based index to the dataset. Therefore, to specify SatelliteRange, select a start offset of 9 (the tenth reference less 1) with the -s option. To see the region reference data, use the -R option.

For example, this command will print out all of the data in the SatelliteRange region reference:

  h5dump -A 0 -d "Data_Products/ATMS-SDR-GEO/ATMS-SDR-GEO_Gran_0" -s 9 
     -R GATMO_npp_d20100906_t0610367_e0611081_b00004_c20111024161800869170_noaa_ops.h5

It displays all of the data in the SatelliteRange:

HDF5 "GATMO_npp_d20100906_t0610367_e0611081_b00004_c20111024161800869170_noaa_ops.h5" {
DATASET "Data_Products/ATMS-SDR-GEO/ATMS-SDR-GEO_Gran_0" {
   DATATYPE  H5T_REFERENCE { H5T_STD_REF_DSETREG }
   DATASPACE  SIMPLE { ( 17 ) / ( H5S_UNLIMITED ) }
   SUBSET {
      START ( 9 );
      STRIDE ( 1 );
      COUNT ( 1 );
      BLOCK ( 1 );
      DATA {
      (9): DATASET /All_Data/ATMS-SDR-GEO_All/SatelliteRange {
         (9): REGION_TYPE BLOCK  (0,0)-(11,95)
         (9): DATATYPE  H5T_IEEE_F32BE
         (9): DATASPACE  SIMPLE { ( 12, 96 ) / ( H5S_UNLIMITED, H5S_UNLIMITED ) }
         (9): DATA {
            (0,0): 1.58728e+06, 1.52747e+06, 1.47382e+06, 1.42535e+06,
            (0,4): 1.3813e+06, 1.34106e+06, 1.30415e+06, 1.27016e+06,
            (0,8): 1.23876e+06, 1.20968e+06, 1.18255e+06, 1.15744e+06,
            (0,12): 1.13402e+06, 1.11216e+06, 1.09172e+06, 1.07259e+06,
            (0,16): 1.05466e+06, 1.03785e+06, 1.02207e+06, 1.00725e+06,

CUT -----------

            (11,92): 1.4293e+06, 1.47811e+06, 1.53216e+06, 1.59244e+06
         (9):  }
      (9): }
      }
   }
}
}

As you can see above by the "CUT" line, not all of the data was shown here, due to the amount of data.

If you leave off the -R option, you will see the subset selection, but not the data:

The quality flags in the NPP file can also be viewed with h5dump.

Editing Datasets