Dear consultants,


My name is Ray Lu.  I work for the HDF5 Group.  I have two issues to report about the ChaMPIon on tungsten.  Unfortunately, I discovered these two problems when I tested the PHDF5 library.  It's too hard for me to create a standalone MPI test.  I can only give you the whole PHDF5 library and a test program to let you try it.  Maybe you or the ChaMPIon people can look into the PHDF5 library and debug the ChaMPIon with it.  I believe that both of the problems are from the ChaMPIon, not the PHDF5.


I. Build PHDF5 library


First, you'll have to build the PHDF5 library.  I used the ChaMPIon 2.1.1-4tgm2 for my test.  The PHDF5 library (hdf5.tar.gz) and the test program (ptest.c) are at ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/slu/champion  


The basic steps are below:


  1. setenv CC "cmpicc -icc -io"
  2. Go into hdf5/ and do "./configure --enable-debug --disable-shared"
  3. gmake
  4. gmake install
  5. Then you have h5pcc and h5dump installed under hdf5/hdf5/bin.  Include this path in your                                
     "path" variable. 


II. MPI_Get_element Problem


Compile the test with "h5pcc ptest.c" and run it with only 2 processors.  Among the debugging information, there is a line saying "H5FD_mpio_write: bytes_written=24, io_size=72".  And the test fails.  The problem happens in the function H5FD_mpio_write in src/H5FDmpio.c.  The "bytes_written" returned by MPI_Get_element is wrong.  This problem was reported by our group 1.5 years ago.  But there's no follow-up.  


III. MPI_Type_struct Problem


If you comment out the lines between 1898 and 1903 in src/H5FDmpio.c and recompile and reinstall the PHDF5 library and the test program, if you run the test, it should work this time.  When you use "h5dump pdata.h5" to display the written data, however, you'll find part of the data is incorrect.  It displays as below:


HDF5 "pdata.h5" {
GROUP "/" {
   DATASET "Data1" {
      DATATYPE  H5T_STD_I32LE
      DATASPACE  SIMPLE { ( 6, 6 ) / ( H5S_UNLIMITED, H5S_UNLIMITED ) }
      DATA {
      (0,0): 1, 2, 3, 4, 5, 6,
      (1,0): 101, 102, 103, 104, 105, 106,
      (2,0): 201, 202, 203, 204, 205, 206,
      (3,0): 301, 302, 303, 304, 305, 306,
      (4,0): 0, 0, 0, 0, 0, 0,
      (5,0): 0, 0, 0, 0, 0, 0
      }
   }
}
}


The correct data should be like this:


HDF5 "pdata.h5" {
GROUP "/" {
   DATASET "Data1" {
      DATATYPE  H5T_STD_I32LE
      DATASPACE  SIMPLE { ( 6, 6 ) / ( H5S_UNLIMITED, H5S_UNLIMITED ) }
      DATA {
      (0,0): 1, 2, 3, 4, 5, 6,
      (1,0): 101, 102, 103, 104, 105, 106,
      (2,0): 201, 202, 203, 204, 205, 206,
      (3,0): 301, 302, 303, 304, 305, 306,
      (4,0): 401, 402, 403, 404, 405, 406,
      (5,0): 501, 502, 503, 504, 505, 506
      }
   }
}
}
  
After my coworker and I examined the PHDF5 library carefully, we think the problem points to the MPI_Type_struct function.  It seems that ChaMPIon doesn't work correctly in this function.  This function is called in line 430 in the function H5S_mpio_hyper_type of H5Smpio.c.  If you need more debugging information, do "setenv HDF5_DEBUG all".


If you need more information, please feel free to contact me.  I apologize for not being able to narrow down the problems.  


Thanks.


Ray