File Handle API Function for VFD

                                                                        By Raymond Lu

                                                                        Sep 16, 2002

 

I.                   Document’s Audience:

Current HDF5 library designers and users who understand HDF5 internal library design.

 

II.                Background:

The ASCI people have requested us to provide an API function to return a pointer to the file handle(file descriptor) of low file Virtual File Driver, especially for MPIO and MPI-POSIX file drivers.  There are also other users looking for the flexibility to retrieve information from the low level file directly. 

 

III.             Design Goals:

Provide general API functions to retrieve file descriptor for all virtual file drivers.

 

IV.              Design Issues:

Currently, the HDF5 library have 11 virtual file drivers.  Each has its own type for file descriptor.  The file descriptor can be found in H5FD_XXX_t struct in H5FDXXX.c files, where XXX represent each file driver’s name.  The following table lists all these file drivers and their file descriptor type.

 

File Driver

Data Type of File Descriptor

core

int

family

Each member has its own descriptor per its actual file driver.

gass

int

mpio

MPI_File

mpiposix

int

multi

Each part has its own descriptor per its actual file driver.

sec2

int

srb

int

stdio

FILE *

stream

int

log

int

 

In general, we can just return the file descriptor directly through an API function.  The exceptions are family and multi file drivers, which may have more than one file descriptors.  In order to handle both the general and exceptional situations, three solutions are discussed below.

 

V.                 Proposals:

 

Option 1:  This proposed API function simply returns file descriptor of the file driver in general.  When a request for family or multi file drivers is made, a NULL pointer is returned.

 

Option 2:      This proposed API function accepts an extra parameter of file access property list identifier.  New API functions for property list will be added to handle special cases of family and multi file drivers.  For family file driver, users can pass in the relative offset of their data in file to this property list function.  Then we can return the file descriptor of the family file member which this offset falls into.  For multi file driver, users can pass in the type of file which they want to query.  Then we can return the file descriptor of the file member where this kind of data is stored.  For other general cases, this property list identifier can be set as a default value.

 

Option 3:      This proposed API function accepts an extra parameter of a structure(struct) type, which contains file driver type and an integer field.  This integer field can be either data offset in file for family driver, or the type of file for multi driver.

 

Option 1 is the most simple but does not let us handle the family and multi file drivers.  If we do want to cover them in the future, this API function has to be modified.  Option 3 can handle all the file drivers we have but still lack of future development abilities.  Option 2 gives us the most flexibility for current file drivers and future development.  It can be easily extended to cover new file driver by adding file access property list functions.  The following proposed API functions are base on Option 2.

 

VI.              New API Functions:

 

Name:            H5Fget_vfd_handle

Signature:    

            Herr_t H5Fget_vfd_handle(hid_t file_id, hid_t fapl, void* file_handle)

Purpose:

            Return a pointer to the file handle from virtual file driver.

Description:

Given a file identifier and file access property list identifier, H5Fget_vfd_handle returns a pointer to the file handle from the low-level file driver, which is used by HDF5 library for file I/O.  Users are not supposed to modify any file through this file handle.  The file handle is only valid during the time the file is opened.        

            Parameters:

                        hid_t file_id

                                    IN: file identifier to be queried.

                        hid_t fapl

IN: file access property list identifier.  For normal file driver, the value of H5P_DEFAULT is passed in.  For family or multi driver, this value should be defined through the property setting functions, H5Pset_family_offset for family driver and H5Pset_multi_type for multi driver.

                        void* file_handle

                                    OUT: pointer to the file handle of the low-level virtual file driver.

            Returns:

                        Non-negative if successful; otherwise returns negative.

 

            Name: H5FDget_vfd_handle 

            Signature:

                        herr_t H5FDget_vfd_handle(H5FD_t *file, hid_t fapl, void *file_handle)

            Purpose:

                        Return a pointer to the file handle from virtual file driver.

            Description:

Given a pointer to the object of file driver structure and file access property list identifier, H5FDget_vfd_handle returns a pointer to the file handle from the low-level file driver, which is used by HDF5 library for file I/O.  Users are not supposed to modify any file through this file handle.  The file handle is only valid during the time the file is opened.        

            Parameters:

                        H5FD_t *file

                                    IN: the object of file driver structure to be queried.

                        hid_t fapl

IN: file access property list identifier.  For normal file driver, the value of H5P_DEFAULT is passed in.  For family or multi driver, this value should be defined through the property setting functions, H5Pset_family_offset for family driver and H5Pset_multi_type for multi driver.

                        void *file_handle

                                    Out: a pointer to the file handle from the low-level file driver.

            Returns:

                        Non-negative if successful; negative otherwise.

 

 

Name:            H5Pset_family_offset

            Signature:

                        herr_t H5Pset_family_offset(hid_t fapl_id, const hsize_t offset)

            Purpose:

Sets the file access property list for family file driver by giving the offset position in a file.

            Description:

H5Pset_family_offset sets the file access property list to identify the data position in a family file users want to query.  This file access property list identifier will be passed into H5Fget_vfd_handle to retrieve the file handle of the family member file where data offset user passes in is located.

            Parameters:

                        hid_t fapl_id

                                    IN: File access property list identifier.

                        hsize_t offset

IN: the offset position in a file for which user wants to query the low-level file handle.

            Returns:

Returns a non-negative value if successful.  Otherwise returns a negative value.

 

            Name:            H5Pset_multi_type

            Signature:

herr_t H5Pset_multi_type(hid_t fapl_id, const H5FD_mem_t type)

            Purpose:

Sets the file access property list for multi file driver by giving the type of data user wants to query.

            Description:

H5Pset_multi_type sets the file access property list to identify the type of data in a multi file for which user wants to query about file handle.  This file access property list identifier can be passed into H5Fget_vfd_handle to retrieve the file handle of the multi file member where the type of data user passes in is located.

            Parameters:

                        hid_t fapl_id

                                    IN: File access property list identifier.

                        const H5FD_mem_t type

IN: the type of data for which user wants to query the low-level file handle.

            Returns:

Returns a non-negative value if successful.  Otherwise returns a negative value. 

 

 

Name:            H5Pget_family_offset

            Signature:

                        herr_t H5Pget_family_offset(hid_t fapl_id, hsize_t *offset)

            Purpose:

Gets the data offset position in file from the file access property list for family file driver.

            Description:

H5Pget_family_offset retrieves the offset position in file from the file access property list to for a family file.  This file access property list identifier will be passed into H5Fget_vfd_handle to retrieve the file handle of the family member file where data offset user passes in is located.

            Parameters:

                        hid_t fapl_id

                                    IN: File access property list identifier.

                        hsize_t *offset

OUT: the offset position in a file for which user wants to query the low-level file handle.

            Returns:

Returns a non-negative value if successful.  Otherwise returns a negative value.

 

 

            Name:            H5Pget_multi_type

            Signature:

herr_t H5Pget_multi_type(hid_t fapl_id, H5FD_mem_t *type)

            Purpose:

Gets the type of data from the file access property list for multi file driver.

            Description:

H5Pget_multi_type retrieves the type of data from the file access property list for a multi file driver.  This file access property list identifier can be passed into H5Fget_vfd_handle to retrieve the file handle of the multi file member where the type of data user passes in is located.

            Parameters:

                        hid_t fapl_id

                                    IN: File access property list identifier.

                        H5FD_mem_t *type

OUT: the type of data for which user wants to query the low-level file handle.

            Returns:

Returns a non-negative value if successful.  Otherwise returns a negative value.