Clients desire to have each write operation from a process in a parallel application perform only one low-level write operation to disk (i.e. only one call to MPI_File_write_at (or MPI_File_write_at_all). When using chunked storage for datasets, this can only be done if the data that each process writes out is exactly aligned with a single chunk.

For example, if the chunks for a dataset are regular and non-overlapping, and are defined like this:

Regular, non-overlapping, identical chunks

Each chunk in the dataset is stored contiguously on disk, like this: (note that the chunks themselves are not necessarily adjacent)

Chunk layout on disk

Assume for the purpose of this example, that the size of the datatype for each element in the dataset is 4 bytes (making each chunk 256 bytes in size) in the file. Also, assume that the client is using 4 processes to access the dataset.

The following table describes the hyperslab selection for each client process:

Aligned disk selection and chunk layout

When I/O is performed by each client process with these selections and chunks defined, each process will perform only one I/O operation, as indicated by the following table:

Now, assume that the following selections have been made:

Unaligned disk selection and chunk layout

Having mis-aligned file selections and chunks results in a much larger number of I/O operations and bytes transferred to disk:

However, if chunks are allowed to be variable-sized, they could be defined like this:

Regular, non-overlapping, variable-sized chunks

Each chunk in the dataset is still stored contiguously on disk, like this: (note that the chunks themselves are not necessarily adjacent)

Chunk layout on disk

Then, the variable-sized selections for each client process (as defined above) would be aligned with the chunk boundaries:

Aligned disk selection and chunk layout for variable-sized chunks

And would have the optimal number of I/O operations and bytes transferred again: