R&D
Decomposed arrays
From a loyal HDF5 user: "As far as we can tell, the array abstraction HDF5 supports is just a "global" abstraction. That is, the array in the file is a single coherent entity, not several pieces that are mapped into a single coherent entity... Of course, HDF5 does support the notion of chunking but that is NOT the kind of "pieces" I am looking for here. I need to store an array in pieces, where each "piece" is probably a different size in each dimension AND where one "piece" might overlap with another and all pieces are some hyper-slab out of the larger, single coherent entity... Basically, you'd have to add to a dataset the notion of a decomposition and the ability to read/write as a single, coherent, whole or as a decomposition."
folder: Variable sized chunks