Various Zlib Issues
How HDF software use zlib
The GZIP organization(http://www.gzip.org) recently issue a security Zlib
advisory.
http://www.gzip.org/zlib/advisory-2002-03-11.txt
It involves a potential security problem due to double free coding error in versions 1.1.3 or older. V1.1.4 has these error code corrected. Though the security is not serious and is not clear how it can be exploited, GZIP advises users to upgrade their zlib to v1.1.4.
If a program code is compiled with one version of zlib.h but linked with different version of libz.a, it will result in a runtime error when zlib code is executed. When such a condition occurs, the zlib routine returns failure. No other actions such as error messages or abort occur.
GZIP.ORG provides the source free to anyone. HDF should direct users to download the zlib
source from the zlib site, http://www.gzip.org/zlib/
HDF website used to provide the binary of zlib 1.1.3 for several platforms that HDF5 library supports. Those binary files are no longer available. Elena has compiled zlib v1.1.4 for various machines and is planning to put them on the HDF download site. The location is not determined yet.
GZIP.ORG also provides zlib binary for various platforms at the same site above. It would be better for the HDF site to direct users to download zlib binary from the zlib site and we will provide zlib binary for platforms that are not available at the zlib binary site but supported by HDF software.
Scenarios 3.1: a user downloads a version of HDF5 library from HDF FTP site but his machine has a different version of libz.a. All HDF5 applications he compiled, using the downloaded HDF5 library, will fail all function involving zlib compression but everything else works correctly. His options are:
Notice that the HDF5 tools (e.g., h5dump) will work independent of the zlib versions if they have been statically linked. That is because the appropriate version of libz.a code have been linked into the h5dump binary files. The external libz.a is not used nor needed during runtime.
Scenarios 3.2: A user downloads a version of HDF5 library
from HDF FTP site and his machine has the same version of libz.a the HDF5
library is built with; or he download the HDF5 source and has built it
according to the zlib files in his machine.
Then the zlib library changes to a different version. All HDF5 application he compiled after the
zlib change will fail all functions involving zlib compression but everything
else works correctly. His options are
similar to Scenarios 3.1. All other
previous compiled code using zlib functions will have the same external library
upgrade problems just like the HDF5 applications.
In summary: we need
to include the version information of zlib we use to build our HDF software for
binary download. Whatever version of
zlib we use to build the software, it will run into compatibility problem in
some machines.
The plan is to not include zlib source as an internal part of the HDF4 source tree but to treat zlib as an external library just like other zlib applications. It will not be the best use of resource to upgrade the zlib code in the hdf/zlib directory. Note that zlib is planning a v1.1.5 beta release soon.
Issue 4: Can our current HDF4 library, which includes zlib source, avoid all zlib version compatibility problems?
It depends. If a user compiles HDF4 applications with
hard coded library paths such as
cc prog.c –I$HOME/hdf4/include
$HOME/hdf4/lib/libdf.a $HOME/hdf4/lib/libz.a …
then his a.out file
is insulated from any external zlib versions issue.
But if he compile
it with the expectation that the HDF4 software are installed in the regular
system places such as
cc prog.c -ldf -lz …
then it depens if
there is another version of libz.so around and if it takes precedence over the
version of zlib from the HDF4 library.
If that happens, all functions involving zlib compression may fail
mysteriously. In that case, the user
has to work on the linking sequence to make sure the right version of zlib is
used.
Issue 5: If we change the HDF4 library to use an external zlib, would it make the situation worse?
If a user builds
the current HDF4 library (has zlib code) in his machine, he will have zlib
version compatibility problem if his machine has a zlib version different from
the one HDF4 includes.
If he builds the
new HDF4 library (uses external zlib) in his machine, he will NOT have any zlib
version compatibility problem.
Issue 6: What about a user who uses both HDF4 and HDF5 libraries in the same application?
Scenario 6.1: If he builds the current
HDF4 library (has zlib code) and HDF5 library in his machine and his machine
has a different version of zlib software, the result will vary, depending on
which copy of zlib is linked in. If the
libz.a in the HDF4 code is used, then all HDF4 calls to zlib compression will
succeed but all HDF5 calls to zlib compress will fail. If the external libz.a is used, then all
HDF4 compression calls fail while all HDF5 calls succeed. Quite confusing.
Scenario 6.2: If he builds the new HDF4
library (uses external zlib) and HDF5 library in his machine, the two built
libraries will work just fine.
Issue 7: What
about upgrading zlib in NCSA machines?
Most,
if not all, NCSA machines have v1.1.3 zlib installed. This is the version with the double-free bug but it is not a
serious bug that requires immediate upgrade.
For most machines, zlib is installed in the /usr/ncsa hierarchy. Couple machines have it installed in
/usr/local/ hierarchy. The /usr/ncsa is
controlled by system staff though it is not clear who, if anyone, is maintaining
it. On the other hand, the worst thing
to happen is for us taking things into our own hand by installing an extra copy
of zlib software in our areas such as the /usr/sdt hierarchy. The disadvantages of doing such are:
1.
Multiple version of zlib in the same machine causing confusions. One would have to figure out the correct
version to compile and link. The
benefit in return is nothing concrete.
2.
If we build with a version of zlib in our own machines while the rest
of NCSA uses a different version, all binary built is not usable for those
other machines. Again, this creates
compatibility problems with no concrete benefit.
It
is better to ask system staff to upgrade the zlib software so that all hosts of
the same platform will move to a new version at the same time, thus eliminating
the version compatibility problem.
Recommendations:
1.
Setup pointers on our HDF website to direct users to go to http://www.gzip.org/zlib
for both source and binary download of zlib software. We will supplement with zlib binary for platforms that HDF software
supported but not provided by the zlib site.
2.
Change HDF4 to use external zlib library. (May do the same for jpeg library but that is an orthogonal
issue.)
3.
Request helpdesk to upgrade all zlib software to the newer
version. May offer to assist them in
acquiring or building the binary.
===
Revision: April 18, 2002
Email: acheng@ncsa.uiuc.edu