From b8e6374808ddc9a69fb47394cfd5c432fb23242d Mon Sep 17 00:00:00 2001 From: Wei-keng Liao Date: Tue, 29 Jul 2025 17:35:00 -0500 Subject: [PATCH] prepare release of 1.14.1 --- DEVELOPER_NOTES.md | 49 ++++++++++++++++----- README.md | 1 + RELEASE_NOTES | 103 +++++++++++++++++++++++++++++++++++++++++++++ configure.ac | 7 ++- sneak_peek.md | 89 +++------------------------------------ 5 files changed, 154 insertions(+), 95 deletions(-) diff --git a/DEVELOPER_NOTES.md b/DEVELOPER_NOTES.md index 558c4fb10..18412a99e 100644 --- a/DEVELOPER_NOTES.md +++ b/DEVELOPER_NOTES.md @@ -7,7 +7,11 @@ codes. PnetCDF uses the same constants and error codes as NetCDF. See "Conform with netCDF" below. - 2. Set the release version + 2. create a new branch, for example named "prepare-1.14.1" + git checkout -B prepare-1.14.1 + git add the bullet points 3 and 4 below to the new branch + + 3. Set the release version In file `configure.ac`, variable `$PNETCDF_VERSION` indicates the version. It is automatically generated from the 2nd argument of `AC_INIT` call. Revise that string to the right release version. For example, @@ -17,7 +21,7 @@ ``` For setting shared library version (ABI versioning), see below. - 3. Update file `RELEASE_NOTES` + 4. Update file `RELEASE_NOTES` Copy the contents of file `sneak_peek.md` to the top of file `RELEASE_NOTES`. Clear up file `sneak_peek.md` (reset all items to none). + 1.9.0 and later - the release version and date in `RELEASE_NOTES` will @@ -25,7 +29,7 @@ + 1.8.1 and priors - add the release version and date at the top of file `RELEASE_NOTES`. - 4. Update the release date (1.8.1 and priors) + 5. Update the release date (1.8.1 and priors only) For more info, see below section "Setting PnetCDF software release date". Run command `svn commit` to get the svn property \$LastChangedDate\$ updated in file `configure.in`. That string will be used as the official @@ -34,12 +38,15 @@ svn commit -m "set release date of version 1.6.1 to today" ``` - 5. Commit all changes to repo servers + 6. Commit all changes to repo servers * 1.8.1 and priors -- run `svn commit` to upload changes to SVN server. - * 1.9.0 and after -- run `git push origin master` to upload changes to - github.com. + * 1.9.0 and after -- + + `git commit -m "prepare release of 1.14.1" + + `git push origin prepare-1.14.1` to upload changes to github.com. + + Create a pull request at github.com + + When all github actions passed, merge the PR. - 6. Generate release tar ball (use it for testing) + 7. Generate release tar ball (use it for testing) * Generate a new `configure` file + Run command `autoreconf` to generate file `configure` to be included in the release (but ignored by the SVN/Git repo). @@ -57,7 +64,7 @@ `make dist`. This setting will be done automatically, unlike step 4 above that manually update the release date. - 7. Test the new release tar ball (not source codes from the repo) + 8. Test the new release tar ball (not source codes from the repo) * build under the same directory as source - run `configure` (with command-line options: `--enable-strict`, `--enable-coverage`, `--disable-cxx`, `--disable-fortran` and their @@ -125,7 +132,7 @@ run instructions. Remember to add new test/example programs added in the new release. - 8. Create a checkpoint + 9. Create a checkpoint * For 1.9.0 and priors - Create a new SVN tag on svn repo, by running command below to duplicate the current trunk to a new tag: ``` @@ -139,7 +146,7 @@ git tag -a checkpoint.1.11.0 -m "Checkpoint right before 1.11.0 release" git push origin checkpoint.1.11.0 ``` - 9. Generate SHA1 checksums + 10. Generate SHA1 checksums * Run command: ``` openssl sha1 pnetcdf-1.11.0.tar.gz` @@ -148,7 +155,7 @@ ``` SHA1(pnetcdf-1.11.0.tar.gz)= 495d42f0a41abbd09d276262dce0f7c1c535968a ``` -10. Update PnetCDF Web Page +11. Update PnetCDF Web Page * https://github.com/Parallel-NetCDF/Parallel-NetCDF.github.io * Create a new file of release note Parallel-NetCDF.github.io/Release_notes/1.11.0.md. * Add a news item in index.html to announce the new release version. @@ -208,6 +215,23 @@ the old version at runtime: set revision to 0, bump current and age. 3. Programs may need to be changed, recompiled, and relinked in order to use the new version. Bump current, set revision and age to 0. + + libtool Chapter 7.2 Libtool’s versioning system + So, libtool library versions are described by three integers: + + current + The most recent interface number that this library implements. + + revision + The implementation number of the current interface. + + age + The difference between the newest and oldest interfaces that this + library implements. In other words, the library implements all the + interface numbers in the range from number current - age to current. + + If two libraries have identical current and age numbers, then the dynamic + linker chooses the library with the greater revision number. ``` --- ### Note on adding new MPI compiler candidates @@ -319,6 +343,9 @@ --- ### config.guess, config.sub, install-sh in directory scripts +There 3 files are automatically added when running command "autoreconf -i" +Below is just FYI. + Copy config.guess, config.sub, and install-sh from GNU libtool. http://www.gnu.org/software/libtool/ diff --git a/README.md b/README.md index 862f14e50..7a4d0fef3 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,7 @@ Northwestern University and Argonne National Laboratory. * [Utility Programs](./src/utils#readme) * [NetCDF4 vs. PnetCDF](./doc/netcdf4_vs_pnetcdf.md) * PnetCDF [blocking vs. non-blocking APIs](./doc/blocking_vs_nonblocking.md) +* [CDL header API references](./doc/cdl_api_guide.md) ### Mailing List * parallel-netcdf@mcs.anl.gov diff --git a/RELEASE_NOTES b/RELEASE_NOTES index 09c24e6b4..f055682a7 100644 --- a/RELEASE_NOTES +++ b/RELEASE_NOTES @@ -6,6 +6,109 @@ PnetCDF Release Notes Version _PNETCDF_VERSION_ (_PNETCDF_RELEASE_DATE_) ------------------------------------- +* New optimization + + When file header extent size grows, moving the data section to a higher + file offset has changed to be done in chunks of 16 MB per process. + See [PR #174](https://github.com/Parallel-NetCDF/PnetCDF/pull/174), + +* Configure options + + For PnetCDF developers, the requirement for libtool version has been + changed to 2.5.4, due to an issue on Mac OS when using OpenMPI. See + [Issue #155](https://github.com/Parallel-NetCDF/PnetCDF/issues/155), + [Issue #163](https://github.com/Parallel-NetCDF/PnetCDF/issues/163), + and [PR #164](https://github.com/Parallel-NetCDF/PnetCDF/pull/164). + +* New APIs + + A set of APIs that read the header of a CDL file header and allow users to + query the metadata defined in the CDL file. These APIs can be useful for + creating a program (such as an I/O benchmark) based on an output netCDF + file from another application (a CDL file can be generated by running the + utility `ncmpidump/ncdump`). See + [PR #177](https://github.com/Parallel-NetCDF/PnetCDF/pull/177). + * `cdl_hdr_open()` opens and parses the CDL file's header + * `cdl_hdr_inq_format()` returns file format version + * `cdl_hdr_inq_ndims()` returns number of dimensions defined in CDL file + * `cdl_hdr_inq_dim()` returns metadata of a dimension + * `cdl_hdr_inq_nvars()` returns number of variables + * `cdl_hdr_inq_var()` returns metadata of a variable defined in CDL file + * `cdl_hdr_inq_nattrs()` returns number of attributes of a given variable + * `cdl_hdr_inq_attr()` returns metadata of an attribute + * `cdl_hdr_close()` closes the CDL file + + The CDL Header C Reference Manual is available in + [cdl_api_guide.md](doc/cdl_api_guide.md). See + [PR #184](https://github.com/Parallel-NetCDF/PnetCDF/pull/184). + +* Bug fixes + + Fix setting of user hint nc_ibuf_size. + See [PR #161](https://github.com/Parallel-NetCDF/PnetCDF/pull/161). + +* New example programs + + examples/C/create_from_cdl.c shows how to call the new CDL header APIs to + create a netCDF file with the same metadata. + +* New programs for I/O benchmarks + + WRF-IO contains an extraction of the I/O kernel of WRF (Wether Research + and Forecast Model, a weather prediction computer simulation program + developed at NCAR) that can be used to evaluate the file I/O performance + of WRF. It's data partitioning pattern is a 2D block-block checkerboard + pattern, along the longitude and latitude. + See [PR #165](https://github.com/Parallel-NetCDF/PnetCDF/pull/165) + and [PR #181](https://github.com/Parallel-NetCDF/PnetCDF/pull/181). + +* New test program + + test/cdf/tst_cdl_hdr_parser.c tests the new CDL header APIs. + + test/testcases/tst_grow_header.c tests header extent growth by re-entering + the define mode multiple times and add more fix-sized and record variables. + +* Clarifications about of PnetCDF hints + + There are three ways in PnetCDF for user to set hints to align the starting + file offset for the data section (header extent) and record variable + section. + 1. through a call to API `nc_header_align_size` by setting arguments of + `h_minfree`, `v_align`, `v_minfree`, and `r_align`. + 2. through an MPI info object passed to calls of `ncmpi_create()` and + `ncmpi_open()`. Hints are `nc_header_align_size`, `nc_var_align_size`, + and `nc_record_align_size`. + 3. through a run-time environment variable `PNETCDF_HINTS`. Hints are + `nc_header_align_size`, `nc_var_align_size`, and `nc_record_align_size`. + + As the same hints may be set by one or more of the above methods, PnetCDF + implements the following hint precedence. + * `PNETCDF_HINTS` > `ncmpi__enddef()` > `MPI info`. + * 1st priority: hints set in the environment variable `PNETCDF_HINTS`, e.g. + `PNETCDF_HINTS="nc_var_align_size=1048576"`. Making this the first + priority is because it allows to run the same application executable + without source code modification using different alignment settings + through a run-time environment variable. + * 2nd priority: hints set in the MPI info object passed to calls of + `ncmpi_create()` and `ncmpi_open()`, e.g. + `MPI_Info_set("nc_var_align_size", "1048576");`. The reasoning is when a + 3rd-party library built on top of PnetCDF implements its codes using + 'ncmpi__enddef'. An application that uses such 3rd-party library can pass + an MPI info object to it, which further passes the info to PnetCDF. This + precedence allows that application to exercise different hints without + changing the 3rd-party library's source codes. + * 3rd priority: hints used in the arguments of `ncmpi__enddef()`, e.g. + `ncmpi__enddef(..., v_align=1048576,...)`. + + PnetCDF I/O hint `nc_header_align_size` is essentially the same as hint + `nc_var_align_size`, but its name appears to be closer to the hint's + intent, i.e. to reserve some space for the header growth in the future when + new data objects are added. Please note when both hints are set, only hint + `nc_var_align_size` will take effect and `nc_header_align_size` ignored. + + When there is no fix-sized variable (i.e. non-record variable) defined, + argument `v_minfree` passed to `ncmpi__enddef()` is ignored. In this + case, users should set `h_minfree`, if an extra header space is desired. + + When there is no fix-sized variables defined and none of hints + `nc_header_align_size`, `nc_var_align_size`, or argument `v_align` is set, + `nc_record_align_size` or `r_align`, if set, will be used to align the + header extent. + + For the above update to the hint precedence, see + PnetCDF See [PR #173](https://github.com/Parallel-NetCDF/PnetCDF/pull/173). + + +------------------------------------- +Version 1.14.0 (November 11, 2024) +------------------------------------- + * New features + Intra-node aggregation for write requests -- When the number of MPI processes allocated to a compute node is large, this feature can diff --git a/configure.ac b/configure.ac index fc9762f9c..e7561bcd2 100644 --- a/configure.ac +++ b/configure.ac @@ -15,7 +15,10 @@ dnl AC_REVISION([$Revision$])dnl dnl autoconf v2.70 and later is required. See https://github.com/Parallel-NetCDF/PnetCDF/issues/94 dnl autoconf v2.70 was released in 2021-01-28 AC_PREREQ([2.70]) -AC_INIT([PnetCDF],[1.14.0],[parallel-netcdf@mcs.anl.gov],[pnetcdf],[https://parallel-netcdf.github.io]) +AC_INIT([PnetCDF], [1.14.1], + [parallel-netcdf@mcs.anl.gov], + [pnetcdf], + [https://parallel-netcdf.github.io]) dnl config.h.in will be created by autoreconf (autoheader) dnl call it right after AC_INIT, as suggested by autoconf @@ -2643,7 +2646,7 @@ dnl Update the version information only immediately before a public release. dnl PnetCDF starts with 1:0:0 (shared library is first supported in 1.9.0) dnl because some package distributors, such as Debian, may have already built dnl PnetCDF with shared libraries. -ABIVERSION="6:0:0" +ABIVERSION="7:0:0" AC_SUBST(ABIVERSION) if test "$enable_versioning" = "yes" ; then ABIVERSIONFLAGS="-version-info \$(ABIVERSION)" diff --git a/sneak_peek.md b/sneak_peek.md index 7c8033e8a..363545063 100644 --- a/sneak_peek.md +++ b/sneak_peek.md @@ -6,19 +6,12 @@ This is essentially a placeholder for the next release note ... + none * New optimization - + When file header extent size grows, moving the data section to a higher - file offset has changed to be done in chunks of 16 MB per process. - See [PR #174](https://github.com/Parallel-NetCDF/PnetCDF/pull/174), + + none * New Limitations + none * Configure options - + For PnetCDF developers, the requirement for libtool version has been - changed to 2.5.4, due to an issue on Mac OS when using OpenMPI. See - [Issue #155](https://github.com/Parallel-NetCDF/PnetCDF/issues/155), - [Issue #163](https://github.com/Parallel-NetCDF/PnetCDF/issues/163), - and [PR #164](https://github.com/Parallel-NetCDF/PnetCDF/pull/164). * Configure updates: + none @@ -27,24 +20,7 @@ This is essentially a placeholder for the next release note ... + none * New APIs - + A set of APIs that read the header of a CDL file header and allow users to - query the metadata defined in the CDL file. These APIs can be useful for - creating a program (such as an I/O benchmark) based on an output netCDF - file from another application (a CDL file can be generated by running the - utility `ncmpidump/ncdump`). See - [PR #177](https://github.com/Parallel-NetCDF/PnetCDF/pull/177). - * `cdl_hdr_open()` opens and parses the CDL file's header - * `cdl_hdr_inq_format()` returns file format version - * `cdl_hdr_inq_ndims()` returns number of dimensions defined in CDL file - * `cdl_hdr_inq_dim()` returns metadata of a dimension - * `cdl_hdr_inq_nvars()` returns number of variables - * `cdl_hdr_inq_var()` returns metadata of a variable defined in CDL file - * `cdl_hdr_inq_nattrs()` returns number of attributes of a given variable - * `cdl_hdr_inq_attr()` returns metadata of an attribute - * `cdl_hdr_close()` closes the CDL file - + The CDL Header C Reference Manual is available in - [cdl_api_guide.md](doc/cdl_api_guide.md). See - [PR #184](https://github.com/Parallel-NetCDF/PnetCDF/pull/184). + + none * API syntax changes + none @@ -77,26 +53,16 @@ This is essentially a placeholder for the next release note ... + none * Bug fixes - + Fix setting of user hint nc_ibuf_size. - See [PR #161](https://github.com/Parallel-NetCDF/PnetCDF/pull/161). + + none * New example programs - + examples/C/create_from_cdl.c shows how to call the new CDL header APIs to - create a netCDF file with the same metadata. + + none * New programs for I/O benchmarks - + WRF-IO contains an extraction of the I/O kernel of WRF (Wether Research - and Forecast Model, a weather prediction computer simulation program - developed at NCAR) that can be used to evaluate the file I/O performance - of WRF. It's data partitioning pattern is a 2D block-block checkerboard - pattern, along the longitude and latitude. - See [PR #165](https://github.com/Parallel-NetCDF/PnetCDF/pull/165) - and [PR #181](https://github.com/Parallel-NetCDF/PnetCDF/pull/181). + + none * New test program - + test/cdf/tst_cdl_hdr_parser.c tests the new CDL header APIs. - + test/testcases/tst_grow_header.c tests header extent growth by re-entering - the define mode multiple times and add more fix-sized and record variables. + + none * Issues with NetCDF library + none @@ -114,46 +80,5 @@ This is essentially a placeholder for the next release note ... + none * Clarifications about of PnetCDF hints - + There are three ways in PnetCDF for user to set hints to align the starting - file offset for the data section (header extent) and record variable - section. - 1. through a call to API `nc_header_align_size` by setting arguments of - `h_minfree`, `v_align`, `v_minfree`, and `r_align`. - 2. through an MPI info object passed to calls of `ncmpi_create()` and - `ncmpi_open()`. Hints are `nc_header_align_size`, `nc_var_align_size`, - and `nc_record_align_size`. - 3. through a run-time environment variable `PNETCDF_HINTS`. Hints are - `nc_header_align_size`, `nc_var_align_size`, and `nc_record_align_size`. - + As the same hints may be set by one or more of the above methods, PnetCDF - implements the following hint precedence. - * `PNETCDF_HINTS` > `ncmpi__enddef()` > `MPI info`. - * 1st priority: hints set in the environment variable `PNETCDF_HINTS`, e.g. - `PNETCDF_HINTS="nc_var_align_size=1048576"`. Making this the first - priority is because it allows to run the same application executable - without source code modification using different alignment settings - through a run-time environment variable. - * 2nd priority: hints set in the MPI info object passed to calls of - `ncmpi_create()` and `ncmpi_open()`, e.g. - `MPI_Info_set("nc_var_align_size", "1048576");`. The reasoning is when a - 3rd-party library built on top of PnetCDF implements its codes using - 'ncmpi__enddef'. An application that uses such 3rd-party library can pass - an MPI info object to it, which further passes the info to PnetCDF. This - precedence allows that application to exercise different hints without - changing the 3rd-party library's source codes. - * 3rd priority: hints used in the arguments of `ncmpi__enddef()`, e.g. - `ncmpi__enddef(..., v_align=1048576,...)`. - + PnetCDF I/O hint `nc_header_align_size` is essentially the same as hint - `nc_var_align_size`, but its name appears to be closer to the hint's - intent, i.e. to reserve some space for the header growth in the future when - new data objects are added. Please note when both hints are set, only hint - `nc_var_align_size` will take effect and `nc_header_align_size` ignored. - + When there is no fix-sized variable (i.e. non-record variable) defined, - argument `v_minfree` passed to `ncmpi__enddef()` is ignored. In this - case, users should set `h_minfree`, if an extra header space is desired. - + When there is no fix-sized variables defined and none of hints - `nc_header_align_size`, `nc_var_align_size`, or argument `v_align` is set, - `nc_record_align_size` or `r_align`, if set, will be used to align the - header extent. - + For the above update to the hint precedence, see - PnetCDF See [PR #173](https://github.com/Parallel-NetCDF/PnetCDF/pull/173). + + none