Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 38 additions & 11 deletions DEVELOPER_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@
codes. PnetCDF uses the same constants and error codes as NetCDF. See
"Conform with netCDF" below.

2. Set the release version
2. create a new branch, for example named "prepare-1.14.1"
git checkout -B prepare-1.14.1
git add the bullet points 3 and 4 below to the new branch

3. Set the release version
In file `configure.ac`, variable `$PNETCDF_VERSION` indicates the version.
It is automatically generated from the 2nd argument of `AC_INIT` call.
Revise that string to the right release version. For example,
Expand All @@ -17,15 +21,15 @@
```
For setting shared library version (ABI versioning), see below.

3. Update file `RELEASE_NOTES`
4. Update file `RELEASE_NOTES`
Copy the contents of file `sneak_peek.md` to the top of file `RELEASE_NOTES`.
Clear up file `sneak_peek.md` (reset all items to none).
+ 1.9.0 and later - the release version and date in `RELEASE_NOTES` will
be automatically set when running command `make dist`.
+ 1.8.1 and priors - add the release version and date at the top
of file `RELEASE_NOTES`.

4. Update the release date (1.8.1 and priors)
5. Update the release date (1.8.1 and priors only)
For more info, see below section "Setting PnetCDF software release date".
Run command `svn commit` to get the svn property \$LastChangedDate\$
updated in file `configure.in`. That string will be used as the official
Expand All @@ -34,12 +38,15 @@
svn commit -m "set release date of version 1.6.1 to today"
```

5. Commit all changes to repo servers
6. Commit all changes to repo servers
* 1.8.1 and priors -- run `svn commit` to upload changes to SVN server.
* 1.9.0 and after -- run `git push origin master` to upload changes to
github.com.
* 1.9.0 and after --
+ `git commit -m "prepare release of 1.14.1"
+ `git push origin prepare-1.14.1` to upload changes to github.com.
+ Create a pull request at github.com
+ When all github actions passed, merge the PR.

6. Generate release tar ball (use it for testing)
7. Generate release tar ball (use it for testing)
* Generate a new `configure` file
+ Run command `autoreconf` to generate file `configure` to be included
in the release (but ignored by the SVN/Git repo).
Expand All @@ -57,7 +64,7 @@
`make dist`. This setting will be done automatically, unlike step 4
above that manually update the release date.

7. Test the new release tar ball (not source codes from the repo)
8. Test the new release tar ball (not source codes from the repo)
* build under the same directory as source
- run `configure` (with command-line options: `--enable-strict`,
`--enable-coverage`, `--disable-cxx`, `--disable-fortran` and their
Expand Down Expand Up @@ -125,7 +132,7 @@
run instructions. Remember to add new test/example programs added in the
new release.

8. Create a checkpoint
9. Create a checkpoint
* For 1.9.0 and priors - Create a new SVN tag on svn repo, by running
command below to duplicate the current trunk to a new tag:
```
Expand All @@ -139,7 +146,7 @@
git tag -a checkpoint.1.11.0 -m "Checkpoint right before 1.11.0 release"
git push origin checkpoint.1.11.0
```
9. Generate SHA1 checksums
10. Generate SHA1 checksums
* Run command:
```
openssl sha1 pnetcdf-1.11.0.tar.gz`
Expand All @@ -148,7 +155,7 @@
```
SHA1(pnetcdf-1.11.0.tar.gz)= 495d42f0a41abbd09d276262dce0f7c1c535968a
```
10. Update PnetCDF Web Page
11. Update PnetCDF Web Page
* https://github.com/Parallel-NetCDF/Parallel-NetCDF.github.io
* Create a new file of release note Parallel-NetCDF.github.io/Release_notes/1.11.0.md.
* Add a news item in index.html to announce the new release version.
Expand Down Expand Up @@ -208,6 +215,23 @@
the old version at runtime: set revision to 0, bump current and age.
3. Programs may need to be changed, recompiled, and relinked in order to
use the new version. Bump current, set revision and age to 0.

libtool Chapter 7.2 Libtool’s versioning system
So, libtool library versions are described by three integers:

current
The most recent interface number that this library implements.

revision
The implementation number of the current interface.

age
The difference between the newest and oldest interfaces that this
library implements. In other words, the library implements all the
interface numbers in the range from number current - age to current.

If two libraries have identical current and age numbers, then the dynamic
linker chooses the library with the greater revision number.
```
---
### Note on adding new MPI compiler candidates
Expand Down Expand Up @@ -319,6 +343,9 @@

---
### config.guess, config.sub, install-sh in directory scripts
There 3 files are automatically added when running command "autoreconf -i"
Below is just FYI.

Copy config.guess, config.sub, and install-sh from GNU libtool.
http://www.gnu.org/software/libtool/

Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ Northwestern University and Argonne National Laboratory.
* [Utility Programs](./src/utils#readme)
* [NetCDF4 vs. PnetCDF](./doc/netcdf4_vs_pnetcdf.md)
* PnetCDF [blocking vs. non-blocking APIs](./doc/blocking_vs_nonblocking.md)
* [CDL header API references](./doc/cdl_api_guide.md)

### Mailing List
* [email protected]
Expand Down
103 changes: 103 additions & 0 deletions RELEASE_NOTES
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,109 @@ PnetCDF Release Notes
Version _PNETCDF_VERSION_ (_PNETCDF_RELEASE_DATE_)
-------------------------------------

* New optimization
+ When file header extent size grows, moving the data section to a higher
file offset has changed to be done in chunks of 16 MB per process.
See [PR #174](https://github.com/Parallel-NetCDF/PnetCDF/pull/174),

* Configure options
+ For PnetCDF developers, the requirement for libtool version has been
changed to 2.5.4, due to an issue on Mac OS when using OpenMPI. See
[Issue #155](https://github.com/Parallel-NetCDF/PnetCDF/issues/155),
[Issue #163](https://github.com/Parallel-NetCDF/PnetCDF/issues/163),
and [PR #164](https://github.com/Parallel-NetCDF/PnetCDF/pull/164).

* New APIs
+ A set of APIs that read the header of a CDL file header and allow users to
query the metadata defined in the CDL file. These APIs can be useful for
creating a program (such as an I/O benchmark) based on an output netCDF
file from another application (a CDL file can be generated by running the
utility `ncmpidump/ncdump`). See
[PR #177](https://github.com/Parallel-NetCDF/PnetCDF/pull/177).
* `cdl_hdr_open()` opens and parses the CDL file's header
* `cdl_hdr_inq_format()` returns file format version
* `cdl_hdr_inq_ndims()` returns number of dimensions defined in CDL file
* `cdl_hdr_inq_dim()` returns metadata of a dimension
* `cdl_hdr_inq_nvars()` returns number of variables
* `cdl_hdr_inq_var()` returns metadata of a variable defined in CDL file
* `cdl_hdr_inq_nattrs()` returns number of attributes of a given variable
* `cdl_hdr_inq_attr()` returns metadata of an attribute
* `cdl_hdr_close()` closes the CDL file
+ The CDL Header C Reference Manual is available in
[cdl_api_guide.md](doc/cdl_api_guide.md). See
[PR #184](https://github.com/Parallel-NetCDF/PnetCDF/pull/184).

* Bug fixes
+ Fix setting of user hint nc_ibuf_size.
See [PR #161](https://github.com/Parallel-NetCDF/PnetCDF/pull/161).

* New example programs
+ examples/C/create_from_cdl.c shows how to call the new CDL header APIs to
create a netCDF file with the same metadata.

* New programs for I/O benchmarks
+ WRF-IO contains an extraction of the I/O kernel of WRF (Wether Research
and Forecast Model, a weather prediction computer simulation program
developed at NCAR) that can be used to evaluate the file I/O performance
of WRF. It's data partitioning pattern is a 2D block-block checkerboard
pattern, along the longitude and latitude.
See [PR #165](https://github.com/Parallel-NetCDF/PnetCDF/pull/165)
and [PR #181](https://github.com/Parallel-NetCDF/PnetCDF/pull/181).

* New test program
+ test/cdf/tst_cdl_hdr_parser.c tests the new CDL header APIs.
+ test/testcases/tst_grow_header.c tests header extent growth by re-entering
the define mode multiple times and add more fix-sized and record variables.

* Clarifications about of PnetCDF hints
+ There are three ways in PnetCDF for user to set hints to align the starting
file offset for the data section (header extent) and record variable
section.
1. through a call to API `nc_header_align_size` by setting arguments of
`h_minfree`, `v_align`, `v_minfree`, and `r_align`.
2. through an MPI info object passed to calls of `ncmpi_create()` and
`ncmpi_open()`. Hints are `nc_header_align_size`, `nc_var_align_size`,
and `nc_record_align_size`.
3. through a run-time environment variable `PNETCDF_HINTS`. Hints are
`nc_header_align_size`, `nc_var_align_size`, and `nc_record_align_size`.
+ As the same hints may be set by one or more of the above methods, PnetCDF
implements the following hint precedence.
* `PNETCDF_HINTS` > `ncmpi__enddef()` > `MPI info`.
* 1st priority: hints set in the environment variable `PNETCDF_HINTS`, e.g.
`PNETCDF_HINTS="nc_var_align_size=1048576"`. Making this the first
priority is because it allows to run the same application executable
without source code modification using different alignment settings
through a run-time environment variable.
* 2nd priority: hints set in the MPI info object passed to calls of
`ncmpi_create()` and `ncmpi_open()`, e.g.
`MPI_Info_set("nc_var_align_size", "1048576");`. The reasoning is when a
3rd-party library built on top of PnetCDF implements its codes using
'ncmpi__enddef'. An application that uses such 3rd-party library can pass
an MPI info object to it, which further passes the info to PnetCDF. This
precedence allows that application to exercise different hints without
changing the 3rd-party library's source codes.
* 3rd priority: hints used in the arguments of `ncmpi__enddef()`, e.g.
`ncmpi__enddef(..., v_align=1048576,...)`.
+ PnetCDF I/O hint `nc_header_align_size` is essentially the same as hint
`nc_var_align_size`, but its name appears to be closer to the hint's
intent, i.e. to reserve some space for the header growth in the future when
new data objects are added. Please note when both hints are set, only hint
`nc_var_align_size` will take effect and `nc_header_align_size` ignored.
+ When there is no fix-sized variable (i.e. non-record variable) defined,
argument `v_minfree` passed to `ncmpi__enddef()` is ignored. In this
case, users should set `h_minfree`, if an extra header space is desired.
+ When there is no fix-sized variables defined and none of hints
`nc_header_align_size`, `nc_var_align_size`, or argument `v_align` is set,
`nc_record_align_size` or `r_align`, if set, will be used to align the
header extent.
+ For the above update to the hint precedence, see
PnetCDF See [PR #173](https://github.com/Parallel-NetCDF/PnetCDF/pull/173).


-------------------------------------
Version 1.14.0 (November 11, 2024)
-------------------------------------

* New features
+ Intra-node aggregation for write requests -- When the number of MPI
processes allocated to a compute node is large, this feature can
Expand Down
7 changes: 5 additions & 2 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ dnl AC_REVISION([$Revision$])dnl
dnl autoconf v2.70 and later is required. See https://github.com/Parallel-NetCDF/PnetCDF/issues/94
dnl autoconf v2.70 was released in 2021-01-28
AC_PREREQ([2.70])
AC_INIT([PnetCDF],[1.14.0],[[email protected]],[pnetcdf],[https://parallel-netcdf.github.io])
AC_INIT([PnetCDF], [1.14.1],
[[email protected]],
[pnetcdf],
[https://parallel-netcdf.github.io])

dnl config.h.in will be created by autoreconf (autoheader)
dnl call it right after AC_INIT, as suggested by autoconf
Expand Down Expand Up @@ -2643,7 +2646,7 @@ dnl Update the version information only immediately before a public release.
dnl PnetCDF starts with 1:0:0 (shared library is first supported in 1.9.0)
dnl because some package distributors, such as Debian, may have already built
dnl PnetCDF with shared libraries.
ABIVERSION="6:0:0"
ABIVERSION="7:0:0"
AC_SUBST(ABIVERSION)
if test "$enable_versioning" = "yes" ; then
ABIVERSIONFLAGS="-version-info \$(ABIVERSION)"
Expand Down
89 changes: 7 additions & 82 deletions sneak_peek.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,12 @@ This is essentially a placeholder for the next release note ...
+ none

* New optimization
+ When file header extent size grows, moving the data section to a higher
file offset has changed to be done in chunks of 16 MB per process.
See [PR #174](https://github.com/Parallel-NetCDF/PnetCDF/pull/174),
+ none

* New Limitations
+ none

* Configure options
+ For PnetCDF developers, the requirement for libtool version has been
changed to 2.5.4, due to an issue on Mac OS when using OpenMPI. See
[Issue #155](https://github.com/Parallel-NetCDF/PnetCDF/issues/155),
[Issue #163](https://github.com/Parallel-NetCDF/PnetCDF/issues/163),
and [PR #164](https://github.com/Parallel-NetCDF/PnetCDF/pull/164).

* Configure updates:
+ none
Expand All @@ -27,24 +20,7 @@ This is essentially a placeholder for the next release note ...
+ none

* New APIs
+ A set of APIs that read the header of a CDL file header and allow users to
query the metadata defined in the CDL file. These APIs can be useful for
creating a program (such as an I/O benchmark) based on an output netCDF
file from another application (a CDL file can be generated by running the
utility `ncmpidump/ncdump`). See
[PR #177](https://github.com/Parallel-NetCDF/PnetCDF/pull/177).
* `cdl_hdr_open()` opens and parses the CDL file's header
* `cdl_hdr_inq_format()` returns file format version
* `cdl_hdr_inq_ndims()` returns number of dimensions defined in CDL file
* `cdl_hdr_inq_dim()` returns metadata of a dimension
* `cdl_hdr_inq_nvars()` returns number of variables
* `cdl_hdr_inq_var()` returns metadata of a variable defined in CDL file
* `cdl_hdr_inq_nattrs()` returns number of attributes of a given variable
* `cdl_hdr_inq_attr()` returns metadata of an attribute
* `cdl_hdr_close()` closes the CDL file
+ The CDL Header C Reference Manual is available in
[cdl_api_guide.md](doc/cdl_api_guide.md). See
[PR #184](https://github.com/Parallel-NetCDF/PnetCDF/pull/184).
+ none

* API syntax changes
+ none
Expand Down Expand Up @@ -77,26 +53,16 @@ This is essentially a placeholder for the next release note ...
+ none

* Bug fixes
+ Fix setting of user hint nc_ibuf_size.
See [PR #161](https://github.com/Parallel-NetCDF/PnetCDF/pull/161).
+ none

* New example programs
+ examples/C/create_from_cdl.c shows how to call the new CDL header APIs to
create a netCDF file with the same metadata.
+ none

* New programs for I/O benchmarks
+ WRF-IO contains an extraction of the I/O kernel of WRF (Wether Research
and Forecast Model, a weather prediction computer simulation program
developed at NCAR) that can be used to evaluate the file I/O performance
of WRF. It's data partitioning pattern is a 2D block-block checkerboard
pattern, along the longitude and latitude.
See [PR #165](https://github.com/Parallel-NetCDF/PnetCDF/pull/165)
and [PR #181](https://github.com/Parallel-NetCDF/PnetCDF/pull/181).
+ none

* New test program
+ test/cdf/tst_cdl_hdr_parser.c tests the new CDL header APIs.
+ test/testcases/tst_grow_header.c tests header extent growth by re-entering
the define mode multiple times and add more fix-sized and record variables.
+ none

* Issues with NetCDF library
+ none
Expand All @@ -114,46 +80,5 @@ This is essentially a placeholder for the next release note ...
+ none

* Clarifications about of PnetCDF hints
+ There are three ways in PnetCDF for user to set hints to align the starting
file offset for the data section (header extent) and record variable
section.
1. through a call to API `nc_header_align_size` by setting arguments of
`h_minfree`, `v_align`, `v_minfree`, and `r_align`.
2. through an MPI info object passed to calls of `ncmpi_create()` and
`ncmpi_open()`. Hints are `nc_header_align_size`, `nc_var_align_size`,
and `nc_record_align_size`.
3. through a run-time environment variable `PNETCDF_HINTS`. Hints are
`nc_header_align_size`, `nc_var_align_size`, and `nc_record_align_size`.
+ As the same hints may be set by one or more of the above methods, PnetCDF
implements the following hint precedence.
* `PNETCDF_HINTS` > `ncmpi__enddef()` > `MPI info`.
* 1st priority: hints set in the environment variable `PNETCDF_HINTS`, e.g.
`PNETCDF_HINTS="nc_var_align_size=1048576"`. Making this the first
priority is because it allows to run the same application executable
without source code modification using different alignment settings
through a run-time environment variable.
* 2nd priority: hints set in the MPI info object passed to calls of
`ncmpi_create()` and `ncmpi_open()`, e.g.
`MPI_Info_set("nc_var_align_size", "1048576");`. The reasoning is when a
3rd-party library built on top of PnetCDF implements its codes using
'ncmpi__enddef'. An application that uses such 3rd-party library can pass
an MPI info object to it, which further passes the info to PnetCDF. This
precedence allows that application to exercise different hints without
changing the 3rd-party library's source codes.
* 3rd priority: hints used in the arguments of `ncmpi__enddef()`, e.g.
`ncmpi__enddef(..., v_align=1048576,...)`.
+ PnetCDF I/O hint `nc_header_align_size` is essentially the same as hint
`nc_var_align_size`, but its name appears to be closer to the hint's
intent, i.e. to reserve some space for the header growth in the future when
new data objects are added. Please note when both hints are set, only hint
`nc_var_align_size` will take effect and `nc_header_align_size` ignored.
+ When there is no fix-sized variable (i.e. non-record variable) defined,
argument `v_minfree` passed to `ncmpi__enddef()` is ignored. In this
case, users should set `h_minfree`, if an extra header space is desired.
+ When there is no fix-sized variables defined and none of hints
`nc_header_align_size`, `nc_var_align_size`, or argument `v_align` is set,
`nc_record_align_size` or `r_align`, if set, will be used to align the
header extent.
+ For the above update to the hint precedence, see
PnetCDF See [PR #173](https://github.com/Parallel-NetCDF/PnetCDF/pull/173).
+ none

Loading