Skip to content

Fix: Correct PW KPAR/BNDPAR occupations and distributed output - #7845

Merged
mohanchen merged 11 commits into
deepmodeling:developfrom
AsTonyshment:fix/bndpar_kpar
Aug 23, 2026
Merged

Fix: Correct PW KPAR/BNDPAR occupations and distributed output#7845
mohanchen merged 11 commits into
deepmodeling:developfrom
AsTonyshment:fix/bndpar_kpar

Conversation

@AsTonyshment

Copy link
Copy Markdown
Collaborator

Summary

This PR fixes several PW KPAR/BNDPAR correctness issues. The changes correct fixed and explicitly supplied occupations under distributed k-point and band layouts, reconstruct complete band data before shared-file output, and fix the real-space grid collection used by charge-density .cube output.

The kpar, bndpar, and ocp_set parameter descriptions are also updated to match the current implementation.

Problems addressed

  • With smearing_method=fixed and bndpar>1, every band group treated its local band zero as global band zero. Multiple groups therefore occupied their first local bands repeatedly, producing incorrect electron counts, charge densities, band energies, and SCF results.
  • ocp_set validated its input length using the pool-local number of k-points and indexed the input with a local-band stride. KPAR could reject a correctly sized input, while BNDPAR could silently assign occupations to the wrong bands.
  • out_chg used source ranks derived from a larger process layout as ranks in POOL_WORLD. Under BPCG this could pass an invalid communicator rank to MPI_Recv after the SCF calculation had completed.
  • out_band and eigenvalue/occupation output treated band-local matrices as if they contained all global bands. This could cause out-of-bounds access, omit bands owned by other band groups, duplicate k-point contributions, or allow multiple groups to write the same shared file.

Validation

A standalone edge-case test was carried out with 21 calculations: 17 CPU cases and 4 GPU cases. PW CPU cases used 6 MPI ranks, 8 k-points, 7 bands, and BPCG. The following layouts were compared:

Layout Coverage
kpar=1,bndpar=1 Reference layout
kpar=2,bndpar=1 KPAR only
kpar=1,bndpar=3 Uneven band ownership, 3+2+2
kpar=2,bndpar=3 Mixed KPAR/BNDPAR
kpar=3,bndpar=2 Uneven k-point pools, 3+3+2, and uneven band ownership, 4+3

The edge-case test covered nspin=1 and nspin=2, fixed occupations, k-point-dependent ocp_set values, PW CPU/GPU execution, and LCAO temporary kpar=1/2/3 diagonalization. Every PW case checked the complete occupation table, total energy, band table, charge-density .cube values, and .cube integral.

All 21 cases completed successfully and reported no failures.

Test group Maximum energy difference / eV Maximum band-table difference Maximum Cube point difference Maximum Cube integral difference
PW CPU 7.190692e-12 1.534826e-10 2.299966e-13 1.199041e-14
LCAO CPU 8.526513e-14 1.163514e-13 1.000589e-14 2.042810e-14
PW GPU 0.000000e+00 3.730349e-14 1.000589e-14 5.329071e-15

@AsTonyshment
AsTonyshment requested review from mohanchen and a lite review from Copilot and removed request for Copilot August 21, 2026 10:06
@AsTonyshment
AsTonyshment requested a lite review from Copilot and removed request for Copilot August 21, 2026 11:58
@mohanchen mohanchen added Refactor Refactor ABACUS codes Bugs Bugs that only solvable with sufficient knowledge of DFT labels Aug 21, 2026
@mohanchen

Copy link
Copy Markdown
Collaborator

Could you add some unit tests or integrate tests?

Copilot AI lite review requested due to automatic review settings August 22, 2026 07:00
@AsTonyshment

Copy link
Copy Markdown
Collaborator Author

Could you add some unit tests or integrate tests?

Added. I also found that get_pchg_pw and get_wf_pw may obtain incorrect results under BNDPAR (I only considered KPAR in the original implementation), and this is fixed as well. Relevant tests are updated.

@AsTonyshment
AsTonyshment removed the request for review from Copilot August 22, 2026 07:02
Copilot AI lite review requested due to automatic review settings August 22, 2026 07:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes PW KPAR/BNDPAR occupation handling and distributed band, charge-density, and cube output.

Changes:

  • Corrects global band and ocp_set occupation mapping.
  • Reconstructs distributed output data across MPI layouts.
  • Updates documentation and adds regression coverage.

Critical blockers remain in source/source_base/parallel_grid.cpp and source/source_estate/elecstate_tools.cpp.

Reviewed changes

Copilot reviewed 51 out of 52 changed files in this pull request and generated 2 comments.

Show a summary per file
File Review status
tests/integrate/tools/catch_properties.sh No final issue reported.
tests/06_SDFT/14_PW_SDFT_10D10S_METHD2/result.ref No final issue reported.
tests/06_SDFT/14_PW_SDFT_10D10S_METHD2/INPUT No final issue reported.
tests/06_SDFT/14_PW_SDFT_10D10S_METHD2/band.txt.ref No final issue reported.
tests/01_PW/CASES_GPU.txt No final issue reported.
tests/01_PW/CASES_CPU.txt No final issue reported.
tests/01_PW/211_PW_BPCG_KB_OCP_CHG/STRU No final issue reported.
tests/01_PW/211_PW_BPCG_KB_OCP_CHG/result.ref No final issue reported.
tests/01_PW/211_PW_BPCG_KB_OCP_CHG/README No final issue reported.
tests/01_PW/211_PW_BPCG_KB_OCP_CHG/KPT No final issue reported.
tests/01_PW/211_PW_BPCG_KB_OCP_CHG/INPUT No final issue reported.
tests/01_PW/211_PW_BPCG_KB_OCP_CHG/chg.cube.ref No final issue reported.
tests/01_PW/089_PW_get_wf_kpar/result.ref No final issue reported.
tests/01_PW/089_PW_get_wf_kpar/README No final issue reported.
tests/01_PW/089_PW_get_wf_kpar_bndpar/STRU No final issue reported.
tests/01_PW/089_PW_get_wf_kpar_bndpar/result.ref No final issue reported.
tests/01_PW/089_PW_get_wf_kpar_bndpar/README No final issue reported.
tests/01_PW/089_PW_get_wf_kpar_bndpar/KPT No final issue reported.
tests/01_PW/089_PW_get_wf_kpar_bndpar/INPUT No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar/result.ref No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar/README No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar/KPT No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar_bndpar/STRU No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar_bndpar/result.ref No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar_bndpar/README No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar_bndpar/KPT No final issue reported.
tests/01_PW/087_PW_get_pchg_kpar_bndpar/INPUT No final issue reported.
source/source_io/test/CMakeLists.txt No final issue reported.
source/source_io/test_serial/CMakeLists.txt No final issue reported.
source/source_io/module_wf/get_wf_pw.h No final issue reported.
source/source_io/module_parameter/read_inp_tddft.cpp No final issue reported.
source/source_io/module_parameter/read_inp_sys.cpp No final issue reported.
source/source_io/module_output/write_cube.cpp No final issue reported.
source/source_io/module_output/band_parallel_output.h No final issue reported.
source/source_io/module_output/band_parallel_output.cpp No final issue reported.
source/source_io/module_energy/write_eig_occ.cpp No final issue reported.
source/source_io/module_energy/write_bands.cpp No final issue reported.
source/source_io/module_ctrl/ctrl_output_pw.cpp No final issue reported.
source/source_io/module_chgpot/get_pchg_pw.h No final issue reported.
source/source_io/CMakeLists.txt No final issue reported.
source/source_estate/test/elecstate_occupy_test.cpp No final issue reported.
source/source_estate/test/elecstate_base_test.cpp No final issue reported.
source/source_estate/occupy.h No final issue reported.
source/source_estate/occupy.cpp No final issue reported.
source/source_estate/elecstate_tools.cpp Critical issue: replicated non-BPCG SDFT band storage receives an incorrect prefix offset.
source/source_cell/parallel_kpoints.cpp No final issue reported.
source/source_cell/cal_atoms_info.h No final issue reported.
source/source_base/parallel_grid.cpp Critical issue: grid distributions do not match the communicator used for KPAR+BNDPAR collection.
source/Makefile.Objects No final issue reported.
docs/parameters.yaml No final issue reported.
docs/advanced/input_files/input-main.md No final issue reported.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread source/source_base/parallel_grid.cpp
Comment thread source/source_estate/elecstate_tools.cpp Outdated
Comment thread tests/01_PW/087_PW_get_pchg_kpar_bndpar/pchgi4s1.cube.ref

@mohanchen mohanchen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mohanchen
mohanchen merged commit 75145da into deepmodeling:develop Aug 23, 2026
17 checks passed
@AsTonyshment
AsTonyshment deleted the fix/bndpar_kpar branch August 23, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bugs Bugs that only solvable with sufficient knowledge of DFT Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants