Skip to content

Commit a6e1a67

Browse files
author
abacus_fixer
committed
refactor(parallel): step-1a introduce ProcessTopology value class + divide_mpi_groups tests
Scope: 5 files changed, +482 lines (all additions; zero behavior change for existing code paths). No existing API touched. Build: cmake --build build exit=0. Test : OMP_NUM_THREADS=1 mpirun -np 4 MODULE_BASE_ProcessTopology -> 7/7 tests passed (5x divide_mpi_groups + 2x ProcessTopology). ProcessTopology (step 1a backbone, 规则1 / 规则2 / 规则3) - New header-only + cpp class ProcessTopology in parallel_topology.h/.cpp: * Immutably holds the six communicators that currently live as raw globals (POOL_WORLD / KP_WORLD / INT_BGROUP / BP_WORLD / GRID_WORLD / DIAG_WORLD) together with kpar / my_pool / rank_in_pool / nproc_in_pool[...] / bndpar / my_bndgroup / rank_in_bgroup / nproc_in_bgroup / world size&rank. * Value semantics (trivially copyable ints + vector, no ref members, no mutable workflow switches) - addresses the two weaknesses observed in the old MPICommGroup (reference aliases between ngroups <-> nprocs_in_inter and lack of RAII). * Head-only-interface-minimal: includes only <vector> and <mpi.h> conditionally; does NOT include parallel_comm.h, parallel_global.h or any GlobalV header. All state flows in via constructor args, so the class is unit-testable in isolation and does not contribute to the 6-global-comm transit-include surface cleaned in step 0. * Default constructor produces a single-process fallback so any non-__MPI build path (including LCAO serial sections) can still take a `const ProcessTopology&` and work. * pool_root_rank(pool) helper computes the world rank of a pool root without reaching for GlobalV arrays - needed by both the upcoming Parallel_Kpoints migration and cube-output rank calculations. - Base library wiring (CMakeLists rule, AGENTS.md deterministic-add requirement): add parallel_topology.cpp to the `base` OBJECT library source list directly after parallel_grid.cpp to keep the parallel_* cluster together. Nothing in the main build links the new object into paths that didn't already include `base`, so the incremental link cost is zero for unchanged modules. Parallel_Global::divide_mpi_groups tests (第一次为纯算术核心补单测) - New target MODULE_BASE_ProcessTopology in test_parallel/CMakeLists.txt, sources match ParaGlobal's transitive list plus parallel_topology.cpp. - parallel_topology_test.cpp covers: * DivideMpiGroups.EvenDivision 8 proc / 4 pools even * DivideMpiGroups.UnevenDivision 5 proc / 2 pools (3+2) * DivideMpiGroups.ExactlyOneProcessPerGroup - N proc / N pools * DivideMpiGroups.OneGroup 5 proc / 1 pool * DivideMpiGroups.UnevenLargePools 24 proc / 5 pools (5+5+5+5+4) Each uses a local helper `divide_all` that enumerates every rank 0..procs-1 against divide_mpi_groups, then `validate_divide` asserts the group sizes sum to procs and every in-group rank is unique within its group. This directly guards against regressions in the upcoming `create_topology` factory because that factory is the one caller that stitches `divide_mpi_groups` outputs into a ProcessTopology object. * ProcessTopology.DefaultConstructorIsSingleProcess (non-MPI fallback) * ProcessTopology.ConstructAndAccessors - explicit 10-proc / KPAR=3 / BNDPAR=2 construction, plus a copy-semantics check. Governance notes: - GlobalV / PARAM / GlobalC budget: added lines = 0 reads of any global. The new class never includes global_variable.h or PARAM headers; the constructor takes everything as explicit args. - Added includes in parallel_topology.h are both self-contained uses (<vector> is for member std::vector<int> nproc_in_pool_; <mpi.h> conditionally declares the MPI_Comm members). They do not leak declarations of the six global MPI_Comm variables - that surface remains isolated to parallel_comm.h. - No documentation change because no external interface (INPUT, CLI, public class API used by non-parallel code) changed. The class is a new internal component used via its constructor and accessors; Parallel_Global::create_topology (next step) wires it at the startup boundary where call sites already take explicit parameters. - C++11 compatible: using only std::vector, assert, int/long long POD types; no auto-return-type deduction, no std::move-only semantics required, no brace-init aggregates in new code beyond ProcessTopology copy test (already allowed in C++11). - One variable per declaration: ProcessTopology members and test helper variables each get their own line; no comma-joined declarations introduced.
1 parent 2a7696f commit a6e1a67

5 files changed

Lines changed: 482 additions & 0 deletions

File tree

‎source/source_base/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ add_library(
5757
parallel_reduce.cpp
5858
parallel_device.cpp
5959
parallel_grid.cpp
60+
parallel_topology.cpp
6061
sph_bessel_tf.cpp
6162
cubic_spline.cpp
6263
parallel_2d.cpp

‎source/source_base/parallel_topology.cpp‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
#include "parallel_topology.h"
2+
3+
#include <cassert>
4+
5+
ProcessTopology::ProcessTopology()
6+
: world_nproc_(1),
7+
my_rank_(0),
8+
kpar_(1),
9+
my_pool_(0),
10+
rank_in_pool_(0),
11+
nproc_in_pool_(1, 1),
12+
bndpar_(1),
13+
my_bndgroup_(0),
14+
rank_in_bgroup_(0),
15+
nproc_in_bgroup_(1)
16+
#ifdef __MPI
17+
,
18+
pool_comm_(MPI_COMM_SELF),
19+
kp_world_comm_(MPI_COMM_NULL),
20+
int_bgroup_comm_(MPI_COMM_SELF),
21+
bp_world_comm_(MPI_COMM_NULL),
22+
grid_comm_(MPI_COMM_NULL),
23+
diag_comm_(MPI_COMM_NULL)
24+
#endif
25+
{
26+
}
27+
28+
ProcessTopology::ProcessTopology(int world_nproc_in,
29+
int my_rank_in,
30+
int kpar_in,
31+
int my_pool_in,
32+
int rank_in_pool_in,
33+
const std::vector<int>& nproc_in_pool_in,
34+
int bndpar_in,
35+
int my_bndgroup_in,
36+
int rank_in_bgroup_in,
37+
int nproc_in_bgroup_in
38+
#ifdef __MPI
39+
,
40+
MPI_Comm pool_comm_in,
41+
MPI_Comm kp_world_comm_in,
42+
MPI_Comm int_bgroup_comm_in,
43+
MPI_Comm bp_world_comm_in,
44+
MPI_Comm grid_comm_in,
45+
MPI_Comm diag_comm_in
46+
#endif
47+
)
48+
: world_nproc_(world_nproc_in),
49+
my_rank_(my_rank_in),
50+
kpar_(kpar_in),
51+
my_pool_(my_pool_in),
52+
rank_in_pool_(rank_in_pool_in),
53+
nproc_in_pool_(nproc_in_pool_in),
54+
bndpar_(bndpar_in),
55+
my_bndgroup_(my_bndgroup_in),
56+
rank_in_bgroup_(rank_in_bgroup_in),
57+
nproc_in_bgroup_(nproc_in_bgroup_in)
58+
#ifdef __MPI
59+
,
60+
pool_comm_(pool_comm_in),
61+
kp_world_comm_(kp_world_comm_in),
62+
int_bgroup_comm_(int_bgroup_comm_in),
63+
bp_world_comm_(bp_world_comm_in),
64+
grid_comm_(grid_comm_in),
65+
diag_comm_(diag_comm_in)
66+
#endif
67+
{
68+
assert(world_nproc_ >= 1);
69+
assert(my_rank_ >= 0 && my_rank_ < world_nproc_);
70+
assert(kpar_ >= 1);
71+
assert(static_cast<int>(nproc_in_pool_.size()) == kpar_);
72+
int total = 0;
73+
for (int s : nproc_in_pool_)
74+
{
75+
assert(s >= 0);
76+
total += s;
77+
}
78+
assert(total == world_nproc_);
79+
assert(my_pool_ >= 0 && my_pool_ < kpar_);
80+
assert(rank_in_pool_ >= 0 && rank_in_pool_ < nproc_in_pool_[my_pool_]);
81+
82+
assert(bndpar_ >= 1);
83+
assert(my_bndgroup_ >= 0 && my_bndgroup_ < bndpar_);
84+
assert(nproc_in_bgroup_ >= 1);
85+
assert(bndpar_ * nproc_in_bgroup_ == world_nproc_);
86+
assert(rank_in_bgroup_ >= 0 && rank_in_bgroup_ < nproc_in_bgroup_);
87+
}
88+
89+
int ProcessTopology::pool_root_rank(int pool) const
90+
{
91+
if (pool < 0 || pool >= kpar_)
92+
{
93+
return -1;
94+
}
95+
int offset = 0;
96+
for (int i = 0; i < pool; ++i)
97+
{
98+
offset += nproc_in_pool_[i];
99+
}
100+
return offset;
101+
}

‎source/source_base/parallel_topology.h‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
#ifndef PARALLEL_TOPOLOGY_H
2+
#define PARALLEL_TOPOLOGY_H
3+
4+
#include <vector>
5+
6+
#ifdef __MPI
7+
#include <mpi.h>
8+
#endif
9+
10+
/**
11+
* @brief Immutable description of the ABACUS process topology.
12+
*
13+
* Replaces the six raw-global MPI_Comm (POOL_WORLD / KP_WORLD /
14+
* INT_BGROUP / BP_WORLD / GRID_WORLD / DIAG_WORLD) together with the
15+
* associated pool / band-group rank & size info from GlobalV.
16+
* Values captured at construction time; the object is copyable
17+
* (value semantics) and exposes no mutable workflow switches.
18+
*
19+
* Consumers currently reading the global MPI_Comm / GlobalV fields
20+
* should migrate to taking a `const ProcessTopology&` from their
21+
* callers. Parallel_Global::create_topology(...) produces one such
22+
* instance at startup whose communicators are also aliased back to
23+
* the legacy globals so existing code keeps working during the
24+
* migration.
25+
*/
26+
class ProcessTopology
27+
{
28+
public:
29+
/// Trivially-constructible serial fallback. Produces a single-
30+
/// process, single-pool topology suitable for non-__MPI builds.
31+
ProcessTopology();
32+
33+
/**
34+
* @brief Full constructor. Intended for Parallel_Global factory.
35+
*
36+
* All inputs are plain values; GlobalV is never read inside the
37+
* constructor. Pass MPI_COMM_NULL for communicators that are
38+
* unused on this rank (same semantics as legacy divide_pools).
39+
*/
40+
ProcessTopology(int world_nproc_in,
41+
int my_rank_in,
42+
int kpar_in,
43+
int my_pool_in,
44+
int rank_in_pool_in,
45+
const std::vector<int>& nproc_in_pool_in,
46+
int bndpar_in,
47+
int my_bndgroup_in,
48+
int rank_in_bgroup_in,
49+
int nproc_in_bgroup_in
50+
#ifdef __MPI
51+
,
52+
MPI_Comm pool_comm_in,
53+
MPI_Comm kp_world_comm_in,
54+
MPI_Comm int_bgroup_comm_in,
55+
MPI_Comm bp_world_comm_in,
56+
MPI_Comm grid_comm_in,
57+
MPI_Comm diag_comm_in
58+
#endif
59+
);
60+
61+
// world ----------------------------------------------------------
62+
int world_size() const { return world_nproc_; }
63+
int world_rank() const { return my_rank_; }
64+
65+
// k-point pools -------------------------------------------------
66+
int kpar() const { return kpar_; }
67+
int my_pool() const { return my_pool_; }
68+
int rank_in_pool() const { return rank_in_pool_; }
69+
const std::vector<int>& nproc_in_pool() const { return nproc_in_pool_; }
70+
int nproc_in_pool(int pool) const { return nproc_in_pool_[pool]; }
71+
/// World rank of the root (rank 0) of a given pool; -1 on bad index.
72+
int pool_root_rank(int pool) const;
73+
74+
// band groups (bndpar) ------------------------------------------
75+
int bndpar() const { return bndpar_; }
76+
int my_bndgroup() const { return my_bndgroup_; }
77+
int rank_in_bgroup() const { return rank_in_bgroup_; }
78+
int nproc_in_bgroup() const { return nproc_in_bgroup_; }
79+
80+
#ifdef __MPI
81+
// communicators -------------------------------------------------
82+
MPI_Comm pool_comm() const { return pool_comm_; }
83+
MPI_Comm kp_world_comm() const { return kp_world_comm_; }
84+
MPI_Comm int_bgroup_comm() const { return int_bgroup_comm_; }
85+
MPI_Comm bp_world_comm() const { return bp_world_comm_; }
86+
MPI_Comm grid_comm() const { return grid_comm_; }
87+
MPI_Comm diag_comm() const { return diag_comm_; }
88+
#endif
89+
90+
private:
91+
int world_nproc_ = 1;
92+
int my_rank_ = 0;
93+
94+
int kpar_ = 1;
95+
int my_pool_ = 0;
96+
int rank_in_pool_ = 0;
97+
std::vector<int> nproc_in_pool_;
98+
99+
int bndpar_ = 1;
100+
int my_bndgroup_ = 0;
101+
int rank_in_bgroup_ = 0;
102+
int nproc_in_bgroup_ = 1;
103+
104+
#ifdef __MPI
105+
MPI_Comm pool_comm_ = MPI_COMM_SELF;
106+
MPI_Comm kp_world_comm_ = MPI_COMM_NULL;
107+
MPI_Comm int_bgroup_comm_ = MPI_COMM_SELF;
108+
MPI_Comm bp_world_comm_ = MPI_COMM_NULL;
109+
MPI_Comm grid_comm_ = MPI_COMM_NULL;
110+
MPI_Comm diag_comm_ = MPI_COMM_NULL;
111+
#endif
112+
};
113+
114+
#endif // PARALLEL_TOPOLOGY_H

‎source/source_base/test_parallel/CMakeLists.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,12 @@ AddTest(
1010
SOURCES parallel_global_test.cpp ../global_variable.cpp ../parallel_global.cpp ../parallel_comm.cpp ../tool_quit.cpp ../global_file.cpp ../global_function.cpp ../memory_recorder.cpp ../timer.cpp ../parallel_reduce.cpp
1111
)
1212

13+
AddTest(
14+
TARGET MODULE_BASE_ProcessTopology
15+
LIBS parameter MPI::MPI_CXX
16+
SOURCES parallel_topology_test.cpp ../global_variable.cpp ../parallel_global.cpp ../parallel_comm.cpp ../tool_quit.cpp ../global_file.cpp ../global_function.cpp ../memory_recorder.cpp ../timer.cpp ../parallel_reduce.cpp ../parallel_topology.cpp
17+
)
18+
1319
AddTest(
1420
TARGET MODULE_BASE_ParaReduce
1521
LIBS parameter MPI::MPI_CXX

0 commit comments

Comments
 (0)