[BUG] pt_expt drops nopbc metadata but retains the zero box placeholder in normalize_batch #6002

Bug summary

For a valid non-periodic DeepMD system containing nopbc and no box.npy,
DeepmdData creates an all-zero box placeholder and
DeepmdDataSystem adds an empty default_mesh to encode non-periodicity.

deepmd.dpmodel.utils.batch.normalize_batch() then drops default_mesh as
metadata while retaining the all-zero box. The pt_expt training path calls
this function and passes the retained box to the model. Downstream DPA4 /
neighbor-graph code treats every non-None box as periodic and attempts to
normalize coordinates with the singular zero cell.

The non-periodic semantic is therefore lost between DeepmdDataSystem and the
model. Depending on the execution/compilation path, this can cause a singular
matrix failure, non-finite intermediates, or silently incorrect training.

DeePMD-kit Version

v3.2.0, official tag and current master commit
8cfd46e37448feca361ff3427a6c2b514ce5a2bb.

The reproducing conda package reports deepmd-kit 3.2.0. Its
deepmd/dpmodel/utils/batch.py and deepmd/utils/data.py are byte-identical to
the files at the official v3.2.0 tag.

Backend and its version

• backend: pt_expt (PyTorch Experimental)
• PyTorch: 2.12.0
• Python: 3.12.14
• NumPy: 2.5.2
• array-api-compat: 1.15.0
• CUDA runtime packages: 13.3
• NVIDIA driver: 580.95.05

The minimal reproducer reaches the faulty shared batch/region code on CPU and
does not require a GPU.

How did you download the software?

Conda (conda-forge build
deepmd-kit 3.2.0 cuda130py312h3eba875_mpi_openmpi_4).

Input Files, Running Commands, Error Log, etc.

Save the following as reproduce_nopbc_zero_box.py:

#!/usr/bin/env python3
"""Minimal reproducer for lost nopbc semantics in normalize_batch()."""

from __future__ import annotations

import tempfile
from pathlib import Path

import numpy as np

from deepmd.dpmodel.utils.batch import normalize_batch, split_batch
from deepmd.dpmodel.utils.region import normalize_coord
from deepmd.utils.data_system import DeepmdDataSystem


with tempfile.TemporaryDirectory() as tmpdir:
    system = Path(tmpdir) / "nopbc"
    set_dir = system / "set.000"
    set_dir.mkdir(parents=True)

    # A valid non-periodic H2 system: nopbc is present and box.npy is absent.
    (system / "type.raw").write_text("0 0\n")
    (system / "type_map.raw").write_text("H\n")
    (system / "nopbc").touch()
    np.save(
        set_dir / "coord.npy",
        np.array([[0.0, 0.0, 0.0, 0.74, 0.0, 0.0]], dtype=np.float64),
    )

    data_system = DeepmdDataSystem(
        [str(system)],
        batch_size=1,
        test_size=1,
        rcut=6.0,
        type_map=["H"],
        trn_all_set=True,
        shuffle_test=False,
    )
    raw = data_system.get_batch()
    normalized = normalize_batch(raw)
    inputs, _ = split_batch(normalized)

    print("pbc:", data_system.data_systems[0].pbc)
    print("raw default_mesh size:", raw["default_mesh"].size)
    print("raw box:", raw["box"])
    print("normalized has default_mesh:", "default_mesh" in normalized)
    print("normalized box is None:", inputs["box"] is None)
    print("normalized box all zero:", np.allclose(inputs["box"], 0.0))

    try:
        normalize_coord(
            inputs["coord"].reshape(1, 2, 3),
            inputs["box"].reshape(1, 3, 3),
        )
    except Exception as exc:
        print("normalize_coord exception:", type(exc).__name__, str(exc))

Run it with:

python reproduce_nopbc_zero_box.py

Observed output:

pbc: False
raw default_mesh size: 0
raw box: [[0. 0. 0. 0. 0. 0. 0. 0. 0.]]
normalized has default_mesh: False
normalized box is None: False
normalized box all zero: True
normalize_coord exception: LinAlgError Singular matrix

Expected behavior:

normalized box is None: True

No periodic coordinate normalization should be attempted for this system.

Steps to Reproduce

1. Create a standard DeepMD NPY system with type.raw, type_map.raw,
   set.000/coord.npy, and an empty nopbc marker, but no box.npy.
2. Load one batch using DeepmdDataSystem.
3. Observe that the raw batch has default_mesh.size == 0 and an all-zero
   box placeholder.
4. Pass the batch through normalize_batch() and split_batch().
5. Observe that default_mesh has been dropped while box remains an
   all-zero array rather than None.
6. Passing that box to normalize_coord() attempts to invert the singular
   zero cell.

The script performs all six steps in a temporary directory and does not use
external data.

Code path

1. deepmd/utils/data.py
   • registers box with must=self.pbc and default 0.0;
   • when nopbc is present and box.npy is absent, _load_data() creates an
     all-zero placeholder with np.full().
2. deepmd/common.py and deepmd/utils/data_system.py
   • make_default_mesh(False, False) returns an empty array;
   • DeepmdDataSystem.get_batch() adds it as default_mesh.
3. deepmd/dpmodel/utils/batch.py
   • _DROP_KEYS contains default_mesh;
   • normalize_batch() drops it but retains box as a model input.
4. deepmd/pt_expt/train/training.py
   • calls normalize_batch(data_sys.get_batch()), then split_batch().
5. deepmd/dpmodel/utils/neighbor_graph/builder.py
   • treats box is not None as periodic and calls normalize_coord().
6. deepmd/dpmodel/utils/region.py
   • calls xp.linalg.inv(cell), which is invalid for the zero cell.

There is consistent handling elsewhere in the same repository:
dpa_adapt/finetuner.py converts nopbc or all-zero boxes to None.

Suggested fix and regression coverage

Before dropping default_mesh, normalize_batch() could translate its encoded
PBC state into the canonical model input:

default_mesh = batch.get("default_mesh")
is_nonperiodic = default_mesh is not None and np.size(default_mesh) in (0, 1)
...
if is_nonperiodic:
    out["box"] = None

The sizes (0, 1) cover standard and mixed-type non-periodic systems according
to make_default_mesh(); periodic systems use sizes (6, 7).

Suggested regression tests should cover standard and mixed-type non-periodic
batches, verify box is None after normalization, and verify that periodic
boxes remain unchanged.

Further information

In a four-replica DPA4 fine-tuning workflow using only valid nopbc systems,
the unconverted zero box caused force errors to degrade immediately from the
first training step. A local compatibility shim that sets box=None when the
raw default_mesh is empty restored normal training in controlled runs; model,
data, optimizer, seed, and learning-rate settings were otherwise held fixed.

GitHub issue searches for normalize_batch default_mesh, zero box nopbc,
non-periodic box pt_expt, and nopbc DPA4 training found no existing report
in deepmodeling/deepmd-kit as of 2026-08-26.