fix(api_c): preserve exact bytes in DP_ReadFileToChar2 (deepmodeling#5763)

## Fix deepmodeling#5620

### Problem

`DP_ReadFileToChar气Char2` reported the original file size but returned a
buffer from `string_to_char`, which trims trailing whitespace before
allocating/copying. The C++ wrapper (`read_file_to_string` in
`deepmd.hpp`) then reconstructed a `std::string` with the reported
(larger) size, causing an over-read of the shorter allocation.

### Fix

Added `string_to_char_exact`, a new helper that preserves every byte
without trimming, and use it in `DP_ReadFileToChar2` and
`DP_ReadFileToChar`. Error-message paths still use the trimming
`string_to_char` since whitespace trimming is desirable there.

### Test

Added both a C++ test (`test_read_file_to_string.cc`) and a Python test
(`test_c_api_readfile.py`) that verify trailing whitespace is preserved.
The Python test uses `ctypes` to call `DP_ReadFileToChar2` directly and
the C API.

Verified: the test fails against the buggy code (over-reads garbage
bytes) and passes with the fix.

### Attribution

Generated with [opencode](https://opencode.ai) using model `glm-5.2`.

### Recommended reviewers

@njzjz (maintainer of the C API)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added per-frame `charge_spin` support to DeepPot and model-deviation C
API computations.
- Introduced “version 3” compute entry points that accept `charge_spin`
(including neighbor-list variants).
  - Added APIs to query the required charge-spin dimension.
- **Bug Fixes**
- File-reading C APIs now preserve exact bytes, including trailing
whitespace/newlines, and add safer size handling.
- **Tests**
- Added regression tests (C++ and Python) validating byte-for-byte file
content preservation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: njzjz-bot <njzjz.bot@gmail.com>

Author: njzjz-bot

source/api_c/include/c_api.h

@@ -682,25 +682,26 @@ extern void DP_DeepPotComputeNListf2(DP_DeepPot* dp,
  * @param[in] dp The DP to use.
  * @param[in] nframes The number of frames.
  * @param[in] natoms The number of atoms.
- * @param[in] coord The coordinates of atoms. The array should be of size natoms
- *x 3.
+ * @param[in] coord The coordinates of atoms. The array should be of size
+ *nframes x natoms x 3.
  * @param[in] atype The atom types. The array should contain natoms ints.
- * @param[in] cell The cell of the region. The array should be of size 9. Pass
- *NULL if pbc is not used.
+ * @param[in] cell The cell of the region. The array should be of size nframes x
+ *9. Pass NULL if pbc is not used.
  * @param[in] fparam The frame parameters. The array can be of size nframes x
  *dim_fparam.
  * @param[in] aparam The atom parameters. The array can be of size nframes x
  *natoms x dim_aparam.
  * @param[in] charge_spin The per-frame charge/spin input. The array can be of
  *size nframes x dim_chg_spin. Pass NULL to use the model's stored
  *default_chg_spin.
- * @param[out] energy Output energy.
- * @param[out] force Output force. The array should be of size natoms x 3.
- * @param[out] virial Output virial. The array should be of size 9.
+ * @param[out] energy Output energy. The array should be of size nframes.
+ * @param[out] force Output force. The array should be of size nframes x natoms
+ *x 3.
+ * @param[out] virial Output virial. The array should be of size nframes x 9.
  * @param[out] atomic_energy Output atomic energy. The array should be of size
- *natoms.
+ *nframes x natoms.
  * @param[out] atomic_virial Output atomic virial. The array should be of size
- *natoms x 9.
+ *nframes x natoms x 9.
  * @warning The output arrays should be allocated before calling this function.
  *Pass NULL if not required.
  * @since API version 27
@@ -727,25 +728,26 @@ extern void DP_DeepPotCompute3(DP_DeepPot* dp,
  * @param[in] dp The DP to use.
  * @param[in] nframes The number of frames.
  * @param[in] natoms The number of atoms.
- * @param[in] coord The coordinates of atoms. The array should be of size natoms
- *x 3.
+ * @param[in] coord The coordinates of atoms. The array should be of size
+ *nframes x natoms x 3.
  * @param[in] atype The atom types. The array should contain natoms ints.
- * @param[in] cell The cell of the region. The array should be of size 9. Pass
- *NULL if pbc is not used.
+ * @param[in] cell The cell of the region. The array should be of size nframes x
+ *9. Pass NULL if pbc is not used.
  * @param[in] fparam The frame parameters. The array can be of size nframes x
  *dim_fparam.
  * @param[in] aparam The atom parameters. The array can be of size nframes x
  *natoms x dim_aparam.
  * @param[in] charge_spin The per-frame charge/spin input. The array can be of
  *size nframes x dim_chg_spin. Pass NULL to use the model's stored
  *default_chg_spin.
- * @param[out] energy Output energy.
- * @param[out] force Output force. The array should be of size natoms x 3.
- * @param[out] virial Output virial. The array should be of size 9.
+ * @param[out] energy Output energy. The array should be of size nframes.
+ * @param[out] force Output force. The array should be of size nframes x natoms
+ *x 3.
+ * @param[out] virial Output virial. The array should be of size nframes x 9.
  * @param[out] atomic_energy Output atomic energy. The array should be of size
- *natoms.
+ *nframes x natoms.
  * @param[out] atomic_virial Output atomic virial. The array should be of size
- *natoms x 9.
+ *nframes x natoms x 9.
  * @warning The output arrays should be allocated before calling this function.
  *Pass NULL if not required.
  * @since API version 27
@@ -772,11 +774,11 @@ extern void DP_DeepPotComputef3(DP_DeepPot* dp,
  * @param[in] dp The DP to use.
  * @param[in] nframes The number of frames.
  * @param[in] natoms The number of atoms.
- * @param[in] coord The coordinates of atoms. The array should be of size natoms
- *x 3.
+ * @param[in] coord The coordinates of atoms. The array should be of size
+ *nframes x natoms x 3.
  * @param[in] atype The atom types. The array should contain natoms ints.
- * @param[in] cell The cell of the region. The array should be of size 9. Pass
- *NULL if pbc is not used.
+ * @param[in] cell The cell of the region. The array should be of size nframes x
+ *9. Pass NULL if pbc is not used.
  * @param[in] nghost The number of ghost atoms.
  * @param[in] nlist The neighbor list.
  * @param[in] ago Update the internal neighbour list if ago is 0.
@@ -787,13 +789,14 @@ extern void DP_DeepPotComputef3(DP_DeepPot* dp,
  * @param[in] charge_spin The per-frame charge/spin input. The array can be of
  *size nframes x dim_chg_spin. Pass NULL to use the model's stored
  *default_chg_spin.
- * @param[out] energy Output energy.
- * @param[out] force Output force. The array should be of size natoms x 3.
- * @param[out] virial Output virial. The array should be of size 9.
+ * @param[out] energy Output energy. The array should be of size nframes.
+ * @param[out] force Output force. The array should be of size nframes x natoms
+ *x 3.
+ * @param[out] virial Output virial. The array should be of size nframes x 9.
  * @param[out] atomic_energy Output atomic energy. The array should be of size
- *natoms.
+ *nframes x natoms.
  * @param[out] atomic_virial Output atomic virial. The array should be of size
- *natoms x 9.
+ *nframes x natoms x 9.
  * @warning The output arrays should be allocated before calling this function.
  *Pass NULL if not required.
  * @since API version 27
@@ -823,11 +826,11 @@ extern void DP_DeepPotComputeNList3(DP_DeepPot* dp,
  * @param[in] dp The DP to use.
  * @param[in] nframes The number of frames.
  * @param[in] natoms The number of atoms.
- * @param[in] coord The coordinates of atoms. The array should be of size natoms
- *x 3.
+ * @param[in] coord The coordinates of atoms. The array should be of size
+ *nframes x natoms x 3.
  * @param[in] atype The atom types. The array should contain natoms ints.
- * @param[in] cell The cell of the region. The array should be of size 9. Pass
- *NULL if pbc is not used.
+ * @param[in] cell The cell of the region. The array should be of size nframes x
+ *9. Pass NULL if pbc is not used.
  * @param[in] nghost The number of ghost atoms.
  * @param[in] nlist The neighbor list.
  * @param[in] ago Update the internal neighbour list if ago is 0.
@@ -838,13 +841,14 @@ extern void DP_DeepPotComputeNList3(DP_DeepPot* dp,
  * @param[in] charge_spin The per-frame charge/spin input. The array can be of
  *size nframes x dim_chg_spin. Pass NULL to use the model's stored
  *default_chg_spin.
- * @param[out] energy Output energy.
- * @param[out] force Output force. The array should be of size natoms x 3.
- * @param[out] virial Output virial. The array should be of size 9.
+ * @param[out] energy Output energy. The array should be of size nframes.
+ * @param[out] force Output force. The array should be of size nframes x natoms
+ *x 3.
+ * @param[out] virial Output virial. The array should be of size nframes x 9.
  * @param[out] atomic_energy Output atomic energy. The array should be of size
- *natoms.
+ *nframes x natoms.
  * @param[out] atomic_virial Output atomic virial. The array should be of size
- *natoms x 9.
+ *nframes x natoms x 9.
  * @warning The output arrays should be allocated before calling this function.
  *Pass NULL if not required.
  * @since API version 27

source/api_c/src/c_api.cc

@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: LGPL-3.0-or-later
 #include "c_api.h"
 
+#include <limits>
 #include <numeric>
 #include <string>
 #include <vector>
@@ -1535,6 +1536,27 @@ const char* string_to_char(std::string& str) {
   return buffer;
 }
 
+/**
+ * @brief Convert std::string to const char without trimming.
+ *
+ * Unlike string_to_char, this helper preserves every byte of the input,
+ * including trailing whitespace.  This is necessary for file-reading
+ * functions (e.g. DP_ReadFileToChar2) where the reported size must
+ * match the allocated buffer exactly; see issue #5620.
+ *
+ * @param[in] str std::string to be converted (not modified)
+ * @return const char* heap-allocated buffer, caller must DP_DeleteChar
+ */
+const char* string_to_char_exact(const std::string& str) {
+  // copy from string to char* without any trimming
+  const std::string::size_type size = str.size();
+  // +1 for '\0'
+  char* buffer = new char[size + 1];
+  std::copy(str.begin(), str.end(), buffer);
+  buffer[size] = '\0';
+  return buffer;
+}
+
 extern "C" {
 
 const char* DP_NlistCheckOK(DP_Nlist* nlist) {
@@ -2642,7 +2664,8 @@ const char* DP_ReadFileToChar(const char* c_model) {
   std::string model(c_model);
   std::string file_content;
   deepmd::read_file_to_string(model, file_content);
-  return string_to_char(file_content);
+  // Preserve exact bytes — see issue #5620 for why trimming is wrong here.
+  return string_to_char_exact(file_content);
 }
 
 const char* DP_ReadFileToChar2(const char* c_model, int* size) {
@@ -2656,8 +2679,20 @@ const char* DP_ReadFileToChar2(const char* c_model, int* size) {
     *size = -error_message.size();
     return string_to_char(error_message);
   }
-  *size = file_content.size();
-  return string_to_char(file_content);
+  // Record the exact file size before any conversion.  We must use
+  // string_to_char_exact (not string_to_char) so that trailing
+  // whitespace is preserved and the returned buffer has exactly *size
+  // bytes — otherwise the C++ wrapper would reconstruct a string that
+  // over-reads the shorter allocation.  See issue #5620.
+  if (file_content.size() >
+      static_cast<std::size_t>(std::numeric_limits<int>::max())) {
+    std::string error_message =
+        "File is too large to be read into a char buffer via this API";
+    *size = -static_cast<int>(error_message.size());
+    return string_to_char(error_message);
+  }
+  *size = static_cast<int>(file_content.size());
+  return string_to_char_exact(file_content);
 }
 
 void DP_SelectByType(const int natoms,

source/api_c/tests/test_read_file_to_string.cc

@@ -6,11 +6,14 @@
 
 #include <algorithm>
 #include <cmath>
+#include <cstdio>
 #include <fstream>
+#include <memory>
 #include <sstream>
 #include <string>
 #include <vector>
 
+#include "c_api.h"
 #include "deepmd.hpp"
 TEST(TestReadFileToString, readfiletostring) {
 #ifndef BUILD_TENSORFLOW
@@ -40,3 +43,42 @@ TEST(TestReadFileToString, readfiletostringerr) {
       },
       deepmd::hpp::deepmd_exception);
 }
+
+TEST(TestReadFileToString, c_api_preserves_exact_bytes) {
+  const std::string file_name = "test_read_file_to_char_exact_bytes.bin";
+  const std::string expected("hello\0world \n", 13);
+  {
+    std::ofstream output(file_name, std::ios::binary);
+    ASSERT_TRUE(output.is_open());
+    output.write(expected.data(), expected.size());
+  }
+
+  int size = 0;
+  std::unique_ptr<const char, decltype(&DP_DeleteChar)> content(
+      DP_ReadFileToChar2(file_name.c_str(), &size), DP_DeleteChar);
+
+  ASSERT_NE(content, nullptr);
+  ASSERT_GE(size, 0);
+  EXPECT_EQ(size, expected.size());
+  EXPECT_EQ(std::string(content.get(), size), expected);
+
+  EXPECT_EQ(std::remove(file_name.c_str()), 0);
+}
+
+TEST(TestReadFileToString, legacy_c_api_preserves_trailing_whitespace) {
+  const std::string file_name = "test_read_file_to_char_whitespace.txt";
+  const std::string expected = "hello world \n";
+  {
+    std::ofstream output(file_name, std::ios::binary);
+    ASSERT_TRUE(output.is_open());
+    output.write(expected.data(), expected.size());
+  }
+
+  std::unique_ptr<const char, decltype(&DP_DeleteChar)> content(
+      DP_ReadFileToChar(file_name.c_str()), DP_DeleteChar);
+
+  ASSERT_NE(content, nullptr);
+  EXPECT_EQ(std::string(content.get()), expected);
+
+  EXPECT_EQ(std::remove(file_name.c_str()), 0);
+}