Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/scripts/partisan-pin
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,10 @@
#
# Recon context lives in the auto-memory entry
# `project_partisan_pin.md` and in issue #57 / #49.
58b56da84ed815063e39acf6616bcc01bc6809a5
#
# Bumped 2026-07-23 (devel HEAD) to pick up partisan PR #349
# ("Add a physical_path attribute to Replica", merged 2026-07-15),
# whose test_replicas asserts physical_path is populated whenever
# client_version() >= 6.1.0 — the assertion this repo's #101 /
# #100 work needs exercised in CI.
1356610d2a195e881ca55351db68f61e53c4edc4
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "baton-rs"
version = "1.0.0-alpha.1"
edition = "2021"
license = "GPL-2.0"
description = "Rust reimplementation of baton, the iRODS metadata client. Wire-compatible with upstream baton 6.0.0."
description = "Rust reimplementation of baton, the iRODS metadata client. Wire-compatible with upstream baton 6.1.0."
repository = "https://github.com/jmtcsngr/baton-rs"
readme = "README.md"

Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A Rust reimplementation of [baton](https://github.com/wtsi-npg/baton), the iRODS
client focused on metadata operations via a single JSON interface. Targets
wire-compat with upstream baton **6.0.0**.
wire-compat with upstream baton **6.1.0**.

**Status:** Under active development. The seven binaries are implemented and
exercised against iRODS 4.2.7 / 4.3.4 / 4.3.5 in CI, plus integration runs
Expand Down Expand Up @@ -57,7 +57,7 @@ without coordinating schema changes with baton-rs. See the

## Compatibility with upstream baton

baton-rs is wire-compatible with upstream baton 6.0.0 for the success
baton-rs is wire-compatible with upstream baton 6.1.0 for the success
path and the in-band error path. Known divergences:

- **Parse-error wire shape** — baton-rs emits a stand-alone synthetic
Expand Down Expand Up @@ -111,7 +111,7 @@ environment variable:
| Env var state | `--version` reports |
|----------------------------------------|------------------------------------------------------------|
| Unset (or empty string) | The baton-rs crate version (`Cargo.toml`'s `version`). |
| Set to any non-empty value | `BATON_COMPAT_VERSION` (e.g. `6.0.0`) — the upstream baton release baton-rs targets wire-compat with. |
| Set to any non-empty value | `BATON_COMPAT_VERSION` (e.g. `6.1.0`) — the upstream baton release baton-rs targets wire-compat with. |

Honest reporting is the default so logs and debugging surfaces aren't misled
about what's actually running. The compat mode exists for downstream consumers
Expand All @@ -125,7 +125,7 @@ $ baton-do --version
1.0.0-alpha.1

$ STRICT_BATON_COMPAT=1 baton-do --version
6.0.0
6.1.0
```

The `STRICT_BATON_COMPAT` toggle is also reserved for future wire-format
Expand Down
7 changes: 6 additions & 1 deletion SESSIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Paste the relevant sections at the start of each new session to restore context

These values do not change during normal development. If they do change, note the reason here.

- **Target baton version for parity:** `6.0.0`
- **Target baton version for parity:** `6.1.0` _(bumped from `6.0.0` 2026-07-23 — upstream released 6.0.1 and 6.1.0; see Changelog for this file, below, for what the re-pin required.)_
- **Reference schema:** <https://wtsi-npg.github.io/baton/>
- **Primary compatibility oracle:** partisan's Python test suite, run with baton-rs's `baton-do` on `PATH`
- **Supported iRODS versions (CI matrix):** 4.2.7, 4.3.4, 4.3.5 _(add 5.0.1 when ready)_
Expand Down Expand Up @@ -583,3 +583,8 @@ Use this space to record non-trivial changes to the plan itself — e.g. changin
- **#82** — extendo pin bumped to `v3.1.0` (`b24b63c…`, against `feat/extendo-ci` base). Tagged release over devel HEAD.
- **#83** — synthetic-`-1` audit cleanup across 17 validation paths. Group A (target-type) and B (local-FS) surface `-310000` (USER_FILE_DOES_NOT_EXIST); Group C (argument validation) surfaces `-403000` (USER_INPUT_OPTION_ERR). Adds `-403000` to `error::descriptive_phrase`.
- **#85** — client-side `verify_checksum` mismatch aligned to `-407000` (CHECK_VERIFICATION_RESULTS). Final audit follow-up; client-side and server-side mismatch now surface under the same code.
- `2026-07-23` — Target baton version for parity bumped `6.0.0` → `6.1.0` on branch `chore/bump-baton-compat-6.1.0` (PR #102), landed as a compat *feature*, not a version-string-only chore. Upstream had shipped two releases we hadn't tracked: `6.0.1` (2026-05-19, mostly CI/dependency churn plus two OOB-write hardening fixes in its own genQuery builder, wtsi-npg/baton#337/#338 — see the #100 note below for why that turned out not to be baton-rs's problem) and `6.1.0` (2026-07-02, adds a `physical_path` field to replicate JSON objects, wtsi-npg/baton#339). Two gaps were filed against the re-pin; only one was real:
- **#100 — filed, then retracted as a false premise.** Initial analysis assumed `genQueryInp_t.sqlCondInp` / `.selectInp` were fixed-size iRODS arrays that `addInxVal` / `addInxIval` could write past, mirroring upstream baton's 6.0.1 OOB-write fix (wtsi-npg/baton#337/#338). A bounds check (`len >= MAX_NUM_CONDITIONS`) was added to the shim — which failed CI immediately: `MAX_NUM_CONDITIONS` isn't an iRODS macro at all. Checking the actual iRODS source (`irods/irods`) showed `inxIvalPair_t` / `inxValPair_t` (`lib/core/include/irods/objInfo.h`) are pointer-based, and `addInxIval` / `addInxVal` (`lib/core/src/rcMisc.cpp`) dynamically grow their backing storage via `malloc` — there is no fixed capacity to overflow. `MAX_NUM_CONDITIONS = 32` is baton's *own* self-imposed limit (`baton/src/query.h`), guarding a fixed-size buffer in baton's own C code, not the shared iRODS client library. Since baton-rs's shim calls `addInxIval` / `addInxVal` directly with no intermediate fixed-size staging array, it never had the vulnerability baton patched. All #100 code (the bounds check, `GenQuery::add_select`'s `Result` return, the 25 call-site `?` additions, and the new test) was reverted; the issue was corrected on GitHub rather than silently dropped, and the finding recorded in memory (`feedback_compat_version_claims.md`'s sibling note) as a reminder to verify upstream C internals before porting a "security fix" description across languages/architectures.
- **#101** — added `SHIM_COL_DATA_PATH` (→ `COL_D_DATA_PATH`) through the standard shim declare/implement/mirror path, a new `Replicate.physical_path: String` field, and wired it into `fetch_replicates`. Extended the existing `--replicate` integration test and the `Replicate` JSON round-trip unit test.
- **Caught mid-flight:** an initial pass bumped `BATON_COMPAT_VERSION` (and the Cargo.toml/README "wire-compatible with X" claims) to 6.1.0 *before* implementing either gap — the user caught this as a compat lie (`STRICT_BATON_COMPAT=1 baton-do --version` is read by real callers, not just documentation) before it merged. Recorded as a standing rule in memory (`feedback_compat_version_claims.md`): never bump the operational compat-version claim ahead of implementing that version's schema/behaviour changes.
- **Partisan pin bumped** (`.github/scripts/partisan-pin`, `58b56da` → `1356610d2a`, devel HEAD) to pick up partisan PR #349 ("Add a physical_path attribute to Replica", merged 2026-07-15), whose `test_replicas` asserts `physical_path is not None` whenever `client_version() >= 6.1.0` — the exact assertion this work needed exercised in CI. Extendo's pin was checked and left alone: no physical_path work exists there yet (`v3.1.0..v3.2.0` diff is dependency bumps only).
1 change: 1 addition & 0 deletions shim/ffi_shim.c
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ static int translate_col(shim_col_t col) {
case SHIM_COL_COLL_USER_ZONE: return COL_COLL_USER_ZONE;
case SHIM_COL_COLL_ACCESS_NAME: return COL_COLL_ACCESS_NAME;
case SHIM_COL_DATA_SIZE: return COL_DATA_SIZE;
case SHIM_COL_DATA_PATH: return COL_D_DATA_PATH;
}
return -1;
}
Expand Down
1 change: 1 addition & 0 deletions shim/ffi_shim.h
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ typedef enum {
SHIM_COL_COLL_USER_ZONE = 23,
SHIM_COL_COLL_ACCESS_NAME = 24,
SHIM_COL_DATA_SIZE = 25,
SHIM_COL_DATA_PATH = 26,
} shim_col_t;

// Allocate a fresh query builder. Returns NULL on out-of-memory.
Expand Down
1 change: 1 addition & 0 deletions src/ffi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ pub const SHIM_COL_COLL_USER_NAME: shim_col_t = 22;
pub const SHIM_COL_COLL_USER_ZONE: shim_col_t = 23;
pub const SHIM_COL_COLL_ACCESS_NAME: shim_col_t = 24;
pub const SHIM_COL_DATA_SIZE: shim_col_t = 25;
pub const SHIM_COL_DATA_PATH: shim_col_t = 26;

// ---- Data object read/write -------------------------------------------------

Expand Down
3 changes: 3 additions & 0 deletions src/operations/list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -466,6 +466,7 @@ fn fetch_replicates(
q.add_select(ffi::SHIM_COL_R_LOC);
q.add_select(ffi::SHIM_COL_D_RESC_NAME);
q.add_select(ffi::SHIM_COL_D_REPL_STATUS);
q.add_select(ffi::SHIM_COL_DATA_PATH);

q.add_where(
ffi::SHIM_COL_COLL_NAME,
Expand All @@ -485,6 +486,7 @@ fn fetch_replicates(
let location = row.get(2).cloned().unwrap_or_default();
let resource = row.get(3).cloned().unwrap_or_default();
let status = row.get(4).cloned().unwrap_or_default();
let physical_path = row.get(5).cloned().unwrap_or_default();

let number: u32 = number_str.parse().map_err(|_| BatonError {
code: -1,
Expand All @@ -497,6 +499,7 @@ fn fetch_replicates(
resource,
number,
valid: status == "1",
physical_path,
})
})
.collect()
Expand Down
16 changes: 12 additions & 4 deletions src/types.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
// Generated by: Claude Opus 4.7 (1M context)
// Generation date: 2026-04-24
// Last modified by: Claude Opus 4.7 (1M context)
// Last modified: 2026-05-12
// Last modified by: Claude Sonnet 5 (1M context)
// Last modified: 2026-07-23

//! JSON data types for the baton-rs API.
//!
//! All types round-trip against the baton 6.0.0 JSON schema
//! All types round-trip against the baton 6.1.0 JSON schema
//! (<https://wtsi-npg.github.io/baton/>).

use serde::de::{self, MapAccess, Visitor};
Expand Down Expand Up @@ -106,6 +106,13 @@ pub struct Replicate {
pub resource: String,
pub number: u32,
pub valid: bool,

/// The replica's physical path on the resource server's
/// filesystem, sourced from `COL_D_DATA_PATH`. Added in baton
/// 6.1.0 (wtsi-npg/baton#339); baton-rs always populates it, so
/// unlike upstream pre-6.1.0 servers there's no "missing key"
/// case to model here.
pub physical_path: String,
}

/// A timestamp for a data object or replicate.
Expand Down Expand Up @@ -1391,13 +1398,14 @@ mod tests {

#[test]
fn replicate_round_trip() {
let json = r#"{"checksum":"d41d8cd98f00b204e9800998ecf8427e","location":"host","resource":"demoResc","number":0,"valid":true}"#;
let json = r#"{"checksum":"d41d8cd98f00b204e9800998ecf8427e","location":"host","resource":"demoResc","number":0,"valid":true,"physical_path":"/var/lib/irods/Vault/home/irods/foo.txt"}"#;
let r: Replicate = serde_json::from_str(json).unwrap();
assert_eq!(r.checksum, "d41d8cd98f00b204e9800998ecf8427e");
assert_eq!(r.location, "host");
assert_eq!(r.resource, "demoResc");
assert_eq!(r.number, 0);
assert!(r.valid);
assert_eq!(r.physical_path, "/var/lib/irods/Vault/home/irods/foo.txt");
assert_eq!(serde_json::to_string(&r).unwrap(), json);
}

Expand Down
2 changes: 1 addition & 1 deletion src/version.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
/// recorded in `SESSIONS.md` whenever we re-pin against a newer
/// upstream baton. The release checklist in #58 calls out this
/// audit step explicitly.
pub const BATON_COMPAT_VERSION: &str = "6.0.0";
pub const BATON_COMPAT_VERSION: &str = "6.1.0";

/// Env var that toggles compat-mode version reporting. See module
/// doc-comment for the rationale and #58 for the design.
Expand Down
2 changes: 1 addition & 1 deletion tests/baton_do_binary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1225,7 +1225,7 @@ fn version_flag_unset_reports_crate_version() {
#[test]
fn version_flag_with_strict_compat_reports_compat_version() {
// STRICT_BATON_COMPAT=1 → report BATON_COMPAT_VERSION instead
// (currently 6.0.0). Lets partisan's `client_version()` parse
// (currently 6.1.0). Lets partisan's `client_version()` parse
// the result and compare it against expected baton X.Y.Z values.
let mut child = Command::new(baton_do_path())
.args(["--version"])
Expand Down
7 changes: 7 additions & 0 deletions tests/list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -628,6 +628,13 @@ fn list_data_object_with_replicate() {
.expect("replica 0 present");
assert!(!r0.resource.is_empty(), "resource set");
assert!(!r0.location.is_empty(), "location set");
// physical_path (baton 6.1.0, #101 / COL_D_DATA_PATH) is a real
// filesystem path under the test container's iRODS vault.
assert!(
r0.physical_path.starts_with("/var/lib/irods"),
"expected physical_path under the iRODS vault, got {:?}",
r0.physical_path
);

// Every replica must be valid right after iput. This is stricter
// than the original "r0 is valid" form: on the 4.2.7 CI image
Expand Down