Labels: enhancement, discovery
Related: #281
Context
When implementing the idempotency policy from #281, the engine needs a stable, cross-process hash of initial_context to compare against the stored checkpoint fingerprint. Python's built-in hash() is
not suitable:
PYTHONHASHSEED randomizes string hashes per process, so two
runs on the same input produce different hash values.
dict and set iteration ordering historically influenced the
hash of structures containing them.
hash() truncates to platform-dependent integer width.
Using hash() would silently invalidate every checkpoint on each process restart.
Concept
Compute the fingerprint as sha256(canonical_json(value)), where canonical_json produces deterministic output regardless of:
- Insertion order of dict keys
- Whitespace
- Python version
- Host architecture
Proposed implementation
# dotflow/core/fingerprint.py
import hashlib
import json
from typing import Any
def canonical_json(value: Any) -> str:
"""JSON output that is byte-identical for equal logical inputs."""
return json.dumps(
value,
sort_keys=True,
ensure_ascii=False,
separators=(",", ":"),
default=_default,
)
def _default(obj):
# Deterministic fallback for non-JSON types
if hasattr(obj, "__dict__"):
return {"__class__": type(obj).__name__, **obj.__dict__}
return repr(obj)
def fingerprint(value: Any) -> str:
"""Stable sha256 of a canonical JSON encoding."""
return hashlib.sha256(canonical_json(value).encode()).hexdigest()
Usage (preview, depends on #281)
from dotflow.core.fingerprint import fingerprint
stored = checkpoint.input_fingerprint
current = fingerprint(task.initial_context)
if stored != current:
# apply on_input_change policy (#281)
...
Acceptance criteria
Why this matters
Issue #281 cannot ship without a deterministic fingerprint. Doing it right once, in a dedicated module, means every future feature (checkpoint cache keys, run replay IDs, task memoization) reuses the same primitive.
Labels:
enhancement,discoveryRelated: #281
Context
When implementing the idempotency policy from #281, the engine needs a stable, cross-process hash of
initial_contextto compare against the stored checkpoint fingerprint. Python's built-inhash()isnot suitable:
PYTHONHASHSEEDrandomizes string hashes per process, so tworuns on the same input produce different hash values.
dictandsetiteration ordering historically influenced thehash of structures containing them.
hash()truncates to platform-dependent integer width.Using
hash()would silently invalidate every checkpoint on each process restart.Concept
Compute the fingerprint as
sha256(canonical_json(value)), wherecanonical_jsonproduces deterministic output regardless of:Proposed implementation
Usage (preview, depends on #281)
Acceptance criteria
dotflow/core/fingerprint.pywithfingerprint()andcanonical_json()sub-processes spawned via
multiprocessing.spawnfingerprint({"a": 1, "b": 2}) == fingerprint({"b": 2, "a": 1})dataclasses, datetime
Why this matters
Issue #281 cannot ship without a deterministic fingerprint. Doing it right once, in a dedicated module, means every future feature (checkpoint cache keys, run replay IDs, task memoization) reuses the same primitive.