Add commercial evidence filesystem observation

Author: ammarheidari

README.md

@@ -75,6 +75,7 @@ The first production milestone is M3UA over a transport abstraction. SCCP, TCAP,
 | Commercial evidence execution orchestration | Phase 37 foundation-complete: evidence execution run identity, stage catalog, operator command plan, execution environment contract, artifact collection manifest, digest/redaction verification, blocker classification, retry/resume policy, status reporting, and final validation are available; real retained execution artifacts are still required before commercial publication |
 | Commercial evidence artifact intake | Phase 38 foundation-complete: artifact intake target identity, artifact source registration, SHA-256 digest coverage, redaction review, completeness evaluation, dossier reporting, promotion handoff, execution-to-dossier bridge, status reporting, and final validation are available, binding a stable intake id, reviewer identity, UTC receipt time, run-scoped dossier root, concrete source paths, unique retained dossier paths, retained digests, trace-bearing artifact approvals, explicit intake blockers, retained Markdown summary, digest-covered handoff, end-to-end intake assembly, and intake readiness status to a governed execution run; real artifact files are still required before commercial publication |
 | Commercial evidence file verification | Phase 39 foundation-complete: retained file evidence item verification, retained file manifest coverage, file verification blocker reporting, retention ledger modeling, integrity sealing, publication attachment planning, verified promotion gating, command planning, status reporting, final validation, and documentation alignment are available, checking file existence, non-empty size, SHA-256 validity, digest match, UTC observation time, unique retained paths, promotion-required handoff coverage, explicit verification blockers, reviewer identity, immutable retention, UTC retention windows, minimum duration, deterministic aggregate ledger digests, sealed ledger attachment coverage, trace-bearing redaction approval, commercial readiness report presence, explicit promotion approval, workflow-ready verification command order, and readiness status separation; real retained file evidence remains required before commercial publication |
+| Commercial evidence filesystem execution | Phase 40 in progress: filesystem observation is available, reading retained files from disk, computing real SHA-256 digests, reporting file existence and size, and mapping observations into the retained file verification model; manifest execution, report execution, artifact writing, ledger execution, seal execution, attachment execution, promotion execution, command materialization, status reporting, and final validation remain |
 
 ## Requirements
 

docs/PHASE40_COMMERCIAL_EVIDENCE_FILESYSTEM_EXECUTION.md

@@ -0,0 +1,27 @@
+# Phase 40 Commercial Evidence Filesystem Execution
+
+Phase 40 turns the file verification contracts from Phase 39 into filesystem-backed execution helpers. It still does not manufacture commercial evidence. It reads retained files from disk, computes real SHA-256 digests, and feeds the existing verification, retention, sealing, attachment, and promotion-gate contracts.
+
+Public APIs use domain names such as `SigtranCommercialEvidenceFileSystemObserver`; phase numbers are intentionally kept out of source type names.
+
+## Unit 1 - Filesystem Observation
+
+`SigtranCommercialEvidenceFileSystemObserver` observes a promotion handoff item against the local filesystem:
+
+- Reads the retained file path or an explicit local path override.
+- Reports existence and file size.
+- Computes the actual SHA-256 digest for existing files.
+- Uses a deterministic missing-file digest marker for absent files.
+- Produces a `SigtranCommercialEvidenceRetainedFile` so the Phase 39 verification pipeline can evaluate the observation.
+
+The observer is the first executable bridge from retained dossier paths to real file verification.
+
+## Validation
+
+Each unit in this phase is validated with:
+
+```powershell
+dotnet build src\Sigtran.NET.sln
+dotnet run --project src\Sigtran.NET.Tests\Sigtran.NET.Tests.csproj
+dotnet pack src\Sigtran.NET\Sigtran.NET.csproj -c Release
+```

docs/PHASE40_SUMMARY.md

@@ -0,0 +1,11 @@
+# Phase 40 Summary - Commercial Evidence Filesystem Execution
+
+Phase 40 prepares the SDK to run commercial evidence file verification against real retained files on disk.
+
+## Completed Capabilities
+
+- Filesystem observer that computes SHA-256 digests from existing retained files and maps missing files into explicit non-verified retained file observations.
+
+## Readiness Position
+
+The phase is in progress. Filesystem observation is available. Manifest execution, report execution, artifact writing, ledger execution, seal execution, attachment execution, promotion execution, command materialization, status reporting, and final validation remain.

docs/PHASE_INDEX.md

@@ -44,6 +44,7 @@ This index is the complete phase map for Sigtran.NET. It links each phase to the
 | 37 | Commercial evidence execution orchestration | [Phase 37 Commercial Evidence Execution Orchestration](PHASE37_COMMERCIAL_EVIDENCE_EXECUTION_ORCHESTRATION.md), [Phase 37 Summary](PHASE37_SUMMARY.md) | Foundation complete; real execution artifacts still required |
 | 38 | Commercial evidence artifact intake | [Phase 38 Commercial Evidence Artifact Intake](PHASE38_COMMERCIAL_EVIDENCE_ARTIFACT_INTAKE.md), [Phase 38 Summary](PHASE38_SUMMARY.md) | Foundation complete; real artifact files still required |
 | 39 | Commercial evidence file verification | [Phase 39 Commercial Evidence File Verification](PHASE39_COMMERCIAL_EVIDENCE_FILE_VERIFICATION.md), [Phase 39 Summary](PHASE39_SUMMARY.md) | Foundation complete; real retained file evidence required |
+| 40 | Commercial evidence filesystem execution | [Phase 40 Commercial Evidence Filesystem Execution](PHASE40_COMMERCIAL_EVIDENCE_FILESYSTEM_EXECUTION.md), [Phase 40 Summary](PHASE40_SUMMARY.md) | In progress; filesystem observation complete |
 
 ## Current Commercial Gate
 

docs/SDK_ROADMAP.md

@@ -349,6 +349,14 @@ Status: Phase 38 is foundation-complete. Artifact intake target identity binds a
 
 Status: Phase 39 is foundation-complete. Retained file evidence item verification checks existence, non-empty size, SHA-256 validity, digest match, and UTC observation time. Retained file manifest coverage verifies that every promotion-required handoff item has a unique verified retained file. File verification reporting exposes missing, empty, invalid digest, digest mismatch, non-UTC observation, duplicate path, and incomplete handoff blockers. Retention ledger modeling binds verified files to reviewer identity, immutable retention, UTC retention windows, and minimum duration checks. Integrity sealing computes and validates a deterministic aggregate SHA-256 digest over the ledger. Publication attachment planning covers sealed ledger entries, validates attachment digests, includes the commercial readiness report, and blocks trace-bearing attachments without redaction approval. Verified promotion gating requires ready attachments, ready integrity seal, ready retention ledger, verified file report, commercial readiness report presence, and explicit approval before evidence can move into release publication decisions. Command planning orders observation, digest computation, comparison, report, ledger, seal, attachment, and promotion-gate work for workflow materialization. Status reporting separates foundation readiness from real retained file evidence and commercial publication readiness. Commercial publication remains blocked until real retained file evidence is captured and approved.
 
+## Phase 40 - Commercial Evidence Filesystem Execution
+
+- Execute retained evidence file observation against the local filesystem.
+- Build verification manifests, reports, retained artifacts, ledger, seal, attachments, promotion gate, and command materialization from observed files.
+- Keep filesystem execution separate from commercial publication: helpers can verify real files, but publication remains blocked until retained evidence comes from an approved commercial run.
+
+Status: Phase 40 is in progress. Filesystem observation now reads retained files from disk, computes real SHA-256 digests, records file existence and size, and maps observations into the retained file verification model. Manifest execution, report execution, retained artifact writing, ledger execution, seal execution, attachment execution, promotion execution, command materialization, status reporting, and final validation remain in progress.
+
 ## Recommended First Deliverable
 
 The first useful SDK release should be an alpha package focused on M3UA over a transport abstraction:

src/Sigtran.NET.Tests/Program.cs

@@ -305,6 +305,7 @@
 Run("SIGTRAN commercial evidence verified promotion gate requires approval", SigtranCommercialEvidenceVerifiedPromotionGateRequiresApproval);
 Run("SIGTRAN commercial evidence file verification command plan orders execution", SigtranCommercialEvidenceFileVerificationCommandPlanOrdersExecution);
 Run("SIGTRAN commercial evidence file verification status summarizes readiness", SigtranCommercialEvidenceFileVerificationStatusSummarizesReadiness);
+Run("SIGTRAN commercial evidence filesystem observer computes retained file digest", SigtranCommercialEvidenceFileSystemObserverComputesRetainedFileDigest);
 Run("SIGTRAN status capabilities use domain documentation labels", SigtranStatusCapabilitiesUseDomainDocumentationLabels);
 Run("Native SCTP platform probe reports socket creation capability", NativeSctpPlatformProbeReportsSocketCreationCapability);
 Run("Native SCTP socket factory creates or reports unsupported platform", NativeSctpSocketFactoryCreatesOrReportsUnsupportedPlatform);
@@ -5157,6 +5158,51 @@ static void SigtranCommercialEvidenceFileVerificationStatusSummarizesReadiness()
     Assert(!blockers.Contains("status-final-validation-pending"), "file verification status should clear final validation blocker");
 }
 
+static void SigtranCommercialEvidenceFileSystemObserverComputesRetainedFileDigest()
+{
+    string tempRoot = Path.Combine(Path.GetTempPath(), "sigtran-commercial-evidence-" + Guid.NewGuid().ToString("N"));
+    Directory.CreateDirectory(tempRoot);
+
+    try
+    {
+        string retainedPath = Path.Combine(tempRoot, "sbom.spdx.json");
+        byte[] content = "sigtran commercial evidence"u8.ToArray();
+        File.WriteAllBytes(retainedPath, content);
+        string expectedSha256 = Convert.ToHexString(System.Security.Cryptography.SHA256.HashData(content)).ToLowerInvariant();
+        SigtranCommercialEvidencePromotionHandoffItem item = new(
+            SigtranCommercialEvidenceChecklistKind.Sbom,
+            retainedPath,
+            expectedSha256,
+            requiredForPromotion: true);
+
+        SigtranCommercialEvidenceFileSystemObservation observation = SigtranCommercialEvidenceFileSystemObserver.Observe(
+            item,
+            observedAtUtc: DateTimeOffset.UtcNow);
+        SigtranCommercialEvidenceFileSystemObservation missing = SigtranCommercialEvidenceFileSystemObserver.Observe(
+            new SigtranCommercialEvidencePromotionHandoffItem(
+                SigtranCommercialEvidenceChecklistKind.Sbom,
+                Path.Combine(tempRoot, "missing.spdx.json"),
+                expectedSha256,
+                requiredForPromotion: true),
+            observedAtUtc: DateTimeOffset.UtcNow);
+
+        Assert(observation.IsVerified, observation.Describe());
+        Assert(observation.FileSystemPathMatchesRetainedPath, "filesystem observation should preserve retained path identity");
+        AssertEqual(expectedSha256, observation.RetainedFile.ActualSha256, "filesystem observation digest");
+        AssertEqual(content.Length, (int)observation.RetainedFile.SizeBytes, "filesystem observation size");
+        Assert(!missing.IsVerified, "missing filesystem evidence should not verify");
+        Assert(!missing.Exists, "missing filesystem evidence should report missing file");
+        AssertEqual(SigtranCommercialEvidenceFileSystemObserver.MissingFileSha256, missing.RetainedFile.ActualSha256, "missing file digest marker");
+    }
+    finally
+    {
+        if (tempRoot.StartsWith(Path.GetTempPath(), StringComparison.OrdinalIgnoreCase) && Directory.Exists(tempRoot))
+        {
+            Directory.Delete(tempRoot, recursive: true);
+        }
+    }
+}
+
 static SigtranCommercialEvidenceFileVerificationCommandPlan CreateDefaultCommercialEvidenceFileVerificationCommandPlan()
 {
     return SigtranCommercialEvidenceFileVerificationCommands.CreateDefault("artifacts/commercial-evidence");

src/Sigtran.NET/Core/Utilities/SigtranCommercialEvidenceFileSystemObservation.cs

@@ -0,0 +1,92 @@
+using System.Security.Cryptography;
+
+namespace Sigtran.NET.Core.Utilities;
+
+/// <summary>
+/// Describes one retained commercial evidence file observation from the local filesystem.
+/// </summary>
+public sealed class SigtranCommercialEvidenceFileSystemObservation
+{
+    /// <summary>Creates a retained commercial evidence filesystem observation.</summary>
+    /// <param name="fileSystemPath">The local filesystem path that was observed.</param>
+    /// <param name="retainedFile">The retained file verification model created from the observation.</param>
+    public SigtranCommercialEvidenceFileSystemObservation(
+        string fileSystemPath,
+        SigtranCommercialEvidenceRetainedFile retainedFile)
+    {
+        FileSystemPath = string.IsNullOrWhiteSpace(fileSystemPath) ? throw new ArgumentException("Filesystem path is required.", nameof(fileSystemPath)) : fileSystemPath;
+        RetainedFile = retainedFile ?? throw new ArgumentNullException(nameof(retainedFile));
+    }
+
+    /// <summary>The local filesystem path that was observed.</summary>
+    public string FileSystemPath { get; }
+
+    /// <summary>The retained file verification model created from the observation.</summary>
+    public SigtranCommercialEvidenceRetainedFile RetainedFile { get; }
+
+    /// <summary>Whether the observed filesystem path matches the retained evidence path.</summary>
+    public bool FileSystemPathMatchesRetainedPath => string.Equals(FileSystemPath, RetainedFile.RetainedPath, StringComparison.OrdinalIgnoreCase);
+
+    /// <summary>Whether the observed filesystem file exists.</summary>
+    public bool Exists => RetainedFile.Exists;
+
+    /// <summary>Whether the observed digest matches the promotion handoff digest.</summary>
+    public bool DigestMatchesHandoff => RetainedFile.DigestMatches;
+
+    /// <summary>Whether the filesystem observation verifies the retained evidence file.</summary>
+    public bool IsVerified => RetainedFile.IsVerified;
+
+    /// <summary>Formats a compact filesystem observation summary.</summary>
+    /// <returns>The filesystem observation summary.</returns>
+    public string Describe()
+    {
+        return $"commercialEvidenceFileSystemObservation={FileSystemPath} verified={IsVerified} exists={Exists}";
+    }
+}
+
+/// <summary>
+/// Observes retained commercial evidence files from the local filesystem.
+/// </summary>
+public static class SigtranCommercialEvidenceFileSystemObserver
+{
+    /// <summary>The deterministic digest marker used when a file is missing.</summary>
+    public const string MissingFileSha256 = "0000000000000000000000000000000000000000000000000000000000000000";
+
+    /// <summary>Observes a retained evidence file and computes its digest when it exists.</summary>
+    /// <param name="item">The promotion handoff item that declares the expected retained file.</param>
+    /// <param name="fileSystemPath">An optional local filesystem path override.</param>
+    /// <param name="observedAtUtc">An optional UTC observation time.</param>
+    /// <returns>The filesystem observation.</returns>
+    public static SigtranCommercialEvidenceFileSystemObservation Observe(
+        SigtranCommercialEvidencePromotionHandoffItem item,
+        string? fileSystemPath = null,
+        DateTimeOffset? observedAtUtc = null)
+    {
+        ArgumentNullException.ThrowIfNull(item);
+
+        string path = string.IsNullOrWhiteSpace(fileSystemPath) ? item.RetainedPath : fileSystemPath;
+        DateTimeOffset observedAt = observedAtUtc ?? DateTimeOffset.UtcNow;
+        observedAt = observedAt.Offset == TimeSpan.Zero ? observedAt : observedAt.ToUniversalTime();
+
+        bool exists = File.Exists(path);
+        long sizeBytes = exists ? new FileInfo(path).Length : 0;
+        string actualSha256 = exists ? ComputeSha256(path) : MissingFileSha256;
+
+        SigtranCommercialEvidenceRetainedFile retainedFile = new(
+            item.Kind,
+            item.RetainedPath,
+            item.Sha256,
+            actualSha256,
+            sizeBytes,
+            observedAt,
+            exists);
+
+        return new(path, retainedFile);
+    }
+
+    private static string ComputeSha256(string path)
+    {
+        using FileStream stream = File.OpenRead(path);
+        return Convert.ToHexString(SHA256.HashData(stream)).ToLowerInvariant();
+    }
+}