Status
Known unresolved failure, recorded from the completed synthetic OMP investigation. This issue tracks retained evidence; it does not restart diagnosis or claim a fix. Related to the multi-provider evaluation work in #46.
Revision and native pin
- Source:
4fa4c6f2578268c60b60c454a3fc1a495dc0b64a on feat/eval-multi-runner.
- OMP
18.1.14, profile omp-18.1.14-acp-v1.
- Native executable SHA-256:
27dd66cb6a2c39fff51e1fe6db40f1f376fee4a3d04457f236edaa5d0290e4d2.
- Harness source SHA-256:
c1b9616436067015ebdf2e0a90f9705fd76cdf67af9b4b9d5ca6679b655b68d6.
Recorded failure
In the synthetic mcp-error-anthropic case, the host made one canonical perps.executeSqlQuery MCP call and received a tool error. Native guard evidence reached terminal completion. The evaluator instead returned PluginEvalOmpHarnessProcessError with reason generation-failed and an empty message.
The trial failed closed. It did not report success, but it produced no canonical failed tool observation. The one-sql-observation and kept-canonical-identity assertions therefore failed. The failed row recorded containerGone: false; the attempt's overall cleanup and final resource cleanup were subsequently recorded as successful. This is not a claim of a persistent resource leak.
Investigation already completed
- Bounded exception instrumentation in a temporary diagnostic source copy.
- Four diagnostic MCP-error passes, one initial pass and three repeats.
- One final unchanged-source pass.
- Those later passes returned observation status
failed, error OMP tool execution failed, one canonical SQL observation with MCP tool call failed, and exactly one host MCP call.
The cause remains unknown. Later passes do not establish that the intermittent failure is fixed. No further reruns are part of filing this issue.
Retained evidence fingerprints
Raw summaries and captures remain private; they are not attached to this public issue. These hashes identify the exact retained files, not independent correctness attestations.
| Receipt |
SHA-256 |
| Closeout |
a20113fc741f6772ddbff07d47e558f9d1277950cdd08b3c6dc1c93d0c2b0a3e |
Failed attempt 666e1572 summary |
ebe3f31dca7decc47abfc33a8c93351b219d3ca0443015ce512b66e057d4f6f2 |
Final unchanged-source attempt 44a4720d summary |
fe72d9dbff9c79d454c79699cf276179897f1d19744d1b1cd2fe734489cd7efa |
All observations above are synthetic local model/MCP protocol evidence, not paid model behavior, production Gina connectivity or measured acceptance. Paid evaluations require a separately approved milestone. This issue does not authorize them.
Status
Known unresolved failure, recorded from the completed synthetic OMP investigation. This issue tracks retained evidence; it does not restart diagnosis or claim a fix. Related to the multi-provider evaluation work in #46.
Revision and native pin
4fa4c6f2578268c60b60c454a3fc1a495dc0b64aonfeat/eval-multi-runner.18.1.14, profileomp-18.1.14-acp-v1.27dd66cb6a2c39fff51e1fe6db40f1f376fee4a3d04457f236edaa5d0290e4d2.c1b9616436067015ebdf2e0a90f9705fd76cdf67af9b4b9d5ca6679b655b68d6.Recorded failure
In the synthetic
mcp-error-anthropiccase, the host made one canonicalperps.executeSqlQueryMCP call and received a tool error. Native guard evidence reached terminal completion. The evaluator instead returnedPluginEvalOmpHarnessProcessErrorwith reasongeneration-failedand an empty message.The trial failed closed. It did not report success, but it produced no canonical failed tool observation. The
one-sql-observationandkept-canonical-identityassertions therefore failed. The failed row recordedcontainerGone: false; the attempt's overall cleanup and final resource cleanup were subsequently recorded as successful. This is not a claim of a persistent resource leak.Investigation already completed
failed, errorOMP tool execution failed, one canonical SQL observation withMCP tool call failed, and exactly one host MCP call.The cause remains unknown. Later passes do not establish that the intermittent failure is fixed. No further reruns are part of filing this issue.
Retained evidence fingerprints
Raw summaries and captures remain private; they are not attached to this public issue. These hashes identify the exact retained files, not independent correctness attestations.
a20113fc741f6772ddbff07d47e558f9d1277950cdd08b3c6dc1c93d0c2b0a3e666e1572summaryebe3f31dca7decc47abfc33a8c93351b219d3ca0443015ce512b66e057d4f6f244a4720dsummaryfe72d9dbff9c79d454c79699cf276179897f1d19744d1b1cd2fe734489cd7efaAll observations above are synthetic local model/MCP protocol evidence, not paid model behavior, production Gina connectivity or measured acceptance. Paid evaluations require a separately approved milestone. This issue does not authorize them.