fix(contracts): report scan FAILED instead of COMPLETED_WITH_ERRORS on pre-results failure

[m1n0]

Fixes DTL-1800. Source incident: SAS-12468.

Problem

When a contract verification fails before producing any check results (e.g. the data source connection fails — Databricks 403 in the incident), core sends an errored, result-less batch to Soda Cloud. The backend maps that to COMPLETED_WITH_ERRORS. For a managed scan still in PENDING, the scan-state machine forbids PENDING -> COMPLETED_WITH_ERRORS (ScanState.PENDING allows only SUBMITTED/FAILED/CANCELED), so the upload returns 400 invalid_scan_state → exit code 4 (RESULTS_NOT_SENT_TO_CLOUD) → the contract launcher raises LibraryExecutionEngineException and pages via Datadog.

Fix

Report such a run as FAILED via mark_scan_as_failed (PENDING -> FAILED is allowed) instead of sending results. The report lands, no false RESULTS_NOT_SENT_TO_CLOUD is raised, and genuine "could not send to Cloud" failures still surface (this only changes the terminal state core requests when the run produced nothing).

• New predicate CheckCollectionResult.errored_without_results (errored and zero check results).
• CheckCollectionImpl.verify() (per-file/non-combine path): route to mark_scan_as_failed.
• session.py combined-upload path (combine_uploads=True subtypes, e.g. Data Standards): skip such results from the combined send and mark the scan FAILED when the whole scan produced nothing to send.

Tests

soda-tests/tests/unit/test_contract_marks_scan_failed_on_connection_error.py — covers both the per-file and combine-upload paths by patching the DuckDB connection to fail. Both assert sodaCoreMarkScanFailed is sent and sodaCoreInsertScanResults is not. Full unit suite: 871 passed, 2 skipped.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adjusts Soda Cloud reporting for contract/check-collection runs that error before producing any check results, to avoid sending a result-less errored batch that Cloud maps to COMPLETED_WITH_ERRORS (an invalid transition from PENDING for managed scans). Instead, such runs are reported as FAILED via mark_scan_as_failed.

Changes:

• Add CheckCollectionResult.errored_without_results to detect ERROR runs with zero check_results.
• Update CheckCollectionImpl.verify() and the session-level combine_uploads executor path to avoid uploading these result-less errored batches and (when appropriate) mark the scan as failed instead.
• Add unit tests covering both per-file and combined-upload paths on a simulated connection failure.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
soda-core/src/soda_core/check_collections/base.py	Adds errored_without_results and changes per-file upload behavior to mark managed scans FAILED when errors occur before any check results exist.
soda-core/src/soda_core/check_collections/session.py	Updates combined-upload grouping to exclude errored_without_results results and optionally mark the scan FAILED if nothing else is sendable.
soda-tests/tests/unit/test_contract_marks_scan_failed_on_connection_error.py	Adds unit tests asserting sodaCoreMarkScanFailed is sent and sodaCoreInsertScanResults is not for pre-results failures (both upload modes).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
15.5% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[Niels-b]

LGTM.

Just for my own understanding. This fixes an issue only when Soda Cloud creates a scan and it fails. This doesn't affect a scan that you when using Soda Core programmatically. Is that correct?

[Niels-b]

Didn't know we had this available in the runner.
As comment says: we don't have this in "local" runs. Would be nice 😄

[m1n0]

Correct — gated on SODA_SCAN_ID, which only the runner sets.
Ad-hoc/programmatic core runs have no scan id and take the unchanged upload path (covered by test_ad_hoc_run_without_scan_id_still_uploads_results). On the runner it only changes the errored with zero results case.