Merge branch 'master' into eval/controlled-ci-failures

Author: vicroms

.github/skills/analyze-ci-failures/SKILL.md

@@ -136,6 +136,12 @@ foreach ($job in $failedJobs) {
 
 Collect all `REGRESSION:` lines from all jobs into a master list before proceeding to artifact download. This list is your **ground truth** for what failed — the failure log artifacts are supplementary evidence for diagnosing *why*.
 
+**Important: Also scan for "unexpected pass" errors.** Jobs can fail with exit code 1 and produce **no REGRESSION lines** when a port passes but is marked as expected to fail in `scripts/ci.feature.baseline.txt`. Scan the step log for lines matching:
+```
+ci.feature.baseline.txt: error: {port}[{features}]:{triplet} passed but was marked expected to fail
+```
+These indicate stale feature baseline entries that must be removed. Report them in the report under a "🟡 Unexpected Passes (Baseline Cleanup Required)" section.
+
 ### Step 3: List Published Artifacts
 
 ```
@@ -258,6 +264,18 @@ libxt/
 └── stdout-x64-windows.log
 ```
 
+**For feature test failures (e.g., arrow-adbc_flightsql):**
+```
+arrow-adbc_flightsql/
+├── issue_body.md          ← markdown summary of the failure (human-readable)
+├── tested-spec.txt        ← exact feature combination tested (e.g., "arrow-adbc[core,flightsql]:x64-linux")
+├── stdout-x64-linux.log   ← may or may not be present
+├── config-x64-linux-*.log ← may or may not be present
+└── build-x64-linux-*.log  ← may or may not be present
+```
+
+Feature test artifacts use naming suffixes to indicate the feature combination: `portname_cpp`, `portname_all_3`, `portname_flightsql`, etc. The `tested-spec.txt` file always contains the exact `portname[features]:triplet` spec that was tested. The `issue_body.md` contains a markdown-formatted summary including the error output — read this first for quick triage.
+
 ### Step 5: Analyze Log Content — Two-Phase Approach
 
 **Phase 1: Classify all failures quickly (one pass)**
@@ -363,17 +381,25 @@ See `references/vcpkg-failure-patterns.md` for a comprehensive pattern catalogue
 
 ### Step 6: Cross-Reference with CI Baseline
 
-Read `scripts/ci.baseline.txt` from the local repository to classify each failure:
+Read **both** baseline files from the local repository to classify each failure:
 
+**`scripts/ci.baseline.txt`** — per-port expected failures:
 ```
 # Format: port-name=fail|skip|pass [# comment]
 boost-filesystem=fail  # needs icu update
 ```
 
+**`scripts/ci.feature.baseline.txt`** — per-feature/triplet expected failures:
+```
+# Format: port-name(condition)=fail|skip [# comment]
+dimcli(windows & static)=fail  # VS2019 PDB issue
+portname[feature]:triplet=feature-fails
+```
+
 **Classification logic:**
-- If the failing port is listed as `fail` or `skip` in the baseline → **Known / Expected** (not a regression)
-- If the failing port is NOT in the baseline, or listed as `pass` → **Regression** (new failure)
-- If a port that is expected to `fail` is now succeeding → **Unexpected Pass** (may need baseline update)
+- If the failing port is listed as `fail` or `skip` in either baseline → **Known / Expected** (not a regression)
+- If the failing port is NOT in any baseline, or listed as `pass` → **Regression** (new failure)
+- If a port that is expected to `fail` is now succeeding → **Unexpected Pass** (baseline entry must be removed)
 
 ### Step 7: Generate the Regression Report
 
@@ -502,6 +528,13 @@ Use the `create` tool to write the full report content to this path. The report
 - **Android artifact sizes (~115–123 MB) typically share root causes with Linux** — if Linux analysis reveals a systemic issue (e.g., compiler header change, Python version), assume Android is affected too without full re-analysis
 - **Downloaded logs are kept for manual review** in `ci-failure-analysis/` (gitignored). ZIP files are removed after extraction to save space, but the extracted logs and report are preserved. If disk space is a concern, old analysis directories can be deleted manually.
 - **Case-sensitive file paths**: Linux and Android file systems are case-sensitive; Windows and macOS are not. A portfile referencing `${SOURCE_PATH}/LICENSE` will succeed on Windows/macOS even if the upstream file is named `License` or `license`, but will fail with "No such file or directory" on Linux/Android. When a BUILD_FAILED occurs only on Linux/Android with a missing file error, always check for a case mismatch against the actual upstream filename.
+- **Two baseline files**: vcpkg CI uses **two** baseline files — `scripts/ci.baseline.txt` (per-port failures) and `scripts/ci.feature.baseline.txt` (per-feature/triplet failures). Always check both when classifying failures. Jobs can fail with exit code 1 and **no REGRESSION lines** when a port unexpectedly passes but is still marked as `fail` in `ci.feature.baseline.txt`. The error pattern in the step log is: `"passed but was marked expected to fail"`.
+- **Android NDK macro collisions**: The Android NDK defines POSIX signal-related names (`si_value`, `si_pid`, `si_uid`, etc.) as preprocessor macros in `<asm-generic/siginfo.h>`. These can collide with C++ parameter names or member names, causing cryptic syntax errors like `error: expected ')'`. This is Android-specific — Linux, Windows, and macOS don't define these as macros.
+- **C++26/libc++ transitive include breakage**: When a library compiles with `-std=c++2c` (C++26) using Clang/libc++ (macOS, Android), previously-transitive standard library includes may no longer be available. A missing `#include <algorithm>` that worked in C++20 may fail in C++26. MSVC and GCC/libstdc++ are more permissive about transitive includes.
+- **Feature test failure artifacts**: When the CI runs feature tests (testing specific feature combinations), failure log artifacts use suffixed names like `portname_cpp`, `portname_all_3` instead of the base port name. These directories may contain `issue_body.md` (with the build error) and `tested-spec.txt` (the feature combination tested) instead of traditional build log files. Read `issue_body.md` first for quick triage — it contains a human-readable summary of the failure including error output.
+- **Version validation failures**: The "Validate version files" task can fail *before any ports are tested* if the PR updated `vcpkg.json` but didn't run `vcpkg x-add-version`. This produces errors like `portname@N was not found in versions database` and `baseline.json: error: portname is assigned M, but the local port is N`. No failure log artifacts are produced. The fix is always `vcpkg x-add-version {portname}`.
+- **Missing build tools (Go, Java, Rust)**: Some port features require external tools not installed on CI agents. Go is not available on macOS/Android agents ("Could not find GO_BIN") and has permission issues on Linux ("mkdir /go: permission denied"). When core port builds pass but specific features fail across all non-Windows triplets with tool-not-found errors, mark them as `feature-fails` in `ci.feature.baseline.txt`.
+- **Feature baseline coverage gaps**: When `ci.feature.baseline.txt` entries only cover a subset of triplets (e.g., only `arm64-linux`), the same features failing on other triplets (e.g., `x64-linux`, `arm64-osx`, Android) will cause unexpected job failures with no REGRESSION lines. Always check whether baseline entries cover *all* affected triplets, not just one.
 - **Portfile anti-patterns — do NOT recommend these as fixes:**
   - `set(VCPKG_BUILD_TYPE release)` — this is **only** appropriate for header-only libraries. Ports that produce binary output (`.lib`, `.a`, `.dll`, `.so`) must build both debug and release configurations. Never suggest adding this to fix mismatched-binary warnings.
   - `set(VCPKG_POLICY_* enabled)` — policy overrides are escape hatches for exceptional cases, not standard practice. Most ports should not set any `VCPKG_POLICY_*` variable. When a post-build check fails, the correct fix is almost always to fix the underlying issue (e.g., correct `vcpkg_cmake_config_fixup()` arguments, install missing files), not to suppress the warning with a policy override.

.github/skills/analyze-ci-failures/references/report-template.md

@@ -148,6 +148,12 @@ Use this as a reference for tone, depth, and structure.
    - 🟠 **High** — Failures on 2+ triplets, popular ports, or clear upstream API breaks
    - 🟡 **Medium** — Single-triplet failures with clear fixes
    - ℹ️ **Baseline/Investigate** — Expected failures, environment issues, or unclear root cause
+14. **Feature baseline unexpected passes**: When a job fails without REGRESSION lines, always scan the step log for `"passed but was marked expected to fail"` from `ci.feature.baseline.txt`. Report these under a separate "🟡 Unexpected Passes (Baseline Cleanup Required)" section, not as regressions or unrelated failures. The fix is to remove the stale entry from `scripts/ci.feature.baseline.txt`.
+15. **Android-specific macro collisions**: When a port fails only on Android with syntax errors and the compiler note says `"expanded from macro"` pointing to an NDK system header, this is a macro name collision — not a code bug in the traditional sense. Report it as an Android-specific issue with the fix being to rename the conflicting identifier upstream.
+16. **C++26 transitive include issues**: When a port compiles with `-std=c++2c` and fails on Clang/libc++ (macOS, Android) but passes on MSVC/GCC, suspect missing standard library includes that were previously available transitively. Check the compile command for the C++ standard flag.
+17. **Version validation failures**: When the "Validate version files" task fails, report it as a separate issue (not a build regression). The fix is always `vcpkg x-add-version {portname}`. No failure log artifacts are produced for this failure type.
+18. **Feature baseline coverage gaps**: When feature tests fail on multiple triplets but `ci.feature.baseline.txt` only covers some, explicitly list which triplets are missing entries and recommend expanding the baseline. Don't just say "update the baseline" — show which entries need to be added.
+19. **Debug/release CRT mismatch**: When a port fails on all Windows triplets except `x64-windows-release` with `LNK2038` runtime library mismatch errors, the root cause is host tools (code generators) mixing debug and release objects. Recommend using `OPTIONS_RELEASE`/`OPTIONS_DEBUG` in `vcpkg_cmake_configure()` to restrict tool building to the release configuration.
 
 ---
 
@@ -183,3 +189,10 @@ When analyzing a new build, scan for these high-frequency patterns before deep-d
 | `must install xorg-macros` | autotools ports | CI image missing xorg-dev package |
 | Build truncated, no error message | OOM or timeout | Use `DISABLE_PARALLEL` or check resources |
 | Many ports → same dependency fails | dep chain | Find root dep; rest are downstream casualties |
+| `passed but was marked expected to fail` | Job exit code 1, no REGRESSION lines | Stale `ci.feature.baseline.txt` entry — remove it |
+| `error: expected ')'` + `expanded from macro` on Android | Android NDK macro collision | POSIX signal macros (`si_value`, etc.) clash with code identifiers |
+| `no member named 'X' in namespace 'std'` + `-std=c++2c` | Clang/libc++ C++26 mode | Missing `#include` — transitive includes removed in C++26 |
+| `was not found in versions database` or `baseline.json: error: ... is assigned M, but the local port is N` | Version validation task | PR didn't run `vcpkg x-add-version` — no port builds attempted |
+| `Could not find GO_BIN` or `mkdir /go: permission denied` | Go-dependent features | Go not available/writable in CI — mark features as `feature-fails` |
+| Feature tests fail on triplets not in `ci.feature.baseline.txt` | Job exit code 1, no REGRESSION lines | Feature baseline entries only cover some triplets — expand them |
+| `LNK2038: mismatch detected for 'RuntimeLibrary'` or `'_ITERATOR_DEBUG_LEVEL'` | MSVC debug build, passes on release-only triplet | Debug/release CRT mismatch in host tools — use `OPTIONS_RELEASE` to build tools only in release |