Commit ff27b2d
committed
Support
`codeql test run --learn` could previously only rewrite `.expected`
files; inline expectations (the `// $ Alert` comments checked by
`InlineExpectationsTest`) had to be fixed by hand. This teaches the
shared test library to compute those source edits so the test runner
can apply them.
The `test-postprocess` query now exposes a `learnEdits` relation
(`file, line, operation, startColumn, endColumn, text`) describing the
minimal source rewrite that would make the inline expectations match the
actual query results. It covers:
- appending a fresh comment carrying every tag learned for a line that
has an unexpected result and no existing comment to merge into;
- rewriting an existing expectation comment as a whole so it matches
the current results: dropping fixed-spurious tags, promoting a
`MISSING:` expectation that now fires, clearing a stale `SPURIOUS:`
annotation, and merging in freshly learned tags, re-rendering the
remaining expectations (or deleting the comment when none remain);
- preserving expectations this test does not own -- e.g. a tag
annotated with a different query's id that shares the source file --
and any trailing regular note (`// $ Alert // note`);
- recording any unexpected result, not just `Alert`.
Edits are emitted as a query predicate rather than applied here: the
engine consumes `learnEdits` only under `--learn` and ignores it
otherwise, so ordinary `test run` output is unchanged.
Comment syntax is provided per language by each
`InlineExpectationsTestQuery.ql`'s `Input` (`getStartCommentMarker`),
which for now renders only line-comment languages; block-comment
languages (XML/YAML) can supply their markers later without changing
this relation's shape.--learn for inline expectations via learnEdits1 parent d1fed84 commit ff27b2d
12 files changed
Lines changed: 611 additions & 26 deletions
File tree
- cpp/ql/lib/utils/test
- csharp/ql/lib/utils/test
- go/ql/lib/utils/test
- javascript/ql/lib/utils/test
- java/ql/lib/utils/test
- python/ql/lib/utils/test
- ql/ql/src/utils/test
- ruby/ql/lib/utils/test
- rust/ql/lib/utils/test
- swift/ql/lib/utils/test
- unified/ql/lib/utils/test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
21 | 30 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
Lines changed: 8 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
21 | 29 | | |
0 commit comments