Skip to content

Add RejectBy* to complement SelectBy* #99

Description

@richardkmichael

Feature Request

I'm migrating a small set of tests from one format to another. I would like to prevent the "old" format tests from running. I think could be useful in other situations as well. For example, excluding slow tests, or those unsuitable for GitHub Runner.

This can be done with SelectByTag by tagging all the tests which should run, such that the untagged tests do not run. Or, rearrange the files and similarly use `SelectByFolder. Both involve a fair bit of churn.

It would be helpful if there were exclusion inputs complementing the inclusion ones. i.e., RejectByTag, RejectByFolder`.

It seems it is not possible to use a "negated tag" string within the select, e.g., SelectByTag('~DoNotRun'). Escaping makes this a literal. I'm need to be able to emit ~HasTag('DoNotRun').


I thought I could contribute the implementation, so I investigated the PR which added SelectBy[Tag|Folder] and discovered it is backed by functionality in the script generator.

It seems changes to the matlab-script-generator will be required. I didn't find the source code in the MathWorks GitHub org.

Can I contribute to the scriptgen?


I explored the scriptgen source with an LLM and considered a few implementation directions. Here is a write-up -- perhaps verbose and already understood by this audience, but thought I'd include it, if helpful.

Exploration

Adding RejectByTag — scriptgen exploration and discussion

Exploration

Based on the exploration below, I believe this cannot be implemented cleanly in run-tests alone —
it needs a new option in the script generator (genscript / the +scriptgen package), because the
tag-to-selector translation lives entirely there and is hardcoded to the positive form. This
document lays out how scriptgen builds the tag selector today, a concrete RejectByTag design that
mirrors SelectByTag, and the design choices around negation.

Artifact reference: observations are against the matlab-script-generator v0 zip from
ssd.mathworks.com/supportfiles/ci/..., with package files dated 2025-12-05. Line numbers refer to
that copy.

Why a scriptgen change is required

run-tests does not implement tag selection itself. src/scriptgen.ts forwards the raw input into a
genscript('Test', ..., 'SelectByTag', '<value>', ...) call, and the selection logic lives entirely in
the downloaded scriptgen package. Four facts from reading that package:

  • The script builder exposes only SelectByFolder, SelectByTag (a scalar), and SelectByName. There
    is no reject/not/exclude option on any builder.

  • The tag selector is hardcoded to the positive form.
    +scriptgen/+internal/+expressions/+test/CreateHasTagSelectorExpressionBuilder.m:15 is
    text = sprintf('HasTag(%s)', obj.Tag) — always HasTag(...), never a negation.

  • Name-value pairs are applied by direct property assignment (the scriptgen.internal.mixin.SetGet
    set method does obj.(name) = value). Passing an unknown pair such as 'RejectByTag' throws
    rather than being ignored.

  • The tag value is quote-escaped, so a ~ cannot be smuggled through select-by-tag. The value is
    wrapped in single quotes and internal quotes are doubled, so it always lands inside HasTag('...')
    as a literal tag string.

MATLAB itself supports the negation directly — suite.selectIf(~HasTag('Broken')), where ~ builds a
matlab.unittest.selectors.NotSelector. TestSuite has no rejectIf; rejection is selectIf with a
negated selector. So the capability exists in the framework; scriptgen simply never emits the ~.

How scriptgen builds the tag selector

scriptgen emits MATLAB source text that run-tests runs in a later phase, so a selector is a string at
generation time and only becomes an object when the generated script executes. Each builder is a
public interface class (declares properties and setters) paired with an internal implementation class
that extends it and provides build() (version-specific). A value flows through a three-stage relay of
these builders, each configured by name-value pairs:

genscript('Test', 'SelectByTag', 'FeatureA')

# property + setter, and pass-through
stage 1  TestScriptBuilder                          .SelectByTag = FeatureA
           build():  createSequence('CreateTestSuite', ..., 'SelectByTag', quote(obj.SelectByTag))

# passed-through property + setter; conditional; emit positive HasTag selector
stage 2  CreateTestSuiteSequenceBuilder             .SelectByTag = FeatureA
           build():  if ~isempty(obj.SelectByTag)
                       buildSelectorStatement(.., 'CreateHasTagSelector', 'Tag', obj.SelectByTag)
                         -> createExpression(...) then selectIf-wrap

# builds the HasTag(...) expression
stage 3  CreateHasTagSelectorExpressionBuilder      .Tag = FeatureA
           build():  sprintf('HasTag(%s)', obj.Tag)   ->  HasTag('FeatureA')

emitted:  suite = suite.selectIf(HasTag('FeatureA'));
import:   matlab.unittest.selectors.HasTag

The property appears at both stage 1 and stage 2 because the value is relayed between two separate
builder objects; each must declare the property it receives. Within each stage the property is declared
once on the public class and inherited by the internal class.

Proposed RejectByTag design

RejectByTag mirrors the relay exactly, with one negation applied at stage 2. Stage 3 is reused
unchanged — it still emits the positive HasTag('...'), which stage 2 negates.

genscript('Test', 'RejectByTag', 'Broken')

# new property + setter, and pass-through
stage 1  TestScriptBuilder                          .RejectByTag = Broken
           build():  createSequence('CreateTestSuite', ..., 'RejectByTag', quote(obj.RejectByTag))

# passed-through new property + setter; new conditional; emit negated HasTag selector
stage 2  CreateTestSuiteSequenceBuilder             .RejectByTag = Broken
           build():  if ~isempty(obj.RejectByTag)
                       expr = createExpression('CreateHasTagSelector', 'Tag', obj.RejectByTag)
                       neg  = scriptgen.Expression(['~' expr.Text], expr.RequiredImports)
                       statements(end+1) = obj.buildSelectIfStatement(neg)

# reused, unchanged
stage 3  CreateHasTagSelectorExpressionBuilder      .Tag = Broken
           build():  sprintf('HasTag(%s)', obj.Tag)   ->  HasTag('Broken')

emitted:  suite = suite.selectIf(~HasTag('Broken'));
import:   matlab.unittest.selectors.HasTag

Files touched (the work is plumbing the new property through, not the ~):

File Change
+scriptgen/+scripts/TestScriptBuilder.m add RejectByTag property + set.RejectByTag
+scriptgen/+internal/+scripts/TestScriptBuilder.m relay 'RejectByTag', quote(obj.RejectByTag) down
+scriptgen/+sequences/+test/CreateTestSuiteSequenceBuilder.m add RejectByTag property + set.RejectByTag
+scriptgen/+internal/+sequences/+test/CreateTestSuiteSequenceBuilder.m add the reject branch in build()

Stage 3 (CreateHasTagSelectorExpressionBuilder) needs no change if the negation is applied in stage 2.
Skipping the stage-1 property makes genscript('Test', 'RejectByTag', ...) throw; skipping the stage-1
relay accepts the option but never passes it to stage 2.

Negation specifics

The negation must be emitted as text, because scriptgen's output is source code. Two constraints
shape how:

  • matlab.unittest.selectors.NotSelector cannot be constructed directly — per its documentation, it is
    instantiated only by the ~ operator. So the generated text must use ~HasTag('...'); a
    NotSelector(HasTag('...')) constructor call is not an option.

  • Emitting an operator in generated text is already established practice. CreateHasBaseFolderSelector
    joins its constraints with the | operator
    (HasBaseFolder(StartsWithSubstring('a') | StartsWithSubstring('b'))), so a leading ~ is consistent
    with how scriptgen already produces selector text.

Design choice: inline negation vs a dedicated not-selector builder

A - Inline negation in the sequence builder (as above)

Reuse CreateHasTagSelector to produce HasTag('...'), wrap its text in ~, and hand the negated
Expression to the existing buildSelectIfStatement. Minimal, and it mirrors how SelectByTag
already flows. For just reject-by-tag, this is the smaller change.

B - A dedicated CreateNotSelectorExpressionBuilder

This slots into the existing Create*Selector* family by name and generalizes: with sibling
CreateAndSelector / CreateOrSelector builders it would express arbitrary selector logic and open
the door to a generic "select by selector" input later. Two caveats: it would be the first
combinator builder (it wraps an inner selector expression rather than taking a value, which none of
the current builders do), and it must emit the ~ operator, not a NotSelector(...) constructor.

For consistency, whichever option is chosen should keep the "feature not available" warning path the
other selectors use (the fallback when an expression cannot be built for the running release). One note
in passing: the existing selector builders are not uniform — CreateHasName (the newest) emits its own
selectIf('Name', names) name-value statement rather than returning a selector object the way
CreateHasTag and CreateHasBaseFolder do — so there is no single rigid idiom to match.

C - Modify genscript result

For completeness: run-tests could implement reject-by-tag without any scriptgen change by augmenting
the script that genscript returns — inserting suite = suite.selectIf(~matlab.unittest.selectors.HasTag('<tag>'))
before the run line. That works and is self-contained, but it couples run-tests to the shape of
scriptgen's generated output (the suite variable name and the run statement), which is why the cleaner
home for the feature is scriptgen itself.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions