Skip to content

Conversation

@CyberiaResurrection
Copy link
Collaborator

@CyberiaResurrection CyberiaResurrection commented Sep 4, 2025

Supplying two mutant specs to mutmut run resulted in the following behaviour:

/home/alex/gitstuf/traveller_pyroute/venv/bin/python /home/alex/gitstuf/traveller_pyroute/venv/bin/mutmut run "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1?"
⠇ Generating mutants
    done in 34228ms
⠸ Listing all tests
Traceback (most recent call last):
  File "/home/alex/gitstuf/traveller_pyroute/venv/bin/mutmut", line 7, in <module>
    sys.exit(cli())
  File "/home/alex/gitstuf/traveller_pyroute/venv/lib64/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/alex/gitstuf/traveller_pyroute/venv/lib64/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/alex/gitstuf/traveller_pyroute/venv/lib64/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/alex/gitstuf/traveller_pyroute/venv/lib64/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/alex/gitstuf/traveller_pyroute/venv/lib64/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/alex/gitstuf/mutmut/mutmut/__main__.py", line 957, in run
    _run(mutant_names, max_children)
  File "/home/alex/gitstuf/mutmut/mutmut/__main__.py", line 1001, in _run
    mutants, source_file_mutation_data_by_path = collect_source_file_mutation_data(mutant_names=mutant_names)
  File "/home/alex/gitstuf/mutmut/mutmut/__main__.py", line 877, in collect_source_file_mutation_data
    assert filtered_mutants, f'Filtered for specific mutants, but nothing matches\n\nFilter: {mutant_names}'
AssertionError: Filtered for specific mutants, but nothing matches

Filter: ('PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1?',)

Process finished with exit code 1

This PR adds tests to verify that "Filtered for specific mutants..." issue exists on mutmut itself (it does), and adds a small, self-contained fix to teach mutmut to handle such multiple mutant specs.

After applying this PR, the same command line results in the following behaviour:

/home/alex/gitstuf/traveller_pyroute/venv/bin/python /home/alex/gitstuf/traveller_pyroute/venv/bin/mutmut run "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1?"
⠇ Generating mutants
    done in 33307ms
⠸ Listing all tests
⠹ Running clean tests
    done
⠧ Running forced fail test
    done
Running mutation testing
⠏ 11/29567  🎉 11 🫥 0  ⏰ 0  🤔 0  🙁 0  🔇 0
0.09 mutations/second

Mutant results
--------------
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_10
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_11
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_12
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_13
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_14
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_15
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_16
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_17
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_18
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_19

On another project, I tried to pass two _separate_ mutant specs
to mutmut run, and it promptly blew up with an assertion error,
saying "Filtered for specific mutants, but nothing matches",
_despite_ having successfully run each mutant spec on their own.

Because that other project is quite large, I figured the best
first step is to see if I can replicate the issue on mutmut's own
test cases.

Turns out I can.  While I was at it, I also added tests for
singleton mutant specs, both explicit and with wildcards.
As at the status quo, calling mutmut run with two or more mutant
specs in quotes (such as "mutant_1 mutant_2") blew up with an
AssertionError in collect_source_file_mutation_data, stating
"Filtered for specific mutants, but nothing matches. ...."

This commit handles that case, by splitting the tuple's element
and replacing the tuple if the resulting split has 2 or more
elements.
@boxed
Copy link
Owner

boxed commented Sep 4, 2025

Shouldn't the command be

mutmut run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1?"

? I'd expect that to work given the current code.

@CyberiaResurrection
Copy link
Collaborator Author

I'm not sure - I tried with both mutation specs in quotes - hmm... how about I try one spec out of quotes and two specs in quotes?

@boxed
Copy link
Owner

boxed commented Sep 4, 2025

The quotes has to do with how the shell interprets the commands, not how mutmut handles them.

@CyberiaResurrection
Copy link
Collaborator Author

With the command line run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_[12]?" passed to mutmut, and rolling back to main HEAD, I got:

/home/alex/gitstuf/traveller_pyroute/venv/bin/python /home/alex/gitstuf/traveller_pyroute/venv/bin/mutmut run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_[12]?
⠇ Generating mutants
    done in 34097ms
⠸ Listing all tests
⠹ Running clean tests
    done
⠙ Running forced fail test
    done
Running mutation testing
⠸ 21/29567  🎉 13 🫥 0  ⏰ 8  🤔 0  🙁 0  🔇 0
0.07 mutations/second

Mutant results
--------------
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_10
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_11
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_12
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_13
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_14
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_15
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_16
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_17
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_18
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_19
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_20
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_21
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_22
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_23
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_24
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_25
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_26
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_27
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_28
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_29

and passing run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1? PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_2?" to mutmut:

/home/alex/gitstuf/traveller_pyroute/venv/bin/python /home/alex/gitstuf/traveller_pyroute/venv/bin/mutmut run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1? PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_2?"
⠇ Generating mutants
    done in 70958ms
⠸ Listing all tests
⠹ Running clean tests
    done
⠏ Running forced fail test
    done
Running mutation testing
⠙ 1/29567  🎉 1 🫥 0  ⏰ 0  🤔 0  🙁 0  🔇 0
0.01 mutations/second

Mutant results
--------------
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1

That's somewhat unexpected - I was expecting either a detonation, or the full 21-mutant run

Further discussion with @boxed revealed that I hadn't considered
the case of a _multiple-element_ mutant_names tuple where at
least one of the elements contains at least one space.

First step, verify the wonkiness exists.
Second step, _fix_ the wonkiness and _verify_ the fix.
@CyberiaResurrection
Copy link
Collaborator Author

As at "Fix wonkiness when mutant names...", passing run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1? PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_2?" to mutmut yields:

/home/alex/gitstuf/traveller_pyroute/venv/bin/python /home/alex/gitstuf/traveller_pyroute/venv/bin/mutmut run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1? PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_2?"
⠇ Generating mutants
    done in 41143ms
⠸ Listing all tests
⠸ Running clean tests
    done
⠼ Running forced fail test
    done
Running mutation testing
⠦ 21/29567  🎉 13 🫥 0  ⏰ 8  🤔 0  🙁 0  🔇 0
0.08 mutations/second

Mutant results
--------------
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_10
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_11
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_12
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_13
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_14
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_15
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_16
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_17
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_18
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_19
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_20
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_21
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_22
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_23
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_24
⏰ PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_25
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_26
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_27
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_28
🎉 PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_29

@Otto-AA
Copy link
Collaborator

Otto-AA commented Sep 4, 2025

I think the current mutmut behaviour was already correct, as long as you pass every filter as a separate argument, eg mutmut run "first" "second filter" "third". If you run mutmut run "first second third" it will only pass the single filter "first second third" to mutmut. That's how CLI args generally work, so I would keep it like that.

With the command line run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_[12]?" passed to mutmut, and rolling back to main HEAD, I got:

Yes, you pass two arguments to the run command (you don't need to quote the first one because it has no spaces. Would be the same if you quoted it). The two filters result in the test selection you reported.

and passing run PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1 "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1? PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_2?" to mutmut:

Here you only pass two arguments:

  • PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1
  • "PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_1? PyRoute.TradeCodes.xǁTradeCodesǁcanonicalise__mutmut_2?"

The first filter only has this single match. The second filter has no match (as it is accidentally a combination of two filters).

@CyberiaResurrection
Copy link
Collaborator Author

Bollocks. Well, closing this PR then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants