Skip to content

Conversation

@Magic-Man-us
Copy link

  • Extend operator_name to flip more built‑ins (len↔sum, min↔max, all↔any, str↔repr, etc.)
  • Add operator_regex to mutate regex literals (quantifier swaps, {1,}↔+, \d↔[0-9], char‐class reversal)
  • Register operator_regex alongside other call mutations
  • Update tests to cover new regex and name‐swap behaviors

@Otto-AA
Copy link
Collaborator

Otto-AA commented Apr 21, 2025

Hi, thanks your contribution, many new mutations that could uncover missing tests :)

There are some mutations, where to me it seems they don't bring much value to mutation testing. If you want, some of them could be reworked, or you can simply drop them for now and we can revisit them later.

In particular, some of the mutations seem like "equivalent mutants", I think we should remove or adapt them. When we would include these mutations, the unit tests won't have a chance to detect them as the mutation behaves identical to the original code. So these are like false positives and we should try to keep their number low.

The following mutations seem like they could produce many equivalent mutants:

  • str <-> repr: in many cases these are equivalent, e.g. for builtin numbers, or also for attrs classes. There are cases where this is not the case, but I feel like it's not worth it.
  • list <-> tuple: If the program does not raise a runtime exception because of e.g. calling .append on a tuple, then I think these two behave similar. And if it throws a runtime exception, we only check code coverage of this line (where tools like coverage.py are more suited)
  • set <-> frozenset: same as for list <-> tuple; in the cases where no exception is raised, they likely behave identical
  • {1,} -> +: From my regex understanding, these are identical, and thus indistinguishable for the tests

The chr <-> ord swap also always (?) raises an Exception. For this, we could map chr(123) -> chr(123 + 1) and ord(c) -> ord(c) + 1, to check if changing the result slightly makes any difference.

Regarding len <-> sum and map <-> filter, I think these are convenient choices that fit into our current name mapping. If we want to spend some more time on the implementation, I would suggest to map len(...) -> len(...) + 1, sum(...) -> sum(...) + 1 and map(fn, arr) -> list(arr), filter(fn, arr) -> list(arr).

m for m in mutants
if "(None)" not in m
and "XX" not in m
]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This filtering seems redundant to me. If want in filtered then want in mutants is also true.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, I will remove this

@Magic-Man-us
Copy link
Author

You are spot on with a lot of this. I apologize for not getting back earlier its been a busy week .
I will remove the equal equivalents, or perhaps do a +1.

In the meantime I was thinking of some problems I saw in the past and of course, I did not experience 😅 ...
One really good candidate I will add to the list is Enum to Str etc. it should cause string comparisons will fail silently >:)
JSON serialization or API payloads will break if not properly handled.
and Database operations or other logic expecting strings or ints will raise exceptions or errors.
Also Enum to Int!

I will update sometime this weekend.
Thank you for the feedback @Otto-AA

@Magic-Man-us Magic-Man-us force-pushed the add-regex-quantifiers branch 2 times, most recently from 9fa1d44 to 286514e Compare June 1, 2025 17:08
@Otto-AA
Copy link
Collaborator

Otto-AA commented Jun 1, 2025

Hi, you can ping me when your changes are review-ready :)
(Though I will probably need some days to find the time for reviewing)

@Magic-Man-us Magic-Man-us force-pushed the add-regex-quantifiers branch from 286514e to ceacaae Compare June 8, 2025 02:28
@Magic-Man-us Magic-Man-us force-pushed the add-regex-quantifiers branch from 3cfbd0f to 66e4162 Compare June 8, 2025 02:42
@Magic-Man-us
Copy link
Author

Magic-Man-us commented Jun 8, 2025

Thanks for the feedback @Otto-AA ! I've addressed your concerns about equivalent mutants.

This introduces improvements to the mutation testing focusing on eliminating equivalent mutations as you had mentioned prior, additionally, adding meaningful function call mutations, and enhancing regex mutation handling. it includes comprehensive updates to tests for better coverage and accuracy. Below is a summary of the most important changes grouped by theme.

Mutation Framework Enhancements

  • Eliminated equivalent mutations for common operations (len ↔ sum, min ↔ max, etc.) to reduce false positives and improve mutation score accuracy. [1] [2]
  • Added new function call mutations for aggregate functions (len(...) → len(...) ± 1) and mapping/filtering functions (map(fn, arr) → list(arr)), ensuring meaningful semantic changes. [1] [2]
  • Improved regex mutation handling by avoiding equivalent mutations ({1,} ↔ +) and introducing meaningful alternatives ({1,} → {2,} or {0,}). [1] [2]

Code Quality Improvements

  • Corrected typos in comments and documentation, such as fixing "indidvidual" to "individual."
  • Preserved function signature integrity in chr/ord mutations to avoid runtime exceptions, ensuring safer mutations. [1] [2]

Test Coverage Enhancements

  • Added extensive parameterized tests for regex mutations to ensure complete coverage of mutation patterns and edge cases.
  • Introduced comprehensive tests for subscript mutations, including edge cases like slices with non-integer bounds and tuple subscripts.
  • Updated assertions in existing tests to use == instead of is for string and integer comparisons, improving correctness. [1] [2]

These changes should enhance the reliability and effectiveness of the mutation testing framework while ensuring even more test coverage.

 o7
/|  
/ \

@Otto-AA
Copy link
Collaborator

Otto-AA commented Jun 8, 2025

Hi @Magic-Man-us , I do not want to spend my free time on reviewing an AI. And btw, the previous PR looked much more useful than this. If you do not have any more time to work on this, then just say so, but please don't let me review an AI output instead.

@Otto-AA Otto-AA closed this Jul 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants