Windows support

[Otto-AA]

Refactor the code to also support windows. Closes #397

This is still a WIP. If anyone wants to finish this PR, feel free to leave a comment and take it over :)
I probably won't get much free time in the next weeks, but wanted to share the current status.

Implementation idea

The main tasks to support Windows are (1) to support the spawn method and (2) to support timeouts properly (if possible).

When directly using the spawn method it is much slower, because we need to startup everything from scratch for each mutant. This includes the python interpreter (I think) and the whole pytest test collection (ie importing the system under test).

To improve this performance, we only create max_children processes as "workers". These workers infinitely loop on:

1. pick the next mutant from a shared queue
2. test the mutant
3. return the result
4. repeat at (1) until the queue is empty

As such we only need to setup a limited number of processes, instead of one per mutant. To support timeouts (where we likely still want to kill processes), we need to restart workers when they time out.

TODO

• support spawn method by passing all globals as args to the new processes
• use a process pool to increase performance (when we reuse the processes, pytest caches all the imported files)
• add Windows tests to CI
• Reimplement timeouts (I've removed them for now for simplicity)
• Debug why this branch outputs slightly different results than on main
• Test on other repos
• Cleanup

Implementing timeouts

Previously we used resource to implement timeouts:

mutmut/mutmut/__main__.py

Lines 1009 to 1011 in f63029b

	estimated_time_of_tests = m.estimated_time_of_tests_by_mutant[mutant_name]
	cpu_time_limit = ceil((estimated_time_of_tests + 1) * 2 + process_time()) * 10
	resource.setrlimit(resource.RLIMIT_CPU, (cpu_time_limit, cpu_time_limit))

And also a timeout checker thread (which I assume is only the backup in case resource does not work):

mutmut/mutmut/__main__.py

Lines 864 to 878 in f63029b

	def timeout_checker(mutants):
	def inner_timout_checker():
	while True:
	sleep(1)
	now = datetime.now()
	for m, mutant_name, result in mutants:
	for pid, start_time in m.start_time_by_pid.items():
	run_time = now - start_time
	if run_time.total_seconds() > (m.estimated_time_of_tests_by_mutant[mutant_name] + 1) * 4:
	try:
	os.kill(pid, signal.SIGXCPU)
	except ProcessLookupError:
	pass
	return inner_timout_checker

Windows does not support resource so maybe we need to only use resource on Linux and the timeout checker on Windows.

Also, we now reuse processes. So when killing a long-running process we need to ensure that we restart that worker, and that we report it as timeouted. The restarting should already work with the current implementation. But it is not reported as a result. So we likely need to track which process starts which task and then in _remove_stopped_workers we need to map the killed processes to the tasks they were executing, and report them as timed out (depending on the exit code).

Also note that I think mutmut reports the timeouted mutant as "killed" accidentally (I did not make a MRE yet, but it seems like it).

Debugging different results

I've tested it on https://github.com/TOD-theses/traces_parser with a # pragma: no mutate on these lines (they would run into an infinite loop as we have no timeout yet): https://github.com/TOD-theses/traces_parser/blob/49d0b9cbc6fad490bba1f8a3136b24cd964336e1/traces_parser/parser/storage/memory.py#L35-L36

The main branch of mutmut outputted slightly different results than this windows-support branch of mutmut. I am not yet sure if this is due to some kind of race condition, or if it is deterministic. Maybe adding some additional logs would be useful to debug this.