Support for running a single model counter?

[latower]

Question from @ydotlai:

Would it possible to provide an interface to let the users run a single tool? Then the tool with the functionality of self-verification can exploit the fuzzer. Currently I need to run at least two model counters on the same instance.

[latower]

The current version in main branch should be able to run just one counter if you comment out line 169 in src/fuzzer_utils.py, but I haven't tested it.

However: that means that you are not fuzzing to find incorrect counts, only fuzzing to find crashes. If you also want to fuzz for incorrect counts, there should be enough information in that branch to also run cpog to formally verify the counts, so you can compare the results. I would suggest that you give cpog plenty of memory resources (at least 8 GB), because I found that it has a tendency to run out of memory.

There is an experimental feature in that branch, src/generate_report.py, which will create a .tex (and optionally .pdf) file that prints some aggregated and detailed results on your fuzzer run. Otherwise, you should be able to find a .csv with the results in the logs/ subdirectory. I'm pointing these out to you because I don't think this version deletes all the instances that don't trigger bugs yet, so it might be hard to find those. I've put it on my todo list to fix that.

I have a deadline tomorrow, but can take a look and make the fuzzer ready to run with just one counter after that.

[ydotlai]

OK. Thanks. Then I can use the fuzzer to test Panini if it works.

[latower]

@ydotlai Can you please check out the main branch, check the README for instructions, and let me know if you can now run a single counter successfully? Also please let me know if you run into problems or find something unclear. Thanks.

[ydotlai]

Sorry, I have been still using old-main to test ExactMC. I will begin to test Panini thereafter. Then will let you know.

Thanks,
Yong

[ydotlai]

The current version in main branch should be able to run just one counter if you comment out line 169 in src/fuzzer_utils.py, but I haven't tested it.

However: that means that you are not fuzzing to find incorrect counts, only fuzzing to find crashes. If you also want to fuzz for incorrect counts, there should be enough information in that branch to also run cpog to formally verify the counts, so you can compare the results. I would suggest that you give cpog plenty of memory resources (at least 8 GB), because I found that it has a tendency to run out of memory.

There is an experimental feature in that branch, src/generate_report.py, which will create a .tex (and optionally .pdf) file that prints some aggregated and detailed results on your fuzzer run. Otherwise, you should be able to find a .csv with the results in the logs/ subdirectory. I'm pointing these out to you because I don't think this version deletes all the instances that don't trigger bugs yet, so it might be hard to find those. I've put it on my todo list to fix that.

I have a deadline tomorrow, but can take a look and make the fuzzer ready to run with just one counter after that.

Hi Anna,

Have you changed the file src/fuzzer_utils.py? The line 169 of 6d42168702ccae90f36c5ccd64fcf17305a477ea is blank.

Best,
Yong