Forked process doing cleanup at_exit can cause duplicate coverage reports

[HerbCSO]

When you run bundle exec rspec on this repo (that is the most minimal test case I can come up with), there will be two SimpleCov reports generated, with very different coverage percentages. The fact that the UtilityConfig class has a forked process that is cleaned up in an at_exit call seems to cause some sort of race condition in SimpleCov (I have not been able to figure out where this occurs).

Sample output on my system:

> bundle exec rspec
.

Finished in 2 seconds
1 example, 0 failures

Randomized with seed 51923

Coverage report generated for RSpec to /opt/pose/test-simplecov/coverage. 23 / 25 LOC (92.0%) covered.
Coverage (92.00%) is below the expected minimum coverage (95.00%).
Coverage report generated for RSpec to /opt/pose/test-simplecov/coverage. 5 / 25 LOC (20.0%) covered.

The second Coverage report is wrong. If it wasn't for the fact that it overwrites the results of the first (correct) one, I wouldn't mind so much... ;]

Please let me know if I can help with any other input/background.

[HerbCSO]

Oh, and I forgot to mention the best bit - if I debug with rspec -d and step into SimpleCov after my tests are done and then continue execution, only a single report gets generated. Sounds a lot like a race condition of some sort, wouldn't you say?

[colszowka]

Hi Carsten, could you please try the 0.8.0 prerelease from rubygems? It implements a lockfile for the resultset merging and should fix your problem

[HerbCSO]

Thanks for the quick response Christoph!

Tried it just now, but unfortunately that doesn't seem to fix the issue. I still get two runs of the coverage report with different percentages:

> bundle exec rspec
.

Finished in 2.01 seconds
1 example, 0 failures

Randomized with seed 55991

Coverage report generated for RSpec to /opt/pose/test-simplecov/coverage. 23 / 25 LOC (92.0%) covered.
Coverage (92.00%) is below the expected minimum coverage (95.00%).
Coverage report generated for RSpec to /opt/pose/test-simplecov/coverage. 5 / 25 LOC (20.0%) covered.

Just to make sure we're on the same page, here's the Ruby version I'm using:

  ruby:
    interpreter:  "ruby"
    version:      "1.9.2p325"
    date:         "2013-04-15"
    platform:     "x86_64-darwin12.3.0"
    patchlevel:   "2013-04-15"
    full_version: "ruby 1.9.2p325 (2013-04-15) [x86_64-darwin12.3.0]"

and a list of my local gems:

>  gem list --local                                                                                                                                                                                       ruby-1.9.2-head@test-simplecov

*** LOCAL GEMS ***

bundler (1.3.5)
diff-lcs (1.2.4)
multi_json (1.7.6)
rake (10.0.4, 0.8.7)
rspec (2.13.0)
rspec-core (2.13.1)
rspec-expectations (2.13.0)
rspec-mocks (2.13.1)
rubygems-bundler (1.1.1)
rubygems-update (1.4.2)
rvm (1.11.3.7, 1.11.3.5)
simplecov (0.8.0.pre, 0.7.1)
simplecov-html (0.7.1)

(I also tried uninstalling 0.7.1 just in case, same result)

I updated my test case repo with the new Gemfile, just in case you want to see that.

Let me know if you need anything else, please.

[HerbCSO]

So I've found a workaround for my code at least. The solution was to add an at_exit { exit! } into my forked process, which caused it to not fire the main at_exit handler in my code anymore, and then gets around SimpleCov triggering its reporting twice. See this diff for what I did in the context of the code.

[colszowka]

@HerbCSO Cool, thanks! I'll keep this open as a placeholder for a standardized solution built into simplecov