Simplecov cannot generate a new coverage report because it does not skip writing already existing asset files.

[chase-stevens]

Hi all,

I was able to generate a coverage/index.html the first time I ran rspec with simplecov. Each subsequent time, I have encountered an error where, when attempting to create the coverage/index.html file, simplecov gets an error that it is not allowed to open an asset file.

1: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1292:in `open' /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1292:in `initialize': Permission denied @ rb_sysopen - /home/chase/eic/coverage/assets/0.10.2/smoothness/images/ui-bg_glass_65_ffffff_1x400.png (Errno::EACCES)

What we believe to be happening is that simplecov is attempting to copy files from the gem kept in our nix/store directory and write them to the coverage/assets directory in the application. The first time writing was successful; however, simplecov does not appear to have the necessary logic to skip writing files that already exist.

Full trace below:

Traceback (most recent call last):
	26: from /nix/store/9gf3q3vzp3f25vvknqb8cf3d68wrfnq2-ruby2.5.5-simplecov-0.17.0/lib/ruby/gems/2.5.0/gems/simplecov-0.17.0/lib/simplecov/defaults.rb:29:in `block in <top (required)>'
	25: from /nix/store/9gf3q3vzp3f25vvknqb8cf3d68wrfnq2-ruby2.5.5-simplecov-0.17.0/lib/ruby/gems/2.5.0/gems/simplecov-0.17.0/lib/simplecov.rb:201:in `run_exit_tasks!'
	24: from /nix/store/9gf3q3vzp3f25vvknqb8cf3d68wrfnq2-ruby2.5.5-simplecov-0.17.0/lib/ruby/gems/2.5.0/gems/simplecov-0.17.0/lib/simplecov/configuration.rb:182:in `block in at_exit'
	23: from /nix/store/9gf3q3vzp3f25vvknqb8cf3d68wrfnq2-ruby2.5.5-simplecov-0.17.0/lib/ruby/gems/2.5.0/gems/simplecov-0.17.0/lib/simplecov/result.rb:48:in `format!'
	22: from /nix/store/jws2anzzrgimgd4v95b05niw2j4q2rrg-ruby2.5.5-simplecov-html-0.10.2/lib/ruby/gems/2.5.0/gems/simplecov-html-0.10.2/lib/simplecov-html.rb:18:in `format'
	21: from /nix/store/jws2anzzrgimgd4v95b05niw2j4q2rrg-ruby2.5.5-simplecov-html-0.10.2/lib/ruby/gems/2.5.0/gems/simplecov-html-0.10.2/lib/simplecov-html.rb:18:in `each'
	20: from /nix/store/jws2anzzrgimgd4v95b05niw2j4q2rrg-ruby2.5.5-simplecov-html-0.10.2/lib/ruby/gems/2.5.0/gems/simplecov-html-0.10.2/lib/simplecov-html.rb:19:in `block in format'
	19: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:392:in `cp_r'
	18: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1461:in `fu_each_src_dest'
	17: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1477:in `fu_each_src_dest0'
	16: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1463:in `block in fu_each_src_dest'
	15: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:393:in `block in cp_r'
	14: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:415:in `copy_entry'
	13: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1392:in `wrap_traverse'
	12: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1392:in `each'
	11: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1393:in `block in wrap_traverse'
	10: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1392:in `wrap_traverse'
	 9: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1392:in `each'
	 8: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1393:in `block in wrap_traverse'
	 7: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1390:in `wrap_traverse'
	 6: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:418:in `block in copy_entry'
	 5: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1259:in `copy'
	 4: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1291:in `copy_file'
	 3: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1291:in `open'
	 2: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1292:in `block in copy_file'
	 1: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1292:in `open'
/nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1292:in `initialize': Permission denied @ rb_sysopen - /home/chase/eic/coverage/assets/0.10.2/smoothness/images/ui-bg_glass_65_ffffff_1x400.png (Errno::EACCES)

I am using RSpec for tests and I run my tests using the following command
nix-shell --run "rspec"

I am requiring and staring SimpleCov in my spec_helper.rb as seen below. Additionally, I have been able to successfully run simplecov once before, so I do not think this is the issue.

require 'simplecov'
SimpleCov.start 'rails'

Software Versions

• Ruby 2.5.5p157 (2019-03-15) [x86_64-linux]
• Rails v5.1.6
• Simplecov (0.17.0)
• Simplecov-html (0.10.2)

Please let me know if you have any questions or if I can provide any additional information and I'll be happy to help.

Best,
Chase

[PragTob]

Hi there @chase-stevens,

thanks for the report 💚

The interesting question for me would be why it doesn't have access to these files any more? Seems like there might be something wrong there?

The reason it writes already existing asset files again is that the gem version and hence the files might have changed. Sure, we could include some hash in the file names or even the version names but that'd require a build process etc... as the time to write the files again isn't too big we just write them new on each run.

I'm also not sure if it'd totally solve your problem. My fear is that if the access rights for the asset files aren't given then simplecov probably also doesn't have the rights to override coverage/index.html so we wouldn't win anything there.

So - I don't think there's much we can do here and it seems more a problem with your file permissions. If you think we can help please let us know :)

[PragTob]

@chase-stevens 👋 hi there, any update on this and the questions from your side? :)

[PragTob]

Closing due to missing feedback

[felixscheinost]

When using Nix all the gems are stored read-only in /nix/store.

As you can see simplecov-html does a simple FileUtils.cp_r which preserves rights.

I would propose that the fix would be to chmod the files, e.g. 644?

[PragTob]

@felixscheinost what is "Nix" ?

I'm not sure whether this is another problem we should fix or that users should fix... like we already integrate with so much. Us messing with chmoding prodocues potential further issues (what if we can't chmod...) so I'm almost inclined to say that this would be user level to chmod them themselves.

[felixscheinost]

Sorry, forgot to explain what Nix is - seems like I am too much in my own filter bubble :)

Nix is a build tool and package manager that focuses on purity and reproducibility.
All packages are stored in /nix and include the hash of the build inputs and outputs in the file names (see the error log in the initial post). Files generated by the build are 444 so that they can’t be changed after the hash is calculated.

I'll try to find a user level solution and will get back to you later.

[PragTob]

@felixscheinost have you found a user level solution yet?

While I generally would like to have things working everywhere, doing this would introduce some overhead. Of course, if we switched the asset compilation on simplecov-html to something more modern, have it easy to attach a hash to the file name then it'd be easy but right now I think there are other issues deserving our attention.

[emptyflask]

I'm a NixOS user too, and I've worked around this by adding the following to test_helper.rb:

require 'simplecov'
Pathname.new(__FILE__).join('..','..','coverage').tap do |cov|
  # clear old coverage results
  FileUtils.rm_rf cov if cov.exist?
end
SimpleCov.start('rails')

But if simplecov skipped rewriting the assets, that would be great!

[PragTob]

I was just thinking, we might introduce asset inlining which would also solve this problem - right?

[slinkp]

Would it hurt anything if simplecov did @emptyflask 's workaround? rm the asset tree if it exists, immediately before writing new ones?

[PragTob]

@slinkp maybe 🤷

Couldn't do it like this because it deletes the whole path. People might (accidentally) throw files in there and we can't just delete them all.

Could we delete all the files we're gonna copy over? Probably. Next thing I know maybe for some reason under some OS we don't have the right to create files there... and then that case starts failing all of a sudden.

So, I'm not very inclined to include it into default. Asset inlining and/or having hashes build style at the end of the assets would be the proper solutions I think. For the time being people can implement this workaround.

Reopening to know/remember it'd be nice to fix these.