Add feature to generate JUnit XML output

[jcoleman]

Background

It's common to use the JUnit plugin on Jenkins to consume machine-readable output about the tests in your CI builds. For example, on our Ruby app tests we use the rspec_junit_formatter to output results from RSpec in a format that Jenkins can consume.

What

This PR adds JUnit style XML output as an option to Spoom so that running Sorbet in CI on Jenkins can output error information that can be displayed natively by Jenkins (and then, in turn, passed along to e.g. Github's interface).

With these changes Sorbet failures on a build look like this:

and a clean Sorbet build looks like this:

TODO

I'm happy to add docs for this feature as well, but I wanted to get initial review of the feature going sooner rather than later.

[Morriar]

Hey @jcoleman,

I think the feature is okay but the code will need to be adjusted.

Thanks!

[Morriar]

Looks like this is happening in both branches, we can extract it before

[jcoleman]

The other's in a loop, but this one isn't, so I don't think that will work?

[Morriar]

What do you think moving this to Error there: https://github.com/Shopify/spoom/blob/main/lib/spoom/sorbet/errors.rb#L118 to error.to_junit_xml_element returns a REXML::Element?

You can also move the write_errors_to_xml to the Errors module: https://github.com/Shopify/spoom/blob/main/lib/spoom/sorbet/errors.rb#L6 maybe renamed as to_junit_xml

[Morriar]

I think you can just make it a REXML::Element.new("testsuite") so you don't have to create the doc for all the method and only create it at the end:

testsuite = REXML::Element.new("testsuite")
if errors.empty?
  #...
else
  errors.each { |error| testsuite << error.to_junit_xml_element }
end

doc = REXML::Document.new
doc << REXML::XMLDecl.new
doc << testsuite

[Morriar]

I don't think we need this?

[jcoleman]

REXML isn't included by default, AFAICT, and if we remove this line we get errors running the tests.

[Morriar]

@project is already a temporary context, you can do something like this:

Suggested change

	Tempfile.create("sorbet_junit") do |tempfile|
	result = @project.spoom("srb tc --junit_output_path=#{tempfile.path}")
	expected_doc = <<~XML.chomp
	<?xml version='1.0'?>
	<testsuite name='Sorbet' failures='0'>
	<testcase name='Typecheck' tests='1'/>
	</testsuite>
	XML
	tempfile.rewind
	actual_doc = tempfile.read
	assert_equal(expected_doc, actual_doc)
	assert(result.status)
	end
	result = @project.spoom("srb tc --junit_output_path=junit.xml")
	assert_equal(<<~XML.chomp, @project.read("junit.xml"))
	<?xml version='1.0'?>
	<testsuite name='Sorbet' failures='0'>
	<testcase name='Typecheck' tests='1'/>
	</testsuite>
	XML
	assert(result.status)

[Morriar]

Same here, we don't need the Tempfile

[jcoleman]

@Morriar I think I've responded to all of your feedback; I usually leave clicking "Resolve" for the reviewer so they can confirm the change first; let me know if you have a different preference.