Module 3C: Advanced Cleaning with Internet Archive Data (Cont'd)
When reconciliation is finished, you should see two new facets. For now, you can ignore the bottom facet -- what we care about is the top one.
The 'Judgement' facet tells us that out of the 273 unique dates from 1977, our SKOS RDF matched all but 3. Let's click on the 'Unmatched' records in that facet.
If we look at the description and title metadata, we can see why these IA shows did not match: they're not, strictly speaking, Dead concerts.
Grateful Dead Live at Arista Studios on 1977-02-17
Grateful Dead Live at Studio (Terrapin Station Outtakes) 1977-02-20
"Tuning '77" - an audio supercut of an entire year of the Grateful
Dead tuning their instruments on stage
Because these three don't fall into the purview of what we are trying to accomplish -- that is, matching live Dead concert dates to baseline data -- we are going to flag (or star) these three rows so that we can discount them from our data later.
Now that we've done one year together, feel free to experiment on other years. Or, if not, read onward!
After cleaning all 30 years of dates and flagging/removing the dates that turned out to be sound checks, studio outtakes, video shoots and so forth, I came up with 1,941 unique dates that matched data in the Internet Archive's collection. Our master SKOS-RDF file contains a total of 2,302 unique Dead concert dates, which puts the Internet Archive's coverage at roughly 84 percent. The remaining unmatched 361 dates mostly between 1965 to 1971, when Dead taping (both official and unofficial) wasn't as widespread as it would become.
Still, keep in mind that these calculations only represent some kind of recording existing for a given date. It isn't meant to imply that the existing recordings are complete, nor does it imply that multiple shows were recorded if more than one exists for a single date. That being said, it's pretty incredible to be able to say that almost 85 percent of known tour dates for a single band are available in one place!