Suppress Redivis DOIs from DataCite

[amyehodge]

Via DataCite we are ingesting records from the other Stanford Redivis organizations that we don't want to include. We need to figure out a way to either not include these or suppress them.

We only want to include Redivis content from SUL, Population Health Sciences (PHS) and the Doerr School (SDSS).

[thatbudakguy]

Needs analysis about the best way to exclude these.

[hudajkhan]

I went to the Stanford Redivis webpage https://stanford.redivis.com/ and looked through the organizations and listed datasets that we do not explicitly request from Redivis.
We explicitly ask for SDSS, SPHS, and SUL org datasets.

For the remaining, here is a breakdown of which datasets we are including and those that don't have any DOIs and would thus not be returned in a DataCite query.

• GSB https://stanfordgsb.redivis.com/ . None of these datasets have DOIs and text searches did not show results in the index. Since these particular datasets are DOI-less, we don't anticipate retrieving these datasets from any DataCite queries. The citation lists the DOI for Redivis itself (10.71778/V2DW-7A53).
• Shah Lab. https://stanford.redivis.com/ShahLab . We have all 4 datasets with "Shah Lab" listed as creator. The index may include canonical versions of the DOIs instead of the most recent version DOI in Redivis. We can add the canonical DOIs and version DOIs to the suppression list.
• DDI. https://stanford.redivis.com/DDI/datasets. None of these datasets have their own DOIs.
• LEVANTE: Most of the items don't have DOIs, but one item does: https://stanford.redivis.com/datasets/czjv-8eg1cc3t3 .This item is also in our index with the canonical DOI 10.57761/eywp-rm76 instead of the latest version DOI on the redivis page for the item. We can add the canonical DOI and the version DOIs to the suppression list
• Salehi Lab. https://stanford.redivis.com/Salehi_Lab . There are zero datasets in this organization.
• Taking the pulse of the planet.https://sdss.redivis.com/Pulse. 1 dataset with the DOI 10.57761/kqv4-9d93 . We do not have this item in the index. This doi does not appear when we try to look it up in commons.datacite.org. Hovering over the DOI on the Redivis page says this is a draft DOI. We can still add the DOI to the suppression list.

Options for automating suppression:
@amyehodge it would be good to get your feedback on whether we could suppress datasets based on the creator names. Redivis uses the "ownership" property, but DataCite datasets don't have this particular field. Instead, owner organizations are listed as creators. For example, https://api.datacite.org/application/vnd.datacite.datacite+json/10.57761/ak51-d519 shows "Shah Lab" as a creator of type organization.

[amyehodge]

Thanks for this detailed analysis @hudajkhan.

If it looks like using the creator names will work, then feel free to give that a try.

[hudajkhan]

Ok @amyehodge I will work on that in a second PR later. I wondered if listing one of the orgs as a creator would lead to suppression of too many datasets but maybe I can add a check for Redivis as publisher in addition so we don't limit too much.

[amyehodge]

@hudajkhan I have a feeling you won't catch too many extra things. But at this point I think it's better that we exclude too many than include these. Hopefully we can come up with a different way to deal with this in the future!

[thatbudakguy]

Pending more investigation about how to do this in an automated way, instead of just suppressing a bunch of DOIs.