missing certain basisOfRecord when using institutionKey for count/download

[veronicar239]

There seems to be a discrepancy (missing occurrences) when using institutionKey vs. institutionCode for occ_count() (and presumably also for occ_download?). I noticed that certain types of basisOfRecord are missing when using institutionKey, specifically there appear to be zero counts for MACHINE_OBSERVATION, HUMAN_OBSERVATION, OBSERVATION, and OCCURRENCE (at least for institution USNM 586ee56e-b0fe-4dff-b7f9-aeb104f3308a).

Does institutionKey include all basisOfRecord types? The institutionKey variable is the primary GBIF identifier and presumably should contain all records for a given institution.

MACHINE_OBSERVATION

occ_count(institutionCode="USNM","basisOfRecord"="MACHINE_OBSERVATION")
[1] 37312
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="MACHINE_OBSERVATION")
[1] 0

HUMAN_OBSERVATION

occ_count(institutionCode="USNM","basisOfRecord"="HUMAN_OBSERVATION")
[1] 33102
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="HUMAN_OBSERVATION")
[1] 0

OBSERVATION

occ_count(institutionCode="USNM","basisOfRecord"="OBSERVATION")
[1] 624
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="OBSERVATION")
[1] 0

OCCURRENCE

occ_count(institutionCode="USNM","basisOfRecord"="OCCURRENCE")
[1] 288
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="OCCURRENCE")
[1] 0

MATERIAL_SAMPLE

occ_count(institutionCode="USNM","basisOfRecord"="MATERIAL_SAMPLE")
[1] 301197
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="MATERIAL_SAMPLE")
[1] 443301

PRESERVED_SPECIMEN

occ_count(institutionCode="USNM","basisOfRecord"="PRESERVED_SPECIMEN")
[1] 4831604
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="PRESERVED_SPECIMEN")
[1] 4831870

FOSSIL_SPECIMEN

occ_count(institutionCode="USNM","basisOfRecord"="FOSSIL_SPECIMEN")
[1] 727292
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="FOSSIL_SPECIMEN")
[1] 727292

MATERIAL_CITATION

occ_count(institutionCode="USNM","basisOfRecord"="MATERIAL_CITATION")
[1] 8531
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="MATERIAL_CITATION")
[1] 9390

LIVING_SPECIMEN

occ_count(institutionCode="USNM","basisOfRecord"="LIVING_SPECIMEN")
[1] 0
occ_count(institutionKey=c('586ee56e-b0fe-4dff-b7f9-aeb104f3308a'),"basisOfRecord"="LIVING_SPECIMEN")
[1] 0

I'm using R version 4.4.1 (2024-06-14). Thanks!

[jhnwllr]

@veronicar239

Thanks for writing.

This video might be helpful in understanding how institutionKey is linked to occurrences.

https://training.gbif.org/en/grscicoll/linking-occurrences

It is a somewhat complex process based off of institutionCodes, institutionID, the country of the institution and publisher and whether they are any explicit mappings in GRSciColl.

Most importantly for your question, certain basisOfRecord values do not get an institutionKey assigned to them.

This can be seen by using facets.

occ_count(facet="institutionKey",basisOfRecord="MACHINE_OBSERVATION")

A tibble: 0 × 0

occ_count(facet="institutionKey",basisOfRecord="PRESERVED_SPECIMEN")

A tibble: 10 × 2
institutionKey count

1 0e919a55-08d3-4bd2-aec1-200858fafd92 8438750
2 6a6ac6c5-1b8a-48db-91a2-f8661274ff80 7896540
3 fa252605-26f6-426c-9892-94d071c2c77f 5889846
4 ab83e5ab-f4f7-4c9b-a1d5-eb779dfb9377 5228784
5 1d808a7c-1f9e-4379-9616-edb749ecf10e 5025815
6 586ee56e-b0fe-4dff-b7f9-aeb104f3308a 4831870
7 be5b9c72-e499-42b8-a5dc-c27505cf232e 4477216
8 1be3aa67-8f90-4309-92f4-bd55426a8c09 4143557
9 29acb160-1a5c-4c58-aba3-93093d60f642 3723382
10 c40e8d01-a969-4fe0-b827-c69679ab42db 3543143

[veronicar239]

@jhnwllr could you please explain why certain basisOfRecord values do not get an institutionKey assigned to them? If institutionKey is the highest level GBIF identifier, shouldn't all data be linked to each individual institutionKey?

[jhnwllr]

@veronicar239 please watch this video explaining how institutionKey is generated

https://training.gbif.org/en/grscicoll/linking-occurrences

"institutions" in GBIF speak are things registered on GriSciCol. So it isn't applicable to all datasets or occurrences. You might be thinking about publishingOrg. All occurrences will have a publishingOrg.