Add API filter options for publication type

[LewisAJones]

I recently came across several instances of occurrences in the database which are linked to unpublished work. Would it be possible to add a filter to the API which could be used to exclude such occurrences? I imagine this might also be a useful default for data downloads? Apologies if this is already possible and I've missed it.

Thanks for all your great work!

[cambro]

The PBDB has long viewed unpublished occurrences, entered by authorized members of the PBDB, as very valuable data that should not be discarded, and certainly not by default. There are many reasons for this. However, there could be utility in identifying these types of occurrences more explicitly, if readily possible (however, these are identified in exactly the way that you point to above: the references of occurrences can be analyzed and filtered based on publication type by the user). We will raise this with the EC.

[LewisAJones]

Thanks for the quick response @cambro! I think just having an additional column specifying the 'publication type' would be really helpful in identifying such cases? The user can then easily identify whether to include such occurrences/collections when working with large datasets?

[cambro]

Helpful, or very misleading, depending on your view. "Unpublished" sounds worse to many people, I would guess. But in actuality, some (most? all?) of these "unpublished" occurrences may be much "better" or "more reliable" or "more expertly grounded" than published occurrences. We'll see what the EC says.

[mmcclenn]

I have just experimented with this on our development server, dev.paleobiodb.org. The API on that server now accepts the parameter "published" when querying for collections, occurrences, specimens, taxa, and opinions. If this parameter is given, only records associated with references with publication type other than 'unpublished' are returned. You can instead specify "published=no", which will return only records associated with references with publication type 'unpublished'.

Please check it out, and see if this does what you want. In order to download using this parameter, you will have to copy the URL generated by the download form and manually add the parameter. Please keep in mind that the database on the development server is not up to date with the main server.

I will wait to take any further action until I hear from the EC.

[LewisAJones]

Hi @mmcclenn - this works exactly how I had hoped, thanks a lot for implementing this so quickly!