AI-FAIR Checklist

AI-FAIR Working Group repo for organization, planning, and development.

Community checklist evaluation: Feedback welcomed on the underlying FAIR4AI-Bio Checklist.

Current Resources

• From FAIR to AI-FAIR Metadata-First Infrastructure for AI-Enabled Science by Elizabeth G. Campolongo, Eric R. Sokol, Marty Downs, Gina Glenne, Gregory E. Maurer, Sydne Record, Dustin R. Rubenstein, Matthew J. Thompson, Christine Laney, Hilmar Lapp, Max Tarjan, and Tanya Berger-Wolf https://doi.org/10.48550/arXiv.2611.03327
  • Introduces the AI-FAIR metadata criteria, placing AI-enabled science in the context of a bidirectional Question-Data-Model triangle, and the agentic AI-FAIR evaluation workflow.

Drafting and Background

• Definitions_FAIR4AI_AI-ready.md
  • Living document for defining AI-FAIR and AI-ready.
  • Currently established as resource to help distinguish as we consider which elements of the checklist are sufficient for FAIR, AI-FAIR, and AI-ready.
  • This is meant to build on the FAIR4AI Working Definitions from 0.5 Workshop, which includes comments from Workshop 0.5 and notes for this group (see tabs on the sidebar).
• DRAFT AI-Ready Checklist
  • Based on combining the Imageomics Data Card and Metadata checklists¹ with the ESIIP Checklist².
• AI-Readiness Defined Slides
  • Presentation prepared for the 0.5 workshop to provide a common understanding/reference upon which to base our conversations.

Guiding Questions and Considerations

• What is the distinction between FAIR and AI-FAIR?
  • Is this different when thinking about biodiversity data?
  • How does it vary by data type/modality?
• What is the distinction between AI-FAIR and AI-Ready?
• What are the requirements and expectations for data providers as compared to users?

Potential Answers or Framing

AI-FAIR is to the point that it can be fed into a pipeline to have an output ready to put into a model. This can be the "AI-enabled" step.

We are working from the idea that AI-Ready is meant as this data can be fed directly into a model, so data need not be published in this format, just in a format that it is "reasonable" to get there. An example being the TreeOfLife-200M dataset is not AI-Ready, but using existing pipelines (once all data is downloaded), it can be transformed into the webdataset format used to train BioCLIP 2. Under this definition, the constituent parts of TreeOfLife-200M are not AI-FAIR.

Dataset Evaluation Agent

The evaluation agent as stored in this repository is archived from updates in Quarter 3 of 2026. Please see the FAIR4AI-Bio/evaluator repository for the most up-to-date version of the evaluation agent.

fair4ai-eval-agent/ — AI agent (v2, configured for use with Claude) that evaluates datasets for AI-readiness using the FAIR4AI-Bio checklist. See fair4ai-eval-agent/README.md for setup instructions.

archived/fair4ai-eval-agent-v1/ — Archived Python script agent (OpenAI/Azure/Anthropic). Superseded by v2.

Footnotes

1. For context, the Imageomics metadata checklist is mostly a generalized version of the Data Card checklist. They fit in the Imageomics Project Lifecycle as part of the iterative process of filling out a dataset card and updating on GitHub with checklists in the GitHub project repo issue. ↩

2. There is also a newer version on GitHub. We incorporated both, and a newer one was released on GitHub in Sept/Oct 2026. ↩