Extracts Conflict of interests / Author contributions statements

[lfoppiano]

Adds the identification of "conflict of interests / declaration of interests" and author contribution / credit statements.

The evaluation requires an updated version of the dataset which can be found here: https://huggingface.co/datasets/sciencialab/grobid-evaluation where the JATS files have been modified to identify the new statements.
We did our best, given the mess of the JATS Jungle.

We also use a dataset for which the author contribution and the data availability statements were already extracted with a different tool and selected documents that had missing or truncated statements, here some results:

	Iteration 0 (grobid 0.8.1)	Iteration 1 (grobid dev)	Iteration 2 (+50 docs)	Iteration 3 (+113 docs)	Iteration 4 (+97 docs)
Docs missing availability statements	2420	1240	805	317	236
Docs missing contribution statements	3737	1205	767	523	483

[coveralls]

coverage: 40.323% (-0.07%) from 40.394%
when pulling 80ec370 on feature/coi-ac
into 01fe109 on master.

[lfoppiano]

@kermitt2 it seems that the default "recipe" for training the header-BidLSTM_ChainCRF_FEATURES does not yield the same results as the current header model in the master repository. Do you remember by any chance which parameters did you use? 🙏

FYI I've solved the issue and obtained similar, or slighly better, results.

[Copilot]

Pull request overview

This PR adds support for extracting conflict of interest statements and author contribution statements from scientific articles. The implementation includes updated model configurations, new tagging labels, and modifications to parsing and processing logic to handle these new statement types.

Changes:

• Added new tagging labels for conflict of interest (<conflict>) and author contribution (<contribution>) statements
• Updated model configurations to accommodate expanded character vocabularies and new tag types
• Modified parsing engines to extract and process the new statement types from both header and body sections

Reviewed changes

Copilot reviewed 27 out of 2468 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
grobid-home/models/*/preprocessor.json	Expanded character vocabularies to support additional Unicode characters encountered in documents
grobid-home/models/*/config.json	Updated character vocabulary sizes and batch sizes to align with preprocessor changes
TaggingLabels.java	Added label constants for conflict of interest and author contribution statements
SegmentationLabels.java	Registered new labels for segmentation model
Segmentation.java	Added handling for conflict and contribution statement extraction in training mode
HeaderParser.java	Implemented extraction logic for conflict and contribution statements from headers
FullTextParser.java	Added processing for conflict and contribution statements from non-header sections
BiblioItem.java	Added fields and accessors for storing conflict and contribution statements
BasicStructureBuilder.java	Removed unused import
doc/training/*.md	Updated documentation to describe new statement types and annotation guidelines
doc/benchmarks/*.md	Updated benchmark results reflecting model performance changes

Comments suppressed due to low confidence (1)

grobid-core/src/main/java/org/grobid/core/engines/HeaderParser.java:1496

• Extra blank line added at line 1443. While not functionally problematic, this inconsistent spacing should be removed to maintain code style consistency.

    }
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.