Skip to content

Commit 94f4db4

Browse files
Update 2026.md
1 parent 18d6d15 commit 94f4db4

1 file changed

Lines changed: 7 additions & 3 deletions

File tree

2026.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,22 +49,26 @@ Participants will be ranked along two primary (character-level) metrics:
4949
1. **Span Identification**: Intersection-over-Union (IoU) of characters marked as hallucinated in the gold reference vs. predicted
5050
2. **Confidence Calibration**: Correlation between the probability assigned by a participant's system that a character is hallucinated and the empirical probability observed in our multi-annotator gold data
5151

52-
Rankings and submissions will be handled **separately per language**. Participants can also download the scoring program [here](./scorer.py) ${\color{red} BROKEN LINK}$ for reference and system development.
52+
Rankings and submissions will be handled **separately per language**.
53+
<!-- Participants can also download the scoring program [here](./scorer.py) ${\color{red} BROKEN LINK}$ for reference and system development. -->
5354

5455
#### Dataset Overview
5556
We provide a curated dataset of 20,000 samples with multiple annotations with a fine-grained, span-level labeling scheme.
5657

5758
| Dataset Split | Size | Composition | Access |
5859
|--------------|------|-------------|--------|
59-
| **Training set** | ~15,200 samples | Outputs from 5 diverse LVLMs, ~3,800 samples per language | [Download](https://a3s.fi/shroom-visions/train.zip) ${\color{red} BROKEN LINK}$ |
60+
| **Training set** | ~15,200 samples | Outputs from 5 diverse LVLMs, ~3,800 samples per language | |
6061
| **Test set** | 4,800 samples | 1,200 samples per language | Closed test set |
6162

63+
Download the annotated training set and the unlabelled test set: [Download data](https://a3s.fi/mickusti-2007780-pub/shroom-visions-data.zip)
64+
Download input images: [Download images](https://a3s.fi/mickusti-2007780-pub/shroom-visions-images.tar.gz)
6265

6366

67+
<!--
68+
6469
Supplementary materials, including annotation guidelines, raw annotations with comments, and image metadata, can be downloaded from [this link](https://a3s.fi/shroom-visions/extra-info.tar.gz) ${\color{red} BROKEN LINK}$.
6570
$
6671
67-
<!--
6872
We are releasing a participant kit containing:
6973
- Scoring program and format checker
7074
- Two baselines: a random baseline and a multimodal transformer-based system

0 commit comments

Comments
 (0)