Hi! I'm a CS student getting into BCI, and I recently built a small open-source tool called EEGauge (https://github.com/YG-paaleee/eegauge) that reads EEGDash metadata. Your contributing guide says to open an issue before starting on a feature, so I wanted to float an idea and get your read before building anything.
EEGDash already handles finding, validating, and loading datasets really well. The part I keep wishing existed is the step after that: how a dataset actually gets evaluated, and whether a result should be trusted. Two concrete examples: flagging when a dataset has multiple sessions or runs per subject (so a naive epoch-level shuffle would leak), and recording the exact cohort that produced a benchmark so someone can reproduce it. EEGauge already spits out a small "evaluation provenance" file like that (here's one for ds002718: https://github.com/YG-paaleee/eegauge/blob/main/examples/ds002718.provenance.json), and it seemed like it might complement your dataset pages instead of overlapping with them.
Mostly I just wanted to ask before doing anything: is this something you'd find useful on the EEGDash side, or is it out of scope and better left to downstream tools like mine? If you're open to it, I'd rather hear how you'd want it shaped (which fields are stable, where it would even fit) than guess and send a PR you didn't ask for.
Either way, thanks for EEGDash. It's been genuinely nice to build on.
Hi! I'm a CS student getting into BCI, and I recently built a small open-source tool called EEGauge (https://github.com/YG-paaleee/eegauge) that reads EEGDash metadata. Your contributing guide says to open an issue before starting on a feature, so I wanted to float an idea and get your read before building anything.
EEGDash already handles finding, validating, and loading datasets really well. The part I keep wishing existed is the step after that: how a dataset actually gets evaluated, and whether a result should be trusted. Two concrete examples: flagging when a dataset has multiple sessions or runs per subject (so a naive epoch-level shuffle would leak), and recording the exact cohort that produced a benchmark so someone can reproduce it. EEGauge already spits out a small "evaluation provenance" file like that (here's one for ds002718: https://github.com/YG-paaleee/eegauge/blob/main/examples/ds002718.provenance.json), and it seemed like it might complement your dataset pages instead of overlapping with them.
Mostly I just wanted to ask before doing anything: is this something you'd find useful on the EEGDash side, or is it out of scope and better left to downstream tools like mine? If you're open to it, I'd rather hear how you'd want it shaped (which fields are stable, where it would even fit) than guess and send a PR you didn't ask for.
Either way, thanks for EEGDash. It's been genuinely nice to build on.