Selective Attention Merging for low resource tasks: A case study of Child ASR

Overview

This github repository contains models, scripts and data splits from our paper accepted at ICASSP 2025

Folder Structure

Source code for our training and evaluating models can be found under /src

Code for our proposed SA Merge can be found under /merge

/egs contains utterance ids for train/test splits for the MyST and CMU Kids databases

Getting Started

1. Install Dependencies: transformers==4.32.1 torch evaluate datasets
2. On older versions of transformers it might be necessary to make minor edits to trainer.py to allow hotloading of Iterable datasets (if streaming is set to True). Follow the instructions in /egs/MyST/README.txt to make the necessary edits.
3. For training SSL based models, it is necessary to clone the fairseq github repo
4. To train/evaluate a model on a particular dataset, edit the corresponding yaml file stored in the /egs/dataset/config directory, specify the train/test list to use, and run the corresponding bash script
5. To merge models using SA Merge, follow instruction in the script under /merge. Other model merging methods are evaluated using the mergekit repo

Trained Models

Model	MyST test WER	Huggingface Link
Whisper tiny - SA Merge	11.52	model
Whisper base - SA Merge	9.87	model
Whisper small - SA Merge	8.85	model
Whisper small - SpecAug + SA Merge	8.69	model
Whisper Medium - SA Merge	8.63	model
Whisper Large v3 - SA Merge	8.74	model

Citation

If you use this code in your research, please cite it as follows:

@INPROCEEDINGS{shankar2025selective,
  author={Shankar, Natarajan Balaji and Wang, Zilai and Eren, Eray and Alwan, Abeer},
  booktitle={ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
  title={Selective Attention Merging for low resource tasks: A case study of Child ASR}, 
  year={2025},
  volume={},
  number={},
  pages={1-5},
  doi={10.1109/ICASSP49660.2025.10887889}
}