HuBERT Fine-tuning for CTC ASR

This project demonstrates how to fine-tune a HuBERT model for Connectionist Temporal Classification (CTC) Automatic Speech Recognition (ASR).

Project Description

This project focuses on training a self-supervised learning (SSL) model and evaluating its performance on a downstream task in Cantonese. It fine-tunes a pre-trained HuBERT model on a speech dataset for automatic speech recognition (ASR) using the Connectionist Temporal Classification (CTC) loss function. The goal is to achieve high accuracy in Cantonese speech recognition.

Setup

1. Install dependencies:

   pip install -r requirements.txt

2. Prepare the dataset:

   • Download and extract the desired speech dataset.
   • Modify the train.py script to point to the correct dataset location and configuration.

Usage

To train the model, run the train.py script:

python train.py