Skip to content

README.md

Latest commit

 

History

History
135 lines (109 loc) · 5.76 KB

README.md

File metadata and controls

135 lines (109 loc) · 5.76 KB
Raw

المُشَكِّل العربي (NLP Arabic Diacritization)

Live Demo

An efficient, highly accurate Arabic text diacritizer built with a Transformer Encoder architecture. This project provides a complete pipeline from model training on the Tashkeela dataset to a serverless, highly optimized production deployment on AWS Lambda. This project utilizes a character-level Transformer model with a classification head to predict the correct diacritics for raw Arabic text. It features an elegant frontend and a robust, serverless backend optimized for fast inference.

Key Features

  • Transformer Architecture: Transformer Encoder trained on over 2 million samples from the Tashkeela dataset
  • Smart Mixed-Language Handling: Seamlessly processes texts containing mixed Arabic and English/Latin characters without hallucinating diacritics on foreign words
  • Serverless Deployment: Backend powered by FastAPI and Mangum, deployed as a Dockerized AWS Lambda function
  • ONNX Optimization: Inference is driven by ONNX Runtime, specifically tuned for Lambda's constrained vCPU environments (handling thread limitations efficiently).
  • UI: A lightweight, vanilla HTML/CSS/JS frontend deployed through Cloudflare pages

Tech Stack

  • Deep Learning & Optimization: PyTorch, Optuna, ONNX Runtime
  • Training Infrastructure: Google Cloud Platform (GCP) — NVIDIA L4 GPU (g2-standard-8)
  • Backend Development: Python, FastAPI, Mangum
  • MLOps & Deployment: MLflow, AWS Lambda, Amazon ECR

Hyperparameter Tuning with Optuna

To achieve our highly optimized Diacritic Error Rate (DER) of 5.43%, I utilized Optuna for automated hyperparameter optimization (HPO). Instead of manual grid searches, Optuna’s Tree-structured Parzen Estimator (TPE) algorithm efficiently explored the hyperparameter landscape while the Median Pruner halted unpromising trials early to save compute resources

Our Search Space: The tuning pipeline optimized the core architectural and training parameters of our Transformer Encoder. HPO Search Space

  1. Objective Function: Designed to minimize the primary metric: DER +CE IUL (Diacritic Error Rate including Case Endings and Undiacritized Letters)
  2. Early Stopping: Trials that performed worse than the median of previous trials at the same epoch were aggressively pruned
  3. Final Selection: The best trial's weights were exported and converted to the highly efficient ONNX format for the serverless backend

MLOps & Experiment Tracking with MLflow

I integrated MLflow as the central system for our training and model management pipeline. Every training run and Optuna trial was automatically logged to MLflow server. This provided a centralized dashboard to compare experiments

  • Parameters Logged: Model architecture config, batch size, learning rate schedule.
  • Metrics Tracked per Epoch: Training Loss, Validation Loss, ..etc
  • Artifact Storage: The pipeline automatically logged the PyTorch .pt checkpoints

Evaluation Results

The model was evaluated using standard metrics: Diacritic Error Rate (DER) and Word Error Rate (WER).

This evaluation metrics combination is based on state-of-the-art paper: BERT-Based Arabic Diacritization: A state-of-the-art approach for improving text accuracy and pronunciation

💡 Legend:

  • +CE: Including Case Ending (syntax + morphology)
  • -CE: Excluding Case Ending (morphology only)
  • IUL: Including Undiacritized Letters
  • EUL: Excluding Undiacritized Letters
  • Primary metrics (paper emphasis): DER +CE IUL, WER +CE IUL

DER (Diacritic Error Rate) - Lower is Better

Metric Score Percentage
DER +CE IUL (Most Challenging) 0.0543 5.43%
DER +CE EUL 0.0792 7.92%
DER -CE IUL 0.0530 5.30%
DER -CE EUL 0.0786 7.86%

WER (Word Error Rate) - Lower is Better

Metric Score Percentage
WER +CE IUL (Most Challenging) 0.1389 13.89%
WER +CE EUL 0.1388 13.88%
WER -CE IUL 0.1083 10.83%
WER -CE EUL 0.1087 10.87%

Local Setup & Installation

Clone the repository

git clone https://github.com/AbdallahSalah003/NLP-Arabic-Diacritization.git
cd NLP-Arabic-Diacritization

Make sure to download the dataset Tashkeela and create the following directory structure

├── data
│ ├── CA
│ │ ├── CA_test.csv
│ │ ├── CA_train.csv
│ │ └── CA_val.csv
│ ├── MSA
│ │ ├── MSA_test.csv
│ │ ├── MSA_train.csv
│ │ └── MSA_val.csv
│ ├── changelog.txt

  1. You can run the following help command
make help

Creating venv and install dependencies

python3 -m venv venv
make setup
  1. To run preprocessing
make preprocess
  1. To start hparams optimization
make tune
  1. To start training the final model using optimized hparams
make train
  1. To export the ONNX model format
make export_onnx
  1. To evaluate the onnx model
make evaluate_onnx
  1. To run tests
make test

Deployment Guide

The following repo contains a detailed guide for optimized onnx runtime deployment through AWS Lambda and AWS ECR Arabic Diacritizer Lambda