Skip to content

README.md

Latest commit

 

History

History
1383 lines (979 loc) · 24.6 KB

README.md

File metadata and controls

1383 lines (979 loc) · 24.6 KB
Raw

AI Meeting Summarizer & Action Tracker

An offline-first web application for automatically transcribing, summarizing, and analyzing meetings using local open-source deep-learning models.

The application accepts either meeting audio or an existing text transcript and automatically generates a structured meeting report containing the main discussion points, decisions, action items, responsible persons, and deadlines.

The system supports both English and Farsi (Persian), with automatic language detection and local processing.

No external AI API, API key, or cloud-based AI service is required.

Overview

AI Meeting Summarizer & Action Tracker is designed to convert unstructured meeting conversations into structured and actionable information.

The application can process meeting audio using Whisper-based speech recognition or directly process an existing transcript.

The processing pipeline includes:

  1. Audio transcription
  2. Language detection
  3. Speaker detection from transcript text
  4. Meeting summarization
  5. Entity extraction
  6. Action-item extraction
  7. Responsible-person identification
  8. Deadline detection
  9. Meeting storage
  10. PDF report generation

All AI models run locally. After the required models have been downloaded for the first time, the application can operate without an internet connection.

Features

Audio Transcription

The application supports meeting recordings in the following formats:

  • MP3
  • WAV
  • M4A

Audio is transcribed locally using faster-whisper.

The transcription system supports multiple languages, including English and Farsi.

Transcript Processing

Users can also bypass audio transcription and provide an existing meeting transcript directly.

This is useful when:

  • The meeting has already been transcribed.
  • A transcript was generated by another system.
  • The user wants to analyze an existing text document.
  • Audio processing is not required.

Automatic Language Detection

The system automatically determines whether the transcript is primarily English or Farsi.

Language detection can also be controlled manually from the user interface when required.

The detected language determines which summarization, NER, and extraction models are used.

Meeting Summarization

The application generates a concise summary of the main topics and discussions.

English

English meetings are summarized using:

facebook/bart-large-cnn

Farsi

Farsi meetings are summarized using:

csebuetnlp/mT5_multilingual_XLSum

For long transcripts, the application uses a map-reduce summarization strategy.

The transcript is divided into manageable sections, each section is summarized separately, and the resulting summaries are combined into a final meeting summary.

This prevents long transcripts from exceeding the context limitations of the underlying models.

Action Item Extraction

The application automatically identifies tasks and action items discussed during meetings.

For each detected action item, the system attempts to extract:

  • Task description
  • Responsible person
  • Deadline
  • Relevant information from the transcript

For example:

Sara will prepare the project report by next Monday.

The system attempts to produce:

{
    "task": "Prepare the project report",
    "owner": "Sara",
    "deadline": "Next Monday"
}

Action extraction combines named-entity recognition with rule-based linguistic patterns.

Responsible Person Detection

The system attempts to associate an action item with the person responsible for completing it.

For example:

Ali should update the API documentation by Thursday.

The extracted action item can be represented as:

Task: Update the API documentation
Owner: Ali
Deadline: Thursday

Because this functionality relies on NLP models and rules, the extracted owner should be reviewed by the user when the transcript contains ambiguous references.

Deadline Detection

The application detects common date and time expressions from meeting transcripts.

Examples include:

Tomorrow
Next Monday
By Friday
Next week
August 25
2026/08/25

For Farsi meetings, the system also supports Persian date expressions and Jalali/Shamsi date conversion.

The following libraries are used:

  • dateparser
  • jdatetime

Speaker Detection

The application provides transcript-based speaker detection.

It identifies common speaker-label patterns such as:

Ali: We need to complete the project this week.

Sara: I will prepare the documentation.

Ali: Let's review it tomorrow.

The system extracts speaker names and associates them with the corresponding dialogue.

This implementation is based on text patterns and speaker labels.

It is important to note that this is not full acoustic speaker diarization. The system does not currently identify speakers from their voices using audio embeddings.

Decision Extraction

Important decisions discussed during the meeting can be identified and included in the generated meeting report.

Examples include:

The team decided to use PostgreSQL.

The deployment will be moved to Friday.

The project deadline was extended by one week.

Decision extraction is based on language patterns and contextual analysis.

Meeting Report

The application generates a structured meeting report containing:

  • Meeting title
  • Meeting date
  • Participants
  • Meeting duration
  • Detected language
  • Summary
  • Key decisions
  • Action items
  • Responsible persons
  • Deadlines
  • Speaker dialogue
  • Original transcript

The report can be exported as a PDF document.

PDF Export

PDF reports are generated using ReportLab.

The application is designed to support both English and Farsi content.

Tahoma or another appropriate Unicode-compatible font can be configured for Persian and English rendering.

The generated report provides a structured representation of the processed meeting and can be used for documentation or internal records.

Technology Stack

Backend

  • Python
  • FastAPI
  • Uvicorn
  • SQLAlchemy
  • SQLite

Frontend

  • HTML5
  • CSS3
  • JavaScript
  • Google Fonts

The main interface is implemented as a single-page HTML application and does not require a frontend build system.

Speech Recognition

faster-whisper

Natural Language Processing

  • Hugging Face Transformers
  • spaCy
  • dateparser
  • jdatetime

Summarization Models

English

facebook/bart-large-cnn

Farsi

csebuetnlp/mT5_multilingual_XLSum

Named Entity Recognition

English

en_core_web_sm

Farsi

HooshvareLab/bert-fa-zwnj-base-ner

Database

SQLite
SQLAlchemy

PDF Generation

ReportLab

System Architecture

                         Web Browser
                              |
                              v
                    HTML / CSS / JavaScript
                              |
                              v
                         FastAPI
                              |
             +----------------+----------------+
             |                |                |
             v                v                v
       Audio Input       Transcript Input   Meeting Data
             |                |                |
             v                +-------+--------+
      faster-whisper                  |
             |                        |
             +-----------+------------+
                         |
                         v
                  Language Detection
                         |
              +----------+----------+
              |                     |
              v                     v
         English Pipeline      Farsi Pipeline
              |                     |
              v                     v
       BART Summarizer       mT5 XLSum
              |                     |
              +----------+----------+
                         |
                         v
                 Entity Extraction
                         |
                         v
               Action Item Extraction
                         |
                         v
                Deadline Detection
                         |
                         v
                 Speaker Detection
                         |
                         v
                  Meeting Database
                         |
              +----------+----------+
              |                     |
              v                     v
        Web Application          PDF Report

Processing Pipeline

The complete processing workflow is:

Audio / Transcript
        |
        v
Transcription
        |
        v
Language Detection
        |
        v
Transcript Preprocessing
        |
        v
Speaker Detection
        |
        v
Named Entity Recognition
        |
        v
Action Item Extraction
        |
        v
Deadline Detection
        |
        v
Meeting Summarization
        |
        v
Result Generation
        |
        +-------------------+
        |                   |
        v                   v
   Web Interface       PDF Report
        |
        v
      SQLite

Installation

Requirements

Before installing the application, make sure the following software is available:

  • Python 3.10 or newer
  • pip
  • Git

For GPU acceleration, a compatible NVIDIA CUDA environment is recommended.

CPU execution is supported but can be significantly slower for large meetings.

1. Clone the Repository

git clone https://github.com/YOUR_USERNAME/YOUR_REPOSITORY.git
cd YOUR_REPOSITORY

Replace YOUR_USERNAME/YOUR_REPOSITORY with the actual GitHub repository path.

2. Create a Virtual Environment

Windows

python -m venv venv

Activate the environment:

venv\Scripts\activate

macOS / Linux

python3 -m venv venv

Activate the environment:

source venv/bin/activate

3. Upgrade pip

python -m pip install --upgrade pip

4. Install Dependencies

pip install -r requirements.txt

5. Install the English spaCy Model

pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl

6. Optional Sample Audio

A sample English audio file can be generated for testing:

python make_sample_audio.py

7. Run the Application

python run.py

The application will start on:

http://localhost:8000

Open this address in a web browser.

Usage

1. Open the Web Application

Open:

http://localhost:8000

The frontend automatically connects to the FastAPI backend.

2. Select the Input Type

The application provides two input modes.

Upload Audio

Upload an audio file in one of the supported formats:

.mp3
.wav
.m4a

The audio will automatically be transcribed using faster-whisper.

Paste Transcript

Alternatively, paste an existing transcript into the application.

3. Enter Meeting Information

The following fields can optionally be provided:

  • Meeting title
  • Participants
  • Language

If the language is not manually selected, the application attempts to detect it automatically.

4. Run Processing

Click the processing button to start the pipeline.

For audio input, the workflow is:

Audio
  |
  v
Whisper Transcription
  |
  v
Language Detection
  |
  v
Speaker Detection
  |
  v
Summarization
  |
  v
NER
  |
  v
Action Extraction
  |
  v
Deadline Detection

The resulting data is then displayed in the web interface.

5. Review the Results

The application provides three main result sections.

Summary

Displays the generated meeting summary and key discussions.

Actions & Deadlines

Displays structured action items including:

  • Task
  • Responsible person
  • Deadline

Dialogue

Displays the detected speaker dialogue and transcript.

6. Save the Meeting

Processed meetings can be saved to the local SQLite database.

Saved meetings can later be retrieved through the meetings API.

7. Download the PDF Report

A structured PDF report can be generated from a saved meeting.

The report includes the main meeting information, summary, decisions, action items, deadlines, speakers, and transcript.

API Documentation

The application provides a REST API through FastAPI.

Health Check

GET /api/health

Response:

{
    "ok": true
}

Transcribe Audio

POST /api/transcribe

Accepts an audio file using multipart form data.

Response:

{
    "transcript": "...",
    "speakers": [],
    "segments": []
}

Summarize Transcript

POST /api/summarize

Request:

{
    "transcript": "Meeting transcript..."
}

Response:

{
    "summary": "Meeting summary..."
}

Extract Action Items

POST /api/extract-actions

Request:

{
    "transcript": "Meeting transcript..."
}

Response:

{
    "items": [
        {
            "task": "Prepare the project report",
            "owner": "Sara",
            "deadline": "2026-08-24"
        }
    ]
}

Create Meeting

POST /api/meetings

Request:

{
    "title": "Weekly Project Meeting",
    "transcript": "Meeting transcript...",
    "summary": "Meeting summary...",
    "items": [],
    "speakers": [],
    "duration": 3600
}

Response:

{
    "id": 1
}

List Meetings

GET /api/meetings

Response:

{
    "meetings": [
        {
            "id": 1,
            "title": "Weekly Project Meeting",
            "date": "2026-08-18",
            "duration": 3600,
            "status": "completed"
        }
    ]
}

Get Meeting

GET /api/meetings/:id

Returns the complete meeting object.

Download Meeting Report

GET /api/meetings/:id/report

Returns the generated PDF meeting report.

Delete Meeting

DELETE /api/meetings/:id

Response:

{
    "ok": true
}

Language Support

The application currently supports English and Farsi.

English Pipeline

English Transcript
        |
        v
English NER
        |
        v
BART Summarization
        |
        v
English Action Rules
        |
        v
Date Parsing

English NER is provided by:

en_core_web_sm

The summarization model is:

facebook/bart-large-cnn

Farsi Pipeline

Farsi Transcript
        |
        v
Persian NER
        |
        v
mT5 XLSum
        |
        v
Farsi Action Rules
        |
        v
Jalali Date Processing

The Persian NER model is:

HooshvareLab/bert-fa-zwnj-base-ner

The summarization model is:

csebuetnlp/mT5_multilingual_XLSum

Jalali/Shamsi date processing uses:

jdatetime

Model Management

The application loads AI models locally.

The first execution may require several gigabytes of model downloads, depending on which language and functionality are used.

The main models include:

Whisper
BART-large-CNN
mT5 XLSum
English spaCy NER
Persian NER

Farsi-specific models are loaded lazily.

This means the Farsi models are downloaded only when a Farsi transcript is processed.

After the models have been downloaded and cached, they can be reused without downloading them again.

Offline Operation

The application is designed to operate locally.

After the initial installation and model downloads:

  • Audio processing runs locally.
  • Transcription runs locally.
  • Summarization runs locally.
  • NER runs locally.
  • Action extraction runs locally.
  • SQLite storage runs locally.
  • PDF generation runs locally.

No OpenAI API, Google API, Azure API, or other external AI API is required.

An internet connection is only required initially for installing Python dependencies and downloading model weights.

Performance

Performance depends on the hardware and meeting length.

CPU

CPU inference is supported.

It is suitable for:

  • Short meetings
  • Testing
  • Development
  • Systems without dedicated GPUs

However, processing long meetings can take significantly longer.

GPU

A compatible NVIDIA GPU is recommended for:

  • Long meeting recordings
  • Faster Whisper transcription
  • Faster Transformer inference
  • Processing multiple meetings

Storage

The application uses SQLite through SQLAlchemy.

The local database can store:

  • Meeting title
  • Date
  • Duration
  • Participants
  • Transcript
  • Summary
  • Decisions
  • Action items
  • Owners
  • Deadlines
  • Speakers

Typical local storage structure:

data/
    meetings.db

Uploaded audio files are stored separately:

uploads/

Project Structure

.
├── index.html
├── run.py
├── make_sample_audio.py
├── requirements.txt
├── README.md
│
├── app/
│   ├── api/
│   │   ├── meetings.py
│   │   ├── reports.py
│   │   └── pipeline.py
│   │
│   ├── core/
│   │   ├── config.py
│   │   ├── models.py
│   │   └── language.py
│   │
│   ├── services/
│   │   ├── transcriber.py
│   │   ├── summarizer.py
│   │   ├── extractor.py
│   │   └── preprocessor.py
│   │
│   ├── db/
│   │   └── database.py
│   │
│   └── templates/
│       ├── meetings.html
│       └── report.html
│
├── static/
│   ├── css/
│   └── js/
│
├── tests/
│   ├── test_pipeline.py
│   └── ...
│
├── uploads/
│
└── data/

Testing

The project includes pipeline tests.

Run the test suite with:

python tests/test_pipeline.py

Before running the tests, make sure that the required dependencies and models have been installed.

Configuration

Application configuration can be centralized in the project's configuration module.

Depending on the implementation, configurable values may include:

  • Whisper model size
  • Device selection
  • Compute type
  • Database location
  • Upload directory
  • Model cache directory
  • Maximum audio size
  • Summarization parameters

For production deployments, configuration values should be provided through environment variables rather than hard-coded values.

Security and Privacy

This project is designed around local processing and privacy.

Meeting recordings and transcripts are not required to leave the local machine for AI processing.

However, users should still protect the application environment appropriately.

Recommended practices include:

  • Do not expose the FastAPI server directly to the public internet without authentication and security controls.
  • Do not commit meeting recordings to Git.
  • Do not commit the SQLite database to Git.
  • Do not commit environment files containing secrets.
  • Restrict access to uploaded meeting files.
  • Use HTTPS when deploying the application remotely.
  • Implement authentication and authorization before using the application in a multi-user production environment.

Recommended .gitignore

The following files and directories should generally not be committed:

venv/
__pycache__/
*.pyc

.env
.DS_Store

uploads/*
data/*
*.db
*.sqlite
*.sqlite3

.pytest_cache/

Model caches should also be excluded if they are stored inside the project directory.

Limitations

The current implementation has several limitations.

Speaker Detection

Speaker identification is based on transcript text and speaker labels.

It is not equivalent to acoustic speaker diarization.

If a transcript does not contain speaker information, the system cannot reliably determine who said each sentence based only on text.

Action Extraction

Action-item extraction combines NER and rule-based patterns.

Natural language can be ambiguous, so extracted tasks, owners, and deadlines should be reviewed by the user.

Deadline Interpretation

Relative expressions such as:

next Friday
tomorrow
by the end of the week
next month

can be ambiguous depending on the meeting date and context.

Summarization

Summarization quality depends on:

  • Transcript quality
  • Language
  • Meeting length
  • Model quality
  • Hardware
  • Content complexity

CPU Performance

Large Transformer models can be slow on CPU, especially when processing long meetings.

Roadmap

Future development may include:

  • Acoustic speaker diarization
  • Speaker identification using voice embeddings
  • Improved Persian NER
  • Improved Persian action-item extraction
  • More accurate deadline normalization
  • Calendar integration
  • Google Calendar integration
  • Microsoft Outlook integration
  • Email notifications
  • Action-item status tracking
  • Meeting search
  • Meeting filtering
  • Meeting analytics
  • User authentication
  • Multi-user support
  • Role-based access control
  • Docker support
  • GPU optimization
  • Real-time transcription
  • Real-time meeting summarization
  • Additional language support
  • Topic and agenda detection
  • Automatic decision extraction improvements
  • Meeting comparison and historical analysis

Future Production Architecture

For a production deployment, the following architecture can be considered:

                    Reverse Proxy
                         |
                         v
                    FastAPI API
                         |
          +--------------+--------------+
          |              |              |
          v              v              v
      Database       Task Queue     File Storage
          |              |              |
          |              v              |
          |        AI Processing        |
          |              |              |
          +--------------+--------------+
                         |
                         v
                  Meeting Reports

For larger deployments, asynchronous task processing should be introduced so that long audio and Transformer workloads do not block HTTP requests.

Contributing

Contributions are welcome.

To contribute:

  1. Fork the repository.
  2. Create a new feature branch.
  3. Implement and test your changes.
  4. Commit your changes.
  5. Push the branch.
  6. Open a Pull Request.

Example:

git checkout -b feature/improved-action-extraction

After making changes:

git add .
git commit -m "Improve action item extraction"
git push origin feature/improved-action-extraction

Then create a Pull Request on GitHub.

License

This project should include an explicit open-source license.

For example, if the project is intended to use the MIT License, add a LICENSE file containing the MIT License text and update this section accordingly.

MIT License

The licenses and usage requirements of all third-party models and libraries should also be reviewed before distributing or deploying the project.

Third-Party Models and Libraries

This project relies on several open-source libraries and machine-learning models, including:

  • faster-whisper
  • Hugging Face Transformers
  • Facebook BART
  • mT5 XLSum
  • spaCy
  • English spaCy models
  • HooshvareLab Persian NER
  • dateparser
  • jdatetime
  • FastAPI
  • SQLAlchemy
  • SQLite
  • ReportLab

Each dependency and model remains subject to its respective license and terms of use.

Project Goal

The primary goal of AI Meeting Summarizer & Action Tracker is to provide a privacy-oriented, locally hosted meeting intelligence system.

Instead of manually reviewing an entire meeting, users can process the conversation and obtain a structured representation:

Meeting Audio or Transcript
            |
            v
       Transcription
            |
            v
    Language Detection
            |
            v
    Speaker Detection
            |
            v
      Summarization
            |
            v
   Entity Extraction
            |
            v
  Action Item Extraction
            |
            v
   Owner Identification
            |
            v
   Deadline Detection
            |
            v
     Meeting Storage
            |
            v
      PDF Reporting

The resulting system transforms unstructured meeting conversations into structured information that can be reviewed, stored, and shared.

Summary

AI Meeting Summarizer & Action Tracker provides a complete local pipeline for meeting analysis:

Audio / Transcript
        ↓
Whisper Transcription
        ↓
Language Detection
        ↓
Speaker Detection
        ↓
NLP Processing
        ↓
Summarization
        ↓
Action Extraction
        ↓
Owner + Deadline Detection
        ↓
SQLite Storage
        ↓
PDF Report

The application supports English and Farsi, operates locally, and does not require external AI APIs or API keys.