Skip to content

Latest commit

 

History

History
315 lines (197 loc) · 17.6 KB

Ellena_summary.md

File metadata and controls

315 lines (197 loc) · 17.6 KB

Ell-ena: AI-Powered Product Management System

Technical Overview

Ell-ena is a sophisticated AI-powered product management system that automates task management, ticket creation, and meeting transcriptions while maintaining full work context. This document provides a comprehensive technical explanation of how Ell-ena works, its architecture, and its key components.

Core Architecture

Ell-ena implements a modern architecture that combines Flutter for the frontend with Supabase for backend services, enhanced by AI processing pipelines:

┌─────────────────────────────────────────────────────────────────────────┐
│                           FRONTEND (Flutter)                            │
├───────────────┬─────────────────┬────────────────────┬─────────────────┤
│  Auth Module  │  Task Manager   │  Meeting Manager   │  Chat Interface │
└───────┬───────┴────────┬────────┴──────────┬─────────┴────────┬────────┘
        │                │                   │                  │
        ▼                ▼                   ▼                  ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Supabase Service Layer                           │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐   ┌─────────────┐  │
│  │ Auth Client │   │ Data Client │   │Storage Client│  │ RPC Client  │  │
│  └──────┬──────┘   └──────┬──────┘   └──────┬──────┘   └──────┬──────┘  │
│         │                 │                 │                 │         │
└─────────┼─────────────────┼─────────────────┼─────────────────┼─────────┘
          │                 │                 │                 │
          ▼                 ▼                 ▼                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          BACKEND (Supabase)                             │
├───────────────┬─────────────────┬────────────────────┬─────────────────┤
│ Authentication│  PostgreSQL DB  │  Object Storage    │  Edge Functions │
└───────┬───────┴────────┬────────┴──────────┬─────────┴────────┬────────┘
        │                │                   │                  │
        ▼                ▼                   ▼                  ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                       AI Processing Pipeline                            │
├───────────────┬─────────────────┬────────────────────┬─────────────────┤
│ NLU Processor │ Vector Database │ Embedding Generator│  AI Summarizer  │
└───────────────┴─────────────────┴────────────────────┴─────────────────┘

Meeting Transcription and AI Summary Pipeline

One of Ell-ena's most powerful features is its automated meeting transcription and summarization system. Here's a detailed breakdown of how it works:

1. Meeting Recording and Transcription

When a user schedules a meeting with a Google Meet URL, Ell-ena's system:

  1. Monitors for meeting activity: The system detects when a meeting starts and ends via the Google Meet integration.

  2. Captures audio: Using the Vexa API, Ell-ena captures the meeting audio in real-time.

  3. Generates transcription: The Vexa API processes the audio and generates a detailed text transcription of the entire meeting conversation, including speaker identification.

  4. Stores raw transcription: The raw transcription is stored in the Supabase database in the meetings table, linked to the specific meeting record.

2. AI Summary Generation

Once a meeting has a transcription, an automated pipeline processes it:

  1. Scheduled processing: A PostgreSQL cron job (process_unsummarized_meetings) runs every minute to check for meetings with transcriptions but no summaries.

  2. Edge Function invocation: For each meeting needing processing, the system calls the summarize-transcription Edge Function.

  3. AI processing: The Edge Function uses the Gemini API to:

    • Analyze the meeting transcription
    • Extract key topics, decisions, and action items
    • Generate a structured summary with follow-up tasks
    • Format the output as a structured JSON object

  4. Summary storage: The AI-generated summary is stored in the meeting_summary_json column of the meetings table.

3. Vector Embedding Generation

After a meeting has been summarized, another pipeline creates semantic embeddings:

  1. Scheduled embedding generation: A PostgreSQL cron job (process_meetings_missing_embeddings) runs every 5 minutes to check for meetings with summaries but no embeddings.

  2. Edge Function invocation: For each meeting needing embeddings, the system calls the generate-embeddings Edge Function.

  3. Vector creation: The Edge Function uses Gemini's embedding-001 model to convert the meeting summary text into a 768-dimensional vector.

  4. Vector storage: The embedding vector is stored in the summary_embedding column (using PostgreSQL's vector extension) of the meetings table.

4. Semantic Search and Retrieval

When a user asks a question about past meetings:

  1. Query detection: The AI service detects if the user's query is meeting-related using keyword analysis.

  2. Query embedding: If meeting-related, the system generates a vector embedding for the user's query using the get-embedding Edge Function.

  3. Vector similarity search: The system performs a cosine similarity search against all meeting summary embeddings to find the most relevant meetings.

  4. Context enrichment: The relevant meeting information is included as context in the prompt to the AI, enabling it to provide accurate, contextual responses.

  5. Response generation: The Gemini model generates a response that incorporates the relevant meeting information, creating the impression of "memory" about past discussions.

Task and Ticket Management

Ell-ena provides sophisticated task and ticket management capabilities:

1. Natural Language Task Creation

Users can create tasks using natural language commands through the chat interface:

  1. Intent recognition: The AI service recognizes when a user is trying to create a task or ticket.

  2. Function calling: The AI generates a structured function call to create_task or create_ticket with appropriate parameters.

  3. Parameter extraction: The system extracts relevant details like title, description, due date, priority, and assignee from the user's natural language input.

  4. Task creation: The Supabase service creates the task/ticket record in the database with the extracted parameters.

  5. Real-time updates: The UI updates in real-time to show the newly created task/ticket.

2. Contextual Enrichment

Ell-ena enriches tasks with contextual information:

  1. Team member awareness: The system understands team member names and can assign tasks appropriately.

  2. Date interpretation: Natural language date references like "tomorrow" or "next week" are automatically converted to proper date formats.

  3. Priority inference: For tickets, the system infers appropriate priority levels based on the context of the request.

  4. Category assignment: Tickets are automatically categorized based on their content.

3. Meeting-to-Task Conversion

Ell-ena can automatically generate tasks from meeting summaries:

  1. Action item extraction: The AI identifies action items and follow-up tasks from meeting transcriptions.

  2. Structured data creation: These are stored as structured data in the meeting summary JSON.

  3. One-click conversion: Users can convert these items to formal tasks or tickets with a single click from the meeting details screen.

  4. Automatic assignment: The system attempts to assign tasks to the appropriate team members based on the meeting context.

Authentication and Team Management

Ell-ena implements a sophisticated multi-account login system:

1. Team-Based Authentication

  1. Team creation: Users can create new teams with unique team codes.

  2. Team joining: Users can join existing teams using team codes.

  3. Role-based access: Users are assigned roles (admin or member) that determine their permissions.

  4. Multi-team support: Users can belong to multiple teams and switch between them.

2. Row-Level Security

  1. Data isolation: Each team's data is completely isolated using Supabase's Row-Level Security policies.

  2. Permission enforcement: Database policies ensure users can only access data from their own teams.

  3. Role-based permissions: Certain actions (like approving tasks or deleting meetings) are restricted to admin users.

User Interface Components

1. Dashboard

The dashboard provides an at-a-glance view of:

  1. Task summary: Shows pending, in-progress, and completed tasks.

  2. Upcoming meetings: Displays scheduled meetings with quick-join links.

  3. Recent activity: Shows recent updates across the team.

  4. Team member status: Indicates which team members are active.

2. Calendar Screen

The calendar screen offers:

  1. Meeting visualization: Shows all scheduled meetings in a calendar view.

  2. Task due dates: Displays task deadlines alongside meetings.

  3. Quick scheduling: Allows users to create new meetings by selecting time slots.

  4. Meeting details: Provides quick access to meeting information and join links.

3. Chat Interface

The AI-powered chat interface:

  1. Natural language interaction: Allows users to interact with the system using everyday language.

  2. Function detection: Automatically detects when users want to create tasks, tickets, or meetings.

  3. Context awareness: Maintains conversation context and understands references to previous messages.

  4. Meeting memory: Can recall and reference information from past meeting transcriptions.

4. Meeting Details Screen

The meeting details screen provides:

  1. Basic information: Shows meeting title, description, date, time, and duration.

  2. Join link: Offers a direct link to join virtual meetings.

  3. Transcription status: Indicates whether transcription is pending, in progress, or complete.

  4. AI summary: Displays the AI-generated summary of the meeting.

  5. Action items: Shows extracted action items with the ability to convert them to tickets.

  6. Follow-up tasks: Lists follow-up tasks with the ability to convert them to formal tasks.

Technical Implementation Details

1. Frontend (Flutter)

The Flutter frontend is organized into:

  1. Screens: UI components for different app sections (auth, tasks, meetings, etc.).

  2. Services: Business logic modules that interact with the backend.

    • supabase_service.dart: Handles all Supabase interactions
    • ai_service.dart: Manages AI processing and function calling
    • meeting_formatter.dart: Formats meeting data for display
    • navigation_service.dart: Manages app navigation

  3. Widgets: Reusable UI components shared across the app.

2. Backend (Supabase)

The Supabase backend consists of:

  1. Database Schema: Tables for users, teams, tasks, tickets, meetings, etc.

  2. Edge Functions: Serverless functions for AI processing:

    • fetch-transcript: Retrieves meeting transcriptions
    • generate-embeddings: Creates vector embeddings for meeting content
    • get-embedding: Retrieves embeddings for specific content
    • search-meetings: Performs semantic search across meeting transcriptions
    • start-bot: Initializes the AI assistant
    • summarize-transcription: Generates AI summaries of meeting transcriptions

  3. SQL Functions: Database functions for various operations:

    • process_unsummarized_meetings: Processes meetings with transcriptions but no summaries
    • process_meetings_missing_embeddings: Processes meetings with summaries but no embeddings
    • search_meeting_summaries: Performs semantic search to find relevant meeting summaries

  4. Cron Jobs: Scheduled tasks that run automatically:

    • Process unsummarized meetings (runs every minute)
    • Generate embeddings for meetings (runs every 5 minutes)

3. AI Integration (Gemini)

The Google Gemini API is used for:

  1. Natural language understanding: Processing user queries and commands
  2. Function calling: Detecting when to create tasks, tickets, or meetings
  3. Meeting summarization: Generating structured summaries from transcriptions
  4. Vector embeddings: Creating semantic embeddings for meeting content
  5. Contextual responses: Generating responses that incorporate meeting context

Data Flow Example: Meeting Lifecycle

To illustrate how all components work together, here's a complete lifecycle of a meeting in Ell-ena:

  1. Meeting Creation:

    • User creates a meeting via chat or calendar interface
    • System stores meeting details in the database
    • Google Meet URL is generated and stored

  2. Meeting Occurrence:

    • User joins the meeting via the stored URL
    • Vexa API captures the audio and generates a transcription
    • Transcription is stored in the database

  3. Automated Processing:

    • Cron job detects the meeting has a transcription but no summary
    • summarize-transcription Edge Function is called
    • Gemini API analyzes the transcription and generates a structured summary
    • Summary is stored in the database

  4. Embedding Generation:

    • Cron job detects the meeting has a summary but no embedding
    • generate-embeddings Edge Function is called
    • Gemini embedding-001 model creates a vector embedding
    • Embedding is stored in the database

  5. User Query:

    • User asks "What did we decide about the marketing budget last week?"
    • System detects this is a meeting-related query
    • get-embedding Edge Function creates an embedding for the query
    • Vector similarity search finds the most relevant meeting summaries
    • Relevant meeting information is included in the AI prompt
    • Gemini generates a response that incorporates the meeting context

  6. Task Creation:

    • User clicks on an action item from the meeting summary
    • System creates a new ticket with details from the action item
    • Ticket is assigned to the appropriate team member
    • Team member receives notification about the new ticket

This end-to-end flow demonstrates how Ell-ena combines real-time transcription, AI processing, vector search, and task management to create a seamless, context-aware productivity system.

Conclusion

Ell-ena represents a sophisticated integration of modern technologies to create an AI-powered productivity assistant that truly understands context and helps teams work more efficiently. By combining Flutter's cross-platform UI capabilities, Supabase's powerful backend services, and Google Gemini's advanced AI capabilities, Ell-ena delivers a seamless experience that feels like working with a smart teammate rather than just another tool.

The system's ability to automatically transcribe meetings, generate summaries, extract action items, and later recall this information when needed represents a significant advancement in AI-assisted productivity tools. The context-aware task and ticket creation further enhances team efficiency by reducing manual data entry and ensuring important details aren't lost.