Feature/Web Svelte GUI Enhancements: Pattern Descriptions, Tags, Favorites, Search Bar, Language Integration, PDF file conversion, etc

[jmd1010]

Enhanced Pattern Selection, Pattern Descriptions, New Pattern TAG System, Language Support and other WEB UI Improvements

This Cumulative PR adds several Web UI and functionality improvements to make pattern management and selection more intuitive and friction free:

• New Pattern Descriptions
• Ability to save favorites
• New Pattern TAG system
• Multilingual capabilities
• Import PDF documents
• Help reference section
• More robust Youtube processing and a variety of ui improvements

Thanks to Jonatan, Daniel, John and others who built this web app. It's a super fondation to continue building upon.

🎥 Demo Video

https://youtu.be/bhwtWXoMASA

🌟 Key Features

1. Web UI and Pattern Selection Improvements

• Enhanced pattern selection interface for better user experience
• New pattern descriptions section accessible via modal
• New pattern favorite list and pattern search functionality
• New Tag system for better pattern organization and filtering
• Seamless PDF file conversion to markdown
• Web UI refinements for clearer interaction
• Help section & Pattern Selection via modal

2. Multilingual Support System

• Language switching via UI dropdown
• Persistent language state management
• Pattern processing now use the selected language
  (Note: translation of pattern section "titles" not 100% for Youtube, anybody?)

3. YouTube Integration Enhancement

• Language handling for YouTube transcript & Pattern output processing
• Youtube required qualifiers or web site text conversion for llms (Jina) handled seamlessly behind the scene

4. Enhanced Tag Management Integration

The tag filtering system has been integrated into the Pattern Selection interface through several UI enhancements:

1. Dual-Position Tag Panel

   • Sliding panel positioned to the right of pattern modal
   • Dynamic toggle button that adapts position and text based on panel state
   • Smooth transitions for opening/closing animations

2. Tag Selection Visibility

   • New dedicated tag display section in pattern modal
   • Visual separation through subtle background styling
   • Immediate feedback showing selected tags with comma separation
   • Inline reset capability for quick tag clearing

3. Improved User Experience

   • Clear visual hierarchy between pattern list and tag filtering
   • Multiple ways to manage tags (panel or quick reset)
   • Consistent styling with existing design language
   • Space-efficient tag brick layout in 3-column grid

4. Technical Implementation

   • Reactive tag state management
   • Efficient tag filtering logic
   • Proper event dispatching between components
   • Maintained accessibility standards
   • Responsive design considerations

HOW TO USE
No change from existing process: from /web run fabric --serve or ./fabric --serve and run npm run dev from two different terminal windows.

These enhancements create a more intuitive and efficient pattern discovery experience, allowing users to quickly filter and find relevant patterns while maintaining a clean, modern interface.

5. Obsidian and Note integration

• Fully operational
• To work out of the box, please maintain this folder structure: (but you can change): web/myfiles/Fabric_obsidian and web/myfiles/inbox for notes.

🛠 Technical Implementation

Language Support Architecture

// Language state management
export const languageStore = writable<string>('');

// Chat input language detection
if (qualifier === 'fr') {
  languageStore.set('fr');
  userInput = userInput.replace(/--fr\s*/, '');
}

// Service layer integration
const language = get(languageStore) || 'en';
const languageInstruction = language !== 'en' 
  ? `. Please use the language '${language}' for the output.` 
  : '';

YouTube Processing Enhancement

// Process stream with language instruction per chunk
await chatService.processStream(
  stream,
  (content: string, response?: StreamResponse) => {
    if (currentLanguage !== 'en') {
      content = `${content}. Please use the language '${currentLanguage}' for the output.`;
    }
    // Update messages...
  }
);

Pattern Descriptions and Tags Management

This document explains the complete workflow for managing pattern descriptions and tags, including how to process new patterns and maintain metadata.

System Overview

The pattern system follows this hierarchy:

1. ~/.config/fabric/patterns/ directory: The source of truth for available patterns
2. pattern_extracts.json: Contains first 500 words of each pattern for reference
3. pattern_descriptions.json: Stores pattern metadata (descriptions and tags)
4. web/static/data/pattern_descriptions.json: Web-accessible copy for the interface

Pattern Processing Workflow

1. Adding New Patterns

• Add patterns to ~/.config/fabric/patterns/
• Run extract_patterns.py to process new additions:

  python extract_patterns.py
  

The Python Script automatically:

• Creates pattern extracts for reference
• Adds placeholder entries in descriptions file
• Syncs to web interface

2. Pattern Extract Creation

The script extracts first 500 words from each pattern's system.md file to:

• Provide context for writing descriptions
• Maintain reference material
• Aid in pattern categorization

3. Description and Tag Management

Pattern descriptions and tags are managed in pattern_descriptions.json:

{
"patterns": [
{
"patternName": "pattern_name",
"description": "[Description pending]",
"tags": []
}
]
}

Completing Pattern Metadata

Writing Descriptions

1. Check pattern_descriptions.json for "[Description pending]" entries

2. Reference pattern_extracts.json for context

3. How to update Pattern short descriptions (one sentence).

You can update your descriptions in pattern_descriptions.json manually or using LLM assistance (prefered approach).

Tell AI to look for "Description pending" entries in this file and write a short description based on the extract info in the pattern_extracts.json file. You can also ask your LLM to add tags for those newly added patterns, using other patterns tag assignments as example.

Managing Tags

1. Add appropriate tags to new patterns
2. Update existing tags as needed
3. Tags are stored as arrays: ["TAG1", "TAG2"]
4. Edit pattern_descriptions.json directly to modify tags
5. Make tags your own. You can delete, replace, amend existing tags.

File Synchronization

The script maintains synchronization between:

• Local pattern_descriptions.json
• Web interface copy in static/data/
• No manual file copying needed

Pattern Updates (adding descriptions and tags)

1. Run extract_patterns.py when:
   • Adding new patterns
   • Updating existing patterns
   • Modifying pattern structure

It will add the pattern extracts for the new patterns in the pattern_extracts.json and create placeholders in the pattern_descriptions.json file to complete later (ie add your pattern descriptions and tags). You can complete this step manually or with AI.

2. Tags Management:
   • See current Tags definition in the Help guide
   • You can modify the Tag list and make it your own in the pattern_descriptions.json file.

Troubleshooting

If patterns are not showing in the web interface:

1. Verify pattern_descriptions.json format
2. Check web/static/data/pattern_descriptions.json copy exists
3. Ensure proper file permissions
4. Run extract_patterns.py to resync

File Structure

fabric/
├── patterns/ # Pattern source files
├── PATTERN_DESCRIPTIONS/
│ ├── extract_patterns.py # Pattern processing script
│ ├── pattern_extracts.json # Pattern content references
│ └── pattern_descriptions.json # Pattern metadata
└── web/
└── static/
└── data/
└── pattern_descriptions.json # Web interface copy

PDF TO MARKDOWN CONVERSION IMPLEMENTATION

This document explains the new PDF to Markdown conversion implementation, detailing its functionality, installation process, and the file changes involved. See and clone from https://github.com/jzillmann/pdf-to-markdown/tree/modularize. The PDF conversion module has been integrated in the svelte web browser interface. Once installed, it will automatically detect pdf files in the chat interface and convert them to markdown automatically for llm processing.

Integration with Svelte

The integration approach focused on using the library's high-level API while maintaining SSR compatibility:

• Create PdfConversionService for PDF processing
• Handle file uploads in ChatInput component
• Convert PDF content to markdown text
• Integrate with existing chat processing flow

How it Works

The PDF to Markdown conversion is implemented as a separate module located in the pdf-to-markdown directory. It leverages the pdf-parse library (likely via PdfParser.ts) to parse PDF documents and extract text content. The core logic resides in PdfPipeline.ts, which orchestrates the PDF parsing and conversion process. Pdf-to-Markdown is a folk from pdf.js - Mozilla's PDF parsing & rendering platform which is used as a raw parser

Here's a simplified breakdown of the process:

1. PDF Parsing: The PdfParser.ts uses pdf-parse to read the PDF file and extract text content from each page.
2. Content Extraction: The extracted text content is processed to identify text elements, formatting, and structure.
3. Markdown Conversion: The PdfPipeline.ts then converts the extracted and processed text content into Markdown format. This involves mapping PDF elements to Markdown syntax, attempting to preserve formatting like headings, lists, and basic text styles.
4. Frontend Integration: The PdfConversionService.ts in the web/src/lib/services directory acts as a frontend service that utilizes the pdf-to-markdown module. It provides a convertToMarkdown function that takes a File object (PDF file) as input, calls the pdf-to-markdown module to perform the conversion, and returns the Markdown output as a string.
5. Chat Input Integration: The ChatInput.svelte component uses the PdfConversionService to convert uploaded PDF files to Markdown before sending the content to the chat service for pattern processing.

Installation

To use the PDF to Markdown conversion functionality, you need to ensure that the necessary dependencies are installed.

1. Navigate to the pdf-to-markdown directory:

   cd pdf-to-markdown

2. Install dependencies:

   npm install

   This will install the required dependencies for the pdf-to-markdown module, including pdf-parse and any other necessary libraries listed in pdf-to-markdown/package.json.

3. Build the pdf-to-markdown module:

   npm run build

   This command will build the TypeScript code in the pdf-to-markdown module and create the necessary JavaScript files in the build directory.

4. Ensure frontend dependencies are installed (web):

File Changes

The following files were added or modified to implement the PDF to Markdown conversion:

New files:

• pdf-to-markdown/: (New directory for the PDF to Markdown module)

  • pdf-to-markdown/package.json: Defines dependencies and build scripts for the PDF to Markdown module.
  • pdf-to-markdown/tsconfig.json: TypeScript configuration for the PDF to Markdown module.
  • pdf-to-markdown/src/: Source code directory for the PDF to Markdown module.
    • pdf-to-markdown/src/index.ts: Entry point of the PDF to Markdown module.
    • pdf-to-markdown/src/PdfPipeline.ts: Core logic for PDF to Markdown conversion pipeline.
    • pdf-to-markdown/src/PdfParser.ts: PDF parsing logic using pdf-parse.

• web/src/lib/services/PdfConversionService.ts: (New file)

  • Frontend service to use the pdf-to-markdown module and expose convertToMarkdown function.

Modified files:

• web/src/lib/components/chat/ChatInput.svelte:
  • Modified to import and use the PdfConversionService in the readFileContent function to handle PDF files.
  • Modified readFileContent to call pdfService.convertToMarkdown for PDF files.

These file changes introduce the new PDF to Markdown conversion functionality and integrate it into the chat input component of the web interface.

🎯 Usage Examples

1. Using Language Qualifiers

User: What is the weather?
AI: The weather information...

User: --fr What is the weather?
AI: Voici les informations météo...

2. Global Settings

1. Select language from dropdown
2. All interactions use selected language
3. Automatic reset to English after each message

3. YouTube Analysis

User: Analyze this YouTube video --fr
AI: [Provides analysis in French, maintaining language throughout the transcript]

💡 Key Benefits

1. Enhanced User Experience

   • Intuitive language switching
   • Consistent language handling
   • Seamless integration with existing features

2. Robust Implementation

   • Simple yet powerful design
   • No complex language detection needed
   • Direct AI instruction approach

3. Maintainable Architecture

   • Clean separation of concerns
   • Stateful language management
   • Easy to extend for new languages

4. YouTube Integration

   • Handles long transcripts effectively
   • Maintains language consistency
   • Robust chunk processing

🔄 Implementation Notes

1. State Management

   • Language persists until changed
   • Handles UI state updates efficiently

2. Error Handling

   • Invalid qualifiers are ignored
   • Unknown languages default to English
   • Proper store reset on errors

3. Next Steps

• Fix "warning" Aria errors with modal
• Reinstate correct language for Pattern output headings when Youtube is used (eg "Main Idea", "Key Take Aways"). Was working fine, lost along the way in some code change. Anyone wants to take a peak @johnconnor-sec ?
• Finalize streaming, could not get this to work ie currently limited to deliver pattern output at once.
• Maybe an MCP type Pattern to handle the Pattern description update process when new patterns are added.
• Else?

[jmd1010]

Hello Daniel and the rest of the Fabric Team! Happy to contribute a series of enhancements to the web front end. I'll let you be the judge if we should add some text to the README file. One of the enhancement is the possibility to convert PDF to Markdown on the fly and users should be aware of the simple steps to set it up. Cheers Jean De Bane, mba Canada On Feb 24, 2025, at 3:09 PM, Eugen Eisler ***@***.***> wrote: Merged #1309 <#1309> into main. — Reply to this email directly, view it on GitHub <#1309 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGCUMWNVMTVNWU3SZTODLTT2RN4ATAVCNFSM6AAAAABXRWMR3CVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJWGQZTOMRVHE4TIMQ>. You are receiving this because you authored the thread.