Skip to content

README_OPENAI_SETUP.md

Latest commit

 

History

History
117 lines (89 loc) · 3.78 KB

README_OPENAI_SETUP.md

File metadata and controls

117 lines (89 loc) · 3.78 KB

OpenAI Integration Setup

Summary of Changes Made

✅ Removed all backup summary logic - The system now only uses OpenAI for summaries ✅ Simplified CommentSummarizer - No more fallback methods or transformer models ✅ Updated to modern OpenAI API - Using the new OpenAI client instead of deprecated methods ✅ Enhanced prompt engineering - More verbose, conversational summaries that highlight commonalities and differences

Setup Instructions

1. Set Your OpenAI API Key

You mentioned you added the OpenAI key to your environment. Make sure it's exported in your current shell session:

export OPENAI_API_KEY="your-openai-api-key-here"

To make this permanent, add it to your ~/.zshrc or ~/.bash_profile:

echo 'export OPENAI_API_KEY="your-openai-api-key-here"' >> ~/.zshrc
source ~/.zshrc

2. Test the Integration

Run the test script to verify everything is working:

python3 test_openai_summarizer.py

You should see:

  • ✅ OpenAI API key is available
  • ✅ CommentSummarizer imported successfully
  • ✅ CommentSummarizer initialized successfully
  • ✅ Sample summarization successful!

3. Test with Your Application

Start the FastAPI server locally:

export USE_FAKE_PIPELINE=1
uvicorn app:app --reload

Then test the summarization endpoint:

curl -X POST "http://localhost:8000/summarize" \
  -H "Content-Type: application/json" \
  -d '{
    "comments": [
      {"text": "This video is amazing! Lady Gaga always delivers.", "likes": 15},
      {"text": "I do not agree with her feminist views but respect her artistry.", "likes": 8},
      {"text": "Great interview, very insightful questions.", "likes": 12}
    ],
    "sentiment": {
      "overall_sentiment": "mixed",
      "sentiment_percentages": {"positive": 60.0, "negative": 30.0, "neutral": 10.0},
      "individual_results": [
        {"predicted_sentiment": "positive", "confidence": 0.9},
        {"predicted_sentiment": "neutral", "confidence": 0.7},
        {"predicted_sentiment": "positive", "confidence": 0.8}
      ]
    },
    "method": "openai"
  }'

What Changed

Before

  • Multiple fallback methods (transformer, objective, OpenAI)
  • Complex initialization with use_openai parameter
  • Old OpenAI API syntax (openai.ChatCompletion.create)

After

  • OpenAI only - No fallback methods
  • Simple initialization: CommentSummarizer()
  • Modern OpenAI client: self.client.chat.completions.create
  • Enhanced prompts for more verbose, conversational summaries
  • Focus on commonalities and differences between comment themes

Expected Output

The new summaries will be:

  • More conversational ("The audience is absolutely loving this!" vs dry statistics)
  • More verbose (3-4 substantial paragraphs vs brief bullet points)
  • More insightful (highlights emotional undercurrents and social dynamics)
  • Better balanced (specifically calls out commonalities and differences in viewer opinions)
  • Uses emojis strategically for emphasis and engagement

Troubleshooting

"OpenAI API key is NOT available"

  • Verify the environment variable: echo $OPENAI_API_KEY
  • Make sure you're in the same terminal session where you set it
  • Try restarting your terminal and setting it again

"Failed to import CommentSummarizer"

  • Make sure you're in the project directory
  • Verify the openai package is installed: pip3 list | grep openai

API Rate Limits

  • OpenAI has rate limits for API calls
  • The system includes caching to reduce API calls
  • Consider upgrading your OpenAI plan if you hit limits frequently

Cost Considerations

  • Each summary request costs approximately $0.001-0.003 (depending on comment length)
  • Caching is enabled for 6 hours to reduce costs
  • Consider implementing additional rate limiting for production use