Skip to content

Commit 32d98ca

Browse files
waleedkadouswaleedkadous
authored
Merge pull request #189 from OdyAsh/develop
Event Loop Safety Enhancement and FastAPI Documentation
2 parents be499a2 + a38a57f commit 32d98ca

8 files changed

Lines changed: 627 additions & 11 deletions

File tree

‎.env.example‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,11 +57,24 @@ template_dir="." # Directory path for templates
5757

5858
WHATSAPP_API_VERSION="<<CURRENT-VERSION-AS-MENTIONED-IN-SOURCE-URL-ABOVE>>"
5959

60-
# NOTE: Contact the team to see whatsapp's 2 phone nums -> one for prod. env. and the other for local/stage testing
60+
# NOTE 1: Contact the team to see whatsapp's 2 phone nums -> one for prod. env. and the other for local/stage testing
61+
# NOTE 2: If you encounter this error:
62+
# "Client error '400 Bad Request' for url 'https://graph.facebook.com/v22.0/<WHATSAPP_BUSINESS_PHONE_NUMBER_ID>/messages'"
63+
# then either:
64+
# (1) the <WHATSAPP_BUSINESS_PHONE_NUMBER_ID> got deleted by Meta due to inactive use for a long time.
65+
# If this happens, then open the page in source 3 (i.e., youtube video URL above) @6:11.
66+
# Then, under "step 1" in the middle of the page, you'll see a selection box with the live WhatsApp number;
67+
# click on it, then click "Get new test number". Then, wait for a while, then you'll see the new number;
68+
# Now copy its number id and paste it in <WHATSAPP_BUSINESS_PHONE_NUMBER_ID>
69+
# (2) Meta updated its version and so now <WHATSAPP_API_VERSION> is outdated.
70+
# You can verify this by opening source 3 @26:13 and checking the version associated with `messages`
71+
# (3) you sent a message with an incorrect format. Check source 3; 26:10->26:28, click on `messages` to see how it should be sent
6172
WHATSAPP_BUSINESS_PHONE_NUMBER_ID="<<YOUR-WHATSAPP-BUSINESS-PHONE-NUMBER-ID>>"
6273

6374
# NOTE 1: check video in source 3 above from 30:45 to 32:15 to see where we get the access token
6475
# NOTE 2: Contact the team to see their 2 access tokens -> one for prod. env. and the other for local/stage testing
76+
# NOTE 3: If you want to debug or check the validity of an access token, check this URL:
77+
# https://developers.facebook.com/tools/debug/accesstoken/?access_token=<YOUR-ACCESS-TOKEN>
6578
WHATSAPP_ACCESS_TOKEN_FROM_SYS_USER="<<YOUR-SYSTEM-USER-ACCESS-TOKEN>"
6679

6780
WHATSAPP_VERIFY_TOKEN_FOR_WEBHOOK="<<The-VERIFIFY-TOKEN-CURRENTLY-USED-TO-VERIFY-META'S-CALLBACK-URL>>"

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# Folders
22
.conda/
3+
.claude/
34
.venv/
45
.vscode/
56
abandoned/

‎docs/fastapi/async_performance_optimization_strategies.md‎

Lines changed: 277 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,277 @@
1+
***TOC:***
2+
3+
- [Async Performance Optimization Strategies](#async-performance-optimization-strategies)
4+
- [Problem Context](#problem-context)
5+
- [Current Architecture Analysis](#current-architecture-analysis)
6+
- [Async Functions in WhatsApp Components](#async-functions-in-whatsapp-components)
7+
- [main\_whatsapp.py](#main_whatsapppy)
8+
- [whatsapp\_presenter.py](#whatsapp_presenterpy)
9+
- [Optimization Strategies](#optimization-strategies)
10+
- [Strategy 1: Convert API Endpoints to Async (Major Code Changes)](#strategy-1-convert-api-endpoints-to-async-major-code-changes)
11+
- [Required Changes](#required-changes)
12+
- [Advantages](#advantages)
13+
- [Disadvantages](#disadvantages)
14+
- [Strategy 2: Convert WhatsApp Endpoints to Sync (Targeted Changes)](#strategy-2-convert-whatsapp-endpoints-to-sync-targeted-changes)
15+
- [Required Changes](#required-changes-1)
16+
- [Advantages](#advantages-1)
17+
- [Disadvantages](#disadvantages-1)
18+
- [Strategy 3: Separate WhatsApp Service (Microservice Architecture)](#strategy-3-separate-whatsapp-service-microservice-architecture)
19+
- [Implementation Details](#implementation-details)
20+
- [Advantages](#advantages-2)
21+
- [Disadvantages](#disadvantages-2)
22+
- [Recommendation](#recommendation)
23+
- [Related Documentation](#related-documentation)
24+
25+
26+
# Async Performance Optimization Strategies
27+
28+
## Problem Context
29+
30+
The `_translate_with_event_loop_safety()` method in `ansari_claude.py:1479` implements a fallback mechanism to handle translation requests in both sync and async contexts. However, the `except` block falls back to sequential translation processing when called from async contexts (like WhatsApp background tasks), which can cause performance degradation on the server.
31+
32+
```python
33+
def _translate_with_event_loop_safety(self, arabic_texts: list[str], context: str = "citation") -> list[str]:
34+
try:
35+
# First try to use asyncio.run() (works when not in event loop)
36+
return asyncio.run(translate_texts_parallel(arabic_texts, "en", "ar"))
37+
except RuntimeError as e:
38+
# If we get RuntimeError, we're already in an event loop
39+
# Falls back to sequential processing - POTENTIAL PERFORMANCE BOTTLENECK
40+
logger.info(f"asyncio.run() failed ({e}), using sequential translation to avoid complexity")
41+
results = []
42+
for text in arabic_texts:
43+
result = translate_text(text, "en", "ar")
44+
results.append(result)
45+
return results
46+
```
47+
48+
## Current Architecture Analysis
49+
50+
### Async Functions in WhatsApp Components
51+
52+
#### main_whatsapp.py
53+
54+
**Async Functions:**
55+
- `verification_webhook()` - GET `/whatsapp/v1` (webhook verification)
56+
- `main_webhook()` - POST `/whatsapp/v1` (main message handler)
57+
58+
#### whatsapp_presenter.py
59+
60+
**Async Functions:**
61+
1. `extract_relevant_whatsapp_message_details()` - Extracts message data from webhook payload
62+
2. `check_and_register_user()` - Checks/registers users in database
63+
3. `send_typing_indicator_then_start_loop()` - Manages typing indicators
64+
4. `_typing_indicator_loop()` - Background typing indicator loop
65+
5. `_send_whatsapp_typing_indicator()` - Sends individual typing indicator
66+
6. `send_whatsapp_message()` - Sends messages to WhatsApp users
67+
7. `handle_text_message()` - Main text message processing (calls translation)
68+
8. `handle_location_message()` - Processes location messages
69+
9. `handle_unsupported_message()` - Handles unsupported message types
70+
71+
## Optimization Strategies
72+
73+
### Strategy 1: Convert API Endpoints to Async (Major Code Changes)
74+
75+
Convert all synchronous endpoints in `main_api.py` to async, along with underlying components.
76+
77+
#### Required Changes
78+
79+
**Files to Convert:**
80+
1. **main_api.py** - Convert all `def` endpoints to `async def`:
81+
- `add_message()` - Main API endpoint
82+
- All other sync endpoints
83+
84+
2. **ansari_claude.py** - Convert core methods to async:
85+
- `_translate_with_event_loop_safety()` → Use `translate_texts_parallel()` directly
86+
- `process_message()` → `async def process_message()`
87+
- `replace_message_history()` → `async def replace_message_history()`
88+
- All citation processing methods
89+
90+
3. **translation.py** - Already has async support:
91+
- `translate_texts_parallel()` (already async)
92+
- Keep `translate_text()` for backward compatibility
93+
94+
4. **Database operations** - Convert to async:
95+
- `ansari_db.py` → Use async database drivers
96+
- All database interactions throughout the codebase
97+
98+
5. **Search tools** - Convert search operations:
99+
- All search tool implementations
100+
- Vector database operations
101+
- Document retrieval operations
102+
103+
**Estimated Changes:** 50+ files, 1000+ lines of code
104+
105+
#### Advantages
106+
- **Optimal Performance**: All operations can use async/await for I/O operations
107+
- **Consistent Architecture**: Single async paradigm throughout the application
108+
- **Better Resource Utilization**: More efficient handling of concurrent requests
109+
- **Future-Proof**: Ready for high-concurrency scenarios
110+
111+
#### Disadvantages
112+
- **Massive Code Changes**: Requires converting the entire application stack
113+
- **High Risk**: Large refactoring increases chance of introducing bugs
114+
- **Development Time**: Weeks/months of development and testing
115+
- **Breaking Changes**: May affect existing API consumers
116+
- **Dependency Updates**: May require async-compatible versions of libraries
117+
118+
### Strategy 2: Convert WhatsApp Endpoints to Sync (Targeted Changes)
119+
120+
Convert WhatsApp-specific async functions to sync equivalents, eliminating the event loop context mismatch.
121+
122+
#### Required Changes
123+
124+
**main_whatsapp.py:**
125+
```python
126+
# Current (async - correctly implemented)
127+
async def main_webhook(request: Request, background_tasks: BackgroundTasks) -> Response:
128+
data = await request.json() # ✅ Valid async operation
129+
# ... other async operations
130+
131+
# Strategy 2 Option: Convert to sync
132+
def main_webhook(request: Request, background_tasks: BackgroundTasks) -> Response:
133+
# Use request.body() and json.loads() instead of await request.json()
134+
# Convert all async operations to sync equivalents
135+
```
136+
137+
**whatsapp_presenter.py - Functions to Convert:**
138+
1. `extract_relevant_whatsapp_message_details()` → Use synchronous JSON processing
139+
2. `check_and_register_user()` → Use synchronous database calls
140+
3. `send_typing_indicator_then_start_loop()` → Use threading instead of asyncio
141+
4. `_typing_indicator_loop()` → Use `threading.Thread` with `time.sleep()`
142+
5. `_send_whatsapp_typing_indicator()` → Use synchronous HTTP client (e.g., `requests`)
143+
6. `send_whatsapp_message()` → Use synchronous HTTP client
144+
7. `handle_text_message()` → Remove `await asyncio.sleep(0)`, use sync processing
145+
8. `handle_location_message()` → Already mostly sync
146+
9. `handle_unsupported_message()` → Use sync message sending
147+
148+
**Key Conversion Examples:**
149+
```python
150+
# Replace httpx AsyncClient with requests
151+
# Before:
152+
async with httpx.AsyncClient() as client:
153+
response = await client.post(url, headers=headers, json=json_data)
154+
155+
# After:
156+
import requests
157+
response = requests.post(url, headers=headers, json=json_data)
158+
159+
# Replace asyncio.create_task() with threading
160+
# Before:
161+
self.typing_indicator_task = asyncio.create_task(self._typing_indicator_loop())
162+
163+
# After:
164+
import threading
165+
self.typing_indicator_thread = threading.Thread(target=self._typing_indicator_loop)
166+
self.typing_indicator_thread.start()
167+
168+
# Replace asyncio.sleep() with time.sleep()
169+
# Before:
170+
await asyncio.sleep(INDICATOR_INTERVAL_SECONDS)
171+
172+
# After:
173+
import time
174+
time.sleep(INDICATOR_INTERVAL_SECONDS)
175+
```
176+
177+
#### Advantages
178+
- **Targeted Changes**: Only affects WhatsApp-specific code (~2 files)
179+
- **Eliminates Event Loop Conflict**: No more async context in WhatsApp processing
180+
- **Faster Implementation**: Can be completed in days rather than weeks
181+
- **Lower Risk**: Limited scope reduces chance of introducing bugs
182+
- **Immediate Performance Gain**: Translation can use parallel processing
183+
184+
#### Disadvantages
185+
- **Mixed Architecture**: Creates inconsistency between API and WhatsApp components
186+
- **Less Scalable**: Sync WhatsApp processing may be less efficient under high load
187+
- **Threading Overhead**: Using threads instead of async tasks for background operations
188+
- **Maintenance Complexity**: Two different paradigms to maintain
189+
190+
### Strategy 3: Separate WhatsApp Service (Microservice Architecture)
191+
192+
Create a dedicated FastAPI service for WhatsApp functionality that communicates with the main backend.
193+
194+
#### Implementation Details
195+
196+
**New Repository Structure:**
197+
```
198+
ansari-whatsapp-service/
199+
├── src/
200+
│ ├── main.py # FastAPI app for WhatsApp
201+
│ ├── whatsapp_handler.py # WhatsApp-specific logic
202+
│ ├── api_client.py # Client to communicate with main backend
203+
│ └── models.py # WhatsApp-specific models
204+
├── requirements.txt
205+
├── Dockerfile
206+
└── README.md
207+
208+
ansari-backend/ (existing)
209+
├── src/ansari/app/
210+
│ ├── main_api.py # Remove WhatsApp routes
211+
│ └── whatsapp_client.py # New: Client to communicate with WhatsApp service
212+
```
213+
214+
**Communication Pattern:**
215+
```python
216+
# In WhatsApp service
217+
@router.post("/process-message")
218+
async def process_message(message_data: MessageData):
219+
# Process WhatsApp message
220+
# Call main backend API for AI processing
221+
async with httpx.AsyncClient() as client:
222+
response = await client.post(
223+
"http://main-backend:8000/api/v1/process",
224+
json={"message": message_data.text, "user_id": message_data.user_id}
225+
)
226+
# Handle response and send back to WhatsApp
227+
228+
# In main backend
229+
@router.post("/api/v1/process")
230+
async def process_ai_request(request: AIRequest):
231+
# Pure AI processing, no WhatsApp-specific logic
232+
# Can be fully async without WhatsApp event loop conflicts
233+
```
234+
235+
#### Advantages
236+
- **Clean Separation**: WhatsApp logic completely isolated from main AI backend
237+
- **Independent Scaling**: Each service can be scaled based on its specific needs
238+
- **Technology Choice Freedom**: Each service can use optimal async/sync patterns
239+
- **Reduced Complexity**: Main backend focuses purely on AI processing
240+
- **Better Testing**: Each service can be tested independently
241+
- **Deployment Flexibility**: Services can be deployed and updated independently
242+
243+
#### Disadvantages
244+
- **Increased Infrastructure**: Need to manage multiple services, databases, deployments
245+
- **Network Latency**: Inter-service communication adds latency
246+
- **Operational Complexity**: Monitoring, logging, and debugging across services
247+
- **Development Overhead**: More complex local development setup
248+
- **Data Consistency**: Need to handle distributed data consistency
249+
- **Initial Development Time**: Significant upfront work to separate services
250+
251+
## Recommendation
252+
253+
**For Immediate Performance Fix: Strategy 2 (Convert WhatsApp to Sync)**
254+
255+
This provides the best balance of:
256+
- **Quick Implementation** (2-3 days)
257+
- **Immediate Performance Improvement**
258+
- **Low Risk** (limited scope)
259+
- **Addresses Core Problem** (eliminates event loop conflict)
260+
261+
**For Long-term Architecture: Strategy 3 (Microservice Separation)**
262+
263+
This should be considered for future development as it provides:
264+
- **Better Scalability**
265+
- **Cleaner Architecture**
266+
- **Independent Development and Deployment**
267+
268+
**Avoid: Strategy 1 (Convert Everything to Async)**
269+
270+
The scope and risk are too high for the current problem. The performance gain doesn't justify the massive refactoring effort.
271+
272+
## Related Documentation
273+
274+
- [FastAPI Sync vs Async Endpoints: Event Loop and Thread Handling](./def_vs_async_def_fastapi_endpoints.md)
275+
- `src/ansari/agents/ansari_claude.py:1479` - `_translate_with_event_loop_safety()` method
276+
- `src/ansari/app/main_whatsapp.py` - WhatsApp endpoint implementations
277+
- `src/ansari/presenters/whatsapp_presenter.py` - WhatsApp processing logic

0 commit comments

Comments
 (0)