|
| 1 | +# SAE Video Conferencing - Connection Architecture |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This document describes the complete wiring between the client (frontend), Frappe server (intermediary), and SFU server (mediasoup) for the SAE video conferencing application. |
| 6 | + |
| 7 | +## Architecture Flow |
| 8 | + |
| 9 | +``` |
| 10 | +┌─────────────┐ Socket.IO ┌─────────────────┐ Socket.IO ┌─────────────┐ |
| 11 | +│ Client │ ◄──────────────► │ Frappe Server │ ◄──────────────► │ SFU Server │ |
| 12 | +│ (Frontend) │ │ (Intermediary) │ │ (mediasoup) │ |
| 13 | +└─────────────┘ └─────────────────┘ └─────────────┘ |
| 14 | +``` |
| 15 | + |
| 16 | +## Components Implemented |
| 17 | + |
| 18 | +### 1. Client-Side (`frontend/src/socket.js`) |
| 19 | + |
| 20 | +- **Socket.IO connection**: Manages connection to Frappe server |
| 21 | +- **Event emission**: Sends WebRTC signals, meeting controls, media controls to server |
| 22 | +- **Event listening**: Receives SFU responses, participant updates, media events |
| 23 | +- **Event handlers**: Configurable handlers for different video conferencing events |
| 24 | + |
| 25 | +**Key Functions:** |
| 26 | + |
| 27 | +- `initSocket()`: Initialize Socket.IO connection |
| 28 | +- `joinMeeting()`, `leaveMeeting()`: Meeting management |
| 29 | +- `sendWebRTCSignal()`: WebRTC signaling |
| 30 | +- `sendMediaControl()`: Audio/video controls |
| 31 | +- `registerWebRTCEventHandlers()`: Register custom event handlers |
| 32 | + |
| 33 | +### 2. Frappe Server Real-time Handlers (`sae/realtime/handlers.js`) |
| 34 | + |
| 35 | +- **Socket event bridging**: Captures client events and forwards to Python handlers |
| 36 | +- **Room management**: Manages Socket.IO rooms for meetings |
| 37 | +- **Real-time relay**: Publishes events to trigger Python processing |
| 38 | +- **Error handling**: Catches and reports client-side errors |
| 39 | + |
| 40 | +**Event Mapping:** |
| 41 | + |
| 42 | +- `webrtc_signal` → `sae_webrtc_signal` |
| 43 | +- `join_meeting` → `sae_join_meeting` |
| 44 | +- `leave_meeting` → `sae_leave_meeting` |
| 45 | +- `media_control` → `sae_media_control` |
| 46 | +- And all SFU-specific events |
| 47 | + |
| 48 | +### 3. Python Socket Handlers (`sae/utils/socket_handlers.py`) |
| 49 | + |
| 50 | +- **WebRTC signaling**: Processes and relays WebRTC offers, answers, ICE candidates |
| 51 | +- **Meeting management**: Handles join/leave operations with SFU coordination |
| 52 | +- **Media control**: Manages audio/video state changes |
| 53 | +- **SFU integration**: Translates client requests to SFU commands |
| 54 | + |
| 55 | +**Handler Functions:** |
| 56 | + |
| 57 | +- `handle_webrtc_signal()`: WebRTC signaling relay |
| 58 | +- `handle_join_meeting()`: Meeting join with SFU room creation |
| 59 | +- `handle_leave_meeting()`: Meeting leave with SFU cleanup |
| 60 | +- `handle_media_control()`: Audio/video control relay |
| 61 | +- `handle_produce_media()`, `handle_consume_media()`: Media production/consumption |
| 62 | +- `handle_create_transport()`, `handle_connect_transport()`: WebRTC transport management |
| 63 | + |
| 64 | +### 4. Real-time Bridge (`sae/utils/realtime_bridge.py`) |
| 65 | + |
| 66 | +- **Event listener setup**: Connects real-time events to socket handlers |
| 67 | +- **User context management**: Sets proper user context for handlers |
| 68 | +- **Cleanup coordination**: Handles user disconnection cleanup |
| 69 | +- **Error handling**: Manages errors in real-time processing |
| 70 | + |
| 71 | +### 5. SFU Connection Manager (`sae/utils/sfu_manager.py`) |
| 72 | + |
| 73 | +- **Socket.IO client**: Maintains connection to SFU server |
| 74 | +- **Event relay**: Bidirectional relay between Frappe and SFU |
| 75 | +- **Room management**: Creates/manages meeting rooms on SFU |
| 76 | +- **Participant tracking**: Tracks participants in each room |
| 77 | +- **Error handling**: Manages SFU connection errors |
| 78 | + |
| 79 | +**Key Methods:** |
| 80 | + |
| 81 | +- `connect_to_sfu()`: Establish SFU connection |
| 82 | +- `create_room()`, `join_room()`, `leave_room()`: Room operations |
| 83 | +- `relay_to_sfu()`: Send events to SFU |
| 84 | +- `relay_to_client()`: Send events to clients |
| 85 | +- Event handlers for all SFU responses |
| 86 | + |
| 87 | +### 6. Integration Hooks (`sae/hooks.py`) |
| 88 | + |
| 89 | +- **Startup initialization**: Initializes real-time bridge on app startup |
| 90 | +- **Boot session**: Ensures bridge is ready when users connect |
| 91 | + |
| 92 | +## Event Flow Examples |
| 93 | + |
| 94 | +### Meeting Join Flow |
| 95 | + |
| 96 | +1. **Client** → `socket.emit('join_meeting', data)` |
| 97 | +2. **Frappe Real-time Handler** → `frappe.publish_realtime('sae_join_meeting', data)` |
| 98 | +3. **Python Handler** → `handle_join_meeting(data)` |
| 99 | +4. **SFU Manager** → `sfu_manager.join_room(meeting_id, user_id, user_data)` |
| 100 | +5. **SFU Server** → Confirms room join |
| 101 | +6. **SFU Manager** → `relay_to_client('meeting_joined_success', data)` |
| 102 | +7. **Client** → Receives `meeting_joined_success` event |
| 103 | + |
| 104 | +### WebRTC Signaling Flow |
| 105 | + |
| 106 | +1. **Client** → `socket.emit('webrtc_signal', {type: 'offer', signal_data: offer})` |
| 107 | +2. **Frappe Handler** → `handle_webrtc_signal(data)` |
| 108 | +3. **SFU Manager** → `relay_to_sfu('webrtc_offer', relay_data)` |
| 109 | +4. **SFU Server** → Processes offer and sends to target user |
| 110 | +5. **SFU Manager** → `relay_to_client('webrtc_offer', data, target_user)` |
| 111 | +6. **Target Client** → Receives offer and can respond with answer |
| 112 | + |
| 113 | +### Media Control Flow |
| 114 | + |
| 115 | +1. **Client** → `socket.emit('media_control', {action: 'mute', meeting_id: 'xyz'})` |
| 116 | +2. **Frappe Handler** → `handle_media_control(data)` |
| 117 | +3. **SFU Manager** → `relay_to_sfu('media_control', relay_data)` |
| 118 | +4. **SFU Server** → Updates media state and notifies participants |
| 119 | +5. **All Participants** → Receive `media_control_update` event |
| 120 | + |
| 121 | +## Error Handling |
| 122 | + |
| 123 | +### Client-Side Errors |
| 124 | + |
| 125 | +- Connection failures: Automatic reconnection attempts |
| 126 | +- Signal errors: Event-specific error handlers |
| 127 | +- Timeout handling: Configurable timeouts for operations |
| 128 | + |
| 129 | +### Server-Side Errors |
| 130 | + |
| 131 | +- Python handler errors: Logged and sent to clients as error events |
| 132 | +- SFU connection errors: Automatic reconnection and fallback |
| 133 | +- Meeting state errors: Graceful degradation and user notification |
| 134 | + |
| 135 | +### SFU Errors |
| 136 | + |
| 137 | +- Connection loss: Automatic reconnection with state recovery |
| 138 | +- Room errors: Proper cleanup and user notification |
| 139 | +- Media errors: Individual participant error handling |
| 140 | + |
| 141 | +## Configuration |
| 142 | + |
| 143 | +### SFU Connection Settings |
| 144 | + |
| 145 | +```python |
| 146 | +# In site_config.json or frappe.conf |
| 147 | +sfu_server_url = "http://localhost" # SFU server URL |
| 148 | +sfu_server_port = 3000 # SFU server port |
| 149 | +``` |
| 150 | + |
| 151 | +### Real-time Settings |
| 152 | + |
| 153 | +```python |
| 154 | +# In hooks.py |
| 155 | +boot_session = ["sae.utils.realtime_bridge.initialize_realtime_bridge"] |
| 156 | +``` |
| 157 | + |
| 158 | +## Security Considerations |
| 159 | + |
| 160 | +### Authentication |
| 161 | + |
| 162 | +- All socket handlers verify user authentication via `frappe.session.user` |
| 163 | +- Meeting access controlled through Frappe permissions |
| 164 | +- SFU operations require valid meeting membership |
| 165 | + |
| 166 | +### Data Validation |
| 167 | + |
| 168 | +- All client data validated before processing |
| 169 | +- JSON parsing with error handling |
| 170 | +- Required field validation for all operations |
| 171 | + |
| 172 | +### Rate Limiting |
| 173 | + |
| 174 | +- API endpoints protected with rate limiting |
| 175 | +- Socket event throttling to prevent abuse |
| 176 | +- Connection limits per user/meeting |
| 177 | + |
| 178 | +## Performance Optimizations |
| 179 | + |
| 180 | +### Connection Management |
| 181 | + |
| 182 | +- Persistent SFU connection with automatic reconnection |
| 183 | +- Connection pooling for multiple concurrent operations |
| 184 | +- Efficient event routing to minimize latency |
| 185 | + |
| 186 | +### Memory Management |
| 187 | + |
| 188 | +- Cleanup on user disconnection |
| 189 | +- Meeting state garbage collection |
| 190 | +- Optimized data structures for participant tracking |
| 191 | + |
| 192 | +### Scalability |
| 193 | + |
| 194 | +- Stateless handler design for horizontal scaling |
| 195 | +- SFU connection sharing across workers |
| 196 | +- Efficient real-time event distribution |
| 197 | + |
| 198 | +## Monitoring and Debugging |
| 199 | + |
| 200 | +### Logging |
| 201 | + |
| 202 | +- Comprehensive logging at all levels |
| 203 | +- Error tracking with stack traces |
| 204 | +- Performance metrics collection |
| 205 | + |
| 206 | +### Debug Events |
| 207 | + |
| 208 | +- Real-time event tracing |
| 209 | +- Connection state monitoring |
| 210 | +- WebRTC state debugging |
| 211 | + |
| 212 | +### Health Checks |
| 213 | + |
| 214 | +- SFU connection health monitoring |
| 215 | +- Meeting state validation |
| 216 | +- Participant synchronization checks |
| 217 | + |
| 218 | +## Future Enhancements |
| 219 | + |
| 220 | +### Planned Features |
| 221 | + |
| 222 | +- Multi-SFU support for load balancing |
| 223 | +- Advanced error recovery mechanisms |
| 224 | +- Enhanced monitoring and analytics |
| 225 | +- Mobile app integration |
| 226 | +- Recording and playback capabilities |
| 227 | + |
| 228 | +### Performance Improvements |
| 229 | + |
| 230 | +- Connection optimization |
| 231 | +- Bandwidth adaptation |
| 232 | +- Quality of service controls |
| 233 | +- Advanced codec support |
0 commit comments