Skip to content

[Feature]: Adaptive Context Engine - Dynamic UI based on Active Host Application #61

@Pranay22077

Description

@Pranay22077

Problem Statement

Is your feature request related to a problem? Please describe.
Currently, the Rein client (mobile) acts as a static trackpad/keyboard regardless of what task the user is performing on the host machine.

  • If I am watching YouTube/Spotify, I have to drag the mouse cursor to tiny "Play/Pause" buttons.
  • If I am giving a PowerPoint presentation, I have to look at my phone to find the "Next Slide" button or use the trackpad blindly.
  • If I am coding in VS Code, I have no quick access to common macros (Run, Terminal, etc.).

Proposed Solution

I propose implementing an Adaptive Context Engine that automatically detects the active window on the host machine and updates the client UI to match the context.

The "Smart Controller" Workflow:

  1. Host (Server): Polls the active window title/class every ~2 seconds.
  2. Protocol: Pushes a context_update event to the client via WebSocket when the application changes.
  3. Client: Dynamically renders a specialized component (e.g., <MediaControls />, <PresentationMode />) instead of the default trackpad.

Proposed Contexts:

  • 🎵 Media Mode: (Spotify, YouTube, VLC) → Big Play/Pause, Volume Slider, Seek.
  • 📊 Presentation Mode: (PowerPoint, Keynote) → Next/Prev Slide buttons + Gyroscopic Air Mouse (optional future addition).
  • 💻 Developer Mode: (VS Code, IntelliJ) → Quick Actions (Run, Debug, Terminal).
  • 🌐 Browser Mode: (Chrome, Firefox) → Tab Management, Refresh, Back/Forward.

Implementation Details (Technical Approach)
I have prototyped a ContextWatcher class using the active-win package, which works reliably across Windows, macOS, and Linux.

Architecture:

  • Backend:
    • src/server/ContextWatcher.ts: Uses active-win to detect window metadata.
    • wss.on('connection'): Subscribes new clients to context updates.
  • Frontend:
    • useRemoteConnection.ts: Listens for type: 'context-update'.
    • src/components/adaptive/: Folder containing <MediaControls />, <IdeControls />, etc.

Alternatives Considered

  • Manual Switching: Adding a dropdown for the user to manually select "Media Mode." This adds friction and is less "smart" than automatic detection.
  • Global Hotkeys: Just sending media keys regardless of app. This fails for app-specific contexts (like "Next Slide" in PowerPoint vs "Next Track" in Spotify).

Additional Context

I have a working Proof-of-Concept (POC) that successfully detects context changes on Windows/Linux. I am happy to submit a PR for the core engine and the first set of UI (Media Controls) to start.

Checklist

  • I have searched for existing feature requests to avoid duplicates

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions