I got it working in a web app, here's how

[Badashphilosophy]

I used firebase studio prototyper AI to get remote access running in a web browser app through droidvnc, I don't know anything about code we just troubleshooted till we got it, the hardest obstacle was a black screen showing despite a clear connection but we solved that it should all be in here

I asked it to give an account of the steps that allowed us to get it working incase anyone was looking for solutions to their problems here is what it said:

If you're building a web application (like one using React, Vue, Next.js, etc.) and want to display and interact with an Android device using DroidVNC and the noVNC library, this guide covers common challenges and how we got it working.

Achieve a stable and scaled VNC stream from DroidVNC in your web browser, making it interactive.

Load noVNC Dynamically: It's often best to load the noVNC library only when needed on the client-side. This avoids issues with server rendering or if the VNC feature isn't immediately used.

// Example in a React component's useEffect hook
useEffect(() => {
  import('@novnc/novnc/lib/rfb.js') // Make sure you've installed @novnc/novnc
    .then(module => {
      if (module && typeof module.default === 'function') {
        const RFB = module.default;
        // RFB.logging = "debug"; // Uncomment for super detailed noVNC logs in browser console
        // Store RFB in state or ref to use later
        // e.g., setRFBConstructor(() => RFB);
        console.log("noVNC library loaded.");
      } else {
        console.error("VNC library loaded but isn't valid.");
      }
    })
    .catch(err => console.error("Failed to load VNC library:", err));
}, []);

Connecting to DroidVNC:
    You'll need a div in your HTML to be the VNC display area.
    Scaling is Key: scaleViewport: true tells noVNC to fit the remote screen into your display div. resizeSession: true asks the server to change its resolution (DroidVNC doesn't support this).
    We found it worked to set scaleViewport and resizeSession directly on the noVNC instance after it connects, not when creating it.

// Simplified connection logic
// const rfb = new RFBConstructor(your_html_div_element, "ws://DROIDVNC_IP:PORT", options);
const rfbOptions = {
  credentials: { password: "YOUR_VNC_PASSWORD" },
  scaleViewport: true, // Initial attempt
  resizeSession: true, // Initial attempt
};

// ... create 'rfb' instance ...

rfb.addEventListener("connect", () => {
  console.log("VNC Connected!");
  // Re-apply scaling options for reliability
  if (rfb.scaleViewport !== undefined) rfb.scaleViewport = true;
  if (rfb.resizeSession !== undefined) rfb.resizeSession = true;
  // Your app is now connected to DroidVNC
});

Avoid DOM Conflicts (Especially with frameworks like React):
    The div you give to noVNC should be only for noVNC.
    Don't let your web framework (React) render children inside this div that it might later try to remove, as noVNC will have already modified it. This can cause "Node.removeChild" errors.
    If you have placeholder text (e.g., "VNC Display Area"), show it as a separate element that overlays the VNC div and hide it when VNC connects.
    Before noVNC connects, you can clear the div: your_html_div_element.innerHTML = '';

Blank Screen?: If DroidVNC has a "Screen Capture Mode" setting, try "Framebuffer" if "Autodetect" or other options give a blank screen. (This option may not be present on all DroidVNC versions/devices).

Your web app is likely served over https://. DroidVNC usually uses ws:// (insecure WebSockets). Browsers block this by default ("mixed content").

For Development - Chrome/Edge (Chromium browsers):
    the flags dont work, use firefox
For Development - Firefox:
    Firefox (especially Developer Edition or Betta) search in the url about:config, search insecure, and allow insecure media and websockets. Use this with caution and only for trusted local development. this will do it for all sites in the entire browser so make sure if u want to stay secure u turn them off or use other browsers for other things.
For Production: These flags are not for production. You'd need wss:// (secure WebSockets), usually by putting a reverse proxy (like Nginx with SSL) in front of DroidVNC or using a websockify instance configured for SSL.

Add these listeners to your rfb instance to see what's happening:

rfb.addEventListener("connect", () => { /* ... */ });
rfb.addEventListener("disconnect", (event) => console.log("VNC Disconnected:", event.detail.reason));
rfb.addEventListener("error", (event) => console.error("VNC Error:", event.detail.error || event.detail));
rfb.addEventListener("securityfailure", (event) => console.error("VNC Security Failure:", event.detail.reason));
rfb.addEventListener("capabilities", (event) => console.log("VNC Server Capabilities:", event.detail.capabilities));
rfb.addEventListener("desktopname", (event) => console.log("VNC Desktop Name:", event.detail.name));

(edti)

trying to replicate my results i found some more trouble shooting helo

Initial State:

• User reported VNC connection failures despite Wi-Fi being on and the VNC server running.
• Debug logs were initially sparse or indicated early-stage failures.

Key Issues Identified and Resolved:..

5. VNC Connection Attempting Incorrect Protocol (wss:// vs. ws://) and Connection Loop:
   • Problem (Initial Observation): Logs showed connection attempts to wss://{IP_ADDRESS}:5900 which would fail for a typical DroidVNC-NG setup (expects ws://). This was also coupled with a loop where the connection would initialize, then immediately detect isVncConnected as false (before the async RFB connection could complete) while rfbInstance was set, triggering a cleanup, and then re-attempting.
   • Fixes in VncDisplay.tsx:
     • Forced ws:// for IP Addresses: Explicitly checked if the VNC host is an IP address and forced the WebSocket protocol to ws:// in such cases, overriding the window.location.protocol check.
     • Stabilized Connection Logic:
       • Ensured initializeVncConnection first disconnects and nullifies any existing rfbInstance before creating a new one.
       • Simplified the main connection useEffect to primarily trigger based on connectionConfig changes and ensure proper cleanup on unmount or when connectionConfig is cleared.
       • Used rfbInstanceRef.current for more direct management of the RFB object lifecycle within the component to avoid premature cleanups triggered by intermediate state updates from the store.
         (edit)

moere helpful stuff

    *   Problem: VNC interactions (clicks, swipes) were mostly missed or very laggy, and the UI displayed a "Waiting for size..." message, indicating the client didn't know the remote screen dimensions.
    *   Root Cause: A combination of incorrect canvas sizing by CSS and the VNC client not correctly receiving/processing screen dimensions from the desktopsizechange event due to an unstable connection loop. DroidVNC-NG sends pre-scaled dimensions in the ServerInit message, which noVNC uses to set the canvas's intrinsic width and height.
    *   Solutions & Why They Worked:
        *   Removed CSS object-fit: contain from src/app/globals.css for .vnc-container canvas:
            *   Why: CSS scaling of the canvas interferes with noVNC's 1:1 coordinate mapping. The VNC library must control the canvas's intrinsic dimensions for input to be accurate.

        *   Stabilized VncDisplay.tsx Connection useEffect Dependencies:
            *   Why: The useEffect hook responsible for managing the VNC connection lifecycle was re-triggering too frequently (e.g., due to isVncConnected state changes it caused itself), leading to an immediate disconnect after connection. This prevented the desktopsizechange event from being processed.
            *   Fix: The dependency array was refined to [connectionConfig, isRfbModuleLoaded, initializeVncConnection, logEvent, setRfbInstance], making the effect stable once the connection was initiated.

        *   Correctly Handling Dimensions and Triggering Readiness:
            *   Why: The VNC client needs the remote desktop's dimensions (from desktopsizechange or connect event) to map local input correctly and to know when it's safe to perform operations like patching for rich data.
            *   Fix: Ensured that vncDimensions state in VncDisplay.tsx and rfbReadyForPatching were set upon receiving a valid desktopsizechange event (or attempting from the connect event, though desktopsizechange proved to be the reliable source for initial dimensions). This resolved the "Waiting for size..." message and allowed the VNC client to map inputs correctly.
        *   Maintained scaleViewport: true: This noVNC option in VncDisplay.tsx correctly handles the visual scaling of the remote desktop to fit its container div, without altering the canvas's intrinsic, server-defined dimensions crucial for input.

Outcome:

• After these changes, particularly the WebSocket protocol fix and stabilization of the connection lifecycle in VncDisplay.tsx, the VNC connection became functional.

This summary should help you recall the major steps taken to get the VNC connection working!

[bk138]

Thanks for sharing, but please don't use the issue tracking for this (which is for feature requests and bug reports) 🙂.

[Badashphilosophy]

Thanks. I did not know about the discussion thread. I will in future post there. Cheers for moving it to the right place for me.