SSH connection: console closing & RemoteHostClosedError

[simon-77]

Dear all

I have setup the docker image dantti/virtlyst on my server to connect to another machine which runs virt-manager & kvm.

After creating SSH keys within the docker image and copying them to my virtualization host, I successfully configured a SSH Connection in the virtylist web UI.

I can see the virtual machines, start them and even connect the console (GUI, via spice), but unfortunatelly after about 30 seconds the following error appears: unexpected close while ready

Even when disconnecting the console and connecting again, it says "ready", but is not showing the GUI of the VM. Only when rebooting the vm, the console will show the GUI again for about 30 seconds.

I have also checked the docker log of virtlyst:

7:7 virtlyst.ws[debug] Create SSH tunnel to "bussard.lan" as "root"

7:7 virtlyst.ws[debug] SSH tunnel established

7:7 virtlyst.ws[debug] Connecting TCP socket to "127.0.0.1" 5900

7:7 virtlyst.ws[warning] Console Proxy socket connected from "127.0.0.1" 5900

7:7 virtlyst.ws[debug] Create SSH tunnel to "bussard.lan" as "root"

7:7 virtlyst.ws[debug] SSH tunnel established

7:7 virtlyst.ws[debug] Connecting TCP socket to "127.0.0.1" 5900

7:7 virtlyst.ws[warning] Console Proxy socket connected from "127.0.0.1" 5900

7:7 virtlyst.ws[warning] Console Proxy error: QAbstractSocket::RemoteHostClosedError "The remote host closed the connection"

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[warning] Console Proxy socket disconnected

7:7 virtlyst.ws[warning] Console Proxy error: QAbstractSocket::RemoteHostClosedError "The remote host closed the connection"

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[warning] Console Proxy socket disconnected

7:7 virtlyst.ws[warning] Console Proxy error: QAbstractSocket::RemoteHostClosedError "The remote host closed the connection"

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[warning] Console Proxy socket disconnected

7:7 virtlyst.ws[warning] Console Proxy error: QAbstractSocket::RemoteHostClosedError "The remote host closed the connection"

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[warning] Console Proxy socket disconnected

7:7 virtlyst.ws[warning] Console Proxy error: QAbstractSocket::RemoteHostClosedError "The remote host closed the connection"

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[warning] Console Proxy socket disconnected

7:7 virtlyst.ws[warning] Console Proxy error: QAbstractSocket::RemoteHostClosedError "The remote host closed the connection"

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[warning] Console Proxy socket disconnected

7:7 virtlyst.ws[warning] Console Proxy error: QAbstractSocket::RemoteHostClosedError "The remote host closed the connection"

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[warning] Console Proxy socket disconnected

7:7 virtlyst.ws[debug] Create SSH tunnel to "bussard.lan" as "root"

7:7 virtlyst.ws[debug] SSH tunnel established

7:7 virtlyst.ws[debug] Connecting TCP socket to "127.0.0.1" 5900

7:7 virtlyst.ws[warning] Console Proxy socket connected from "127.0.0.1" 5900

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

7:7 virtlyst.ws[debug] Create SSH tunnel to "bussard.lan" as "root"

7:7 virtlyst.ws[debug] SSH tunnel established

7:7 virtlyst.ws[debug] Connecting TCP socket to "127.0.0.1" 5900

7:7 virtlyst.ws[warning] Console Proxy socket connected from "127.0.0.1" 5900

7:7 virtlyst.ws[debug] Shutdown ssh tunnel: true

And I have also checked the log on my virtualization host (SSH server):

 sudo journalctl -u sshd
Mai 14 19:09:06 Bussard sshd[6164]: pam_unix(sshd:session): session closed for user root
Mai 14 19:09:06 Bussard sshd[6164]: pam_kwallet5(sshd:session): pam_kwallet5: pam_sm_close_session
Mai 14 19:09:06 Bussard sshd[6164]: pam_kwallet5(sshd:setcred): pam_kwallet5: pam_sm_setcred
Mai 14 19:09:06 Bussard sshd[6251]: pam_unix(sshd:session): session opened for user root by (uid=0)
Mai 14 19:09:06 Bussard sshd[6251]: pam_kwallet5(sshd:session): pam_kwallet5: pam_sm_open_session
Mai 14 19:09:06 Bussard sshd[6251]: pam_kwallet5(sshd:session): pam_kwallet5: not a graphical session, skipping. Use force_run parameter to ignore this.
Mai 14 19:09:06 Bussard sshd[6287]: Accepted publickey for root from 10.70.0.71 port 57110 ssh2: ED25519 SHA256:<xxx>
Mai 14 19:09:06 Bussard sshd[6287]: pam_kwallet5(sshd:setcred): pam_kwallet5: pam_sm_setcred
Mai 14 19:09:06 Bussard sshd[6287]: pam_unix(sshd:session): session opened for user root by (uid=0)
Mai 14 19:09:06 Bussard sshd[6287]: pam_kwallet5(sshd:session): pam_kwallet5: pam_sm_open_session
Mai 14 19:09:06 Bussard sshd[6287]: pam_kwallet5(sshd:session): pam_kwallet5: not a graphical session, skipping. Use force_run parameter to ignore this.
Mai 14 19:10:49 Bussard sshd[6438]: Accepted publickey for root from 10.70.0.71 port 46018 ssh2: ED25519 <xxx>
Mai 14 19:10:49 Bussard sshd[6438]: pam_kwallet5(sshd:setcred): pam_kwallet5: pam_sm_setcred
Mai 14 19:10:49 Bussard sshd[6438]: pam_unix(sshd:session): session opened for user root by (uid=0)
Mai 14 19:10:49 Bussard sshd[6438]: pam_kwallet5(sshd:session): pam_kwallet5: pam_sm_open_session
Mai 14 19:10:49 Bussard sshd[6438]: pam_kwallet5(sshd:session): pam_kwallet5: not a graphical session, skipping. Use force_run parameter to ignore this.
Mai 14 19:11:03 Bussard sshd[6500]: Accepted publickey for root from 10.70.0.71 port 35802 ssh2: ED25519 <xxx>
Mai 14 19:11:03 Bussard sshd[6500]: pam_kwallet5(sshd:setcred): pam_kwallet5: pam_sm_setcred
Mai 14 19:11:03 Bussard sshd[6500]: pam_unix(sshd:session): session opened for user root by (uid=0)
Mai 14 19:11:03 Bussard sshd[6500]: pam_kwallet5(sshd:session): pam_kwallet5: pam_sm_open_session
Mai 14 19:11:03 Bussard sshd[6500]: pam_kwallet5(sshd:session): pam_kwallet5: not a graphical session, skipping. Use force_run parameter to ignore this.
Mai 14 19:11:03 Bussard sshd[6539]: Accepted publickey for root from 10.70.0.71 port 35810 ssh2: ED25519 <xxx>

I get that the ssh connection is opening and closing all the time and that this might impact the work of the virtlyst spice console - but I can't make out why this would happen.

Does anyone have a hint where to look next? What furtether logs would be relevant?

Cheers
Simon

[dantti]

maybe add keepalive to ssh config

[misery]

@simon-77
Did this help in your case?

Host remotehost
    HostName remotehost.com
    ServerAliveInterval 10

If this helps we can add -o ServerAliveInterval=10 to virtlyst command.

Or maybe try this on server side:

ClientAliveInterval 10
ClientAliveCountMax 2

[simon-77]

Hi, thank you for making these suggestions.

I have tried both options with an alive Interval of 10

• Client Side (inside Virtlyst container): create an alias in /root/.ssh/config (as suggested by @misery) and use this alias as SSH connection in Virtlyst web-UI
• Server Side: add the config lines to /etc/ssh/sshd_config and restart the sshd service with systemctl

Unfortunately still the same thing happens to the "Graphical Console" of Virtlyst web-UI (after about a minute). Controlling the remote host (e.g. restarting the VM) is still possible.
To get the "Graphical Console" back, I have to restart the VM - only restarting the Virtlyst docker container will not resolve the issue.

Best wishes,
Simon