Unable to connect to Linux VM through Guacamole

[jonnyry]

Describe the bug

Unable to connect to Linux VM through Guacamole.

Receive a "The remote desktop server is currently unavailable" from the Guacamole shell.

I've tried:

• Deploying a new Linux VM
• Deploying both Ubuntu images (18.04 + 18.04 Data Science)
• Deploying on Azure TRE 0.15.2 & 0.16.0
• Stopping and starting the VM

Things that do work:

• Connecting to the Linux VM via Azure bastion
• Connecting to a Windows VM in the same workspace

Appreciate any pointers in where to investigate.

Steps to reproduce

1. Create Linux VM in workspace
2. Observe inability connect via Guacamole

Azure TRE release version (e.g. v0.14.0 or main):

0.15.2 + 0.16.0

[marrobi]

Check the cloud init logs on the VM - https://microsoft.github.io/AzureTRE/latest/troubleshooting-faq/cloud-init/

In production we recommend using VM images with minimal startup scripts. As we can't publish VM images we rely on scripts in this repository. Let us know what you find.

[jonnyry]

Thanks @marrobi

Re-running the cloud-init scripts has resolved the issue.

Not entirely clear on what caused the issue in the first place, though I could see a number of connection issues to the nexus package mirror - the nexus VM was stopped when I originally created the Linux VM - does this sound like a possibility?

[marrobi]

Yes, the Linux VM requires Nexus for packages.

[jonnyry]

Just to add more context - I've run a few extra tests on this, and it appears to occur when:

• a Linux VM is created (from the TRE portal) while the package mirror is not running

Bringing the package mirror up afterwards and restarting the Linux VM does not automatically resolve the issue, without re-running the cloud-init scripts.

Conversely creating a Linux VM while the package mirror IS running, then stopping the package mirror, does not cause an issue.

---

I've amended our TRE start runbook script to start the package mirror if it exists.

@marrobi Would you like a PR to update the script on this page https://microsoft.github.io/AzureTRE/v0.16.0/tre-admins/start-stop/ ?

    # Start package mirror (if exists)
    $PackageMirror = Get-AzVM -ResourceGroupName $Group.ResourceGroupName -Name "nexus-$azureTreId" -ErrorVariable packageMirrorNotPresent -ErrorAction SilentlyContinue
    if (!$packageMirrorNotPresent)
    {
	    Write-Output "Starting Nexus Package Mirror VM '$($PackageMirror.Name)'"
	    Start-AzVM -ResourceGroupName $Group.ResourceGroupName -Name $PackageMirror.Name
    }

[marrobi]

Ah, yes, or might be best to start all VMs, in core, that's what the make commands do:

  az vm list -d --resource-group "${core_rg_name}" --query "[?powerState!='VM running'].name" -o tsv |
  while read -r vm_name; do
    echo "Starting VM ${vm_name}"
    az vm start --resource-group "${core_rg_name}" --name "${vm_name}" &
  done

[jonnyry]

Actually I would say I prefer that the workspace VMs to remain stopped, and started on demand since it reduces cost, and it's easy to start the workspace VMs from TRE portal with minimal privileges (unlike the package mirror VM).

[marrobi]

@jonnyry they are outside of the core resource group hence would not be started. I'm suggesting starting all VMs in the core RG. Which at the moment includes shared services.

Albeit I'm not convinced shared services should be in the core resource group long term :)

[jonnyry]

Oh I see right - yes OK, that makes sense. Let me take a look at the PowerShell code again.

[jonnyry]

Here's the more generic version to start all VMs in the core resource group, rather than just the nexus package mirror:

    # Start VMs
    $VM = Get-AzVM -ResourceGroupName $Group.ResourceGroupName
    foreach ($item in $VM) {
      Write-Output "Starting VM '$($item.Name)'"
      Start-AzVm -ResourceGroupName $item.ResourceGroupName -Name $item.Name
    }

I'll put together a PR to update the start script in the docs.