Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User Resource Ubuntu update #3893

Closed
wants to merge 65 commits into from
Closed
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
2716749
Ubuntu update (#1)
wojciechcloudkubed Apr 2, 2024
c92494e
Merge branch 'main' into ubuntu-update
Apr 4, 2024
7aa64ba
update
Apr 9, 2024
33d14be
update image options
Apr 9, 2024
30ecfa2
add in image to enum
Apr 9, 2024
072cfdc
add in microsoft-apt/ubuntu/20.04 to source-list
Apr 9, 2024
19316a1
update install UI
Apr 10, 2024
18df98c
update version
Apr 10, 2024
c7ebc5b
update linux vms
Apr 10, 2024
9c2211b
update core
Apr 10, 2024
d1beb03
updaye
Apr 10, 2024
1ea1159
update
Apr 10, 2024
271c7e0
update pypi source
Apr 10, 2024
a09af34
update apt-source-list
Apr 10, 2024
bc414b7
update config
Apr 11, 2024
af50b39
update apt-get
Apr 11, 2024
d4a57a3
update Readme for user resources
Apr 11, 2024
d75e442
update to gen2
Apr 11, 2024
3b7522e
version for guac
Apr 11, 2024
3a9eecb
testing
Apr 11, 2024
9fc272b
Merge branch 'main' into ubuntu-update
Apr 11, 2024
875f2f4
Ubuntu update (#1)
wojciechcloudkubed Apr 2, 2024
a3d9109
update readme and changelog
Apr 11, 2024
3a30d23
ds install_ui: false
Apr 11, 2024
be7212b
update version
Apr 11, 2024
8ff7f69
update script
Apr 11, 2024
7db0e5f
update ds install_ui: true
Apr 11, 2024
7a43e65
update vm config
Apr 11, 2024
ce4dc82
update version
Apr 11, 2024
1049390
remove moby-tini
Apr 11, 2024
c08a2b7
update version
Apr 11, 2024
e06dbcd
moby-tini || true
Apr 11, 2024
39aa284
resolving comments
Apr 12, 2024
aa4b948
Update Resource processor Ubuntu Image (#3902)
Apr 12, 2024
9af4f05
Ubuntu update (#1)
wojciechcloudkubed Apr 2, 2024
5e2cdb5
update readme and changelog
Apr 11, 2024
d8fa5d4
Update Resource processor Ubuntu Image (#3902)
Apr 12, 2024
b17bfab
Ubuntu update (#1)
wojciechcloudkubed Apr 2, 2024
85cdb98
update readme and changelog
Apr 11, 2024
f3b4efb
Update Resource processor Ubuntu Image (#3902)
Apr 12, 2024
e8ba5fd
update readme and changelog
Apr 11, 2024
e193943
changelog update
Apr 12, 2024
4a2036b
update resource proccesor
Apr 12, 2024
9a28c42
reset
Apr 12, 2024
b1bb5c9
Merge branch 'main' into ubuntu-update
Apr 12, 2024
c5ce11b
update config
Apr 12, 2024
86909a3
vm-config
Apr 12, 2024
81801f8
update version
Apr 12, 2024
ba9ae50
update
Apr 12, 2024
eff7050
edit
Apr 12, 2024
cd1a4db
updaet vm config
Apr 12, 2024
2082559
update to gen2
Apr 15, 2024
aa4713c
update image version
Apr 15, 2024
d5a4045
update
Apr 15, 2024
f9d9ffc
update
Apr 15, 2024
a3e5492
update
Apr 15, 2024
6980ecb
update
Apr 15, 2024
4354ca0
format
Apr 16, 2024
770e4c7
fix linting
Apr 16, 2024
7a4d250
update linux
Apr 18, 2024
943e07b
add in ms download and other apps
Apr 23, 2024
760f783
updat
Apr 23, 2024
c4e7094
update
Apr 25, 2024
060b6fd
vm update
Apr 25, 2024
f1d68da
update
Apr 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ FEATURES:
ENHANCEMENTS:

BUG FIXES:

* Update Guacomole Linux VM Images to 22.04 and 20.04 Data Science VM. Part of ([#3523](https://github.com/microsoft/AzureTRE/issues/3523))

COMPONENTS:

## 0.17.0
Expand Down
4 changes: 2 additions & 2 deletions core/terraform/resource_processor/vmss_porter/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ resource "azurerm_linux_virtual_machine_scale_set" "vm_linux" {

source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "18.04-LTS"
offer = "0001-com-ubuntu-server-jammy"
sku = "22_04-lts"
version = "latest"
}

Expand Down
2 changes: 1 addition & 1 deletion e2e_tests/test_performance.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ async def test_bulk_updates_to_ensure_each_resource_updated_in_series(verify) ->
"properties": {
"display_name": "Perf test VM",
"description": "",
"os_image": "Ubuntu 18.04"
"os_image": "Ubuntu 22.04 LTS"
}
}

Expand Down
3 changes: 2 additions & 1 deletion templates/shared_services/sonatype-nexus-vm/terraform/vm.tf
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ resource "azurerm_user_assigned_identity" "nexus_msi" {
location = data.azurerm_resource_group.rg.location
resource_group_name = local.core_resource_group_name
tags = local.tre_shared_service_tags

lifecycle { ignore_changes = [tags] }
}

Expand Down Expand Up @@ -112,7 +113,7 @@ resource "azurerm_linux_virtual_machine" "nexus" {
source_image_reference {
publisher = "Canonical"
offer = "0001-com-ubuntu-server-jammy"
sku = "22_04-lts-gen2"
sku = "22_04-lts"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for not using gen2?

version = "latest"
}

Expand Down
27 changes: 16 additions & 11 deletions templates/workspace_services/guacamole/user_resources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@

This folder contains user resources that can be deployed with the Guacamole workspace service:

- linuxvm - a Linux-based virtual machine (expects an Ubuntu 18.04-based VM)
- linuxvm - a Linux-based virtual machine
- windowsvm - A Windows-based virtual machine


## Customising the user resources

The `guacamole-azure-linuxvm` and `guacamole-azure-windowsvm` folders follow a consistent layout.
Expand All @@ -29,21 +28,29 @@ custom:
"8 CPU | 32GB RAM": Standard_D8s_v5
"16 CPU | 64GB RAM": Standard_D16s_v5
image_options:
"Ubuntu 18.04":
"Ubuntu 22.04 LTS":
source_image_reference:
publisher: canonical
offer: ubuntuserver
sku: 18_04-lts-gen2
offer: 0001-com-ubuntu-server-jammy
sku: 22_04-lts-gen2
version: latest
install_ui: true
conda_config: false
"Ubuntu 18.04 Data Science VM":
"Ubuntu 20.04 LTS":
source_image_reference:
publisher: canonical
offer: 0001-com-ubuntu-server-focal
sku: 20_04-lts-gen2
version: latest
install_ui: true
conda_config: false
"Ubuntu 20.04 LTS Data Science VM":
source_image_reference:
publisher: microsoft-dsvm
offer: ubuntu-1804
sku: 1804-gen2
offer: ubuntu-2004
sku: 2004-gen2
version: latest
install_ui: false
install_ui: true
conda_config: true
# "Custom Image From Gallery":
# source_image_name: your-image
Expand All @@ -68,8 +75,6 @@ When specifying images using `source_image_name`, the image must be stored in an
To enable re-using built user resource templates across environments where the image may vary, the image gallery is configured via the `RP_BUNDLE_VALUES` environment variable when deploying the TRE.
The `RP_BUNDLE_VALUES` variable is a JSON object, and the `image_gallery_id` property within it identifies the image gallery that contains the images specified by `source_image_name`:


```bash
RP_BUNDLE_VALUES='{"image_gallery_id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/<your-rg>/providers/Microsoft.Compute/galleries/<your-gallery-name>"}
```

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
schemaVersion: 1.0.0
name: tre-service-guacamole-linuxvm
version: 0.6.9
version: 0.6.17
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if semantic versioning is used for these bundles, but it feels like this should be at least a minor version increase, if not a major.
(E.g., how will the Update command work?)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed will change then to 1.0.0 when I've finished testing

description: "An Azure TRE User Resource Template for Guacamole (Linux)"
dockerfile: Dockerfile.tmpl
registry: azuretre
Expand All @@ -14,29 +14,39 @@ custom:
"8 CPU | 32GB RAM": Standard_D8s_v5
"16 CPU | 64GB RAM": Standard_D16s_v5
image_options:
"Ubuntu 18.04":
"Ubuntu 22.04 LTS":
source_image_reference:
publisher: canonical
offer: ubuntuserver
sku: 18_04-lts-gen2
offer: 0001-com-ubuntu-server-jammy
sku: 22_04-lts-gen2
version: latest
apt-sku: 22.04
install_ui: true
conda_config: false
"Ubuntu 18.04 Data Science VM":
"Ubuntu 20.04 LTS":
source_image_reference:
publisher: canonical
offer: 0001-com-ubuntu-server-focal
sku: 20_04-lts-gen2
version: latest
apt-sku: 20.04
install_ui: true
conda_config: false
"Ubuntu 20.04 LTS Data Science VM":
source_image_reference:
publisher: microsoft-dsvm
offer: ubuntu-1804
sku: 1804-gen2
offer: ubuntu-2004
sku: 2004-gen2
version: latest
install_ui: false
apt-sku: 20.04
install_ui: true
conda_config: true
# For information on using custom images, see README.me in the guacamole/user-resources folder
# "Custom Image From Gallery":
# source_image_name: your-image
# install_ui: true
# conda_config: true


credentials:
- name: azure_tenant_id
env: ARM_TENANT_ID
Expand Down Expand Up @@ -91,7 +101,7 @@ parameters:
default: "public"
- name: os_image
type: string
default: "Ubuntu 18.04 Data Science VM"
default: "Ubuntu 22.04 LTS Data Science VM"
- name: vm_size
type: string
default: "2 CPU | 8GB RAM"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@
"title": "Linux image",
"description": "Select Linux image to use for VM",
"enum": [
"Ubuntu 18.04",
"Ubuntu 18.04 Data Science VM"
"Ubuntu 22.04 LTS",
"Ubuntu 20.04 LTS",
"Ubuntu 20.04 LTS Data Science VM"
]
},
"vm_size": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ apt:
deb [trusted=yes] $PRIMARY $RELEASE main restricted universe multiverse
deb [trusted=yes] $PRIMARY $RELEASE-updates main restricted universe multiverse
deb [trusted=yes] $SECURITY $RELEASE main restricted universe multiverse
deb [signed-by=/etc/apt/trusted.gpg.d/microsoft.gpg] ${nexus_proxy_url}/repository/microsoft-apt/ubuntu/18.04/prod $RELEASE main
deb [signed-by=/etc/apt/trusted.gpg.d/microsoft.gpg] ${nexus_proxy_url}/repository/microsoft-apt/ubuntu/${sku}/prod $RELEASE main
deb [signed-by=/etc/apt/trusted.gpg.d/microsoft.gpg] ${nexus_proxy_url}/repository/microsoft-apt/repos/edge stable main
deb [signed-by=/etc/apt/trusted.gpg.d/docker-archive-keyring.gpg] ${nexus_proxy_url}/repository/docker/ $RELEASE stable
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ data "template_file" "apt_sources_config" {
template = file("${path.module}/apt_sources_config.yml")
vars = {
nexus_proxy_url = local.nexus_proxy_url
sku = local.apt_sku
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ locals {
tre_user_resource_id = var.tre_resource_id
}
nexus_proxy_url = "https://nexus-${data.azurerm_public_ip.app_gateway_ip.fqdn}"

# Load VM SKU/image details from porter.yaml
porter_yaml = yamldecode(file("${path.module}/../porter.yaml"))
vm_sizes = local.porter_yaml["custom"]["vm_sizes"]
Expand All @@ -26,4 +25,5 @@ locals {
# selected_image_source_refs is an array to enable easy use of a dynamic block
selected_image_source_refs = lookup(local.selected_image, "source_image_reference", null) == null ? [] : [local.selected_image.source_image_reference]
selected_image_source_id = lookup(local.selected_image, "source_image_name", null) == null ? null : "${var.image_gallery_id}/images/${local.selected_image.source_image_name}"
apt_sku = local.selected_image_source_refs[0]["apt-sku"]
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
sudo tee /etc/pip.conf > dev/null <<'EOF'
sudo tee /etc/pip.conf > /dev/null <<'EOF'
[global]
index = ${nexus_proxy_url}/repository/pypi/pypi
index-url = ${nexus_proxy_url}/repository/pypi/simple
Expand Down
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my experience, this file required additional changes for the Ubuntu 20 DSVM to work correctly. Specifically, the line to set the xcfe session config file had to be executed in the context of the user that will be logging in.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting. so i wonder if running something like this from within the bootstrap will work

su - username -c "command;command;command"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it does. This is what my line of code looks like (I am not a Linux expert, took my several tries to get it right. There might be a better way):
sudo -u ${USERNAME} -i bash -c 'echo xfce4-session > ~/.xsession'

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, noting that I had to do this even for the DSVM marketplace image where the UI is already installed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SvenAelterman could you possibly share your bootstrap script please?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but be aware I modified it only for the DSVM Ubuntu 20 image. I did not get around to updating for plain Ubuntu 20/22.
Also, some of the commented out lines at the bottom might be necessary but I was just trying to get the basics to work.
https://gist.github.com/SvenAelterman/4771b6accf1fe82ac782e5c0e00efa0b

Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ fi

# Docker install and config
sudo apt-get install -y ca-certificates curl gnupg lsb-release
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin jq
sudo apt-get install -y docker-compose-plugin docker-ce-cli containerd.io jq
sudo apt-get install -y docker-ce
jq -n --arg proxy "${NEXUS_PROXY_URL}:8083" '{"registry-mirrors": [$proxy]}' > /etc/docker/daemon.json
sudo systemctl daemon-reload
sudo systemctl restart docker
Expand Down
Loading