provision-openssh.ps1 fails when run from autounattend

[velvetant]

I noticed OpenSSH not being installed even though the builds ran through fine. In the setupact.log everything seems to be fine. When run manually the script runs through without problems as well. I don't know how to get more verbose logging :/

My workaround so far is to comment it out in provision-autounattend.ps1 and run it as a packer provision step.
Edit: oh, that fails as well with curious reasons for doing so:

==> proxmox-iso.windows-10-21h2-amd64: Provisioning with powershell script: ../scripts/provision-openssh.ps1
2023/04/11 19:48:12 packer-provisioner-powershell plugin: Opening ../scripts/provision-openssh.ps1 for reading
2023/04/11 19:48:12 packer-provisioner-powershell plugin: Uploading env vars to c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1
2023/04/11 19:48:12 packer-provisioner-powershell plugin: [INFO] 118 bytes written for 'uploadData'
2023/04/11 19:48:12 [INFO] 118 bytes written for 'uploadData'
2023/04/11 19:48:12 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:48:12 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1'
2023/04/11 19:48:22 packer-provisioner-powershell plugin: [INFO] 6062 bytes written for 'uploadData'
2023/04/11 19:48:22 [INFO] 6062 bytes written for 'uploadData'
2023/04/11 19:48:22 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:48:22 Uploading file to 'c:/Windows/Temp/script-64359d5c-f051-c72a-b84f-495c81f719c3.ps1'
2023/04/11 19:48:57 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:48:57 [INFO] starting remote command: pwsh -executionpolicy bypass -command "& { if (Test-Path variable:global:ProgressPreference){set-variable -name variable:global:ProgressPreference -value 'SilentlyContinue'};. c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1; &'c:/Windows/Temp/script-64359d5c-f051-c72a-b84f-495c81f719c3.ps1'; exit $LastExitCode }"
2023/04/11 19:48:58 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:48:58 [INFO] command 'pwsh -executionpolicy bypass -command "& { if (Test-Path variable:global:ProgressPreference){set-variable -name variable:global:ProgressPreference -value 'SilentlyContinue'};. c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1; &'c:/Windows/Temp/script-64359d5c-f051-c72a-b84f-495c81f719c3.ps1'; exit $LastExitCode }"' exited with code: 0
2023/04/11 19:48:58 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:48:58 [INFO] RPC endpoint: Communicator ended with: 0
2023/04/11 19:48:58 [INFO] 0 bytes written for 'stdout'
2023/04/11 19:48:58 [INFO] 303 bytes written for 'stderr'
2023/04/11 19:48:58 [INFO] RPC client: Communicator ended with: 0
2023/04/11 19:48:58 [INFO] RPC endpoint: Communicator ended with: 0
2023/04/11 19:48:58 packer-provisioner-powershell plugin: [INFO] 0 bytes written for 'stdout'
2023/04/11 19:48:58 packer-provisioner-powershell plugin: [INFO] 303 bytes written for 'stderr'
2023/04/11 19:48:58 packer-provisioner-powershell plugin: [INFO] RPC client: Communicator ended with: 0
==> proxmox-iso.windows-10-21h2-amd64: .: The term 'c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1' is not recognized as a name of a cmdlet, function, script file, or executable program.
2023/04/11 19:48:58 packer-provisioner-powershell plugin: c:/Windows/Temp/script-64359d5c-f051-c72a-b84f-495c81f719c3.ps1 returned with exit code 0
==> proxmox-iso.windows-10-21h2-amd64: Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
2023/04/11 19:48:58 packer-provisioner-powershell plugin: [INFO] 511 bytes written for 'uploadData'
2023/04/11 19:48:58 [INFO] 511 bytes written for 'uploadData'
2023/04/11 19:48:58 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:48:58 Uploading file to 'c:/Windows/Temp/packer-cleanup-64359d5c-9dcd-971a-6265-5b6602982e70.ps1'
2023/04/11 19:49:04 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:49:04 [INFO] starting remote command: pwsh -executionpolicy bypass -command "& { if (Test-Path variable:global:ProgressPreference){set-variable -name variable:global:ProgressPreference -value 'SilentlyContinue'};. c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1; &'c:/Windows/Temp/packer-cleanup-64359d5c-9dcd-971a-6265-5b6602982e70.ps1'; exit $LastExitCode }"
2023/04/11 19:49:05 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:49:05 [INFO] command 'pwsh -executionpolicy bypass -command "& { if (Test-Path variable:global:ProgressPreference){set-variable -name variable:global:ProgressPreference -value 'SilentlyContinue'};. c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1; &'c:/Windows/Temp/packer-cleanup-64359d5c-9dcd-971a-6265-5b6602982e70.ps1'; exit $LastExitCode }"' exited with code: 0
2023/04/11 19:49:05 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/11 19:49:05 [INFO] RPC endpoint: Communicator ended with: 0
2023/04/11 19:49:05 [INFO] 0 bytes written for 'stdout'
2023/04/11 19:49:05 [INFO] 303 bytes written for 'stderr'
2023/04/11 19:49:05 [INFO] RPC client: Communicator ended with: 0
2023/04/11 19:49:05 [INFO] RPC endpoint: Communicator ended with: 0
2023/04/11 19:49:05 packer-provisioner-powershell plugin: [INFO] 0 bytes written for 'stdout'
2023/04/11 19:49:05 packer-provisioner-powershell plugin: [INFO] 303 bytes written for 'stderr'
2023/04/11 19:49:05 packer-provisioner-powershell plugin: [INFO] RPC client: Communicator ended with: 0
==> proxmox-iso.windows-10-21h2-amd64: .: The term 'c:/Windows/Temp/packer-ps-env-vars-64359d5c-af82-c4bb-31a9-195b1e873f36.ps1' is not recognized as a name of a cmdlet, function, script file, or executable program.
2023/04/11 19:49:05 [INFO] (telemetry) ending powershell
==> proxmox-iso.windows-10-21h2-amd64: Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
==> proxmox-iso.windows-10-21h2-amd64: Stopping VM

After switching on the debug flags, it shows the script silently exiting after

    proxmox-iso.windows-10-21h2-amd64: DEBUG:   70+  >>>> $windowsOpenSshCapabilities = Get-WindowsCapability -Online -Name 'OpenSSH.*' | Where-Object {$_.State -ne 'NotPresent'}
    proxmox-iso.windows-10-21h2-amd64: 
2023/04/12 00:13:00 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/12 00:13:00 [INFO] command 'pwsh -executionpolicy bypass -command "& { if (Test-Path variable:global:ProgressPreference){set-variable -name variable:global:ProgressPreference -value 'SilentlyContinue'};Set-PsDebug -Trace 1;. c:/Windows/Temp/packer-ps-env-vars-6435db5a-c007-eecc-99ad-e36c813e4adf.ps1; &'c:/Windows/Temp/script-6435db5a-586f-9950-b5e0-c9f1c873fcf8.ps1'; exit $LastExitCode }"' exited with code: 0
2023/04/12 00:13:00 packer-plugin-proxmox_v1.1.2_x5.0_linux_amd64 plugin: 2023/04/12 00:13:00 [INFO] RPC endpoint: Communicator ended with: 0
2023/04/12 00:13:00 [INFO] 4119 bytes written for 'stdout'
2023/04/12 00:13:00 [INFO] 0 bytes written for 'stderr'
2023/04/12 00:13:00 [INFO] RPC client: Communicator ended with: 0
2023/04/12 00:13:00 [INFO] RPC endpoint: Communicator ended with: 0
2023/04/12 00:13:00 packer-provisioner-powershell plugin: [INFO] 4119 bytes written for 'stdout'
2023/04/12 00:13:00 packer-provisioner-powershell plugin: [INFO] 0 bytes written for 'stderr'
2023/04/12 00:13:00 packer-provisioner-powershell plugin: [INFO] RPC client: Communicator ended with: 0
2023/04/12 00:13:00 packer-provisioner-powershell plugin: c:/Windows/Temp/script-6435db5a-586f-9950-b5e0-c9f1c873fcf8.ps1 returned with exit code 0
2023/04/12 00:13:00 [INFO] (telemetry) ending powershell
==> proxmox-iso.windows-10-21h2-amd64: Stopping VM

as said, manually, there are no hickups. I wonder which account actually runs the script, a) in the autoattend.xml b) as a packer provisioner step.

[rgl]

That's a strange error. It seems the provision-openssh.ps1 script is not even executed. When the script is executed, and there is an unknown error, it would print a stack trace and wait for 60m, which does not seem to be the case here.

I would try to execute a simple script, and go from there.

[velvetant]

Hey Rui, all the other scripts are working, and, at least in the provisioner step, the script gets successfully executed until the line mentioned above.

Edit: to add on to that, it works when lines 70-74 are commented out in provision-openssh.ps1 -- this is really weird!? that command works flawlessly in all the different user / privilege / shells - foo combinations, that I tried.

[rgl]

To confirm, if you are running that script from packer, which is using the ssh communicator, that might terminate the ssh session when it uninstalls the built in ssh server, which might explain why packer ends up failing. You should change the packer communicator to winrm.

If you are running that script from autounatted, I have no idea why that uninstall might be failing, maybe opening the vm console (which would have a desktop session open in the administrator account), and manually execute it there might give some clue.

The autounatend is running as the administrator user account. It runs in a auto logon desktop session which you can see if you connect to the launched packer vm.

[velvetant]

Hey Rui, thanks for your answers!

Packer's communicator has always been set to WinRM so far. Packer builds run through, that script just outputs a premature Exit Code 0, so to speak and thus never executes the steps after installing rsync.
In autounattend as well as a packer provisioning stop, when lines 70-74 are commented out, everything executes successfully, but old OpenSSH remains installed (obviously).
When I set only the uninstall part as a packer provisioning step (after the majority of the script has run in autounattend) it shows the same faulty behavior. Packer runs through successfully, but old OpenSSH remains installed with the script being shown as having successfully run.

Hope that was understandable.

[rgl]

There are extra limitations imposed on applications that execute from a remote session, so sometimes, one has to reshort to run the commands indirectly from a high privileged scheduled task (e.g. Run as system). So maybe that's why running from packer might be failing. But, I was expecting things to run without problems from autounatend because those command run locally as you would manually do, so that's quite puzzling to understand why it's failing.

Which windows 10 iso URL are you using? Maybe I can try to reproduce that problem here.

[velvetant]

I have been using win10_21h2_EnglishInternational_x64.iso with Windows 10 Pro as Edition, as well as 17763.737.190906-2324.rs5_release_svc_refresh_SERVERESSENTIALS_OEM_x64FRE_en-us_1.iso for Windows 2019 Server Essentials. Both installed without OpenSSH.

My (working) workaround thus far is: running provision-pwsh, -openssh and -psremoting in SynchronousCommands in the specialize pass in my Autounattend and provision-winrm as well as provision-vmtools in the oobe pass. I came to try this as I read, that oobe will be run as the user, while specialize will be run as SYSTEM - apparently openssh need SYSTEM permissions, while vmtools (though I added qemu-ga to that script) and winrm need user permissions.

Edit: which, heads up, results in the insecure vagrant public key being placed into: C:\Window\System32\config\systemprofile\.ssh\authorized_keys

[rgl]

Thanks for the clarifications.

Maybe, now, when OpenSSH is installed as a windows feature it does something differently than what is available at https://github.com/PowerShell/Win32-OpenSSH/releases. I say "now", because at some point, I've tried to remove the OpenSSH windows feature (using what is in this repo) and it worked, so, it seems something changed in the mean time.

Unfortunately, I do not use or have access the Windows versions/flavors that you've mentioned. I can only use the ones that are in this repository.

Disclaimer: Be aware that this repository is intended to be used to create a vagrant box, that's why some things (like adding the public vagrant key and default vagrant user/password) are configured to work as vagrant normally expects them. YMMV when you take them out-of-context.

[velvetant]

Well, fwiw, I could reproduce the behavior with the Win10 Evaluation 20H2 you are using in this repo. My autounattend is a tiny bit different, and I use proxmox as a builder, so the error must be lying somewhere with these.

[rgl]

Its indeed strange, but I'm afraid I cannot reproduce it. I've just tried Win10 Evaluation 20H2 and it worked here. It uninstalled the OpenSSH Windows feature without any problem. I'm doing this in a Ubuntu 22.04 with the kvm/libvirt builder.