schema: add XS 8.3, 8.4 hypervisor capabilities

[rohityadavcloud]

This aims to add support for XenServer 8.4 and XCP-ng 8.3.

https://www.xenserver.com/downloads
https://mirrors.xcp-ng.org/isos/8.3/xcp-ng-8.3.0.iso?https=1

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI
• test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

How Has This Been Tested?

Testing by developing support for hypervisor template with mbx, also later on with Trillian.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 15.98%. Comparing base (2d00933) to head (31123e3).
Report is 53 commits behind head on 4.20.

Additional details and impacted files

@@            Coverage Diff            @@
##               4.20   #10483   +/-   ##
=========================================
  Coverage     15.98%   15.98%           
- Complexity    13085    13086    +1     
=========================================
  Files          5649     5649           
  Lines        495739   495739           
  Branches      60021    60021           
=========================================
  Hits          79252    79252           
  Misses       407633   407633           
  Partials       8854     8854           

Flag	Coverage Δ
uitests	4.01% <ø> (ø)
unittests	16.82% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[DaanHoogland]

this is double to #10470

[rohityadavcloud]

@DaanHoogland my bad didn't know you were working on that, I had this started as roadmap work;

Currently I've added support for the missing guest OSs and built mbx templates and verified an env is created with mbx:
https://download.cloudstack.org/templates/mbx/?new

mbx users can do a git pull and then call mbx init to refresh new templates - https://github.com/shapeblue/mbx

xcp-ng 8.3 mbx vm:

XS 8.4 mbx vm:

Currently the following issues are observed when adding the xcp or xs hosts in the zone:

1. For XenServer 8.4.0, I see this in mgmt server logs:

2025-02-28 08:08:13,845 WARN  [c.c.h.x.d.XcpServerDiscoverer] (ApiServer-10:[ctx-02a7c0a1, ctx-fe2ae55b]) (logid:03343254) Unable to setup agent 1 due to callHostPlugin failed for cmd: setIptables with args  due to There was a failure communicating with the plugin.

2. For XCP-ng 8.3, I see this in mgmt server logs:

2025-02-28 08:08:51,858 INFO  [c.c.r.ResourceManagerImpl] (ApiServer-10:[ctx-3f036c76, ctx-8ba28abd]) (logid:9b4870b5) Trying to add a new host at http://172.20.0.117 in data center Zone {"id": "1", "name": "DC", "uuid": "6495fda0-dc6e-4404-823c-cf7d5130049e"}
2025-02-28 08:08:51,955 WARN  [c.c.h.x.d.XcpServerDiscoverer] (ApiServer-10:[ctx-3f036c76, ctx-8ba28abd]) (logid:9b4870b5) Xml Rpc Exception org.apache.xmlrpc.client.XmlRpcClientException: Failed to parse server's response: Duplicate name: restrict_vtpm

... which seems to be discussed here https://xcp-ng.org/forum/topic/9839/java-sdk-xmlrpcclientexception-failed-to-parse-server-s-response-duplicate-name-restrict_vtpm/2?_=1740730465521 and other related thread on xcp-ng forums.

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[DaanHoogland]

np @rohityadavcloud (just hate doing double work ;)

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12615

[rohityadavcloud]

@blueorangutan help

[blueorangutan]

@rohityadavcloud [SL] I understand these words: "help", "hello", "thanks", "package", "test"
Test command usage: test [mgmt os] [hypervisor] [keepEnv] [qemuEv] [basicZone|securityGroups]
Mgmt OS options: ['ol8', 'ol9', 'debian12', 'rocky8', 'alma9', 'suse15', 'centos7', 'centos6', 'alma8', 'ubuntu18', 'ubuntu22', 'ubuntu20', 'ubuntu24']
Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-rocky8', 'kvm-ol8', 'kvm-ol9', 'kvm-alma8', 'kvm-alma9', 'kvm-ubuntu18', 'kvm-ubuntu20', 'kvm-ubuntu22', 'kvm-ubuntu24', 'kvm-debian12', 'kvm-suse15', 'vmware-55u3', 'vmware-60u2', 'vmware-65u2', 'vmware-67u3', 'vmware-70u1', 'vmware-70u2', 'vmware-70u3', 'vmware-80', 'vmware-80u1', 'vmware-80u2', 'vmware-80u3', 'xenserver-65sp1', 'xenserver-71', 'xenserver-74', 'xcpng74', 'xcpng76', 'xcpng80', 'xcpng81', 'xcpng82']
Note: when keepEnv is passed, you need to specify mgmt server os and hypervisor or use the matrix command.
when qemuEv is passed, it will deploy KVM hyperviosr hosts with qemu-kvm-ev, else it will default to stock qemu.
When basicZone and/or securityGroups are passed it will create a zone of the last type specified (default is Advanced)
Package command usage: package [all(default value),kvm,xen,vmware,hyperv,ovm] - a comma separated list can be passed with package command to bundle the required hypervisor's systemVM templates. Not passing any argument will bundle all - kvm,xen and vmware templates.

Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'shwstppr', 'vishesh92', 'Pearl1594', 'harikrishna-patnala', 'nvazquez', 'DaanHoogland', 'weizhouapache', 'borisstoyanov', 'vladimirpetrov', 'kiranchavala', 'andrijapanicsb', 'NuxRo', 'rajujith', 'alexandremattioli', 'sureshanaparti', 'abh1sar', 'sudo87']

[rohityadavcloud]

@blueorangutan test ol8 xcpng82

[blueorangutan]

@rohityadavcloud a [SL] Trillian-Jenkins test job (ol8 mgmt + xcpng82) has been kicked to run smoke tests

[rohityadavcloud]

@blueorangutan test ol8 xcpng82

[blueorangutan]

@rohityadavcloud a [SL] Trillian-Jenkins test job (ol8 mgmt + xcpng82) has been kicked to run smoke tests

[Pearl1594]

@blueorangutan test ol8 xenserver-84

[blueorangutan]

@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + xenserver-84) has been kicked to run smoke tests

[Pearl1594]

@blueorangutan test ol8 xcpng83

[blueorangutan]

@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + xcpng83) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian Build Failed (tid-12772)

[Pearl1594]

@blueorangutan test ol8 xenserver-84 keepEnv

[Pearl1594]

@rohityadavcloud adding XCP hosts tend to fail with the following exception :

2025-03-19 13:34:58,727 INFO  [c.c.r.ResourceManagerImpl] (qtp1390913202-18:[ctx-4be47ca3, ctx-752e7a77]) (logid:0a02de25) Trying to add a new host at http://10.0.32.176 in data center Zone {"id": "1", "name": "pr10483-t12773-xcpng83", "uuid": "dddf459a-8a67-4b06-8c69-7430b694a12a"}
2025-03-19 13:34:58,783 WARN  [c.c.h.x.d.XcpServerDiscoverer] (qtp1390913202-18:[ctx-4be47ca3, ctx-752e7a77]) (logid:0a02de25) Xml Rpc Exception org.apache.xmlrpc.client.XmlRpcClientException: Failed to parse server's response: Duplicate name: restrict_vtpm
	at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:188)
	at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:156)
	at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143)
	at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:69)
	at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56)
	at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167)
	at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137)
	at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126)
	at com.xensource.xenapi.Connection.dispatch(Connection.java:330)
	at com.xensource.xenapi.Host.getAllRecords(Host.java:3251)
	at com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer.find(XcpServerDiscoverer.java:236)
	at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:829)

Seems similar to https://xcp-ng.org/forum/topic/9839/java-sdk-xmlrpcclientexception-failed-to-parse-server-s-response-duplicate-name-restrict_vtpm/2

Edit: Noticed comment from earlier - seems like you hit the same issue - in that case I don't think we should add this hypervisor verison.
Xenserver 8.4 hosts also don't get added succesfully. If they do they are in alert state.

[blueorangutan]

[SF] Trillian Build Failed (tid-12783)

[rohityadavcloud]

Hi @olivierlambert we're trying to add support for newer XCP-ng/XS to ACS. Any advise for us, are there any sdk/client changes in XCP-ng 8.3 (or XS 8.4)? And, is this something we should address (client-wise) or will you guys be working to maintain backward compatibility for sdk/client usage (such as in the ACS hypervisor plugin)?

[weizhouapache]

Hi @olivierlambert we're trying to add support for newer XCP-ng/XS to ACS. Any advise for us, are there any sdk/client changes in XCP-ng 8.3 (or XS 8.4)? And, is this something we should address (client-wise) or will you guys be working to maintain backward compatibility for sdk/client usage (such as in the ACS hypervisor plugin)?

I created a draft PR #10644 to use a different library for xen/xcp-ng
if there is no regression, we can merge into this PR

[olivierlambert]

@rohityadavcloud Hi! Thanks for the PR. Sadly, IDK which SDK you are referring to. It seems like you are talking about the Java one? So far, at Vates, we only own/maintain a Go SDK, soon a Python SDK but no Java SDK. So as @weizhouapache did, using the existing one from Citrix is probably the right move 👍 (I'm confident that we are almost 100% compatible with it).

[alsko-icom]

Hello everyone, any news on the xcp-ng 8.3? I plan to test it under a new cloudstack 4.20 and was wondering if there is any progress to the error logo mentioned above? The suggestion offered in the xcp forum that you mentioned above is not clear for sysadmins https://xcp-ng.org/forum/topic/9839/java-sdk-xmlrpcclientexception-failed-to-parse-server-s-response-duplicate-name-restrict_vtpm/6?_=1740730465521

[rohityadavcloud]

Thanks for advising @olivierlambert our @Pearl1594 (and Wei/Hari) will continue the work forward. Hopefully we should be able to support both XCP-ng 8.3 & XS8.4 in the next release.

@alsko-icom we're working on it, like you've found this PR - it's not functioning well due to dependency & python-version changes. We're working on it with #10684

@Pearl1594 @weizhouapache @harikrishna-patnala closing this PR, let's continue with #10684