guestos: add el9 distros by shwstppr · Pull Request #7059 · apache/cloudstack

shwstppr · 2023-01-05T12:32:04Z

Description

Adds EL9 distros as guest OS for supported hypervisors

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

codecov · 2023-01-05T15:54:23Z

Codecov Report

Merging #7059 (eaa9fde) into main (5c20e78) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main    #7059   +/-   ##
=========================================
  Coverage     12.63%   12.63%           
  Complexity     8610     8610           
=========================================
  Files          2711     2711           
  Lines        255748   255748           
  Branches      39871    39871           
=========================================
  Hits          32312    32312           
  Misses       219337   219337           
  Partials       4099     4099

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

yadvr · 2023-01-06T06:46:30Z


 CALL `cloud`.`IDEMPOTENT_ADD_KEY`('i_user_ip_address_state','user_ip_address', '(state)');
+
+CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'AlmaLinux 9.0', 'KVM', 'default', 'AlmaLinux 9.0');


Leave it just 9, so in future we need not add for all 9.0, 9.1, 9.2…. Also should we find and add guest os mappings for other hypervisors? (Could be a separate issue or PR)

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2023-01-09T09:15:13Z

+CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'Red Hat Enterprise Linux 9', 'KVM', 'default', 'Red Hat Enterprise Linux 9');
+CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'Rocky Linux 9', 'KVM', 'default', 'Rocky Linux 9');
+
+CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'AlmaLinux 9', 'VMware', '7.0.3.0', 'otherLinux64Guest');


@rohityadavcloud For vmware, should we add otherLinux64Guest for OS which doesn't return ID? I tried to log guest OS descriptors for VMware 7u3 and can't find some of the OSes.
vmware-guestos-descriptors.txt

But could see them listed here, https://www.vmware.com/resources/compatibility/search.php?deviceCategory=software

For XCP-ng/Xen, I don't see any EL9 distro here, https://docs.citrix.com/en-us/citrix-hypervisor/system-requirements/guest-os-support.html

LGTM @shwstppr can we move this to top of the sql (or move the sql related down) - where we add support for vSphere 8; so that these mappings are also copied for vSphere 8.

github-actions · 2023-01-11T06:55:20Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions · 2023-01-20T09:10:48Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

shwstppr · 2023-01-25T08:21:38Z

@blueorangutan package

blueorangutan · 2023-01-25T08:22:03Z

@shwstppr a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

shwstppr · 2023-01-25T08:25:02Z

@blueorangutan help

blueorangutan · 2023-01-25T08:26:03Z

@shwstppr I understand these words: "help", "hello", "thanks", "package", "test"
Test command usage: test [mgmt os] [hypervisor] [keepEnv] [qemuEv]
Mgmt OS options: ['centos7', 'centos6', 'suse15', 'alma8', 'ubuntu18', 'ubuntu22', 'ubuntu20', 'rocky8']
Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-rocky8', 'kvm-alma8', 'kvm-ubuntu18', 'kvm-ubuntu20', 'kvm-ubuntu22', 'kvm-suse15', 'vmware-55u3', 'vmware-60u2', 'vmware-65u2', 'vmware-67u3', 'vmware-70u1', 'vmware-70u2', 'vmware-70u3', 'xenserver-65sp1', 'xenserver-71', 'xenserver-74', 'xcpng74', 'xcpng76', 'xcpng80', 'xcpng81', 'xcpng82']
Note: when keepEnv is passed, you need to specify mgmt server os and hypervisor or use the matrix command.
when qemuEv is passed, it will deploy KVM hyperviosr hosts with qemu-kvm-ev, else it will default to stock qemu.
Package command usage: package [all(default value),kvm,xen,vmware,hyperv,ovm] - a comma separated list can be passed with package command to bundle the required hypervisor's systemVM templates. Not passing any argument will bundle all - kvm,xen and vmware templates.

Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'nvazquez', 'borisstoyanov', 'DaanHoogland', 'shwstppr', 'andrijapanicsb', 'rahulbcn27', 'davidjumani', 'harikrishna-patnala', 'vladimirpetrov', 'weizhouapache', 'NuxRo', 'mlsorensen']

yadvr

Lgtm, needs manual QA or verification though plus smoketests on supported hypervisors

blueorangutan · 2023-01-25T09:16:27Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5393

github-actions · 2023-01-27T10:24:47Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

shwstppr · 2023-01-27T10:41:13Z

@blueorangutan package

blueorangutan · 2023-01-27T10:42:03Z

@shwstppr a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-01-27T11:38:06Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5421

vladimirpetrov · 2023-01-27T16:30:18Z

I tested with the following images:

https://yum.oracle.com/ISOS/OracleLinux/OL9/u0/x86_64/OracleLinux-R9-U0-x86_64-boot.iso
https://download.rockylinux.org/pub/rocky/9/isos/x86_64/Rocky-9.1-x86_64-minimal.iso
http://mirrors.storpool.com/almalinux/9.0/isos/x86_64/AlmaLinux-9.0-x86_64-boot.iso

And got kernel panic on KVM (Rocky 8) and hanging VMs on VMWare 7.0u3.

shwstppr · 2023-01-30T09:57:07Z

@blueorangutan help

blueorangutan · 2023-01-30T09:58:03Z

@shwstppr I understand these words: "help", "hello", "thanks", "package", "test"
Test command usage: test [mgmt os] [hypervisor] [keepEnv] [qemuEv]
Mgmt OS options: ['centos7', 'centos6', 'suse15', 'alma8', 'ubuntu18', 'ubuntu22', 'ubuntu20', 'rocky8', 'alma9']
Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-rocky8', 'kvm-alma8', 'kvm-alma9', 'kvm-ubuntu18', 'kvm-ubuntu20', 'kvm-ubuntu22', 'kvm-suse15', 'vmware-55u3', 'vmware-60u2', 'vmware-65u2', 'vmware-67u3', 'vmware-70u1', 'vmware-70u2', 'vmware-70u3', 'xenserver-65sp1', 'xenserver-71', 'xenserver-74', 'xcpng74', 'xcpng76', 'xcpng80', 'xcpng81', 'xcpng82']
Note: when keepEnv is passed, you need to specify mgmt server os and hypervisor or use the matrix command.
when qemuEv is passed, it will deploy KVM hyperviosr hosts with qemu-kvm-ev, else it will default to stock qemu.
Package command usage: package [all(default value),kvm,xen,vmware,hyperv,ovm] - a comma separated list can be passed with package command to bundle the required hypervisor's systemVM templates. Not passing any argument will bundle all - kvm,xen and vmware templates.

Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'nvazquez', 'borisstoyanov', 'DaanHoogland', 'shwstppr', 'andrijapanicsb', 'rahulbcn27', 'davidjumani', 'harikrishna-patnala', 'vladimirpetrov', 'weizhouapache', 'NuxRo', 'mlsorensen']

blueorangutan · 2023-02-01T02:36:32Z

Trillian test result (tid-6051)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41071 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7059-t6051-kvm-centos7.zip
Smoke tests completed. 107 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

shwstppr · 2023-02-01T12:24:51Z

@blueorangutan test rocky8 vmware-67u3

blueorangutan · 2023-02-01T12:26:03Z

@shwstppr a Trillian-Jenkins test job (rocky8 mgmt + vmware-67u3) has been kicked to run smoke tests

DaanHoogland

sqllgtm

blueorangutan · 2023-02-02T01:10:33Z

Trillian test result (tid-6066)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 44553 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7059-t6066-vmware-67u3.zip
Smoke tests completed. 107 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

weizhouapache

code lgtm

@shwstppr do you know if rl9 is supported in other vmware 7 minor versions (7.0, 7.0.1.0, 7.0.2.0) ?

shwstppr · 2023-02-02T08:14:50Z

@weizhouapache looks like they are supported from 7.0 onwards,
https://www.vmware.com/resources/compatibility/search.php?deviceCategory=software&details=1&partner=1192&releases=448&page=1&display_interval=10&sortColumn=Partner&sortOrder=Asc&testConfig=16

weizhouapache · 2023-02-02T08:26:47Z

@weizhouapache looks like they are supported from 7.0 onwards, https://www.vmware.com/resources/compatibility/search.php?deviceCategory=software&details=1&partner=1192&releases=448&page=1&display_interval=10&sortColumn=Partner&sortOrder=Asc&testConfig=16

@shwstppr thanks Ahbishek.

do we need to support other vmware 7.0.x.0 ?

…00.sql

shwstppr · 2023-02-03T06:02:01Z

@blueorangutan package

blueorangutan · 2023-02-03T06:03:04Z

@shwstppr a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-02-03T07:06:55Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5492

github-actions · 2023-02-03T07:44:10Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland · 2023-02-03T08:34:16Z

@blueorangutan package

blueorangutan · 2023-02-03T08:35:03Z

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

DaanHoogland · 2023-02-03T09:11:35Z

I will regard packaging and GHA as sufficient proof of this PR. awaiting those.

blueorangutan · 2023-02-03T09:25:48Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5496

sonarqubecloud · 2023-02-03T09:50:55Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

el9 and variants were introduced by apache#7059 they are supported with guest os identifiers since VMware 8.0 see https://vdc-repo.vmware.com/vmwb-repository/dcr-public/c476b64b-c93c-4b21-9d76-be14da0148f9/04ca12ad-59b9-4e1c-8232-fd3d4276e52c/SDK/vsphere-ws/docs/ReferenceGuide/vim.vm.GuestOsDescriptor.GuestOsIdentifier.html

* VMware: add support for 8.0b (8.0.0.2) * VMware 8: add new guest os mappings in VirtualMachineGuestOsIdentifier The full list can be found at https://developer.vmware.com/apis/1355/vsphere * VMware: get guest os mappings of parent version * VMware8: remove guest os mappings for 8.0.0.2 * VMware8: fix code smells * vmware: remove annotations in VmwareVmImplementerTest which caused 0.0% code coverage * VMware8: add a unit test case * VMware: add support for 8.0c (8.0.0.3) * VMware8: move to CloudStackVersion.getVMwareParentVersion * VMware: add support for 8.0u1 (8.0.1.0) * Copy engine/schema/src/main/java/com/cloud/upgrade/GuestOsMapper.java from PR 6979 * Copy engine/schema/src/main/java/com/cloud/storage/dao/GuestOSHypervisorDao.java from PR 6979 * VMware: ignore the last number in VMware versions * VMware: copy guest os mapping from 8.0 to 8.0.1 * VMware: add unit tests in VmwareVmImplementerTest.java * Copy engine/schema/src/test/java/com/cloud/upgrade/GuestOsMapperTest.java from PR 6979 * VMware8: retry vm poweron if fails due to exception "File system specific implementation of Ioctl[file] failed" This fixes a weird issue on vmware8. When power on a vm, sometimes it fails due to error 2023-04-27 07:04:43,207 ERROR [c.c.h.v.r.VmwareResource] (DirectAgent-442:ctx-cdd42b03 10.0.32.133, job-105/job-106, cmd: StartCommand) (logid:8a24a607) StartCommand failed due to [Exception: java.lang.RuntimeException Message: File system specific implementation of Ioctl[file] failed ]. java.lang.RuntimeException: File system specific implementation of Ioctl[file] failed at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:426) at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.powerOn(VirtualMachineMO.java:288) in vmware.log on ESXi host, it shows 2023-04-27T09:20:41.713Z In(05)+ vmx - Power on failure messages: File system specific implementation of Ioctl[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed 2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to lock the file 2023-04-27T09:20:41.713Z In(05)+ vmx - Cannot open the disk '/vmfs/volumes/7b29c876-ac102328/i-2-167-VM/ROOT-167.vmdk' or one of the snapshot disks it depends on. 2023-04-27T09:20:41.713Z In(05)+ vmx - Module 'Disk' power on failed. 2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to start the virtual machine. There is a KB article for it, but I still do not know why and how to fix it. https://kb.vmware.com/s/article/1004232 * VMware: extract to method powerOnVM * vmware: fix mistake in logs * vmware8: use curl instead of wget to fix test failures Traceback (most recent call last): File "/root/test_internal_lb.py", line 555, in test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 self.execute_internallb_roundrobin_tests(vpc_offering) File "/root/test_internal_lb.py", line 641, in execute_internallb_roundrobin_tests client_vm, applb.sourceipaddress, max_http_requests) File "/root/test_internal_lb.py", line 497, in run_ssh_test_accross_hosts (e, clienthost.public_ip)) AssertionError: list index out of range: SSH failed for VM with IP Address: 10.0.52.187 and sshClient: DEBUG: {Cmd: /usr/bin/wget -T3 -qO- --user=admin --password=password http://10.1.2.253:8081/admin?stats via Host: 10.0.52.188} {returns: ["/usr/bin/wget: '/usr/lib/libpcre.so.1' is not an ELF file", "/usr/bin/wget: can't load library 'libpcre.so.1'"]} * VMware: correct guest OS names in hypervisor mappings for VMware 8.0 el9 and variants were introduced by #7059 they are supported with guest os identifiers since VMware 8.0 see https://vdc-repo.vmware.com/vmwb-repository/dcr-public/c476b64b-c93c-4b21-9d76-be14da0148f9/04ca12ad-59b9-4e1c-8232-fd3d4276e52c/SDK/vsphere-ws/docs/ReferenceGuide/vim.vm.GuestOsDescriptor.GuestOsIdentifier.html * VMware: add Ubuntu 20.04 and 22.04 support for vmware 7.0+ * PR7380: only add guest os mappings for Ubuntu 20.04 * PR7380: Correct RHEL9 guest os names and others for VMware 8.0 * PR7380: correct guest os names on 8.0.0.1 as well * PR7380: remove Windows 12 and Windows Server 2025 which are not released yet

guestos: add el9 distros

b4c7fb1

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

yadvr reviewed Jan 6, 2023

View reviewed changes

shwstppr added 2 commits January 9, 2023 13:50

fix

6824c38

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

vmware mappings

a458bf5

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr commented Jan 9, 2023

View reviewed changes

github-actions Bot added the status:has-conflicts label Jan 11, 2023

Merge branch 'main' into add-el8-guest-os

6f22457

github-actions Bot added status:has-conflicts and removed status:has-conflicts labels Jan 17, 2023

Merge branch 'main' into add-el8-guest-os

ffdfaa5

github-actions Bot removed the status:has-conflicts label Jan 24, 2023

yadvr approved these changes Jan 25, 2023

View reviewed changes

github-actions Bot added the status:has-conflicts label Jan 27, 2023

Merge branch 'main' into add-el8-guest-os

5043d1a

github-actions Bot removed the status:has-conflicts label Jan 27, 2023

yadvr requested review from DaanHoogland and weizhouapache February 1, 2023 08:48

DaanHoogland approved these changes Feb 1, 2023

View reviewed changes

weizhouapache approved these changes Feb 2, 2023

View reviewed changes

shwstppr commented Feb 2, 2023

View reviewed changes

Comment thread engine/schema/src/main/resources/META-INF/db/schema-41720to41800.sql

Update engine/schema/src/main/resources/META-INF/db/schema-41720to418…

ebaaa85

…00.sql

github-actions Bot added the status:has-conflicts label Feb 3, 2023

Merge branch 'main' into add-el8-guest-os

eaa9fde

github-actions Bot removed the status:has-conflicts label Feb 3, 2023

DaanHoogland added the status:ready-for-merge label Feb 3, 2023

DaanHoogland merged commit 6d28779 into apache:main Feb 3, 2023

weizhouapache mentioned this pull request Jul 27, 2023

Problem with guest os mappings for Red Hat Enterprise Linux 9 in ACS 4.18.0.0 #7781

Closed

DaanHoogland removed the status:ready-for-merge label Dec 4, 2024


		CALL `cloud`.`IDEMPOTENT_ADD_KEY`('i_user_ip_address_state','user_ip_address', '(state)');

		CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'AlmaLinux 9.0', 'KVM', 'default', 'AlmaLinux 9.0');

Uh oh!

Conversation

shwstppr commented Jan 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

codecov Bot commented Jan 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

yadvr Jan 6, 2023

Choose a reason for hiding this comment

Uh oh!

shwstppr Jan 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yadvr Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jan 11, 2023

Uh oh!

github-actions Bot commented Jan 20, 2023

Uh oh!

shwstppr commented Jan 25, 2023

Uh oh!

blueorangutan commented Jan 25, 2023

Uh oh!

shwstppr commented Jan 25, 2023

Uh oh!

blueorangutan commented Jan 25, 2023

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Jan 25, 2023

Uh oh!

github-actions Bot commented Jan 27, 2023

Uh oh!

shwstppr commented Jan 27, 2023

Uh oh!

blueorangutan commented Jan 27, 2023

Uh oh!

blueorangutan commented Jan 27, 2023

Uh oh!

vladimirpetrov commented Jan 27, 2023

Uh oh!

shwstppr commented Jan 30, 2023

Uh oh!

blueorangutan commented Jan 30, 2023

Uh oh!

blueorangutan commented Feb 1, 2023

Uh oh!

shwstppr commented Feb 1, 2023

Uh oh!

blueorangutan commented Feb 1, 2023

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Feb 2, 2023

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

shwstppr commented Feb 2, 2023

Uh oh!

weizhouapache commented Feb 2, 2023

Uh oh!

Uh oh!

shwstppr commented Feb 3, 2023

Uh oh!

blueorangutan commented Feb 3, 2023

Uh oh!

shwstppr commented Jan 5, 2023 •

edited

Loading

codecov Bot commented Jan 5, 2023 •

edited

Loading

shwstppr Jan 9, 2023 •

edited

Loading