Skip to content

Commit

Permalink
Merge branch 'stackhpc/2024.1' into cross-arch-builds-2024.1
Browse files Browse the repository at this point in the history
  • Loading branch information
bbezak authored Nov 18, 2024
2 parents 4844d72 + bc562dd commit 5931c4c
Show file tree
Hide file tree
Showing 28 changed files with 386 additions and 45 deletions.
22 changes: 6 additions & 16 deletions .github/workflows/stackhpc-all-in-one.yml
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ jobs:
VM_NETWORK: ${{ inputs.vm_network }}
VM_SUBNET: ${{ inputs.vm_subnet }}
VM_INTERFACE: ${{ inputs.vm_interface }}
VM_VOLUME_SIZE: ${{ inputs.upgrade && '55' || '40' }}
VM_VOLUME_SIZE: ${{ inputs.upgrade && '65' || '50' }}
VM_TAGS: '["skc-ci-aio", "PR=${{ github.event.number }}"]'

- name: Terraform Plan
Expand Down Expand Up @@ -214,22 +214,12 @@ jobs:
- name: Write Terraform network config
run: |
cat << EOF > etc/kayobe/environments/$KAYOBE_ENVIRONMENT/tf-networks.yml
admin_oc_net_name: admin
admin_cidr: "{{ access_cidr.value }}"
admin_allocation_pool_start: 0.0.0.0
admin_allocation_pool_end: 0.0.0.0
admin_gateway: "{{ access_gw.value }}"
admin_bootproto: dhcp
admin_ips:
admin_oc_net_name: ethernet
ethernet_cidr: "{{ access_cidr.value }}"
ethernet_allocation_pool_start: 0.0.0.0
ethernet_allocation_pool_end: 0.0.0.0
ethernet_ips:
controller0: "{{ access_ip_v4.value }}"
admin_zone: admin
EOF
- name: Write Terraform network interface config
run: |
cat << EOF > etc/kayobe/environments/$KAYOBE_ENVIRONMENT/inventory/group_vars/controllers/tf-network-interfaces
admin_interface: "{{ access_interface.value }}"
EOF
- name: Write all-in-one scenario config
Expand Down
34 changes: 34 additions & 0 deletions doc/source/operations/upgrading-openstack.rst
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,13 @@ to ``default``. Whilst this does not have any negative impact on services
that utilise Redis it will feature prominently in any preview of the overcloud
configuration.

AvailabilityZoneFilter removal
------------------------------

Support for the ``AvailabilityZoneFilter`` filter has been dropped in Nova.
Remove it from any Nova config files before upgrading. It will cause errors in
Caracal and halt the Nova scheduler.

Known issues
============

Expand All @@ -130,6 +137,24 @@ Known issues
around this in custom config, see the SMS PR for an example:
https://github.com/stackhpc/smslab-kayobe-config/pull/354

* Due to a `security-related change in the GRUB package on Rocky Linux 9
<https://access.redhat.com/security/cve/CVE-2023-4001>`__, the operating
system can become unbootable (boot will stop at a ``grub>`` prompt). Remove
the ``--root-dev-only`` option from ``/boot/efi/EFI/rocky/grub.cfg`` after
applying package updates. This will happen automatically as a post hook when
running the ``kayobe overcloud host package update`` command.

* After upgrading OpenSearch to the latest 2023.1 container image, we have seen
cluster routing allocation be disabled on some systems. See bug for details:
https://bugs.launchpad.net/kolla-ansible/+bug/2085943.
This will cause the "Perform a flush" handler to fail during the 2024.1
OpenSearch upgrade. To workaround this, you can run the following PUT request
to enable allocation again:

..code-block:: console

curl -X PUT "https://<kolla-vip>:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d '{ "transient" : { "cluster.routing.allocation.enable" : "all" } } '

Security baseline
=================

Expand Down Expand Up @@ -863,6 +888,15 @@ To update all eligible packages, use ``*``, escaping if necessary:
kayobe overcloud host package update --packages "*" --limit <host>
.. note::

Due to a `security-related change in the GRUB package on Rocky Linux 9
<https://access.redhat.com/security/cve/CVE-2023-4001>`__, the operating
system can become unbootable (boot will stop at a ``grub>`` prompt). Remove
the ``--root-dev-only`` option from ``/boot/efi/EFI/rocky/grub.cfg`` after
applying package updates. This will happen automatically as a post hook when
running the ``kayobe overcloud host package update`` command.

If the kernel has been upgraded, reboot the host or batch of hosts to pick up
the change:

Expand Down
1 change: 1 addition & 0 deletions etc/kayobe/ansible/advise-run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
name:
- git+https://github.com/stackhpc/ADVise
state: latest
virtualenv_command: "python3 -m venv"

- name: Create data directory
file:
Expand Down
151 changes: 151 additions & 0 deletions etc/kayobe/ansible/deploy-radosgw-usage-exporter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
---
- name: Deploy RADOS gateway usage exporter
hosts: monitoring
gather_facts: false
tags: radosgw_usage_exporter
vars:
venv: "{{ virtualenv_path }}/openstack"
tasks:
- name: Deploy RADOS gateway usage exporter
when: stackhpc_enable_radosgw_usage_exporter
block:
- name: Set up openstack cli virtualenv
ansible.builtin.pip:
virtualenv: "{{ venv }}"
virtualenv_command: "/usr/bin/python3 -m venv"
name:
- python-openstackclient
state: latest
extra_args: "{% if pip_upper_constraints_file %}-c {{ pip_upper_constraints_file }}{% endif %}"
run_once: true
delegate_to: "{{ groups['controllers'][0] }}"
vars:
ansible_host: "{{ hostvars[groups['controllers'][0]].ansible_host }}"

- name: Read admin-openrc credential file
ansible.builtin.command:
cmd: "cat {{ lookup('ansible.builtin.env', 'KOLLA_CONFIG_PATH') }}/admin-openrc.sh"
delegate_to: localhost
register: credential
changed_when: false

- name: Set facts for admin credentials
ansible.builtin.set_fact:
openstack_auth_env:
OS_PROJECT_DOMAIN_NAME: "{{ credential.stdout_lines | select('match', '.*OS_PROJECT_DOMAIN_NAME*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_USER_DOMAIN_NAME: "{{ credential.stdout_lines | select('match', '.*OS_USER_DOMAIN_NAME*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_PROJECT_NAME: "{{ credential.stdout_lines | select('match', '.*OS_PROJECT_NAME*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_USERNAME: "{{ credential.stdout_lines | select('match', '.*OS_USERNAME*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_PASSWORD: "{{ credential.stdout_lines | select('match', '.*OS_PASSWORD*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_AUTH_URL: "{{ credential.stdout_lines | select('match', '.*OS_AUTH_URL*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_INTERFACE: "{{ credential.stdout_lines | select('match', '.*OS_INTERFACE*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_IDENTITY_API_VERSION: "{{ credential.stdout_lines | select('match', '.*OS_IDENTITY_API_VERSION*.') | first | split('=') | last | replace(\"'\", '') }}"
OS_CACERT: "{{ '/etc/ssl/certs/ca-certificates.crt' if os_distribution == 'ubuntu' else '/etc/pki/tls/certs/ca-bundle.crt' }}"

- name: Check ec2 credential for ceph_rgw
ansible.builtin.command: >
{{ venv }}/bin/openstack
ec2 credentials list --user ceph_rgw
--format json
environment: "{{ openstack_auth_env }}"
register: credential_check
delegate_to: "{{ groups['controllers'][0] }}"
changed_when: false
vars:
ansible_host: "{{ hostvars[groups['controllers'][0]].ansible_host }}"
run_once: true

- name: Create ec2 credential if there's none
ansible.builtin.command: >
{{ venv }}/bin/openstack
ec2 credentials create --user ceph_rgw --project service
--format json
environment: "{{ openstack_auth_env }}"
delegate_to: "{{ groups['controllers'][0] }}"
changed_when: true
vars:
ansible_host: "{{ hostvars[groups['controllers'][0]].ansible_host }}"
run_once: true
when: credential_check.stdout == []

- name: Query ec2 credential for ceph_rgw
ansible.builtin.command: >
{{ venv }}/bin/openstack
ec2 credentials list --user ceph_rgw
--format json
environment: "{{ openstack_auth_env }}"
delegate_to: "{{ groups['controllers'][0] }}"
changed_when: false
vars:
ansible_host: "{{ hostvars[groups['controllers'][0]].ansible_host }}"
register: credential
run_once: true

- name: Get object storage endpoint
ansible.builtin.command: >
{{ venv }}/bin/openstack
endpoint list --service object-store --interface internal
--format json
environment: "{{ openstack_auth_env }}"
delegate_to: "{{ groups['controllers'][0] }}"
changed_when: false
vars:
ansible_host: "{{ hostvars[groups['controllers'][0]].ansible_host }}"
register: endpoint
run_once: true

- name: Process object storage endpoint
ansible.builtin.set_fact:
radosgw_server: "{{ scheme + '://' + hostname + ':' + radosgw_port }}"
vars:
swift: "{{ endpoint.stdout | from_json | first }}"
hostname: "{{ swift.URL | urlsplit('hostname') }}"
scheme: "{{ swift.URL | urlsplit('scheme') }}"
radosgw_port: "{{ swift.URL | urlsplit('port') }}"
run_once: true

- name: Ensure radosgw_usage_exporter container is running
community.docker.docker_container:
name: radosgw_usage_exporter
image: ghcr.io/stackhpc/radosgw_usage_exporter:v0.1.1
network_mode: host
env:
RADOSGW_SERVER: "{{ radosgw_server }}"
ADMIN_ENTRY: admin
ACCESS_KEY: "{{ ec2.Access }}"
SECRET_KEY: "{{ ec2.Secret }}"
VIRTUAL_PORT: "{{ stackhpc_radosgw_usage_exporter_port | string }}"
entrypoint: "{{ ['python', '-u', './radosgw_usage_exporter.py', '--insecure'] if not stackhpc_radosgw_usage_exporter_verify else omit }}"
vars:
ec2: "{{ credential.stdout | from_json | first }}"
become: true

- name: Ensure that the internal TLS certificate is trusted by the exporter
when: stackhpc_radosgw_usage_exporter_cacert | length > 0
block:
- name: Create radosgw-usage-exporter directory
ansible.builtin.file:
path: /opt/kayobe/radosgw-usage-exporter/
state: directory
mode: 0755

- name: Copy CA certificate to RADOS gateway usage exporter nodes
ansible.builtin.copy:
src: "{{ stackhpc_radosgw_usage_exporter_cacert }}"
dest: "/opt/kayobe/radosgw-usage-exporter/{{ stackhpc_radosgw_usage_exporter_cacert | basename }}"
mode: 0644
register: copy_to_node_result

- name: Copy CA certificate to RADOS gateway usage exporter container
community.docker.docker_container_copy_into:
container: radosgw_usage_exporter
path: "{{ copy_to_node_result.dest }}"
container_path: "/usr/local/share/ca-certificates/{{ copy_to_node_result.dest | basename }}"
become: true

- name: Update CA certificate of RADOS gateway usage exporter container
community.docker.docker_container_exec:
container: radosgw_usage_exporter
command: update-ca-certificates
user: root
become: true
21 changes: 21 additions & 0 deletions etc/kayobe/ansible/fix-grub-rl9.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
- name: Remove "--root-dev-only" from grub.cfg if OS is Rocky Linux 9
hosts: overcloud
become: yes
gather_facts: true
tasks:
- block:
- name: Check that /boot/efi/EFI/rocky/grub.cfg exists
ansible.builtin.stat:
path: /boot/efi/EFI/rocky/grub.cfg
register: stat_result

- name: Remove "--root-dev-only" from /boot/efi/EFI/rocky/grub.cfg
ansible.builtin.replace:
path: /boot/efi/EFI/rocky/grub.cfg
regexp: '--root-dev-only\s?'
replace: ''
when: stat_result.stat.exists
when:
- ansible_facts['distribution'] == 'Rocky'
- ansible_facts['distribution_major_version'] == '9'
4 changes: 0 additions & 4 deletions etc/kayobe/environments/ci-aio/automated-setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,6 @@ fi
sudo ip l set dummy1 up
sudo ip l set dummy1 master breth1

if type apt; then
sudo cp /run/systemd/network/* /etc/systemd/network
fi

export KAYOBE_VAULT_PASSWORD=$(cat $BASE_PATH/vault-pw)
pushd $BASE_PATH/src/kayobe-config
source kayobe-env --environment ci-aio
Expand Down
3 changes: 3 additions & 0 deletions etc/kayobe/environments/ci-aio/controllers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
# to setup the Kayobe user account. Default is {{ os_distribution }}.
controller_bootstrap_user: "{{ os_distribution if os_distribution == 'ubuntu' else 'cloud-user' }}"

controller_extra_network_interfaces:
- ethernet

# Controller lvm configuration. See intentory/group_vars/controllers/lvm.yml
# for the exact configuration.
controller_lvm_groups:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
###############################################################################
# Network interface definitions for the controller group.

# Ethernet interface is the `primary` or `physical` interface associated
# with the instance that the AIO deployment runs inside of. It is the interface used
# to reach the instance.
ethernet_interface: "{{ ansible_facts['default_ipv4']['interface'] }}"

# Controller interface on all-in-one network.
aio_interface: breth1
# Use dummy1 if it exists, otherwise the bridge will have no ports.
Expand Down
6 changes: 6 additions & 0 deletions etc/kayobe/environments/ci-aio/networks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,12 @@ cleaning_net_name: aio
###############################################################################
# Network definitions.

# This network is required to be defined within `ci-aio` environment to ensure that
# the network interface files are created appropriately and to provide easy inclusion
# within the firewall configuration.
ethernet_bootproto: dhcp
ethernet_zone: trusted

# All-in-one network.
aio_cidr: 192.168.33.0/24
aio_allocation_pool_start: 192.168.33.3
Expand Down
29 changes: 13 additions & 16 deletions etc/kayobe/kolla-image-tags.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,35 +5,32 @@
kolla_image_tags:
openstack:
rocky-9: 2024.1-rocky-9-20240903T113235
ubuntu-jammy: 2024.1-ubuntu-jammy-20240725T165045
ubuntu-jammy: 2024.1-ubuntu-jammy-20240917T091559
heat:
rocky-9: 2024.1-rocky-9-20240805T142526
ubuntu-jammy: 2024.1-ubuntu-jammy-20240805T142526
nova:
rocky-9: 2024.1-rocky-9-20240918T164436
ubuntu-jammy: 2024.1-ubuntu-jammy-20240918T164436
rocky-9: 2024.1-rocky-9-20241004T094540
ubuntu-jammy: 2024.1-ubuntu-jammy-20241004T094540
neutron:
rocky-9: 2024.1-rocky-9-20240918T164436
ubuntu-jammy: 2024.1-ubuntu-jammy-20240918T164436
rocky-9: 2024.1-rocky-9-20241025T090323
ubuntu-jammy: 2024.1-ubuntu-jammy-20241025T090323
octavia:
rocky-9: 2024.1-rocky-9-20240926T144142
ubuntu-jammy: 2024.1-ubuntu-jammy-20240926T144142
rocky-9: 2024.1-rocky-9-20241004T094540
ubuntu-jammy: 2024.1-ubuntu-jammy-20241004T094540
horizon:
rocky-9: 2024.1-rocky-9-20240909T144917
ubuntu-jammy: 2024.1-ubuntu-jammy-20240909T144917
bifrost_deploy:
rocky-9: 2024.1-rocky-9-20240725T165045
prometheus:
rocky-9: 2024.1-rocky-9-20240910T072617
ubuntu-jammy: 2024.1-ubuntu-jammy-20240910T072617
rabbitmq:
rocky-9: 2024.1-rocky-9-20240927T152945
ironic:
rocky-9: 2024.1-rocky-9-20241022T090648
ubuntu-jammy: 2024.1-ubuntu-jammy-20241022T090648
rocky-9: 2024.1-rocky-9-20241023T143407
ubuntu-jammy: 2024.1-ubuntu-jammy-20241023T143407
ironic_dnsmasq:
rocky-9: 2024.1-rocky-9-20241022T090648
ubuntu-jammy: 2024.1-ubuntu-jammy-20241022T090648
rocky-9: 2024.1-rocky-9-20241023T143407
ubuntu-jammy: 2024.1-ubuntu-jammy-20241023T143407
ironic_neutron_agent:
rocky-9: 2024.1-rocky-9-20241022T090648
ubuntu-jammy: 2024.1-ubuntu-jammy-20241022T090648
rocky-9: 2024.1-rocky-9-20241023T143407
ubuntu-jammy: 2024.1-ubuntu-jammy-20241023T143407
2 changes: 2 additions & 0 deletions etc/kayobe/kolla.yml
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,8 @@ kolla_build_customizations_common:
nova_compute_packages_append:
- python3-libvirt
- python3-ethtool
neutron_mlnx_agent_pip_packages_override:
- networking-mlnx@git+https://github.com/stackhpc/networking-mlnx@stackhpc/{{ openstack_release }}

kolla_build_customizations_rocky:
kolla_toolbox_packages_remove:
Expand Down
6 changes: 6 additions & 0 deletions etc/kayobe/kolla/config/ironic/policy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Retrieve multiple Node records
# GET /nodes
# GET /nodes/detail
# Intended scope(s): system, project
# Overridden: added role:admin
"baremetal:node:list_all": "role:admin or (role:reader and system_scope:all) or (role:service and system_scope:all) or rule:service_role"
Loading

0 comments on commit 5931c4c

Please sign in to comment.