Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add redhat_ocp_virt platform to sap_hypervisor_node_preconfigure #468

Merged
merged 45 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
3d4ae5b
added redhat_ocp_virt platform to sap_hypervisor_node_preconfigure
Jun 21, 2023
0cb059a
Merge branch 'sap-linuxlab:dev' into dev
newkit Sep 26, 2023
4a8f670
Merge branch 'sap-linuxlab:dev' into dev
newkit Sep 26, 2023
82ac5dc
Merge branch 'dev' of github.com:newkitlabs/community.sap_install int…
Oct 12, 2023
d455bbc
updated documentation
Oct 16, 2023
5600c6e
Update roles/sap_hypervisor_node_preconfigure/README.md
newkit Oct 24, 2023
0e1ce92
Update roles/sap_hypervisor_node_preconfigure/README.md
newkit Oct 24, 2023
8c80b9e
include hpp
Oct 24, 2023
68f775d
added install-hpp.yml
Oct 24, 2023
f4fd42e
Merge branch 'hpp' into dev
Oct 24, 2023
b743620
Update roles/sap_hypervisor_node_preconfigure/README.md
newkit Oct 24, 2023
af869ef
Merge branch 'dev' of github.com:newkitlabs/community.sap_install int…
Oct 24, 2023
8ac014c
updated doc
Oct 26, 2023
1000071
WIP: restructure sap_hypervisor_node_preconfigure
Oct 26, 2023
2edc92a
variable name fix (minor)
Oct 30, 2023
34a8dca
fixed missing vars
Oct 30, 2023
ec5b083
removed unnecessary files
Oct 31, 2023
4afbe84
added label cpumanager
Nov 2, 2023
1bf351b
fixed worker node name when labeling for cpumanger
Nov 2, 2023
288b76f
removed misplaced kubernetes_reserved_cpu
Nov 6, 2023
ea875e6
added info on required dependencies
Nov 6, 2023
ce45a58
implemented ansible-lint suggestions
Nov 6, 2023
a98efd5
removed obsolete file roles/sap_hypervisor_node_preconfigure/files/pl…
Nov 6, 2023
fe45274
renamed setup-worker-node -> setup-worker-nodes
Nov 6, 2023
f40c0ea
Merge branch 'sap-linuxlab:dev' into dev
newkit Nov 6, 2023
8ecce64
updated README.md; name template.yml after instanciation
Nov 6, 2023
62d34b8
removed startup dependency for HPP
Nov 6, 2023
37440c1
fixed creation of cpumanager KR
Nov 7, 2023
3ca31d5
updated docu
Nov 13, 2023
724e1ad
include defaults platform dependent
Nov 13, 2023
860f712
Revert "include defaults platform dependent"
Nov 13, 2023
ae4456a
Revert "include defaults platform dependent"
Nov 14, 2023
ce82be3
create VM namespace, remove uneeded waits
Nov 16, 2023
8308ccc
fixed merge conflict
Nov 16, 2023
1db86b8
fixed typo
Nov 16, 2023
dbc634f
added missing whitespace before linebreak
Nov 23, 2023
24ccfa7
fixed typo and indentation
Nov 23, 2023
bc4a43c
Ansible lint wants role name to start with a capital letter
Dec 6, 2023
a89f7da
Made clear that memory unit used is GiB
Dec 7, 2023
a0410b6
fixed memory variable renaming and removed cating to int
Dec 7, 2023
88adb96
cast integers when used
Dec 7, 2023
7a94e65
label nodes via k8s
Dec 8, 2023
b225ee9
labeling did not work
Dec 8, 2023
53be961
reverted not working PR suggestions: booleans in trident template; te…
Dec 11, 2023
b072cb5
use kubernetes.core.k8s rather than ansible k8s module
Dec 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions playbooks/sample-sap-hypervisor-redhat_ocp_virt-preconfigure.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
- hosts: all
gather_facts: true
serial: 1
vars:
sap_hypervisor_node_platform: redhat_ocp_virt

tasks:
- name: Include Role
ansible.builtin.include_role:
name: sap_hypervisor_node_preconfigure
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
sap_hypervisor_node_preconfigure_cluster_config:

# URL under which the OCP cluster is reachable
cluster_url: ocpcluster.domain.org

# namespace under which the VMs are created, note this has to be
# openshift-sriov-network-operator in case of using SRIOV network
# devices
vm_namespace: sap

# Optional, configuration for trident driver for Netapp NFS filer
trident:
management: management.domain.org
data: datalif.netapp.domain.org
svm: sap_svm
backend: nas_backend
aggregate: aggregate_Name
username: admin
password: xxxxx
storage_driver: ontap-nas
storage_prefix: ocpv_sap_

# CPU cores which will be reserved for kubernetes
worker_kubernetes_reserved_cpus: "0,1"

# Storage device used for host path provisioner as local storage.
worker_localstorage_device: /dev/vdb

# detailed configuration for every worker that should be configured
#
workers:
- name: worker-0 # name must match the node name
networks: # Example network config
- name: sapbridge # using a bridge
description: SAP bridge
state: up
type: linux-bridge
ipv4:
enabled: false
auto-gateway: false
auto-dns: false
bridge:
options:
stp:
enabled: false
port:
- name: ens1f0 # network IF name
- name: storage # an SRIOV device
interface: ens2f0 # network IF name
type: sriov

- bridge: # another bridge
options:
stp:
enabled: false
port:
- name: ens2f0 # network IF name
description: storage
mtu: 9000
ipv4:
address:
- ip: 192.168.1.51 # IP config
prefix-length: 24
auto-dns: false
auto-gateway: false
enabled: true
name: storagebridge
state: up
type: linux-bridge
- name: multi # another SRIOV device
interface: ens2f1 # network IF name
type: sriov

- name: worker-1 # second worker configuration
networks: # Example network config
- name: sapbridge # using a bridge
description: SAP bridge
state: up
type: linux-bridge
ipv4:
enabled: false
auto-gateway: false
auto-dns: false
bridge:
options:
stp:
enabled: false
port:
- name: ens1f0 # network IF name
- name: storage # an SRIOV device
interface: ens2f0 # network IF name
type: sriov
211 changes: 189 additions & 22 deletions roles/sap_hypervisor_node_preconfigure/README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,177 @@
`EXPERIMENTAL`

sap_hypervisor_node_preconfigure
=======================
# sap_hypervisor_node_preconfigure

This role will configure the following hypervisors in order to run SAP workloads:
newkit marked this conversation as resolved.
Show resolved Hide resolved
* Red Hat OpenShift Virtualization (OCPV)
* Red Hat Enterprise Virtualization (RHV)

## Platform: Red Hat OpenShift Virtualization

Will configure a plain vanilla OpenShift cluster so it can be used for SAP workloads.

### Requirements
* An OpenShift cluster, best without any previous customization.
* The worker nodes should have > 96GB of memory.
* Worker nodes need to have Intel CPUs that provide TSX feature.
* Storage is required, e.g. via NFS, OpenShift Data Foundation or local storage. This role can setup access to a Netapp Filer via Trident storage connector.
Local storage will be configures using host path provisioner.
* Point the `KUBECONFIG` environment variable to your `kubeconfig`.
* Make the role available in case you didn't install it already in an ansible roles directory, e.g.
* Make sure to install the dependencies mentioned below are installed.
* To ensure your local checkout it found by ansible:
```
mkdir -p ~/.ansible/roles/
ln -sf ~/community.sap_install/roles/sap_hypervisor_node_preconfigure ~/.ansible/roles/
```
### Dependencies

This role will set and check the required settings and parameters for a hypervisor running VMs for SAP HANA.
Needs the ansible kubernetes module and the python3 kubernetes binding. On a RHEL based system the are named
* python3-kubernetes
* ansible-collection-kubernetes-core

Requirements
------------
A RHV hypervisor.
Needs `oc` binary available in path.

Role Variables
--------------
### Role Variables
General variables are defined in sap_hypervisor_node_preconfigure/vars/platform_defaults_redhat_ocp_virt.yml
```
# Install the trident NFS storage provider. If yes, expects configuration details under
# sap_hypervisor_node_preconfigure_cluster_config.trident, see example config.
sap_hypervisor_node_preconfigure_install_trident: True|False
# URL of the trident installer package to use
sap_hypervisor_node_preconfigure_install_trident_url: https://github.com/NetApp/trident/releases/download/v23.01.0/trident-installer-23.01.0.tar.gz

# should SRIOV be enabled for unsupported NICs
sap_hypervisor_node_preconfigure_sriov_enable_unsupported_nics: True|False

# Amount of memory [GB] to be reserved for the hypervisor on hosts >= 512GB
sap_hypervisor_node_preconfigure_hypervisor_reserved_ram_host_ge_512: 64 #GB
# Amount of memory [GB] to be reserved for the hypervisor on hosts < 512GB
sap_hypervisor_node_preconfigure_hypervisor_reserved_ram_host_lt_512: 32 #GB

# Should the check for the minimal amount of memory be ignored? Minimal amount is 96 GB
# If ignored, the amount of $hostmemory - $reserved is allocated with a lower bound of 0 in case $reserved > $hostmemory
sap_hypervisor_node_preconfigure_ignore_minimal_memory_check: True|False

# Define if the host path provisioner should be installed in order to use a local disk as storage device.
# Uses the following variable to be set to the storage device to be used, e.g.:
# sap_hypervisor_node_preconfigure_cluster_config.worker_localstorage_device: /dev/sdb
sap_hypervisor_node_preconfigure_install_hpp: True|False
```
The following variables are describing the nodes and networks to be used. It can make sense to have them in a seperate file, e.g. see `playbooks/vars/sample-variables-sap-hypervisor-node-preconfigure-rh_ocp_virt.yml` for an example.
```
sap_hypervisor_node_preconfigure_cluster_config:
# URL under which the OCP cluster is reachable
cluster_url: ocpcluster.domain.org

# namespace under which the VMs are created, note this has to be
# openshift-sriov-network-operator in case of using SRIOV network
# devices
vm_namespace: sap

# Optional, configuration for trident driver for Netapp NFS filer
trident:
management: management.domain.org
data: datalif.netapp.domain.org
svm: sap_svm
backend: nas_backend
aggregate: aggregate_Name
username: admin
password: xxxxx
storage_driver: ontap-nas
storage_prefix: ocpv_sap_

# CPU cores which will be reserved for kubernetes
worker_kubernetes_reserved_cpus: "0,1"

# Storage device used for host path provisioner as local storage.
worker_localstorage_device: /dev/vdb

# detailed configuration for every worker that should be configured
workers:
- name: worker-0 # name must match the node name
networks: # Example network config
- name: sapbridge # using a bridge
description: SAP bridge
state: up
type: linux-bridge
ipv4:
enabled: false
auto-gateway: false
auto-dns: false
bridge:
options:
stp:
enabled: false
port:
- name: ens1f0 # network IF name
- name: storage # an SRIOV device
interface: ens2f0 # network IF name
type: sriov

- bridge: # another bridge
options:
stp:
enabled: false
port:
- name: ens2f0 # network IF name
description: storage
mtu: 9000
ipv4:
address:
- ip: 192.168.1.51 # IP config
prefix-length: 24
auto-dns: false
auto-gateway: false
enabled: true
name: storagebridge
state: up
type: linux-bridge
- name: multi # another SRIOV device
interface: ens2f1 # network IF name
type: sriov

- name: worker-1 # second worker configuration
networks: # Example network config
- name: sapbridge # using a bridge
description: SAP bridge
state: up
type: linux-bridge
ipv4:
enabled: false
auto-gateway: false
auto-dns: false
bridge:
options:
stp:
enabled: false
port:
- name: ens1f0 # network IF name
- name: storage # an SRIOV device
interface: ens2f0 # network IF name
type: sriov
```
### Example Playbook
See `playbooks/sample-sap-hypervisor-redhat_ocp_virt-preconfigure.yml` for an example.

### Example Usage
Make sure to set the `KUBECONFIG` environment variable, e.g.
```
export KUBECONFIG=~/.kubeconfig
```
To invoke the example playbook with the example configuration using your localhost as ansible host use the following command line:
```
ansible-playbook --connection=local -i localhost, playbooks/sample-sap-hypervisor-redhat_ocp_virt-preconfigure.yml -e @s/sample-sap-hypervisor-redhat_ocp_virt-preconfigure.yml
newkit marked this conversation as resolved.
Show resolved Hide resolved
```

newkit marked this conversation as resolved.
Show resolved Hide resolved
## Platform: RHEL KVM
This Ansible Role allows preconfigure of Red Hat Virtualization (RHV), formerly called Red Hat Enterprise Virtualization (RHEV) prior to version 4.4 release. Red Hat Virtualization (RHV) consists of 'Red Hat Virtualization Manager (RHV-M)' and the 'Red Hat Virtualization Host (RHV-H)' hypervisor nodes that this Ansible Role preconfigures. Please note, Red Hat Virtualization is discontinued and maintenance support will end mid-2024. Extended life support for RHV ends mid-2026.
This Ansible Role does not preconfigure RHEL KVM (RHEL-KVM) hypervisor nodes. Please note that RHEL KVM is standalone, and does not have Management tooling (previously provided by RHV-M).

### Requirements
* A RHV hypervisor.

### Role Variables
`sap_hypervisor_node_preconfigure_reserved_ram (default: 100)` Reserve memory [GB] for hypervisor host. Depending in the use case should be at least 50-100GB.

`sap_hypervisor_node_preconfigure_reserve_hugepages (default: static)` Hugepage allocation method: {static|runtime}.
Expand Down Expand Up @@ -43,30 +203,37 @@ runtime: done with hugeadm which is faster, but can in some cases not ensure all
`sap_hypervisor_node_preconfigure_run_grub2_mkconfig (default: yes)` Update the grub2 config.


Example Playbook
----------------

### Example Playbook
Simple example that just sets the parameters.
```
---
- hosts: all
roles:
- sap_hypervisor_node_preconfigure
gather_facts: true
serial: 1
vars:
sap_hypervisor_node_platform: redhat_rhel_kvm
tasks:
- name: Include Role
ansible.builtin.include_role:
name: sap_hypervisor_node_preconfigure
```

Run in assert mode to verify that parameters have been set.
```
---
- hosts: all
roles:
- sap_hypervisor_node_preconfigure
gather_facts: true
serial: 1
vars:
- sap_hypervisor_node_preconfigure_assert: yes
sap_hypervisor_node_platform: redhat_rhel_kvm
sap_hypervisor_node_preconfigure_assert: yes
tasks:
- name: Include Role
ansible.builtin.include_role:
name: sap_hypervisor_node_preconfigure
```
License
-------

### License
Apache 2.0

Author Information
------------------

### Author Information
Nils Koenig ([email protected])
Loading