Add flow for using agent subcommand of openshift installer

deploy_cluster.yml

@@ -1,19 +1,6 @@
 ---
-- import_playbook: playbooks/generate_ssh_key_pair.yml
-  when: generate_ssh_keys | default(True) == True
+- import_playbook: playbooks/deploy_cluster_agent_based_installer.yml
+  when: (use_agent_based_installer | default(false)) | bool
 
-- import_playbook: playbooks/create_cluster.yml
-
-- import_playbook: playbooks/generate_discovery_iso.yml
-
-- import_playbook: playbooks/mount_discovery_iso_for_pxe.yml
-
-- import_playbook: playbooks/boot_iso.yml
-  vars:
-    boot_iso_hosts: masters,workers
-
-- import_playbook: playbooks/install_cluster.yml
-
-- import_playbook: playbooks/monitor_hosts.yml
-
-- import_playbook: playbooks/monitor_cluster.yml
+- import_playbook: playbooks/deploy_cluster_assisted_installer.yml
+  when: not ((use_agent_based_installer | default(false)) | bool)

deploy_prerequisites.yml

@@ -21,5 +21,6 @@
 - import_playbook: playbooks/deploy_registry.yml
 
 - import_playbook: playbooks/deploy_assisted_installer_onprem.yml
+  when: not ((use_agent_based_installer | default(false)) | bool)
 
 - import_playbook: playbooks/deploy_sushy_tools.yml

docs/crucible_installer_features_comparison.md

@@ -0,0 +1,29 @@
+# Crucible Features
+
+This is a comparison of the features available through crucible depending on which installer is used
+
+| Feature                                            | Assisted installer (on-prem)  | Agent based installer    |
+| -------------------------------------------------- | ----------------------------- | ------------------------ |
+| Compact cluster                                    | Y                             | Y                        |
+| Workers                                            | Y                             | Y                        |
+| SNO                                                | Y                             | Y                        |
+| 2 day workers                                      | Y                             | N[1]                     |
+| Set Network type                                   | Y                             | Y                        |
+| DHCP                                               | Y                             | Y[2]                     |
+| IPV6                                               | Y                             | Y                        |
+| Dual Stack                                         | Y                             | Y                        |
+| NMState network config                             | Y                             | Y                        |
+| Mirror Registry support                            | Y                             | Y                        |
+| Set hostname                                       | Y                             | Y                        |
+| Set role                                           | Y                             | Y                        |
+| Proxy                                              | Y                             | Y                        |
+| Install OLM Operators (LSO, ODF, CNV)              | Y                             | N[3]                     |
+| Patitions                                          | Y                             | N[4]                     |
+| Discovery iso password                             | Y                             | N[4]                     |
+| -                                                  | -                             | -                        |
+
+Footnotes:
+[1] There are plans for the agent based method to install the [multicluster engine operator](https://docs.openshift.com/container-platform/4.12/architecture/mce-overview-ocp.html) which crucible could then leverage to add day2 workers.
+[2] A `network_config` is still required however you could provide a raw nmstate, which configures the interfaces for dhcp and the corisponding `mac_interface_map`. If you are not using the DHCP provided by crucible you would need to provide the correct IP for the bootstrap node (by default the first node in the masters group).
+[3] It is possible to apply extra manifests to deploy those operators as part of the install. The MCE deploy ment mentioned in [1] will likely expose this feature as well.
+[4] This feature of crucible is done by modifing an iginition file which is not currently possible in the agent based flow.

docs/network_config.md

@@ -0,0 +1,135 @@
+# Network configuration
+Network configuration can currently be used in two places in the inventory to configure the network config of a node and the network config of a vm_host.
+
+The `network_config` entry on a node is a simplified version of the `nmstate`([nmstate.io](http://nmstate.io/)) required by the [assisted installer api](https://github.com/openshift/assisted-service/blob/3bcaca8abef5173b0e2175b5d0b722e851e39cee/docs/user-guide/restful-api-guide.md).
+
+#### Static IPs
+
+To activate static IPs in the discovery iso and resulting cluster there is some configuration required in the inventory.
+
+```yaml
+network_config:
+  interfaces:
+    - name: "{{ interface }}"
+      mac: "{{ mac }}"
+      addresses:
+        ipv4:
+          - ip: "{{ ansible_host}}"
+            prefix: "{{ mask }}"
+  dns_server_ips:
+    - "{{ dns }}"
+    - "{{ dns2 }}"
+  routes: # optional
+    - destination: 0.0.0.0/0
+      address: "{{ gateway }}"
+      interface: "{{ interface }}"
+```
+
+where the variables are as follows:
+
+- `ip`: The static IP is set
+- `dns` & `dns2`: IPs of the DNS servers
+- `gateway`: IP of the gateway
+- `mask`: Length of subnet mask (e.g. 24)
+- `interface`: The name of the interface you wish to configure
+- `mac`: Mac address of the interface you wish to configure
+
+## Examples
+
+### Link Aggregation
+
+```yaml
+network_config:
+  interfaces:
+    - name: bond0
+      type: bond
+      state: up
+      addresses:
+        ipv4:
+          - ip: 172.17.0.101
+            prefix: 24
+      link_aggregation:
+        mode: active-backup
+        options:
+          miimon: "1500"
+        slaves:
+          - ens7f0
+          - ens7f1
+    - name: ens1f0
+      type: ethernet
+      mac: "40:A6:B7:3D:B3:70"
+      state: up
+    - name: ens1f1
+      type: ethernet
+      mac: "40:A6:B7:3D:B3:71"
+      state: up
+    dns_server_ips:
+        - 10.40.0.100
+  routes:
+    - destination: 0.0.0.0/0
+      address: 172.17.0.1
+      interface: bond0
+```
+
+### Dual Stack:
+``` yaml
+network_config:
+    interfaces:
+    - name: "enp1s0"
+      mac: "{{ mac }}"
+      addresses:
+      ipv4:
+        - ip: "{{ ansible_host }}"
+          prefix: "{{ ipv4.mask }}"
+      ipv6:
+        - ip: "{{ ipv6_address }}"
+          prefix: "{{ ipv6.mask }}"
+    dns_server_ips:
+      - "{{ ipv6.dns }}"
+      - "{{ ipv4.dns }}"
+    routes:
+      - destination: "0:0:0:0:0:0:0:0/0"
+        address: "{{ ipv6.gateway }}"
+        interface: "enp1s0"
+      - destination: 0.0.0.0/0
+        address: "{{ ipv4.gateway }}"
+        interface: "enp1s0"
+```
+
+## Advanced
+
+### Raw nmstate
+
+ If you wish to write the `nmstate` by hand you can use the `network_config.raw` entry, however you will also need to add `mac_interface_map`, the following is static ipv4 address
+
+```yaml
+mac_interface_map:
+  - logical_nic_name: "enp1s0"
+    mac_address: "{{ mac }}"
+network_config:
+  raw:
+    dns-resolver:
+      config:
+        server:
+        - "{{ dns }}"
+    interfaces:
+      - name: enp1s0
+        state: up
+        type: ethernet
+        ipv4:
+          address:
+            - ip: "{{ ansible_host }}"
+              prefix-length: "{{ mask }}"
+          dhcp: false
+          enabled: true
+    routes:
+      config:
+        - destination: 0.0.0.0/0
+          next-hop-address: "{{ gateway }}"
+          next-hop-interface: enp1s0
+          table-id: 254
+```
+
+
+### Custom template
+If you wish to use your own template you can set `network_config.template` with a path to your desired template the default can be found [here](../roles/generate_discovery_iso/templates/nmstate.yml.j2).

playbooks/deploy_cluster_agent_based_installer.yml

@@ -0,0 +1,15 @@
+---
+- import_playbook: extract_agent_based_installer.yml
+  when: agent_based_installer_path is not defined
+
+- import_playbook: generate_ssh_key_pair.yml
+  when: generate_ssh_keys | default(True) == True
+
+- import_playbook: generate_manifests.yml
+- import_playbook: generate_agent_iso.yml
+
+- import_playbook: boot_iso.yml
+  vars:
+    boot_iso_hosts: masters,workers
+
+- import_playbook: monitor_agent_based_installer.yml

playbooks/deploy_cluster_assisted_installer.yml

@@ -0,0 +1,19 @@
+---
+- import_playbook: generate_ssh_key_pair.yml
+  when: generate_ssh_keys | default(True) == True
+
+- import_playbook: create_cluster.yml
+
+- import_playbook: generate_discovery_iso.yml
+
+- import_playbook: mount_discovery_iso_for_pxe.yml
+
+- import_playbook: boot_iso.yml
+  vars:
+    boot_iso_hosts: masters,workers
+
+- import_playbook: install_cluster.yml
+
+- import_playbook: monitor_hosts.yml
+
+- import_playbook: monitor_cluster.yml

playbooks/extract_agent_based_installer.yml

@@ -0,0 +1,15 @@
+---
+- name: Extract openshift installer
+  hosts: bastion
+  gather_facts: false
+  vars:
+    destination_hosts:
+      - bastion
+  pre_tasks:
+    - name: pre-compute need to get hashes
+      set_fact:
+        run_get_hash: "{{ assisted_installer_release_images | default({}) | length == 0  }}"
+  roles:
+    - role: get_image_hash
+      when: run_get_hash | bool
+    - extract_openshift_installer

playbooks/generate_agent_iso.yml

@@ -0,0 +1,5 @@
+---
+- name:  Generate agent iso using agent_based_installer
+  hosts: bastion
+  roles:
+    - generate_agent_iso

playbooks/generate_manifests.yml

@@ -0,0 +1,14 @@
+---
+- name: Generate manfiests for agent_based_installer
+  hosts: bastion
+  vars:
+    destination_hosts:
+      - bastion
+  pre_tasks:
+    - name: pre-compute need to get hashes
+      set_fact:
+        run_get_hash: "{{ image_hashes | default({}) | length == 0  }}"
+  roles:
+    - role: get_image_hash
+      when: run_get_hash
+    - generate_manifests

playbooks/monitor_agent_based_installer.yml

@@ -0,0 +1,5 @@
+- name: Monitor install process of agent_based_installer
+  hosts: bastion
+  gather_facts: False
+  roles:
+    - monitor_agent_based_installer

roles/extract_openshift_installer/defaults/main.yml

@@ -0,0 +1,6 @@
+arch: x86_64
+version_filter: "[?(openshift_version == '{{ openshift_version }}') && (cpu_architecture == '{{ arch }}')]"
+release_image: "{{ (assisted_installer_release_images | json_query(version_filter))[0].url }}"
+extact_dest_path: /tmp/wip/extract/
+pull_secret_file: "{{ extact_dest_path }}/pull_secret.txt"
+openshift_installer_path: "{{ extact_dest_path }}/openshift-install"

roles/extract_openshift_installer/tasks/main.yml

@@ -0,0 +1,40 @@
+---
+- name: "Create {{ extact_dest_path }}"
+  ansible.builtin.file:
+    path: "{{ extact_dest_path }}"
+    state: directory
+    mode: 0755
+    recurse: true
+
+- name: Create pull_secret_file
+  ansible.builtin.copy:
+    content: "{{ pull_secret }}"
+    dest: "{{ pull_secret_file }}"
+    mode: "0600"
+
+- name: Extract openshift_installer
+  ansible.builtin.shell:
+    cmd: >
+      oc adm
+      -a {{ pull_secret_file }}
+      release extract
+      --command=openshift-install
+      {{ release_image }}
+      --to={{ extact_dest_path }}
+
+- name: Check extracted installer has agent subcommand
+  ansible.builtin.shell:
+    cmd: "{{ openshift_installer_path }} agent --help"
+  register: res
+  failed_when: false
+
+- name: Check agent sub-commmand output
+  ansible.builtin.fail:
+    msg: >
+      Version of openshift install extracted from
+      release image does not have agent subcommand
+  when: "'unknown command' in res.stderr"
+
+- name: Set agent_based_installer_path
+  ansible.builtin.set_fact:
+    agent_based_installer_path: "{{ openshift_installer_path }}"

roles/generate_agent_iso/defaults/main.yml

@@ -0,0 +1,9 @@
+generated_dir: "{{ repo_root_path }}/generated"
+manifests_dir: "{{ generated_dir }}/{{ cluster_name }}"
+download_agent_dest_file: "{{ discovery_iso_name }}"
+download_dest_path: "{{ iso_download_dest_path | default('/opt/http_store/data') }}"
+config_file_path: /tmp/wip/config
+arch: x86_64
+version_filter: "[?(openshift_version == '{{ openshift_version }}') && (cpu_architecture == '{{ arch }}')]"
+release_image: "{{ (assisted_installer_release_images | json_query(version_filter))[0].url }}"
+use_local_mirror_registry: false

roles/generate_agent_iso/tasks/main.yml

@@ -0,0 +1,36 @@
+- name: Create podman auth dir
+  ansible.builtin.file:
+    path: "{{ ansible_env.HOME }}/.docker"
+    state: directory
+    mode: 0755
+    recurse: true
+
+- name: Copy pull_secret file.
+  ansible.builtin.copy:
+    content: "{{ pull_secret }}"
+    dest: "{{ ansible_env.HOME }}/.docker/config.json"
+    mode: 0644
+
+- name: Generate ISO
+  ansible.builtin.shell:
+    cmd: "{{ agent_based_installer_path }} --log-level=debug agent create image"
+    chdir: "{{ manifests_dir }}"
+  environment:
+    XDG_RUNTIME_DIR: "{{ config_file_path }}"
+    REGISTRY_AUTH_FILE: "{{ config_file_path }}/containers/auth.json"
+
+- name: Put discovery iso in http store
+  delegate_to: http_store
+  become: true
+  block:
+    - name: Create discovery directory
+      ansible.builtin.file:
+        path: "{{ download_dest_path }}/{{ download_agent_dest_file | dirname }}"
+        recurse: true
+        state: directory
+
+    - name: Copy agent iso to discovery directory
+      ansible.builtin.copy:
+        src: "{{ manifests_dir }}/agent.{{ arch | default('x86_64') }}.iso"
+        dest: "{{ download_dest_path }}/{{ download_agent_dest_file }}"
+        mode: 0644

roles/generate_manifests/defaults/main.yml

@@ -0,0 +1,18 @@
+generated_dir: "{{ repo_root_path }}/generated"
+manifests_dir: "{{ generated_dir }}/{{ cluster_name }}"
+cluster_manifest_dir: "{{ manifests_dir }}/cluster-manifests"
+extra_manifest_dir: "{{ manifests_dir }}/openshift"
+mac_interface_default_mapping: "interfaces[?(name != null && mac != null)].{logical_nic_name: name, mac_address: mac}"
+static_network_config: {}
+arch: x86_64
+version_filter: "[?(openshift_version == '{{ openshift_version }}') && (cpu_architecture == '{{ arch }}')]"
+release_image: "{{ (assisted_installer_release_images | json_query(version_filter))[0].url }}"
+mirror_registry: "{{ hostvars['registry_host']['registry_fqdn'] }}:{{ hostvars['registry_host']['registry_port'] }}"
+agent_based_installer_bootstrap_node: "{{ groups['masters'][0] }}"
+host_ip_keyword: ansible_host
+use_local_mirror_registry: "{{ setup_registry_service | default(true) }}"
+single_node_openshift_enabled: "{{ is_valid_single_node_openshift_config | default(false) }}"
+
+manifests: true
+extra_manifests: []
+manifest_templates: "{{ extra_manifests }}"