Ansible Provisioner Installer (#12)

Ansible Provisioner Installer

.gitignore

@@ -4,3 +4,4 @@
 *.inventory
 *.output
 downloads/
+credentials/

LICENSE

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright {yyyy} {name of copyright owner}
+   Copyright 2018 Jeremy Poulin
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1,45 +1,19 @@
 # multiarch-ci-provisioner
 The RedHat Multi-Arch QE provisioner project is an effort to provide Jenkins CI users with an easy way build and test packages across all architectures. It is meant to go hand in hand with the multiarch-test-template project so that users can get up and running with their tests quickly. Currently this project is only expected to work in RedHat's internal enviroment, we are making active progress towards supporting general usage.
 
-## Getting Started
-The first step to using the multiarch-ci-provisioner is where you need to do a full install on the system, or just install the OpenShift components into your existing cluster.
-
-### Full Install
-1. Create a standalone VM.
-2. Run the `install/install-all.sh'
-
-### Install Using Pre-Existing OpenShift Cluster
-1. 
-2. 
+## Table of Contents
+- [Getting Started](#getting-started)
+- [License](#license)
+- [Authors](#authors)
 
-## Jenkins Setup
-### Jenkins Plugin Dependencies
-Beyond the OpenShift Jenkins image plugins, you'll need to install these plugins to get the provisioner running.
-1. AnsiColor
-2. Timestamper
-3. Self-Organizing Swarm Plugin
-
-### Jenkins Secrets
-In order to run the provisioner, you'll need to have 3 secret files uploaded as Jenkins credentials:
-1. `KEYTAB`
-Kerberos keytab that will be used to authenticate to beaker.
-2. `SSHPUBKEY`
-SSH public key that will be installed on the provisioned node.
-3. `SSHPRIVKEY`
-SSH private key to connect to the provisioned node.
-
-It is recommended that the key pair be used only to connect your Jenkins node to the provisioned host.
+## Getting Started
+For directions on how to get started using the provisioner, please visit our [wiki](https://github.com/RedHat-MultiArch-QE/multiarch-ci-provisioner/wiki).
 
-### Cinch Configuration
-Depending on whether the Jenkins instance is run in an OpenShift cluster, you'll need to override the following variables to connect via JSwarm.
-1. `JENKINS_MASTER_URL`
-The URL belonging of the Jenkins master.
-2. `JENKINS_SLAVE_USERNAME`
-Username or key ID that will be used to connect the slave.
-3. `JENKINS_SLAVE_PASSWORD`
-Password or key that will be used to connect the slave.
-4. `JSWARM_EXTRA_ARGS`
-In the case of running Jenkins in an OpenShift cluster, you'll first want to expose the JNLP port by creating a route object for the JNLP service. From there, this should be set to '-tunnel routeURL'.
+## License
+This project is licensed under the Apache 2.0 License - see the LICENSE file for details.
 
-### OpenShift Service Account
-The OpenShift service account used to run the provisioner container image needs to have permission to run priveleged containers. In a full install, the Jenkins service account is given this elevated privileges. The step that must be done beyond this in Jenkins is to ensure that the OpenShift oauth token being used to connect to the OpenShift cluster is the Jenkins service account token. This is output to the console as the last step in the full installation process.
+## Authors
+This project would not be possible without the work of following people.
+- [jaypoulz](https://github.com/jaypoulz/) - *Develops and maintains the current provisioner.*
+- [detiber](https://github.com/detiber) - *Helped guide the development of the provisioner use case and upstream support.*
+- [dbenoit](https://github.com/dbenoit17) - *Developed the Ansible installer and is working to add support for multi-arch containers.*

images/provisioner/Dockerfile

@@ -3,41 +3,34 @@ FROM openshift/jenkins-slave-base-centos7
 # Set home to root directory for caching purposes
 # Disable host key checking explicitly - see https://github.com/ansible/ansible/issues/30720
 ENV HOME=/root \
-    ANSIBLE_HOST_KEY_CHECKING=False
-
-# Set WORKDIR to HOME
-WORKDIR /root
+    ANSIBLE_HOST_KEY_CHECKING=False \
+    WORKON_HOME=/home/jenkins/envs
 
 # Install all dependencies from yum
 RUN yum-config-manager --add-repo=http://download.lab.eng.bos.redhat.com/beakerrepos/client/CentOS7/ && \
-yum upgrade -y --skip-broken && \
-yum install -y epel-release && \
-yum install -y python python-devel python-pip && \
-yum install -y beaker-common beakerlib beaker-client beaker-redhat python-beaker --nogpgcheck && \
-yum install -y krb5-workstation libvirt-devel libyaml-devel openssl-devel libffi-devel gcc && \
-yum clean all
-
-# Install all pip dependencies
-RUN pip install -U pip setuptools && \
-    pip install -U beaker-common beaker-client && \
-    pip install -U linchpin cinch
-
-# Copy over configuration files
-COPY beaker/client.conf /etc/beaker/client.conf
-COPY krb5/krb5.conf /etc/krb5.conf
-RUN chmod 644 /etc/krb5.conf && \
-    chmod 644 /etc/beaker/client.conf
+    yum install -y epel-release && \
+    yum install -y python python-devel python-pip && \
+    yum install -y beaker-common beakerlib beaker-client beaker-redhat python-beaker --nogpgcheck && \
+    yum install -y krb5-workstation libvirt-devel libyaml-devel openssl-devel libffi-devel gcc && \
+    yum clean all
+
+# Install virutalenvwrapper && pip dependencies
+RUN pip install virtualenvwrapper && \
+    mkdir -p $WORKON_HOME && \
+    source $(which virtualenvwrapper.sh) && \
+    mkvirtualenv --no-site-packages provisioner && \
+    workon provisioner && \
+    pip install -U pip && \
+    pip install -U setuptools && \
+    pip install -U linchpin[beaker]==1.5.4 cinch
 
 # Setup certificates for RedHat
 RUN cd /etc/pki/ca-trust/source/anchors && \
     curl -o RedHat_CA.crt --insecure -L https://password.corp.redhat.com/cacert.crt && \
     curl -o PnTDevOps_CA.crt --insecure -L https://engineering.redhat.com/Eng-CA.crt && \
     curl -o RH-IT-Root-CA.crt --insecure -L https://password.corp.redhat.com/RH-IT-Root-CA.crt && \
+    cp /etc/pki/ca-trust/source/anchors/RH-IT-Root-CA.crt /etc/beaker/RH-IT-Root-CA.crt && \
     update-ca-trust extract
 
-# Configure beaker
-RUN cp /etc/pki/ca-trust/source/anchors/RH-IT-Root-CA.crt /etc/beaker/RH-IT-Root-CA.crt
-
-# Set home dir and env HOME var for SSH
-WORKDIR /home/jenkins
+# Set env HOME var for SSH
 ENV HOME=/home/jenkins

install/ansible/default.inventory

@@ -0,0 +1,2 @@
+[openshift_master]
+localhost

install/ansible/group_vars/openshift_master.yml

@@ -0,0 +1,38 @@
+# For openshift cluster deployment
+deploy_cluster: true
+# For multiarch-qe provisioner deployment
+deploy_provisioner: true
+
+# openshift configuration options
+openshift_master_external_ip: "{{ (inventory_hostname is match('(?:[0-9]{1,3}\\.){3}[0-9]{1,3}')) | ternary(inventory_hostname, ansible_default_ipv4.address) }}"
+openshift_master_hostname: "{{ (inventory_hostname == 'localhost') | ternary(openshift_master_external_ip, inventory_hostname) }}"
+openshift_master_fqdn: "{{ openshift_master_external_ip }}.xip.io }}"
+openshift_host_data_dir: /var/lib/origin/local
+openshift_install_mode: binary
+
+# container template locations
+templates_dir: "{{ playbook_dir }}/../../templates"
+jenkins_template_path: "{{ templates_dir }}/jenkins-persistent.yml"
+provisioner_template_path: "{{ templates_dir }}/provisioner.yml"
+
+# github repositories
+github_clone_org: redhat-multiarch-qe
+github_repo_ref: v1.0
+
+# openshift_urls
+openshift_version: 3.9.0
+openshift_3_9_0_id: 191fece
+openshift_3_7_2_id: 282e43f
+openshift_3_7_1_id: ab0f056
+openshift_3_7_0_id: 7ed6862
+openshift_3_6_1_id: 008f2d5
+openshift_3_6_0_id: c4dd4cf
+openshift_id: "{{ lookup('vars', 'openshift_' + openshift_version | replace('.', '_') + '_id') }}"
+openshift_download_url: https://github.com/openshift/origin/releases/download/v{{ openshift_version }}/openshift-origin-client-tools-v{{ openshift_version }}-{{ openshift_id }}-linux-64bit.tar.gz
+
+# kerberos and beaker credentials
+credentials_dir: "{{ playbook_dir }}/../../credentials"
+secrets_dir: "{{ playbook_dir }}/../secrets"
+kerberos_keytab_path: "{{ credentials_dir }}/{{ kerberos_principal }}.keytab"
+beaker_private_key_path: "{{ credentials_dir }}/id_rsa"
+beaker_public_key_path: "{{ credentials_dir }}/id_rsa.pub"

install/ansible/install.yml

@@ -0,0 +1,21 @@
+---
+- name: deploy openshift cluster
+  hosts: openshift_master
+  vars_prompt:
+    - name: kerberos_principal
+      private: no
+      prompt: |
+        Call ansible-playbook with '-e "kerberos_principal=<your kerberos username>"'
+        to skip this prompt. Please enter your kerberos principal.
+  tasks:
+    - name: check ansible version is 2.5 or higher
+      local_action:
+        module: fail
+        msg: requires ansible version 2.5 or higher
+      when: ansible_version.full is version('2.5', '<=')
+    - name: deploy openshift cluster
+      include_tasks: ./playbooks/deploy_cluster.yml
+      when: deploy_cluster
+    - name: set up multiarch ci provisioner
+      include_tasks: ./playbooks/setup_provisioner.yml
+      when: deploy_provisioner