Merge pull request #21 from wildsubnet/23-pre

23 pre

Dockerfile

@@ -35,7 +35,7 @@ WORKDIR /tmp
 COPY requirements.txt requirements.txt
 RUN pip3 install -r requirements.txt
 
-RUN ansible-galaxy install Juniper.junos -p /etc/ansible/roles/
+#RUN ansible-galaxy install Juniper.junos -p /etc/ansible/roles/
 COPY requirements.yml .
 RUN ansible-galaxy collection install -r requirements.yml
 

README.md

@@ -1,16 +1,16 @@
-# NITA Ansible 22.8
+# NITA Ansible 23.p
 
-Welcome to NITA 22.8.
+Welcome to NITA 23.p pre-release. This is a testing release that includes scripts for building ansible execution environment for use with AWX. The execution environment is based on the nita-ansible docker build. Also included is a script to install AWX. See the individual readme files under the ansible-ee and awx folders for more information.
 
-Packages built from this branch will be nita-*-22.8-x where x is the packaging release.
+Packages built from this branch will be nita-*-22.p-x where x is the packaging release.
 This branch also contains patches from other branches or minor modifications as required to support the stability and usability of the release.
 There are also some backwards compatibility packages here for ansible and robot that allow projects written for NITA 3.0.7 to work without having to make any changes.
 
-Note that NITA 22.8 is backward compatible with NITA 21.7 projects, provided the correct ansible and robot containers are installed.
+Note that NITA 23.p should be backward compatible with NITA 22.8 and 21.7 projects, provided the correct ansible and robot containers are installed. However, see the note below about ansible 2.10 changes.
 
 # Copyright
 
-Copyright 2021, Juniper Networks, Inc.
+Copyright 2023, Juniper Networks, Inc.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 
@@ -29,7 +29,13 @@ No containers tagged as "latest" are provided by the package.
 
 ## Ansible 2.10 and greater
 
-Starting with Ansible 2.10, juniper.device ansible collection is added. This collection has been included in the requirements.yml file and will be built-in the container automatically. Existing juniper.junos role is still supported and works with Ansible 2.10. For more information on this transition see https://www.juniper.net/documentation/us/en/software/junos-ansible/ansible/topics/concept/junos-ansible-modules-overview.html and https://galaxy.ansible.com/juniper/device. As of juniper.device collection 1.0.1 it is still currently recommended to continue to use the juniper.junos roles versus juniper.device collection per the galxy website. 
+Starting with Ansible 2.10, juniper.device and junipernetworks.junos ansible collections are added. This collection has been included in the requirements.yml file and will be built-in the container automatically. Existing juniper.junos role is still supported and works with Ansible 2.10. For more information on this transition see https://www.juniper.net/documentation/us/en/software/junos-ansible/ansible/topics/concept/junos-ansible-modules-overview.html and https://galaxy.ansible.com/juniper/device. Juniper.junos roles has been removed and superceded by the newer collections. If the old roles are still required, simply uncomment the line from the ``Dockerfile``:
+
+```
+RUN ansible-galaxy install Juniper.junos -p /etc/ansible/roles/
+```
+
+The network-roles the container copies over to support the NITA Project have been updated to remove the dependencies on the old roles and will use the collections.
 
 # Installing
 
@@ -89,11 +95,6 @@ When invoking these built-in roles the build-folder needs to be passed to the an
 ```
 ansible-playbook -i hosts playbook.yaml --extra-vars "build_dir=container_build_folder"
 ```
-
-### Juniper.junos
-
-This is the main Junos Ansible 2.9 role and is included in the nita-ansible container. This role allows Ansible to communicate with Junos devices. Information on the functionality that is provided with this Ansible role can found here: https://www.juniper.net/documentation/en_US/junos-ansible/topics/reference/general/junos-ansible-modules-overview.html
-
 ### ebgp_ip_fabric
 
 Builds the configuration stanzas that make up a Junos eBGP IP fabric underlay. It includes configurations for forwarding, routing, and policy options as well as interfaces participating in the fabric and BGP protocol configuration. In the sample Excel file included with the NITA package for EVPN data center mentioned above, the tabs that hold the variable information include "underlay_ebgp+", "core_interfaces+", "underlay_ebgp+"

ansible-ee/README.md

@@ -0,0 +1,88 @@
+<H1> AWX Execution Environment </H1>
+
+Files located in this directory are used to create ansible execution environment for AWX/Ansible Tower. the ```build-container.sh``` script calls ansible-builder, which needs to be installed using pip (see https://ansible-builder.readthedocs.io/en/stable/installation/). Ansible-builder requires python3.8 or later to execute. [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) and [Docker](https://docs.docker.com/engine/install/ubuntu/) also need to be installed.  You can also build the container directly by issuing the following commands:
+
+```
+cd context
+docker build  --tag "juniper/nita-ansible-ee:22.9" .
+```
+
+AWX requires ansible EEs to be pulled from a repository. For testing purposes you can setup a local repository. See the instructions at https://www.docker.com/blog/how-to-use-your-own-registry-2/. After building the container, you can add a local repository as follows:
+
+```
+docker run -d -p 5000:5000 --restart always --name registry registry
+docker tag <Ansible-ee-container-image-id> "localhost:5000/ansible-ee"
+docker push localhost:5000/ansible-ee
+```
+
+After this simply define the localhost:5000/ansible-ee in your definition for an Execution Environment in AWX:
+
+![image](https://user-images.githubusercontent.com/6110061/187557638-8b0e00bf-9cfc-4f53-9ef3-c97e7fdf0ad0.png)
+
+<H2> Notes </H2>
+
++ The file ``bindep.txt`` and related configuration in ``nita-ansible-ee.yml`` are here for completeness. AWX documentation and some blogs indicated it is required although container seemed to build and operate fine without it.
++ This container is being tested against a version of the ebgp-wan example from the NITA project. Current files for the AWX test version can be found at  https://github.com/wildsubnet/awx-test. There are certain incompatiabilities between NITA environment and AWX need to be worked out (see todo list below)
+
+## To-Do
+
+See https://github.com/users/wildsubnet/projects/1 for current list. Issues can be found at https://github.com/wildsubnet/nita-awx/issues
+
+
+## AWX Screenshots
+
+How the NITA example data is being loaded into AWX for testing (as of September 22, 2022).
+
+### Template
+<img width="1255" alt="image" src="https://user-images.githubusercontent.com/6110061/236244860-1bf9d65b-f034-4d35-88c4-b904def590c1.png">
+
+
+### Project
+
+![image](https://user-images.githubusercontent.com/6110061/191846831-1f8644e1-96e2-496d-b77e-5a127d46ea61.png)
+
+### Inventories
+
+Note the variable data from ``groups_vars``. Also AWX requires an additional variable definition here ``ansible_python_interpreter: "{{ ansible_playbook_python }}"`` otherwise it will toss a missing module error because it runs the wrong python environment. See [this](https://www.reddit.com/r/ansible/comments/rb80pv/execution_environments_and_pip_module_locations/) thread.
+
+<img width="1269" alt="image" src="https://github.com/wildsubnet/nita-awx/assets/6110061/958f3a4e-c262-4c8d-b92a-b1d56ea40204">
+
+### Hosts
+
+Each individual host has ``host_vars`` defined here 
+
+<img width="1277" alt="image" src="https://user-images.githubusercontent.com/6110061/235500135-b8bf91ca-a3f5-49a4-a625-a5820bba60e4.png">
+<img width="1277" alt="image" src="https://user-images.githubusercontent.com/6110061/191847262-1cefe0fa-5960-4513-8c06-a37247fa4aa3.png">
+
+### Instance Groups
+
+In order to mount /var/tmp into ansible execution environment, you need to update the yaml file for the container group that creates the AWX worker.  
+<img width="1274" alt="image" src="https://user-images.githubusercontent.com/6110061/235755603-e99723e2-d46a-4533-bd7e-c392d94c8f17.png">
+
+Example file:
+```
+apiVersion: v1
+kind: Pod
+metadata:
+  namespace: awx
+spec:
+  serviceAccountName: default
+  automountServiceAccountToken: false
+  containers:
+    - image: quay.io/ansible/awx-ee:latest
+      name: worker
+      args:
+        - ansible-runner
+        - worker
+        - '--private-data-dir=/runner'
+      volumeMounts:
+        - mountPath: /var/tmp
+          name: nb-volume
+          readOnly: false
+  volumes:
+    - hostPath:
+        path: /var/tmp
+        type: ""
+      name: nb-volume
+```
+

ansible-ee/ansible.cfg

@@ -0,0 +1,7 @@
+[galaxy]
+server_list = automation_hub
+
+[galaxy_server.automation_hub]
+url=https://cloud.redhat.com/api/automation-hub/
+auth_url=https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token
+token=my_ah_token

ansible-ee/bindep.txt

@@ -0,0 +1,4 @@
+python38-devel [platform:rpm compile]
+subversion [platform:rpm]
+subversion [platform:dpkg]
+git-lfs [platform:rpm]

ansible-ee/context/Dockerfile

@@ -0,0 +1,41 @@
+ARG EE_BASE_IMAGE=quay.io/ansible/ansible-runner:latest
+ARG EE_BUILDER_IMAGE=quay.io/ansible/ansible-builder:latest
+
+FROM $EE_BASE_IMAGE as galaxy
+ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS=-vvv
+USER root
+
+ADD _build /build
+WORKDIR /build
+
+RUN ansible-galaxy role install -r requirements.yml --roles-path "/usr/share/ansible/roles"
+RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path "/usr/share/ansible/collections"
+
+FROM $EE_BUILDER_IMAGE as builder
+
+COPY --from=galaxy /usr/share/ansible /usr/share/ansible
+
+ADD _build/requirements.txt requirements.txt
+ADD _build/bindep.txt bindep.txt
+RUN ansible-builder introspect --sanitize --user-pip=requirements.txt --user-bindep=bindep.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
+RUN assemble
+
+FROM $EE_BASE_IMAGE
+USER root
+RUN whoami
+RUN cat /etc/os-release
+
+COPY --from=galaxy /usr/share/ansible /usr/share/ansible
+
+COPY --from=builder /output/ /output/
+RUN /output/install-from-bindep && rm -rf /output/wheels
+COPY network-roles network-roles
+RUN cd network-roles/ebgp-ip-fabric && bash install.sh
+RUN cd network-roles/evpn-vxlan-common && bash install.sh
+RUN cd network-roles/evpn-vxlan-fi && bash install.sh
+RUN cd network-roles/evpn-vxlan-erb && bash install.sh
+RUN cd network-roles/evpn-vxlan-hb && bash install.sh
+RUN cd network-roles/evpn-vxlan-sb && bash install.sh
+COPY network-roles/mx_common/ /etc/ansible/roles/mx_common
+RUN mkdir -p /var/nita_configs
+RUN mkdir -p /var/tmp/build

ansible-ee/context/_build/bindep.txt

@@ -0,0 +1,4 @@
+python38-devel [platform:rpm compile]
+subversion [platform:rpm]
+subversion [platform:dpkg]
+git-lfs [platform:rpm]

ansible-ee/context/_build/requirements.txt

@@ -0,0 +1,5 @@
+junos-eznc
+pynetbox
+jxmlease
+xmltodict
+dos2unix

ansible-ee/context/_build/requirements.yml

@@ -0,0 +1,12 @@
+---
+  collections:
+    # Install a collection from Ansible Galaxy.
+    - name: netbox.netbox
+      version: '3.1.1'
+    # Install juniper.device collection from Ansible Galaxy.
+    - name: juniper.device
+      version: '1.0.1'
+    # Install junipernetworks collection from Ansible Galaxy.
+    - name: junipernetworks.junos
+      version: '3.1.0'
+

ansible-ee/context/network-roles/ebgp-ip-fabric/.gitignore

@@ -0,0 +1,129 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/

ansible-ee/context/network-roles/ebgp-ip-fabric/README.md

@@ -0,0 +1,2 @@
+# ebgp-ip-fabric
+Ansible roles to configure a Juniper QFX with an EBGP based IP fabric

ansible-ee/context/network-roles/ebgp-ip-fabric/install.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -e
+
+ANSIBLEROLES=${ANSIBLEROLES:=/etc/ansible/roles}
+
+mkdir -p ${ANSIBLEROLES}
+cp -r roles/* ${ANSIBLEROLES}

ansible-ee/context/network-roles/ebgp-ip-fabric/roles/ebgp_ip_fabric/tasks/main.yaml

@@ -0,0 +1,16 @@
+---
+- name: Creating base interfaces config
+  template: src="interfaces.j2" dest="{{ tmp_dir }}/interfaces.cfg"
+
+- name: Creating protocols config
+  template: src="protocols.j2" dest="{{ tmp_dir }}/protocols.cfg"
+
+- name: Creating policy-options config
+  template: src="policy-options.j2" dest="{{ tmp_dir }}/policy-options.cfg"
+
+- name: Creating forwarding-options config
+  template: src="forwarding-options.j2" dest="{{ tmp_dir }}/forwarding-options.cfg"
+
+- name: Creating routing-options config
+  template: src="routing-options.j2" dest="{{ tmp_dir }}/routing-options.cfg"
+