Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: integration of argoexec rock #10

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,20 @@
# argo-workflows-rocks
ROCKs for Argoi Workflows
# ROCKs for Argo Workflows

This repository contains collection of ROCKs for Argo Workflows.

- argocli/ contains Rockcraft project for Argo Workflows CLI (TO BE DEPRECATED?)
- argoexec/ contains Rockcraft project for Argo Workflows executor.
- workflow-controller/ contains Rockcraft project for Argo Workflows controller.

To build ROCK execute the following command in correspomnding directory:

```
rockcraft pack
```

To run sanity tests on the ROCK execute the following command in correspomnding directory:

```
tox -e sanity
```

2 changes: 2 additions & 0 deletions argoexec/requirements-sanity.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
charmed_kubeflow_chisme
pytest
94 changes: 94 additions & 0 deletions argoexec/requirements-sanity.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
#
# This file is autogenerated by pip-compile with Python 3.8
# by the following command:
#
# pip-compile requirements-sanity.in
#
anyio==3.7.1
# via httpcore
attrs==23.1.0
# via jsonschema
certifi==2023.7.22
# via
# httpcore
# httpx
# requests
charmed-kubeflow-chisme==0.2.0
# via -r requirements-sanity.in
charset-normalizer==3.2.0
# via requests
deepdiff==6.2.1
# via charmed-kubeflow-chisme
exceptiongroup==1.1.3
# via
# anyio
# pytest
h11==0.14.0
# via httpcore
httpcore==0.17.3
# via httpx
httpx==0.24.1
# via lightkube
idna==3.4
# via
# anyio
# httpx
# requests
importlib-resources==6.0.1
# via jsonschema
iniconfig==2.0.0
# via pytest
jinja2==3.1.2
# via charmed-kubeflow-chisme
jsonschema==4.17.3
# via serialized-data-interface
lightkube==0.14.0
# via charmed-kubeflow-chisme
lightkube-models==1.27.1.4
# via lightkube
markupsafe==2.1.3
# via jinja2
ops==2.5.1
# via
# charmed-kubeflow-chisme
# serialized-data-interface
ordered-set==4.1.0
# via deepdiff
packaging==23.1
# via pytest
pkgutil-resolve-name==1.3.10
# via jsonschema
pluggy==1.2.0
# via pytest
pyrsistent==0.19.3
# via jsonschema
pytest==7.4.0
# via -r requirements-sanity.in
pyyaml==6.0.1
# via
# lightkube
# ops
# serialized-data-interface
requests==2.31.0
# via serialized-data-interface
ruamel-yaml==0.17.32
# via charmed-kubeflow-chisme
ruamel-yaml-clib==0.2.7
# via ruamel-yaml
serialized-data-interface==0.7.0
# via charmed-kubeflow-chisme
sniffio==1.3.0
# via
# anyio
# httpcore
# httpx
tenacity==8.2.3
# via charmed-kubeflow-chisme
tomli==2.0.1
# via pytest
urllib3==1.26.16
# via requests
websocket-client==1.6.2
# via ops
zipp==3.16.2
# via importlib-resources
46 changes: 46 additions & 0 deletions argoexec/tests/test_rock.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Copyright 2023 Canonical Ltd.
# See LICENSE file for licensing details.
#
# Tests for required artifacts to be present in the ROCK image.
#

from charmed_kubeflow_chisme.rock import CheckRock
from pathlib import Path

import os
import logging
import random
import pytest
import string
import subprocess
import yaml
from pytest_operator.plugin import OpsTest

@pytest.fixture()
def rock_test_env():
"""Yelds random docker container name, then cleans it up after."""
container_name = "".join([str(i) for i in random.choices(string.ascii_lowercase, k=8)])
yield container_name

try:
subprocess.run(["docker", "rm", container_name], check=True, capture_output=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a docker Python SDK, maybe you want to explore that option

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is the size of the dependencies needed for testing. Last time we introduced Chisme for testing and it was rejected, because it was a large dependency. IMO the same rationale applies to this Python Docker SDK.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that rationale applies here, as this is in fact, a Python script, and not a command that runs inside a tox file. It's up to you if you want to use the SDK, it was a suggestion to make this script cleaner.

except Exception as error:
print(f"Failed to remove Docker container {container_name} error: {str(error)}")


@pytest.mark.abort_on_fail
def test_rock_artefacts(ops_test: OpsTest, rock_test_env):
"""Test that all artefacts are present in the ROCK: binaries."""
check_rock = CheckRock("rockcraft.yaml")
container_name = rock_test_env
LOCAL_ROCK_IMAGE = f"{check_rock.get_image_name()}:{check_rock.get_version()}"

# verify that all artifacts are in correct locations
try:
subprocess.run(
["docker", "run", "--name", container_name, LOCAL_ROCK_IMAGE, "exec", "pebble", "ls", "/bin/argoexec"],
check=True
)
except subprocess.CalledProcessError as error:
print(f"Failed to execute docker run for container {container_name} error: {str(error)}")
assert 0
38 changes: 38 additions & 0 deletions argoexec/tox.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Copyright 2023 Canonical Ltd.
# See LICENSE file for licensing details.
[tox]
skipsdist = True
skip_missing_interpreters = True

[testenv]
setenv =
PYTHONPATH={toxinidir}
PYTHONBREAKPOINT=ipdb.set_trace
CHARM_REPO=https://github.com/canonical/argo-operators.git
CHARM_BRANCH=main
LOCAL_CHARM_DIR=charm_repo

[testenv:update-requirements]
allowlist_externals =
pip-compile
commands =
pip-compile requirements-sanity.in
deps =
pip-tools
description = Update requirements files by executing pip-compile on all requirements*.in files, including those in subdirs.

[testenv:sanity]
passenv = *
allowlist_externals =
bash
deps =
-r requirements-sanity.txt
commands =
bash -c 'NAME=$(yq eval .name rockcraft.yaml) && \
VERSION=$(yq eval .version rockcraft.yaml) && \
ARCH=$(yq eval ".platforms | keys" rockcraft.yaml | awk -F " " '\''{ print $2 }'\'') && \
ROCK="$\{NAME\}_$\{VERSION\}_$\{ARCH\}" && \
sudo skopeo --insecure-policy copy oci-archive:$ROCK.rock docker-daemon:$ROCK:$VERSION && \
docker save $ROCK > $ROCK.tar'
# run rock tests
pytest -v --tb native --show-capture=all --log-cli-level=INFO {posargs} {toxinidir}/tests