Code

Test kubernetes in CI#3482

Code

Draft

benclifford wants to merge 76 commits intomasterParsl/parsl:masterfrom benc-k8s-kind-ciParsl/parsl:benc-k8s-kind-ciCopy head branch name to clipboard

+184-5

Commits

Commits on Jun 7, 2024

• Added options to set annotations and a service account in the Kubernetes worker pod configuration

  shishichen

  committed
  8e3d7cf

• Correct punctuation in debug message. hack out tests that won't fail-as-expected as root

  benclifford

  committed
  45269ed

• Fix a couple of docstrings

  benclifford

  committed
  7ceec7a

• a bit of name sanitization for default pod names

  benclifford

  committed
  0cdece2

Commits on Jun 8, 2024

• fiddle with markings to deal with no shared fs and no staging

  Show description for 87d3454
  benclifford

  committed
  87d3454

Commits on Jun 10, 2024

• add config file i've been using

  benclifford

  committed
  954bad7

• Merge remote-tracking branch 'shishichen/add-k8s-pod-options'

  benclifford

  committed
  d3e3828

• add the dockerfile i've been using

  benclifford

  committed
  17a00dd

• beginning of kubernetes-in-CI

  benclifford

  committed
  62e0e36

• push docker image? upgrade ubuntu

  benclifford

  committed
  9086e19

• fiddle with default name

  benclifford

  committed
  26869e6

• Add kubernetes, needed for submitting from inside a cluster

  benclifford

  committed
  16c0a49

• Add more bits for running everything in a kubernetes cluster

  benclifford

  committed
  b03615a

• fix syntax error in github workflow definition

  benclifford

  committed
  e122d19

• Tighten timeout, add some debugging info at the end

  benclifford

  committed
  ee14f6e

• Correct pod name from my test

  benclifford

  committed
  120cf78

• try to stop Job from recreating pod on failure, but instead abort fast

  benclifford

  committed
  5c55fe6

• Randomise test order to see if a test failure is specific to a particular test, or something else

  benclifford

  committed
  b56dfd9

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  dd0f66c

• Add some memory logging

  benclifford

  committed
  f8f5a27

• Allocate more memory to workers

  benclifford

  committed
  f4a7300

• Add a staging_required marker that apparently wasn't breaking things always in k8s

  benclifford

  committed
  ffdb021

• messing with backoff limits and restart policy

  benclifford

  committed
  21711e9

• remove apparently invalid restart policy

  benclifford

  committed
  fb1733e

• Flush out some more staging_required tests (by setting storage_access=[] in local htex test...)

  benclifford

  committed
  73b3e1d

Commits on Jun 12, 2024

• Switch the Kubernetes client call to read_namespaced_pod_status() to read_namespaced_pod(), which is functionally the same but requires fewer permissions.

  shishichen

  committed
  31bc958

Commits on Jun 13, 2024

• Fixed Kubernetes worker container launch command to remove trailing semicolon.

  shishichen

  committed
  8b39024

Commits on Jun 14, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  4538763

• Merge remote-tracking branch 'shishichen/fix-k8s-launch-cmd' into benc-k8s-kind-ci

  benclifford

  committed
  5f43aeb

• Merge remote-tracking branch 'shishichen/swap-k8s-pod-status' into benc-k8s-kind-ci

  benclifford

  committed
  299de99

Commits on Jun 18, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  68e3a5d

Commits on Jun 24, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  fe3c55e

Commits on Jul 2, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  064b833

• Merge remote-tracking branch 'origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  9c6a04e

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  ba5f047

Commits on Jul 7, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  69fbf03

Commits on Jul 31, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  75b7c02

• Merge remote-tracking branch 'origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  780fbb0

Commits on Aug 5, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  2324744

Commits on Aug 19, 2024

• function data in temp

  colinthomas-z80

  committed
  b75a3ae

• use getpass for username

  colinthomas-z80

  committed
  2c18d6c

Commits on Aug 20, 2024

• use tempfile module

  colinthomas-z80

  committed
  c201ec1

• flake etc

  colinthomas-z80

  committed
  9f6b037

Commits on Aug 21, 2024

• Merge branch 'master' into tmp_function_data

  benclifford

  authored
  5ec7cdb

Commits on Aug 22, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  c3f6d45

Commits on Aug 26, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  edf870f

Commits on Sep 3, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  811b8e5

Commits on Sep 4, 2024

• Merge remote-tracking branch 'refs/remotes/origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  7347f64

• Merge branch 'master' into tmp_function_data

  benclifford

  authored
  cd7229f

Commits on Sep 5, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  08f8ce9

• Merge remote-tracking branch 'refs/remotes/origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  5967f01

• Build cctools and run a probably-broken taskvine vs kubernetes test config

  benclifford

  committed
  4938dbf

• fix repr in taskvine

  benclifford

  committed
  98d7693

• install cloudpickle explicitly for taskvine

  benclifford

  committed
  dfc94a8

• Add more time onto job timeout, because more is happening in job with taskvine now

  benclifford

  committed
  47378f3

• revert to 180s test time

  benclifford

  committed
  43af8ef

• Log more to the console, kubernetes style

  benclifford

  committed
  6a32f0f

• Note a (documentation?) bug in taskvine address selection

  benclifford

  committed
  d4fab6a

• force hostname based address config, in line with comment in previous commit

  benclifford

  committed
  21dcae6

• now we're starting taskvine test successfully, give it time to complete

  benclifford

  committed
  e1cce03

• Make taskvine shutdown scale-in more like htex shutdown scale-in

  benclifford

  committed
  4d4b4ba

• enable staging_required tests in taskvine, because taskvine might be able to deal with staging those files properly (or it might be good to flush out more test problems)

  benclifford

  committed
  2e42e5c

• Output timestamps in kubernetes log to help diagnose hangs

  benclifford

  committed
  3ba7e12

• failed to get non-staging tests working, made a note in comments

  benclifford

  committed
  084d797

• correct duplicated 'and' in pytest -k option

  benclifford

  committed
  f34f2b8

Commits on Sep 6, 2024

• Merge remote-tracking branch 'colinthomas-z80/tmp_function_data' into benc-k8s-kind-ci

  benclifford

  committed
  60a8611

Commits on Oct 21, 2024

• Add utils to sanitize strings for DNS compliance

  Show description for 1f09e5c
  rjmello

  committed
  1f09e5c

• Ensure k8s pod names/labels are RFC 1123 compliant

  Show description for 83278c2
  rjmello

  committed
  83278c2

• Use hex value for k8s job ID instead of pod name

  rjmello

  committed
  86ade32

• Add tests for KubernetesProvider submit

  rjmello

  committed
  0c4d541

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  08693ab

• Merge remote-tracking branch 'origin/rjmello-kube-pod-names' into benc-k8s-kind-ci, with conflicts

  benclifford

  committed
  415f780

• Fix some bad merge

  benclifford

  committed
  c78defa

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  54ea143

Commits on Oct 31, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  535289f

Commits on Nov 1, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  fd26ddd