create a catlin-validate task to lint task.yaml submitted to tekton c…

…atalog
tektoncd · Aug 6, 2023 · 069debb · 069debb
1 parent df36b38
commit 069debb
Show file tree

Hide file tree

Showing 13 changed files with 517 additions and 0 deletions.
diff --git a/task/catlin-validate/0.1/README.md b/task/catlin-validate/0.1/README.md
@@ -0,0 +1,145 @@
+# Catlin Validate
+
+* [Catlin](https://github.com/tektoncd/catlin) is a command-line tool that Lints Tekton Resources and Catalogs.
+* Catlin Validate task supports running catlin validate for
+    * An isolated task yaml that you want to submit to tekton catalog
+    * A set of tasks that you want to submit to tekton catalog
+
+## Catlin validate use cases
+
+### Run catlin validate to quickly validate a sample task
+* Imagine you are developing a task and you quickly want to run a catlin validate to check if this task can be submitted to the tekton catalog
+* Without downloading catlin or having any knowledge on how to run catlin you can check if your task passes all catlin checks by simply running a taskrun
+* A working taskrun is provided in the [tests](../0.1/tests/taskrun-success.yaml) folder. 
+* [This](../0.1/tests/taskrun-success.yaml) taskrun validates a sample hello task. 
+* The hello task yaml which we want to validate is injected in the source workspace in the appropriate folder structure using  [this](../0.1/tests/source-success-configmap.yaml) configMap.
+* The filepath of the hello.yaml is supplied as a catlin-input using [this](../0.1/tests/input-configmap.yaml) configMap
+* Thus without knowing anything about how to run catlin validate you can quickly lint your task yaml by simply following above steps
+
+### Integrate catlin as a part of CI/CD pipelines for task
+* You may have an internal tekton catalog or want to contribute to open source tekton catalog and before submitting your task to these you might want to run catlin validate checks as a part of a PR pipeline
+* To acheive this you can easily integrate catlin-validate task to validate your task yamls as part of a PR CI/CD tekton pipeline. 
+
+
+### Run catlin validate for multiple task submissions in your tekton catalog
+* You might have multiple tasks submitted as a part of a PR to the tekton catalog
+* Catlin Validate Task can be run to validate all the task files in the inputFile provided to the task
+* Simply put the paths of all tasks in $(params.inputFile) and catlin validate task will lint all the tasks
+* Check [Results](#results) to see what is the final result of the task when multiple tasks are provided as input to catlin-validate
+
+
+
+## Parameters
+
+### outputFile 
+Name of the file that holds catlin validate output for each task. This file will be available in [catlin-output](#catlin-output) workspace
+### inputFile
+File that contains catlin input. This file contains relative paths to the task yaml's delimited by newline. Paths should adhere to the tekton task catalog standards. Check [input-configmap](../0.1/tests/input-configmap.yaml) for an example of an inputFile
+
+### ignoreWarnings
+Ignores any warnings in catlin output. If catlin output has only success and warnings, all warnings are ignored and final catlin output will be success
+
+
+## Workspaces
+
+### source
+This workspace contains files on which catlin will be run. This can be the git repo of your tekton catalog. task yamls in this folder should have a path that follows tekton-catalog folder structure. Check [this](../0.1/tests/source-success-configmap.yaml) configmap and how it is injected into the cource workspace in [this](../0.1/tests/taskrun-success.yaml) taskrun as an example
+### catlin-input
+A workspace that contains [$(params.inputFile)](#inputfile) Check [this](../0.1/tests/input-configmap.yaml) configmap and how it is injected onto the catlin-input workspace in [this](../0.1/tests/taskrun-success.yaml) taskrun as an example
+### catlin-output
+A workspace that contains output of catlin validate. Output of each catlin validate command is sent to [$(params.outputFle)](#outputfile) in this workspace
+
+## Results
+
+### catlin-status
+* catlin-status contains the final catlin-validate task result Success, Warning or Failure  which will depend on the cumulative output of catlin validate on all tasks
+    * Final result will be success if there is no error and no warning for any of the task specified in [inputFile](#inputfile). 
+    * Final result will be warning if there is no error and a warning for any of the task specified in [inputFile](#inputfile).
+    * Final result will be failure if there an error in even one of the tasks specified in [inputFile](#inputfile)
+    * Output for each catlin validate commmand is stored in [outFile](#outputfile) in [catlin-output](#catlin-output) workspace
+
+
+## Platforms
+
+The Task can be run on `linux/amd64` platform.
+
+## Example
+* [tests](../0.1/tests) folder contains an example which you can use as a reference to create a taskrun that will lint your task before submitting it to a tekton catalog
+* You will need a kubernetes cluster to test the catlin task. You can use [minikube](https://minikube.sigs.k8s.io/docs/start/) or [Kind](https://kind.sigs.k8s.io/) to install a kubernetes cluster locally.
+
+### Catlin Validate Success
+* Follow the below steps to run the sample taskrun in your local cluster which validates a properly constructed task that adheres to catlin linting standards
+```
+git clone https://github.com/tektoncd/catalog.git
+cd catalog/task/catlin-validate/0.1/tests
+kubectl apply -f input-configmap.yaml
+kubectl apply -f source-success-configmap.yaml
+kubectl create -f taskrun-success.yaml
+kubectl logs catlin-run-pod 
+```
+* Pod log output
+```Defaulted container "step-catlin" out of: step-catlin, prepare (init), place-scripts (init), working-dir-initializer (init)
+-------catlin output--------
+FILE: /workspace/source/task/hello/0.1/hello.yaml
+
+-------catlin output end--------
+```
+
+
+
+### Catlin Validate Warn
+* Follow the below steps to run the sample taskrun in your local cluster which validates a task that has a warning.
+* **Note** Warning will not fail the task. Task will still succeed but with [catlin-status](#catlin-status) as warning
+```
+git clone https://github.com/tektoncd/catalog.git
+cd catalog/task/catlin-validate/0.1/tests
+kubectl apply -f input-configmap.yaml
+kubectl apply -f source-warn-configmap.yaml
+kubectl create -f taskrun-warn.yaml
+kubectl logs catlin-run-pod 
+```
+* Pod log output
+```Defaulted container "step-catlin" out of: step-catlin, prepare (init), place-scripts (init), working-dir-initializer (init)
+-------catlin output--------
+FILE: /workspace/source/task/hello/0.1/hello.yaml
+WARN : Step "echo" uses image "$(params.image)" that contains variables; skipping validation
+
+-------catlin output end--------
+```
+* Catlin output logs are also stored in [$(params.outputFile)](#outputfile) in [catlin-output](#catlin-output) workspace. This workspace can be passed on to other tasks in case other tasks need to consume this output. An example would be git-comment task can use this workspace to post the catlin output logs to the PR created to add a task to tekton catalog
+
+
+### Catlin Validate Error
+* Follow the below steps to run the sample taskrun in your local cluster which validates a task that is non-conforming to the tekton-catalog standards
+* **Note** Error will result in [catlin-status](#catlin-status) having the value as failure.
+```
+git clone https://github.com/tektoncd/catalog.git
+cd catalog/task/catlin-validate/0.1/tests
+kubectl apply -f input-configmap.yaml
+kubectl apply -f source-error-configmap.yaml
+kubectl create -f taskrun-error.yaml
+kubectl logs catlin-run-pod 
+```
+* Pod logs output
+```
+Defaulted container "step-catlin" out of: step-catlin, prepare (init), place-scripts (init), working-dir-initializer (init)
+Error: /workspace/source/task/hello/0.1/hello.yaml failed validation
+-------catlin output--------
+FILE: /workspace/source/task/hello/0.1/hello.yaml
+ERROR: Resource path is invalid; expected path: task/hello/hello.yaml
+ERROR: Task: tekton.dev/v1beta1 - name: "hello" must have a label "app.kubernetes.io/version" to indicate version
+ERROR: Task: tekton.dev/v1beta1 - name: "hello" is missing a mandatory annotation for minimum pipeline version("tekton.dev/pipelines.minVersion")
+ERROR: Task: tekton.dev/v1beta1 - name: "hello" is missing a mandatory annotation for category("tekton.dev/categories")
+ERROR: Category not defined
+You can choose from the categories present at location: https://raw.githubusercontent.com/tektoncd/hub/main/config.yaml"
+HINT : Task: tekton.dev/v1beta1 - name: "hello" is missing a readable display name annotation("tekton.dev/displayName")
+ERROR: Task: tekton.dev/v1beta1 - name: "hello" must have a description
+HINT : Task: tekton.dev/v1beta1 - name: "hello" is easily discoverable if it has annotation for tag "tekton.dev/tags"
+HINT : Task: tekton.dev/v1beta1 - name: "hello" is more usable if it has "tekton.dev/platforms" annotation about platforms to run
+WARN : Step "echo" uses image "alpine"; consider using a fully qualified name - e.g. docker.io/library/ubuntu:1.0
+ERROR: Step "echo" uses image "alpine" which must be tagged with a specific version
+
+-------catlin output end--------
+```
+
+* Catlin output is also stored in [$(params.outputFile)](#outputfile) in [catlin-output](#catlin-output) workspace. This workspace can be passed on to other tasks in case other tasks need to consume this output. An example would be binding the output workspace to a workspace in git-comment to post the catlin output as a comment for a PR of a new task submission
diff --git a/task/catlin-validate/0.1/catlin-validate.yaml b/task/catlin-validate/0.1/catlin-validate.yaml
@@ -0,0 +1,112 @@
+apiVersion: tekton.dev/v1beta1
+kind: Task
+metadata:
+  name: catlin-validate
+  labels:
+    app.kubernetes.io/version: "0.1"
+  annotations:
+    tekton.dev/categories: Testing
+    tekton.dev/pipelines.minVersion: "0.17.0"
+    tekton.dev/tags: "tekton task yaml validation"
+    tekton.dev/platforms: "linux/amd64"
+    tekton.dev/displayName: catlin
+spec:
+  description: >-
+    Catlin(https://github.com/tektoncd/catlin) is a command-line tool that Lints Tekton Resources and Catalogs.
+    Catlin Validate task supports running catlin validate for
+      1. An isolated task yaml that you want to submit to tekton catalog
+      2. A set of tasks that you want to submit to tekton catalog
+  params:
+    - name: outputFile
+      description: File that contains final catlin output. Catlin output will be stored in /workspaces/catlin-output/$(params.outputFile)
+      type: string
+      default: catlin-output.txt
+    - name: inputFile
+      description: |
+        File that contains catlin input. This file contains relative paths to the task yaml's delimited by newline
+        this file will be read from /workspaces/catlin-input/$(params.inputFile) and catlin validate will be applied to each file
+        example if file contains below paths (as you can see the paths are separated by newlines)
+        task/git-clone/0.9/git-clone.yaml
+        task/git-clone/0.1/git-comment.yaml
+        it will run catlin on the above paths in the source workspace one after another and store its output in catlin-output workspace
+        final command run will be 
+        'catlin validate $(workspaces.source.path)/task/git-clone/0.9/git-clone.yaml >> $(workspaces.catlin-output.path)/output.txt'
+        'catlin validate $(workspaces.source.path)/task/git-clone/0.1/git-comment.yaml >> $(workspaces.catlin-output.path)/output.txt'
+      type: string
+    - name: ignoreWarnings
+      description: |
+        Ignores any warnings in catlin output. 
+        If catlin output has only success and warnings, all warnings are ignored and final catlin output will be success
+      type: string
+      default: "false"
+  results:
+    - name: catlin-status
+      description: |
+        final status of catlin command. If all catlin validate command succeeds status will be set to 'success', if any one command validate command fails 
+        status will be set to 'failure', If there is no failure and atleast one catline validate command is warning the status will be set to 'warning'
+  workspaces:
+  - name: source
+    description: |
+      This workspace contains files on which catlin will be run. task yamls in this folder should have a path that follows tekton-catalog folder structure.
+      example of folder structure: task/git-clone/0.9/git-clone.yaml
+      refer https://github.com/tektoncd/catalog for more details
+  - name: catlin-input
+    description: |
+      This workspace contains a file $(params.inputFile) delimited by newline containing paths to task yaml's on which catlin validate will be applied
+  - name: catlin-output
+    description: This workspace is where catlin output will be stored
+  steps:
+    - name: catlin-validate
+      image: gcr.io/tekton-releases/dogfooding/catlin:v20230721-0755082305@sha256:8db5e290b9a1b7794c536dbc826bb9cf3eeb82c41b251b7caee12204bcfb6664
+      env:
+      - name: OUTPUT_FILE
+        value: $(params.outputFile)
+      - name: INPUT_FILE
+        value: $(params.inputFile)
+      - name: CATLIN_OUTPUT_WORKSPACE_PATH
+        value: $(workspaces.catlin-output.path)
+      - name: IGNORE_WARNINGS
+        value: $(params.ignoreWarnings)
+      workingDir: $(workspaces.catlin-input.path)
+      script: |
+        #!/usr/bin/env bash
+        rm -rf "${CATLIN_OUTPUT_WORKSPACE_PATH:?}"/"$OUTPUT_FILE"
+        while IFS="" read -r p || [ -n "$p" ]
+        do
+          catlin validate "$(workspaces.source.path)"/"$p" >> "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE" 2>&1
+          echo "" >> "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE"
+        done < "$INPUT_FILE"
+
+        echo ""
+        echo "--------------------------------CATLIN OUTPUT START------------------------------------------"
+        echo ""
+        cat "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE"
+        echo "--------------------------------CATLIN OUTPUT END--------------------------------------------"
+        echo ""
+
+        STR=$(cat "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE")
+        ERROR='ERROR'
+        WARN='WARN'
+        
+        if [[ "$IGNORE_WARNINGS" == "true" ]]; then
+          echo "Ignore Warnings is set to true, any warnings in catlin output will be ignored"
+        fi
+
+        if [[ "$STR" == *"$ERROR"* || "$STR" == *"Error"* ]]; then
+          echo "failure" | tr -d "\n" > "$(results.catlin-status.path)"
+          echo ""
+          echo "Final output of catlin validate is failed since one of the files Errored out."
+          exit 1
+        elif [[ "$STR" == *"$WARN"* && "$IGNORE_WARNINGS" == "false" ]]; then
+          echo "warning" | tr -d "\n" > "$(results.catlin-status.path)"
+          echo ""
+          echo "Final output of catlin validate is warning since one of the files has WARNING in its output."
+        else
+          echo "success" | tr -d "\n" > "$(results.catlin-status.path)"
+          echo ""
+          echo "Final output of catlin validate is success."
+        fi
+       
+
+
+      
diff --git a/task/catlin-validate/0.1/tests/input-configmap.yaml b/task/catlin-validate/0.1/tests/input-configmap.yaml
@@ -0,0 +1,8 @@
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: catlin-input
+data:
+  input.txt: |
+    task/hello/0.1/hello.yaml
diff --git a/task/catlin-validate/0.1/tests/multiple-input-configmap.yaml b/task/catlin-validate/0.1/tests/multiple-input-configmap.yaml
@@ -0,0 +1,9 @@
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: catlin-input-multiple
+data:
+  input.txt: |
+    task/hello/0.1/hello.yaml
+    task/bye/0.1/bye.yaml
diff --git a/task/catlin-validate/0.1/tests/source-error-configmap.yaml b/task/catlin-validate/0.1/tests/source-error-configmap.yaml
@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: source-error
+data:
+  hello.yaml: |
+    apiVersion: tekton.dev/v1beta1
+    kind: Task
+    metadata:
+      name: hello
+    spec:
+      steps:
+        - name: echo
+          image: alpine
+          script: |
+            #!/bin/sh
+            echo "Hello World"   
diff --git a/task/catlin-validate/0.1/tests/source-multiple-configmap.yaml b/task/catlin-validate/0.1/tests/source-multiple-configmap.yaml
@@ -0,0 +1,50 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: source-multiple
+data:
+  hello.yaml: |
+    apiVersion: tekton.dev/v1beta1
+    kind: Task
+    metadata:
+      name: hello
+      labels:
+        app.kubernetes.io/version: "0.1"
+      annotations:
+        tekton.dev/categories: Testing
+        tekton.dev/pipelines.minVersion: "0.17.0"
+        tekton.dev/tags: "testing"
+        tekton.dev/platforms: "linux/amd64"
+        tekton.dev/displayName: hello
+    spec:
+      description: "hello task"
+      steps:
+        - name: echo
+          image: docker.io/library/ubuntu:18.04
+          script: |
+            #!/bin/sh
+            echo "Hello World"
+  bye.yaml: |
+    apiVersion: tekton.dev/v1beta1
+    kind: Task
+    metadata:
+      name: bye
+      labels:
+        app.kubernetes.io/version: "0.1"
+      annotations:
+        tekton.dev/categories: Testing
+        tekton.dev/pipelines.minVersion: "0.17.0"
+        tekton.dev/tags: "testing"
+        tekton.dev/platforms: "linux/amd64"
+        tekton.dev/displayName: bye
+    spec:
+      description: "bye task"
+      params:
+        - name: image
+          type: string
+      steps:
+        - name: echo
+          image: $(params.image)
+          script: |
+            #!/bin/sh
+            echo "bye"
diff --git a/task/catlin-validate/0.1/tests/source-success-configmap.yaml b/task/catlin-validate/0.1/tests/source-success-configmap.yaml
@@ -0,0 +1,26 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: source-success
+data:
+  hello.yaml: |
+    apiVersion: tekton.dev/v1beta1
+    kind: Task
+    metadata:
+      name: hello
+      labels:
+        app.kubernetes.io/version: "0.1"
+      annotations:
+        tekton.dev/categories: Testing
+        tekton.dev/pipelines.minVersion: "0.17.0"
+        tekton.dev/tags: "testing"
+        tekton.dev/platforms: "linux/amd64"
+        tekton.dev/displayName: hello
+    spec:
+      description: "hello task"
+      steps:
+        - name: echo
+          image: docker.io/library/ubuntu:18.04
+          script: |
+            #!/bin/sh
+            echo "Hello World"