README.md

Hitman

A daemon for Kubernetes to kill target resources under user-defined templated conditions

Motivation

In today's fast-paced environments, Kubernetes clusters often manage systems that dynamically create and destroy resources automatically. Examples of these are pipelines and cronjobs.

However, these automated processes can sometimes get stuck, causing disruptions that affect the smooth operation of the entire system. Often, simply terminating some of these objects can restore normalcy.

There is a need for a solution that empowers Kubernetes administrators to automate this cleanup process efficiently. This project exists to provide a robust agent for automating the deletion of potential stuck resources, ensuring your Kubernetes clusters run smoothly and reliably.

Flags

As every configuration parameter can be defined in the config file, there are only few flags that can be defined. They are described in the following table:

Name	Description	Default	Example
--config	Path to the YAML config file	hitman.yaml	--config ./hitman.yaml
--log-level	Verbosity level for logs	info	--log-level info
--disable-trace	Disable showing traces in logs	false	--disable-trace

Output is thrown always in JSON as it is more suitable for automations

hitman run \
    --log-level=info
    --config="./hitman.yaml"

Examples

Here you have a complete example. More up-to-date one will always be maintained in docs/prototypes directory here

version: v1alpha1
kind: Hitman
metadata:
  name: killing-sample
spec:
  synchronization:
    time: 1m

  resources:

    - target:
        group: ""
        version: v1
        resource: pods

        # Select the resources by their name
        # Choose one of the following options
        name:
          matchRegex: ^(coredns-)
          #matchExact: "coredns-xxxxxxxxxx-yyyyy"

        # Select the namespace where the resources are located
        # Choose one of the following options
        namespace:
          matchRegex: ^(kube-system)
          #matchExact: kube-system

      conditions:

      # Delete the resources when they are older than 10 minutes
      - key: |-
          {{/* Define some variables */}}
          {{- $maxAgeMinutes := 10 -}}

          {{- $nowTimestamp := (now | unixEpoch) -}}
          {{- $podStartTime := (toDate "2006-01-02T15:04:05Z07:00" .object.status.startTime) | unixEpoch -}}

          {{/* Calculate the age of the resource in minutes */}}
          {{- $minutedFromNow := int (round (div (sub $nowTimestamp $podStartTime) 60) 0) -}}

          {{/* Print true ONLY if the resource is older than 5 minutes */}}
          {{- printf "%v" (ge $minutedFromNow $maxAgeMinutes) -}}
        value: true

As you probably noticed in the previous example, conditions are made using vitamin Golang template (better known as Helm template), so all the functions available in Helm are available here too. This way you start creating wonderful deletions from first minute.

Sometimes it's needed to process the whole list of targets before being evaluated for deletion. This is useful to analyze the resources deeper, and group or drop some of them based on user-defined desires.

Because of that, there is a special optional preStep section that is evaluated just one time before evaluating conditions. In that section an object .targets is injected.

It's also possible to store custom content into variables that will be alive during the whole evaluation lifecycle, so stored content will be available inside conditions templates into .vars object.

To store content, there is a function called setVar. It can be used as follows:

version: v1alpha1
kind: Hitman
metadata:
  name: killing-sample
spec:
  resources:

    - ...

      # (Optional) Define a preStep
      preStep: |
        {{/* The whole list of targets is available */}}
        {{ $retrievedTargets := .targets }}


        {{/* Let's define example data */}}
        {{- $someDataForLater := dict "name" "example-name" "namespace" "example-namespace" -}}


        {{/* Store your data under your desired key. You can use as many keys as needed */}}
        {{- setVar "example" $someDataForLater -}}

      conditions:

      # Delete the resources when they are older than 10 minutes
      - key: |-
          {{/* Retrieve a previously defined variable if needed */}}
          {{ $myStoredKey := .vars.example }}

          {{/* Add some logic here to evaluate your conditions */}}
        value: true

Tip

Another useful function that can be used in templates is logPrintf. It accepts the same params as printf but throw the result in controller's logs instead of returning it

How to deploy

This project is designed specially for Kubernetes, but also provides binary files and Docker images to make it easy to be deployed however wanted

Binaries

Binary files for most popular platforms will be added to the releases

Kubernetes

You can deploy hitman in Kubernetes using Helm as follows:

helm repo add hitman https://achetronic.github.io/hitman/

helm upgrade --install --wait hitman \
  --namespace hitman \
  --create-namespace achetronic/hitman

More information and Helm packages here

Docker

Docker images can be found in GitHub's packages related to this repository

Do you need it in a different container registry? I think this is not needed, but if I'm wrong, please, let's discuss it in the best place for that: an issue

How to contribute

We are open to external collaborations for this project: improvements, bugfixes, whatever.

For doing it, open an issue to discuss the need of the changes, then:

• Fork the repository
• Make your changes to the code
• Open a PR and wait for review

The code will be reviewed and tested (always)

We are developers and hate bad code. For that reason we ask you the highest quality on each line of code to improve this project on each iteration.

License

Copyright 2022.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.