A daemon for Kubernetes to kill target resources under user-defined templated conditions
In today's fast-paced environments, Kubernetes clusters often manage systems that dynamically create and destroy resources automatically. Examples of these are pipelines and cronjobs.
However, these automated processes can sometimes get stuck, causing disruptions that affect the smooth operation of the entire system. Often, simply terminating some of these objects can restore normalcy.
There is a need for a solution that empowers Kubernetes administrators to automate this cleanup process efficiently. This project exists to provide a robust agent for automating the deletion of potential stuck resources, ensuring your Kubernetes clusters run smoothly and reliably.
As every configuration parameter can be defined in the config file, there are only few flags that can be defined. They are described in the following table:
| Name | Description | Default | Example |
|---|---|---|---|
--config |
Path to the YAML config file | hitman.yaml |
--config ./hitman.yaml |
--log-level |
Verbosity level for logs | info |
--log-level info |
--disable-trace |
Disable showing traces in logs | false |
--disable-trace |
Output is thrown always in JSON as it is more suitable for automations
hitman run \
--log-level=info
--config="./hitman.yaml"Here you have a complete example. More up-to-date one will always be maintained in
docs/prototypes directory here
version: v1alpha1
kind: Hitman
metadata:
name: killing-sample
spec:
synchronization:
time: 1m
resources:
- target:
group: ""
version: v1
resource: pods
# Select the resources by their name
# Choose one of the following options
name:
matchRegex: ^(coredns-)
#matchExact: "coredns-xxxxxxxxxx-yyyyy"
# Select the namespace where the resources are located
# Choose one of the following options
namespace:
matchRegex: ^(kube-system)
#matchExact: kube-system
conditions:
# Delete the resources when they are older than 10 minutes
- key: |-
{{/* Define some variables */}}
{{- $maxAgeMinutes := 10 -}}
{{- $nowTimestamp := (now | unixEpoch) -}}
{{- $podStartTime := (toDate "2006-01-02T15:04:05Z07:00" .object.status.startTime) | unixEpoch -}}
{{/* Calculate the age of the resource in minutes */}}
{{- $minutedFromNow := int (round (div (sub $nowTimestamp $podStartTime) 60) 0) -}}
{{/* Print true ONLY if the resource is older than 5 minutes */}}
{{- printf "%v" (ge $minutedFromNow $maxAgeMinutes) -}}
value: trueAs you probably noticed in the previous example, conditions are made using vitamin Golang template (better known as Helm template), so all the functions available in Helm are available here too. This way you start creating wonderful deletions from first minute.
Sometimes it's needed to process the whole list of targets before being evaluated for deletion. This is useful to analyze the resources deeper, and group or drop some of them based on user-defined desires.
Because of that, there is a special optional preStep section that is evaluated just one time before evaluating conditions.
In that section an object .targets is injected.
It's also possible to store custom content into variables that will be alive during the whole evaluation lifecycle, so stored content
will be available inside conditions templates into .vars object.
To store content, there is a function called setVar. It can be used as follows:
version: v1alpha1
kind: Hitman
metadata:
name: killing-sample
spec:
resources:
- ...
# (Optional) Define a preStep
preStep: |
{{/* The whole list of targets is available */}}
{{ $retrievedTargets := .targets }}
{{/* Let's define example data */}}
{{- $someDataForLater := dict "name" "example-name" "namespace" "example-namespace" -}}
{{/* Store your data under your desired key. You can use as many keys as needed */}}
{{- setVar "example" $someDataForLater -}}
conditions:
# Delete the resources when they are older than 10 minutes
- key: |-
{{/* Retrieve a previously defined variable if needed */}}
{{ $myStoredKey := .vars.example }}
{{/* Add some logic here to evaluate your conditions */}}
value: trueTip
Another useful function that can be used in templates is logPrintf. It accepts the same params as printf
but throw the result in controller's logs instead of returning it
This project is designed specially for Kubernetes, but also provides binary files and Docker images to make it easy to be deployed however wanted
Binary files for most popular platforms will be added to the releases
You can deploy hitman in Kubernetes using Helm as follows:
helm repo add hitman https://achetronic.github.io/hitman/
helm upgrade --install --wait hitman \
--namespace hitman \
--create-namespace achetronic/hitmanMore information and Helm packages here
Docker images can be found in GitHub's packages related to this repository
Do you need it in a different container registry? I think this is not needed, but if I'm wrong, please, let's discuss it in the best place for that: an issue
We are open to external collaborations for this project: improvements, bugfixes, whatever.
For doing it, open an issue to discuss the need of the changes, then:
- Fork the repository
- Make your changes to the code
- Open a PR and wait for review
The code will be reviewed and tested (always)
We are developers and hate bad code. For that reason we ask you the highest quality on each line of code to improve this project on each iteration.
Copyright 2022.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.