feat: conditional profiles (#726)

* build(pip): (deps-dev): bump pytest from 7.1.2 to 7.2.1 (#684) Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.1.2 to 7.2.1. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](pytest-dev/pytest@7.1.2...7.2.1) --- updated-dependencies: - dependency-name: pytest dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: update poetry version and generate new lock file (#698) * chore: add missing setuptools for mike (#699) * chore: add gnome-keyring (#700) * fix: refactor varBinds mechanism and implement compound indexes (#704) * fix: refactor varBinds mechanism and implement compound indexes * fix: add docstrings and fix unit tests * fix: mapping method for profiles * fix: looking for a parent in a map * fix: run pre-commit * fix: bring back the previous behavior of IF-MIB and SNMPv2-MIB during walk profile operation * chore(release): 1.8.7-beta.1 ## [1.8.7-beta.1](v1.8.6...v1.8.7-beta.1) (2023-02-23) ### Bug Fixes * refactor varBinds mechanism and implement compound indexes ([#704](#704)) ([f4007f4](f4007f4)) * fix: cleanup helm structure different traps and polling (#688) * fix: helm cleanup, add more render manifests, cleanup values.yaml file, document configuration * fix: run make render and update Makefile with render_manifests.sh * fix: delete loggers, run pre-commit * fix: selector labels * fix: selector labels for yaml cm used by scheduler * fix: update documentation * fix: add polling and trap configuration to the offline and lightweight examples --------- Co-authored-by: semantic-release-bot <[email protected]> Co-authored-by: Wojciech Zyla <[email protected]> * chore(release): 1.8.7-beta.2 ## [1.8.7-beta.2](v1.8.7-beta.1...v1.8.7-beta.2) (2023-02-28) ### Bug Fixes * cleanup helm structure different traps and polling ([#688](#688)) ([86957f9](86957f9)) * fix: conditional profiles (#713) * fix: implement conditional profiles feature, add unit tests, refactor adding to assigned_profiles listm improve varBinds validation, change tagging conditional profiles in mongo, add documentation * fix: apply pre-commit * fix: documentation * chore: update CHANGELOG.md * chore(release): 1.8.7-beta.3 ## [1.8.7-beta.3](v1.8.7-beta.2...v1.8.7-beta.3) (2023-03-17) ### Bug Fixes * conditional profiles ([#713](#713)) ([7d8d5fc](7d8d5fc)) * fix: do not poll IF-MIB by default (#714) * fix: do not poll IF-MIB by default, add NOTE about IF-MIB not being polled by default * fix: update CHANGELOG and add unit test * fix: unit test * chore(release): 1.8.7-beta.4 ## [1.8.7-beta.4](v1.8.7-beta.3...v1.8.7-beta.4) (2023-03-21) ### Bug Fixes * do not poll IF-MIB by default ([#714](#714)) ([3d2a1f6](3d2a1f6)) * fix: tooBig error (#717) * fix: tooBig error * chore(release): 1.8.7-beta.5 ## [1.8.7-beta.5](v1.8.7-beta.4...v1.8.7-beta.5) (2023-03-24) ### Bug Fixes * tooBig error ([#717](#717)) ([d964746](d964746)) * test: adding syntax from poetry >= 1.2 (#716) * test: adding syntax from poetry >= 1.2 * fix: treating not existing group name as a hostname (#721) * chore(release): 1.8.7-beta.6 ## [1.8.7-beta.6](v1.8.7-beta.5...v1.8.7-beta.6) (2023-03-28) ### Bug Fixes * treating not existing group name as a hostname ([#721](#721)) ([5e46474](5e46474)) * fix: prepare release (#722) * doc: improve documentation * chore: improve GH workflow permissions * doc: update docs, delete outdated informations * chore(release): 1.8.7-beta.7 ## [1.8.7-beta.7](v1.8.7-beta.6...v1.8.7-beta.7) (2023-04-03) ### Bug Fixes * prepare release ([#722](#722)) ([b169d5d](b169d5d)) * fix: workflow permissions (#724) * chore(release): 1.8.7-beta.8 ## [1.8.7-beta.8](v1.8.7-beta.7...v1.8.7-beta.8) (2023-04-03) ### Bug Fixes * workflow permissions ([#724](#724)) ([7ffbc35](7ffbc35)) * fix: GitHub actions Helm permissions (#725) * fix: workflow permissions * chore(release): 1.8.7-beta.9 ## [1.8.7-beta.9](v1.8.7-beta.8...v1.8.7-beta.9) (2023-04-03) ### Bug Fixes * GitHub actions Helm permissions ([#725](#725)) ([dec2fef](dec2fef)) * doc: minor enhancements (#727) * doc: shorten step by step polling * doc: add search option to the documentation and update values.yaml with the new profiles * doc: add logLevel to the main values.yaml, run make render * doc: change values.yaml lint * doc: fix title (#728) * feat: prepare release, modify changelog (#729) * chore(release): 1.9.0-beta.1 # [1.9.0-beta.1](v1.8.7-beta.9...v1.9.0-beta.1) (2023-04-04) ### Features * prepare release, modify changelog ([#729](#729)) ([a373477](a373477)) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: semantic-release-bot <[email protected]> Co-authored-by: Wojciech Zyla <[email protected]> Co-authored-by: Lukasz Loboda <[email protected]>
splunk · Apr 4, 2023 · 3db5f1c · 3db5f1c
1 parent bc2e8d1
commit 3db5f1c
Show file tree

Hide file tree

Showing 121 changed files with 3,003 additions and 1,389 deletions.
diff --git a/.github/workflows/agreements.yaml b/.github/workflows/agreements.yaml
@@ -8,6 +8,9 @@ on:
 jobs:
   call-workflow-agreements:
     uses: splunk/addonfactory-github-workflows/.github/workflows/[email protected]
+    permissions:
+      contents: read
+      packages: write
     secrets:
       GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       PERSONAL_ACCESS_TOKEN: ${{ secrets.PAT_CLATOOL }}
diff --git a/.github/workflows/cd-pages.yaml b/.github/workflows/cd-pages.yaml
@@ -11,6 +11,9 @@ jobs:
   mike:
     name: Release
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
     steps:
       - name: Checkout
         uses: actions/checkout@v2
@@ -70,6 +73,9 @@ jobs:
   #          fi
   helm:
     runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      packages: write
     needs:
       - mike
     steps:

diff --git a/.github/workflows/ci-main.yaml b/.github/workflows/ci-main.yaml
@@ -136,7 +136,7 @@ jobs:
     if: "contains(needs.integration-tests-check.outputs.commit_message, '[run-int-tests]')"
     steps:
       - uses: actions/checkout@v2
-      - name: Archieve repository
+      - name: Archive repository
         working-directory: /home/runner/work/splunk-connect-for-snmp
         run: tar -czf splunk-connect-for-snmp.tgz splunk-connect-for-snmp
       - name: Copy archived repository to scripts directory

diff --git a/.github/workflows/ci-release-pr.yaml b/.github/workflows/ci-release-pr.yaml
@@ -27,6 +27,9 @@ jobs:
   release:
     name: Release
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
     steps:
       - uses: actions/checkout@v2
         with:

diff --git a/.github/workflows/ci-release.yaml b/.github/workflows/ci-release.yaml
@@ -28,6 +28,9 @@ jobs:
   release:
     name: Release
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
     steps:
       - uses: actions/checkout@v2
         with:

diff --git a/.github/workflows/mike.yaml b/.github/workflows/mike.yaml
@@ -37,9 +37,11 @@ jobs:
 
       - name: Upload Docs
         run: |
+          sudo apt update
+          sudo apt install gnome-keyring
           BRANCH=$(echo $GITHUB_REF | cut -d / -f 3)
           echo $BRANCH
-          pip3 install poetry=="1.1.15"
+          pip3 install poetry=="1.2.2"
           poetry install
           poetry run pip install 'setuptools==65.6.3'
           poetry run mike deploy -p $BRANCH
diff --git a/.github/workflows/offline-installation.yaml b/.github/workflows/offline-installation.yaml
@@ -8,6 +8,9 @@ jobs:
   build:
     name: Build
     runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      packages: write
     steps:
       - name: Check out code
         uses: actions/checkout@v1

diff --git a/.github/workflows/release-notes.yaml b/.github/workflows/release-notes.yaml
@@ -7,6 +7,9 @@ on:
 
 jobs:
   call-workflow-preview:
+    permissions:
+      contents: write
+      packages: write
     uses: splunk/addonfactory-github-workflows/.github/workflows/[email protected]
     secrets:
       GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,18 @@
 
 ## Unreleased
 
+## [1.9.0]
+
+### Changed
+- add possibility to poll compound indexes (more than one value, ex. `['IP-MIB', 'ipAddressStatus', 'ipv4', '172.31.27.144']`)
+- add option to automatically poll SNMP objects based on provided conditions with conditional profiles
+- remove IF-MIB from the scope of the default small walk
+
+### Fixed
+- possibility to use hostname instead of the bare ip address in polling
+- getting rid off `An error of SNMP isWalk=False for a host 54.91.99.113 occurred: tooBig at ?` with limiting maximum 
+number of varBinds polled at once `maxOidToProcess`
+
 ## [1.8.6]
 
 ### Changed

diff --git a/Makefile b/Makefile
@@ -1,4 +1,6 @@
 .PHONY: render
 render:
+	rm -rf rendered/manifests
 	helm template -n default --values rendered/values.yaml --output-dir rendered/manifests/tests charts/splunk-connect-for-snmp
 	rm -rf rendered/manifests/tests/splunk-connect-for-snmp/charts
+	./render_manifests.sh
diff --git a/README.md b/README.md
@@ -15,5 +15,3 @@ There are plenty of versions you can browse:
 
 # Contact
 Feel free to contact us via [#splunk-connect-for-snmp](https://splunk-usergroups.slack.com/archives/C01K4V86WV7) slack channel.
-
-#
diff --git a/charts/splunk-connect-for-snmp/Chart.yaml b/charts/splunk-connect-for-snmp/Chart.yaml
@@ -14,12 +14,12 @@ type: application
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
 
-version: 1.8.6
+version: 1.9.0-beta.1
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.8.6"
+appVersion: "1.9.0-beta.1"
 #
 dependencies:
   - name: mongodb

diff --git a/charts/splunk-connect-for-snmp/templates/NOTES.txt b/charts/splunk-connect-for-snmp/templates/NOTES.txt
@@ -1,9 +1,2 @@
-Version 1.7 of SC4SNMP add new feature which enables horizontal worker pods autoscaling, in order to use it you will need to turn on microk8s metrics-server addon:
-
-microk8s enable metrics-server
-
-and you should also update worker configuration in values.yaml file according to the documentation:
-https://splunk.github.io/splunk-connect-for-snmp/main/configuration/worker-configuration
-
-values.yaml template is available here:
-https://splunk.github.io/splunk-connect-for-snmp/main/gettingstarted/sc4snmp-installation
+Walk profiles no longer include IF-MIB family by default.
+If you've used this functionality before, please update the walk profile with ['IF-MIB'] varBind.
diff --git a/charts/splunk-connect-for-snmp/templates/_helpers.tpl b/charts/splunk-connect-for-snmp/templates/_helpers.tpl
@@ -14,6 +14,84 @@
 {{- printf "redis://%s-redis-headless:6379/1" .Release.Name }}
 {{- end }}
 
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "splunk-connect-for-snmp.fullname" -}}
+{{- if .Values.worker.fullnameOverride }}
+{{- .Values.worker.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default (printf "%s-%s" .Chart.Name "user") .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Selector labels
+*/}}
+{{- define "splunk-connect-for-snmp.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "splunk-connect-for-snmp.name" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "splunk-connect-for-snmp.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "splunk-connect-for-snmp.labels" -}}
+helm.sh/chart: {{ include "splunk-connect-for-snmp.chart" . }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end }}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "splunk-connect-for-snmp.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create }}
+{{- default (include "splunk-connect-for-snmp.fullname" .) .Values.serviceAccount.name }}
+{{- else }}
+{{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}
+
 {{- define "splunk-connect-for-snmp.name" -}}
 {{- default (printf "%s" .Chart.Name ) .Values.nameOverride | trunc 63 | trimSuffix "-" }}
 {{- end }}
+
+
+{{/*
+Whether enable traps
+*/}}
+{{- define "splunk-connect-for-snmp.traps.enable" -}}
+{{- if or (and (eq .Values.traps.service.type "LoadBalancer") .Values.traps.loadBalancerIP ) (and (eq .Values.traps.service.type "NodePort") .Values.traps.service.nodePort) }}
+{{- printf "true" }}
+{{- else }}
+{{- printf "false" }}
+{{- end -}}
+{{- end }}
+
+{{/*
+Whether enable polling
+*/}}
+{{- define "splunk-connect-for-snmp.polling.enable" -}}
+{{- if .Values.poller.inventory }}
+{{- printf "true" }}
+{{- else }}
+{{- printf "false" }}
+{{- end -}}
+{{- end }}
diff --git a/charts/splunk-connect-for-snmp/templates/common/scheduler-inventory.yaml b/charts/splunk-connect-for-snmp/templates/common/scheduler-inventory.yaml
@@ -1,3 +1,4 @@
+{{- if eq (include "splunk-connect-for-snmp.polling.enable" .) "true" }}
 apiVersion: v1
 kind: ConfigMap
 metadata:
@@ -11,3 +12,4 @@ data:
 {{ else }}
     address,port,version,community,secret,securityEngine,walk_interval,profiles,SmartProfiles,delete
 {{- end }}
+{{- end -}}
diff --git a/charts/splunk-connect-for-snmp/templates/common/traps-config.yaml b/charts/splunk-connect-for-snmp/templates/common/traps-config.yaml
@@ -1,3 +1,4 @@
+{{- if eq (include "splunk-connect-for-snmp.traps.enable" .) "true" }}
 apiVersion: v1
 kind: ConfigMap
 metadata:
@@ -14,3 +15,4 @@ data:
       usernameSecrets:
 {{- toYaml .Values.traps.usernameSecrets | nindent 8 }}
 {{ end }}
+{{- end -}}
diff --git a/charts/splunk-connect-for-snmp/templates/inventory/_helpers.tpl b/charts/splunk-connect-for-snmp/templates/inventory/_helpers.tpl
@@ -30,18 +30,6 @@ Create chart name and version as used by the chart label.
 {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
 {{- end }}
 
-{{/*
-Common labels
-*/}}
-{{- define "splunk-connect-for-snmp.inventory.labels" -}}
-helm.sh/chart: {{ include "splunk-connect-for-snmp.inventory.chart" . }}
-{{ include "splunk-connect-for-snmp.inventory.selectorLabels" . }}
-{{- if .Chart.AppVersion }}
-app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
-{{- end }}
-app.kubernetes.io/managed-by: {{ .Release.Service }}
-{{- end }}
-
 {{/*
 Selector labels
 */}}
@@ -51,12 +39,10 @@ app.kubernetes.io/instance: {{ .Release.Name }}
 {{- end }}
 
 {{/*
-Create the name of the service account to use
+Common labels
 */}}
-{{- define "splunk-connect-for-snmp.inventory.serviceAccountName" -}}
-{{- if .Values.inventory.serviceAccount.create }}
-{{- default (include "splunk-connect-for-snmp.inventory.fullname" .) .Values.inventory.serviceAccount.name }}
-{{- else }}
-{{- default "default" .Values.inventory.serviceAccount.name }}
-{{- end }}
+{{- define "splunk-connect-for-snmp.inventory.labels" -}}
+{{ include "splunk-connect-for-snmp.inventory.selectorLabels" . }}
+{{ include "splunk-connect-for-snmp.labels" . }}
 {{- end }}
+
diff --git a/charts/splunk-connect-for-snmp/templates/inventory/job.yaml b/charts/splunk-connect-for-snmp/templates/inventory/job.yaml
@@ -1,3 +1,4 @@
+{{- if eq (include "splunk-connect-for-snmp.polling.enable" .) "true" }}
 apiVersion: batch/v1
 kind: Job
 metadata:
@@ -81,4 +82,5 @@ spec:
           emptyDir: {}    
         - name: tmp
           emptyDir: {}                        
-      restartPolicy: OnFailure
+      restartPolicy: OnFailure
+{{- end -}}
diff --git a/charts/splunk-connect-for-snmp/templates/scheduler/_helpers.tpl b/charts/splunk-connect-for-snmp/templates/scheduler/_helpers.tpl
@@ -30,18 +30,6 @@ Create chart name and version as used by the chart label.
 {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
 {{- end }}
 
-{{/*
-Common labels
-*/}}
-{{- define "splunk-connect-for-snmp.scheduler.labels" -}}
-helm.sh/chart: {{ include "splunk-connect-for-snmp.scheduler.chart" . }}
-{{ include "splunk-connect-for-snmp.scheduler.selectorLabels" . }}
-{{- if .Chart.AppVersion }}
-app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
-{{- end }}
-app.kubernetes.io/managed-by: {{ .Release.Service }}
-{{- end }}
-
 {{/*
 Selector labels
 */}}
@@ -51,12 +39,9 @@ app.kubernetes.io/instance: {{ .Release.Name }}
 {{- end }}
 
 {{/*
-Create the name of the service account to use
+Common labels
 */}}
-{{- define "splunk-connect-for-snmp.scheduler.serviceAccountName" -}}
-{{- if .Values.scheduler.serviceAccount.create }}
-{{- default (include "splunk-connect-for-snmp.scheduler.fullname" .) .Values.scheduler.serviceAccount.name }}
-{{- else }}
-{{- default "default" .Values.scheduler.serviceAccount.name }}
-{{- end }}
-{{- end }}
+{{- define "splunk-connect-for-snmp.scheduler.labels" -}}
+{{ include "splunk-connect-for-snmp.scheduler.selectorLabels" . }}
+{{ include "splunk-connect-for-snmp.labels" . }}
+{{- end }}
diff --git a/charts/splunk-connect-for-snmp/templates/scheduler/deployment.yaml b/charts/splunk-connect-for-snmp/templates/scheduler/deployment.yaml
@@ -1,13 +1,12 @@
+{{- if eq (include "splunk-connect-for-snmp.polling.enable" .) "true" }}
 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: {{ include "splunk-connect-for-snmp.scheduler.fullname" . }}
   labels:
     {{- include "splunk-connect-for-snmp.scheduler.labels" . | nindent 4 }}
 spec:
-  {{- if not .Values.scheduler.autoscaling.enabled }}
-  replicas: {{ .Values.scheduler.replicaCount }}
-  {{- end }}
+  replicas: 1
   selector:
     matchLabels:
       {{- include "splunk-connect-for-snmp.scheduler.selectorLabels" . | nindent 6 }}
@@ -24,13 +23,19 @@ spec:
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      serviceAccountName: {{ include "splunk-connect-for-snmp.scheduler.serviceAccountName" . }}
+      serviceAccountName: {{ include "splunk-connect-for-snmp.serviceAccountName" . }}
       securityContext:
-        {{- toYaml .Values.scheduler.podSecurityContext | nindent 8 }}
+        fsGroup: 10001
       containers:
         - name: {{ .Chart.Name }}-scheduler
           securityContext:
-            {{- toYaml .Values.scheduler.securityContext | nindent 12 }}
+            capabilities:
+              drop:
+                - ALL
+            readOnlyRootFilesystem: true
+            runAsNonRoot: true
+            runAsUser: 10001
+            runAsGroup: 10001
           image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
           imagePullPolicy: {{ .Values.image.pullPolicy }}
           args:
@@ -108,4 +113,5 @@ spec:
         - name: pysnmp-cache-volume
           emptyDir: {}    
         - name: tmp
-          emptyDir: {}                        
+          emptyDir: {}
+{{- end -}}
Original file line number	Diff line number	Diff line change
Expand Up		@@ -15,5 +15,3 @@ There are plenty of versions you can browse:

		# Contact
		Feel free to contact us via [#splunk-connect-for-snmp](https://splunk-usergroups.slack.com/archives/C01K4V86WV7) slack channel.

		#