Update snap to 3.2.0.742 and event-collector for logsv2 (AO-16825) (#25)

* Update snap to 3.2.0.742 and event-collector for logsv2 (AO-16825)

Makefile

@@ -1,5 +1,5 @@
-DOCKERFILE_VERSION=3.4.0
-SWISNAP_VERSION=3.1.1.717
+DOCKERFILE_VERSION=4.0.0
+SWISNAP_VERSION=3.2.0.742
 TAG=$(DOCKERFILE_VERSION)-$(SWISNAP_VERSION)
 USER=solarwinds
 REPOSITORY=solarwinds-snap-agent-docker

README.md

@@ -240,9 +240,77 @@ The following environment parameters are available:
 If you use `SWISNAP_ENABLE_<plugin_name>` set to `true`, then keep in mind that SolarWinds Snap Agent will use default plugins configs and task manifests. For custom configuration see [Custom plugins configuration and tasks manifests](#custom-plugins-configuration-and-tasks-manifests).
 
 ## Integrating Kubernetes Cluster Events Collection With Loggly
+This documentaton can be also found in [Documentation for SolarWinds](https://documentation.solarwinds.com/en/Success_Center/appoptics/Content/kb/host_infrastructure/host_agent/kubernetes_ha.htm#integrating-kubernetes-cluster-events-collection-with-loggly) webpage.
+Version 22 of Kubernetes collector allows you to collect cluster events and push them to Loggly using logs collector under the hood. To utilise this functionality there is a need to create corresponding configmaps in your cluster, with proper plugins configration. The example config files can be found in  [Event collector configs](examples/event-collector-configs). To enable event collection in your deployment, follow below steps:
 
-Version 22 of Kubernetes collector allows you to collect cluster events and push them to Loggly using logs collector under the hood. To enable event collection in your deployment, follow below steps:
-* Create `kubernetes.yaml` file that will configure kubernetes collector. This config should contain `collector.kubernetes.all.events` field with specified filter. Following example config will watch for `normal` events in `default` namespace:
+* Create Kubernetes secret for `SOLARWINDS_TOKEN`:
+  ```shell
+  kubectl create secret generic solarwinds-token -n kube-system --from-literal=SOLARWINDS_TOKEN=<REPLACE WITH TOKEN>
+  ```
+
+* Update Loggly Token in [logs-v2.yaml](examples/event-collector-configs/logs-v2.yaml) file under `v2:collector:logs:all:logging_service:loggly:token`.
+  ```yaml
+  v2:
+    collector:
+      logs:
+        all:
+          #  [...]
+          logging_service:
+            # [..]
+            ## Sign up for a Loggly account at: https://www.loggly.com
+            loggly:
+  
+              ## Loggly API token and host
+              token: "LOGGLY_TOKEN
+              host: "logs-01.loggly.com"
+  
+              ## Loggly API port and protocol
+              ## use 6514 with TLS or 514 with TCP
+              port: 6514
+              protocol: tls
+  
+  ```
+* [task-logs-k8s.yaml](examples/event-collector-configs/logs-v2.yaml) file configures the logs collector plugin. It is telling logs collector to look for `/var/log/SolarWinds/Snap/events.log` file:
+  ```
+  ---
+  version: 2
+
+  schedule:
+    type: cron
+    interval: "0 * * * * *"
+
+  plugins:
+    - plugin_name: logs
+
+      metrics:
+        - /logs/lines_total
+        - /logs/lines_forwarded
+        - /logs/bytes_forwarded
+        - /logs/lines_skipped
+        - /logs/lines_failed
+        - /logs/bytes_failed
+        - /logs/lines_succeeded
+        - /logs/bytes_succeeded
+        - /logs/attempts_total
+        - /logs/failed_attempts_total
+
+      config:
+        sources:
+          log_files:
+
+            # [...]
+            file_paths:
+              - Path: /var/log/SolarWinds/Snap/events.log
+
+            # [...]
+            exclude_lines_patterns:
+              - ".*self-skip-logs-collector.*"
+
+      publish:
+        - plugin_name: publisher-appoptics
+  ```
+
+* [kubernetes.yaml](examples/event-collector-configs/kubernetes.yaml) file configures the kubernetes collector plugin. This config contain `collector.kubernetes.all.events` field with specified filter. With this example filter event collector will watch for `normal` events in `default` namespace.
   ```yaml
   collector:
     kubernetes:
@@ -263,24 +331,19 @@ Version 22 of Kubernetes collector allows you to collect cluster events and push
     plugin: snap-plugin-collector-aokubernetes
     task: task-aokubernetes.yaml
   ```
-* If you want to monitor events count in AppOptics, then edit your current `task-aokubernetes.yaml` task manifest so it contains `/kubernetes/events/count` metric in `workflow.collect.metrics` list, and copy it to working directory:
+* If you want to monitor events count in AppOptics, then edit [task-aokubernetes.yaml](examples/event-collector-configs/task-aokubernetes.yaml) task manifest so it contains `/kubernetes/events/count` metric in `workflow.collect.metrics` list, and copy it to working directory:
   ```yaml
   ---
   version: 1
-
   schedule:
     type: streaming
-
   deadline: "55s"
-
   workflow:
     collect:
-
       config:
         /kubernetes:
           MaxCollectDuration: "2s"
           MaxMetricsBuffer: 250
-
       metrics:
         /kubernetes/events/count: {}
         /kubernetes/pod/*/*/*/status/phase/Running: {}
@@ -290,42 +353,13 @@ Version 22 of Kubernetes collector allows you to collect cluster events and push
           period: 60
           floor_seconds: 60
   ```
-* Create `logs.yaml` file configuring the logs collector. Make sure that logs collector looks for `/var/log/SolarWinds/Snap/events.log` file:
-  ```yaml
-  collector:
-    logs:
-      all:
-        loggly_token: <your loggly token>
-        api_host: "logs-01.loggly.com"
-
-        api_port: 514
-        api_protocol: "tcp"
-
-        connect_timeout: "30s"
-
-        write_timeout: "30s"
-
-        files: |
-          /var/log/SolarWinds/Snap/events.log
-
-        exclude_patterns: |
-          .*self-skip-logs-collector.*
-
-  load:
-    plugin: snap-plugin-collector-logs
-    task: task-logs.yaml
-  ```
-* Copy your current `task-logs.yaml` task manifest to working directory.
-* Once all 4 files are ready (`kubernetes.yaml`, `logs.yaml`, `task-aokubernetes.yaml` and `task-logs.yaml`), create 2 configmaps:
+* Once above steps are finished, create 3 configmaps:
   ```shell
-  kubectl create configmap plugin-configs --from-file=./logs.yaml --from-file=./kubernetes.yaml --namespace=kube-system
-  kubectl create configmap task-manifests --from-file=./task-logs.yaml --from-file=./task-aokubernetes.yaml --namespace=kube-system
+  kubectl create configmap plugin-configs --from-file=./examples/event-collector-configs/logs-v2.yaml --from-file=./examples/event-collector-configs/kubernetes.yaml --namespace=kube-system
+  kubectl create configmap task-manifests --from-file=./examples/event-collector-configs/task-aokubernetes.yaml --namespace=kube-system
+  kubectl create configmap task-autoload --from-file=./examples/event-collector-configs/task-logs-k8s.yaml --namespace=kube-system
 
-  kubectl describe configmaps -n kube-system plugin-configs task-manifests
-  ```
-* Create Kubernetes secret for `SOLARWINDS_TOKEN`:
-  ```shell
-  kubectl create secret generic solarwinds-token -n kube-system --from-literal=SOLARWINDS_TOKEN=<REPLACE WITH TOKEN>
+  kubectl describe configmaps -n kube-system plugin-configs task-manifests task-autoload
   ```
 * Create Events Collector Deployment (it will automatically create corresponding ServiceAccount):
   ```shell

deploy/base/events-collector/swisnap-agent-deployment-event-collector.yaml

@@ -23,17 +23,17 @@ spec:
           imagePullPolicy: IfNotPresent
           volumeMounts:
             - name: plugins-logs
-              mountPath: /opt/SolarWinds/Snap/etc/plugins.d/logs.yaml
-              subPath: logs.yaml
+              mountPath: /opt/SolarWinds/Snap/etc/plugins.d/logs-v2.yaml
+              subPath: logs-v2.yaml
             - name: plugins-kubernetes
               mountPath: /opt/SolarWinds/Snap/etc/plugins.d/kubernetes.yaml
               subPath: kubernetes.yaml
             - name: tasks-aokubernetes
               mountPath: /opt/SolarWinds/Snap/etc/tasks.d/task-aokubernetes.yaml
               subPath: task-aokubernetes.yaml
-            - name: tasks-logs
-              mountPath: /opt/SolarWinds/Snap/etc/tasks.d/task-logs.yaml
-              subPath: task-logs.yaml
+            - name: tasks-autoload-logs
+              mountPath: /opt/SolarWinds/Snap/etc/tasks-autoload.d/task-logs-k8s.yaml
+              subPath: task-logs-k8s.yaml
           env:
             - name: APPOPTICS_TOKEN
               valueFrom:
@@ -74,20 +74,20 @@ spec:
           configMap:
             name: plugin-configs
             items:
-              - key: logs.yaml
-                path: logs.yaml
+              - key: logs-v2.yaml
+                path: logs-v2.yaml
         - name: plugins-kubernetes
           configMap:
             name: plugin-configs
             items:
               - key: kubernetes.yaml
                 path: kubernetes.yaml
-        - name: tasks-logs
+        - name: tasks-autoload-logs
           configMap:
-            name: task-manifests
+            name: task-autoload
             items:
-              - key: task-logs.yaml
-                path: task-logs.yaml
+              - key: task-logs-k8s.yaml
+                path: task-logs-k8s.yaml
         - name: tasks-aokubernetes
           configMap:
             name: task-manifests

deploy/overlays/stable/daemonset/kustomization.yaml

@@ -8,4 +8,4 @@ bases:
 
 images:
   - name: solarwinds/solarwinds-snap-agent-docker
-    newTag: 3.4.0-3.1.1.717
+    newTag: 4.0.0-3.2.0.742

deploy/overlays/stable/deployment/kustomization.yaml

@@ -7,4 +7,4 @@ bases:
 
 images:
   - name: solarwinds/solarwinds-snap-agent-docker
-    newTag: 3.4.0-3.1.1.717
+    newTag: 4.0.0-3.2.0.742

deploy/overlays/stable/events-collector/kustomization.yaml

@@ -7,4 +7,4 @@ bases:
 
 images:
   - name: solarwinds/solarwinds-snap-agent-docker
-    newTag: 3.4.0-3.1.1.717
+    newTag: 4.0.0-3.2.0.742

examples/event-collector-configs/kubernetes.yaml

@@ -0,0 +1,18 @@
+collector:
+  kubernetes:
+    all:
+      incluster: true
+      kubeconfigpath: ""
+      interval: "60s"
+
+      events: |
+        # Embedded YAML (as a multiline string literal)
+        filters:
+        - namespace: default
+          type: normal
+
+      grpc_timeout: 30
+
+load:
+  plugin: snap-plugin-collector-aokubernetes
+  task: task-aokubernetes.yaml

examples/event-collector-configs/logs-v2.yaml

@@ -0,0 +1,54 @@
+v2:
+  collector:
+    logs:
+      all:
+
+        sources:
+          ## Define common settings for Loggly or Papertrail.
+          common:
+
+            ## Maximum line length to be written at once (in utf-8 characters)
+            ## 0 means no limit
+            ## 1024 is the default
+            #max_line_length: 1024
+
+        ## Tail and send logs from this host to Loggly or Papertrail.
+        ## Loggly and Papertrail are also from the SolarWinds DevOps Cloud Product Family Suite.
+        logging_service:
+
+          ## Override the hostname used for logs reported by this agent. Defaults to the OS-provided hostname.
+          #hostname: "myhost"
+
+          timeouts:
+
+            ## A timeout for connecting to logs-accepting API
+            #connect: 30s
+
+            ## A timeout for writing to logs-accepting API
+            #write: 30s
+
+          ## Sign up for a Loggly account at: https://www.loggly.com
+          loggly:
+
+            ## Loggly API token and host
+            token: "LOGGLY_TOKEN
+            host: "logs-01.loggly.com"
+
+            ## Loggly API port and protocol
+            ## use 6514 with TLS or 514 with TCP
+            port: 6514
+            protocol: tls
+
+            ## Path to Loggly public CA certificate. See https://www.loggly.com/docs/rsyslog-tls-configuration/ for reference.
+            ## Uncomment this line if you want to use custom host certificate store.
+            # ca_certificate_path: /path/to/your/certificate
+
+          ## Sign up for a Papertrail account at: https://papertrailapp.com
+          #papertrail:
+            ## Papertrail API token
+            #token: "PAPERTRAIL_TOKEN"
+
+            ## Papertrail host and port details: change this to YOUR papertrail host.
+            #host: "HOST.papertrailapp.com"
+            #port: 12345
+            #protocol: tls

examples/event-collector-configs/task-aokubernetes.yaml

@@ -0,0 +1,24 @@
+---
+version: 1
+
+schedule:
+  type: streaming
+
+deadline: "55s"
+
+workflow:
+  collect:
+
+    config:
+      /kubernetes:
+        MaxCollectDuration: "2s"
+        MaxMetricsBuffer: 250
+
+    metrics:
+      /kubernetes/events/count: {}
+      /kubernetes/pod/*/*/*/status/phase/Running: {}
+    publish:
+    - plugin_name: publisher-appoptics
+      config:
+        period: 60
+        floor_seconds: 60

examples/event-collector-configs/task-logs-k8s.yaml

@@ -0,0 +1,60 @@
+---
+version: 2
+
+schedule:
+  type: cron
+  interval: "0 * * * * *"
+
+plugins:
+  - plugin_name: logs
+
+    metrics:
+      - /logs/lines_total
+      - /logs/lines_forwarded
+      - /logs/bytes_forwarded
+      - /logs/lines_skipped
+      - /logs/lines_failed
+      - /logs/bytes_failed
+      - /logs/lines_succeeded
+      - /logs/bytes_succeeded
+      - /logs/attempts_total
+      - /logs/failed_attempts_total
+
+    config:
+      ## Define sources from which logs will be read.
+      sources:
+
+        ## Define files from which logs will be read and sent to the Loggly or Papertrail.
+        log_files:
+
+          ## An interval for looking for new files matching given pattern(s)
+          #new_file_check_interval: 30s
+
+          ## An array of files or filename patterns to watch.
+          ##
+          ## NOTE: Be careful when attempting to handle snapteld logs
+          ## as those might also contain log entries of logs collector
+          ## to avoid infinite recurrence effect you should apply exclude pattern below by adding
+          ## ".*self-skip-logs-collector.*"
+          file_paths:
+            - Path: /var/log/SolarWinds/Snap/events.log
+
+          #  - Path: /var/log/*.log
+          #  - Path: /var/log/httpd/access_log
+          #    Tag: HTTPD Access log
+
+          ## Provide one or more regular expressions to prevent certain files from being matched.
+          #exclude_files_patterns:
+          #  - \.\d$
+          #  - \.bz2
+          #  - \.gz
+
+          ## There may be certain log messages that you do not want to be sent.
+          ## These may be repetitive log lines that are "noise" that you might
+          ## not be able to filter out easily from the respective application.
+          ## To filter these lines, use exclude_patterns with an array or regexes.
+          exclude_lines_patterns:
+            - ".*self-skip-logs-collector.*"
+
+    publish:
+      - plugin_name: publisher-appoptics