From f9eae0a9c65f7fca88c68884439472245b3b653e Mon Sep 17 00:00:00 2001 From: Grigorii Merkushev Date: Tue, 17 Oct 2023 14:55:25 +0200 Subject: [PATCH] remove manual packaging tests --- test/manual/system_acceptance_test.md | 80 ------- test/manual/use-case/apt_validations.md | 243 --------------------- test/manual/use-case/docker_validations.md | 151 ------------- test/manual/use-case/win_validations.md | 202 ----------------- test/manual/use-case/yum_validations.md | 222 ------------------- test/manual/use-case/zypper_validations.md | 228 ------------------- 6 files changed, 1126 deletions(-) delete mode 100644 test/manual/system_acceptance_test.md delete mode 100644 test/manual/use-case/apt_validations.md delete mode 100644 test/manual/use-case/docker_validations.md delete mode 100644 test/manual/use-case/win_validations.md delete mode 100644 test/manual/use-case/yum_validations.md delete mode 100644 test/manual/use-case/zypper_validations.md diff --git a/test/manual/system_acceptance_test.md b/test/manual/system_acceptance_test.md deleted file mode 100644 index 337d978a4..000000000 --- a/test/manual/system_acceptance_test.md +++ /dev/null @@ -1,80 +0,0 @@ -### Why: -As moving forward to support many OS distribution and version we want to unsure agent main functionalities covered and working in each new release. - -### How: -Under script folder You will find all commands and assert criteria to run locally this suit case separated by each OS dist package manager. - -### Setup: -In order to pass this manual test some requirements are needed: -- Install [newrelic-cli](https://github.com/newrelic/newrelic-cli) with your license key and api key (can be obtained though NR user information) -This tools will allow You to get data from your agent host on NR and validate some use cases. -- Add env var: - ```shell script - export NR_LICENSE_KEY=*** NR_API_KEY=*** NR_ACCOUNT_ID=*** DISPLAY_NAME=*** NR_REGION=*** - ``` - -### Acceptance and Criteria: -- ***Scenario 1. Package naming follow dist convention*** \ -Given a new newrelic-infra agent version to install, \ -When inspect package name, \ -Then name match with the previous version. - -- ***Scenario 2. Package contains all required files*** \ -Given the latest newrelic-infra agent version installed, \ -When I list all files installed to my system, \ -Then all match with previous version. - -- ***Scenario 3. Check agent version*** \ -Given the latest newrelic-infra agent version installed, \ -When I review the version number, \ -Then it matches with latest released version. - -- ***Scenario 4. Service is working*** -Given the latest newrelic-infra agent version installed, \ -When I run the agent, \ -Then all host data is sent to NR. - -- ***Scenario 5. Package metadata is valid*** \ -Given the latest newrelic-infra agent version installed, \ -When I inspect package metadata, \ -Then all Company/team/version is present. - -- ***Scenario 6. Package signature is valid*** \ -Given the latest newrelic-infra agent version installed, \ -When I inspect package signature, \ -Then it matches with previous released version. - -- ***Scenario 7. Agent privileged mode is working*** \ -Given the latest newrelic-infra agent version installed as privileged mode, \ -When I run the agent, \ -Then all host data is sent to NR. - -- ***Scenario 8. Agent unprivileged mode is working*** \ -Given the latest newrelic-infra agent version installed as unprivileged mode, \ -When I run the agent, \ -Then all host data is sent to NR. - -- ***Scenario 9. Package uninstall*** \ -Given the latest newrelic-infra agent version installed as unprivileged mode, \ -When I uninstall the newrelic-infra agent package, \ -Then no agent is present. - -- ***Scenario 10. Package upgrade*** \ -Given an old newrelic-infra agent version installed, \ -When I upgrade to latest released version, \ -Then all host data is sent to NR. - -- ***Scenario 11. Built in Flex integration is working*** \ -Given the latest newrelic-infra agent version installed, \ -When agent run nri-flex integration \ -Then I flexSample is sent to NR. - -- ***Scenario 11. Built in Log-forwarded integration is working*** \ -Given the latest newrelic-infra agent version installed, \ -When I enable the log-forwarder mode, \ -Then my configured logs are sent to NR. - -- ***Scenario 12. Built in Prometheus integration is working*** \ -Given the latest newrelic-infra agent version installed, \ -When I run nri-prometheus integration binary, \ -Then the integration returns a message. \ No newline at end of file diff --git a/test/manual/use-case/apt_validations.md b/test/manual/use-case/apt_validations.md deleted file mode 100644 index a424ca451..000000000 --- a/test/manual/use-case/apt_validations.md +++ /dev/null @@ -1,243 +0,0 @@ -### Scope -This document share which are the commands and validations needed to passed all acceptance criteria for the manual system acceptance test. -It covers support for use case of: -* Ubuntu 20, 18, 16 and 14 -* Debian 10 - -### Setup -- For [Ubuntu 20.04](https://app.vagrantup.com/generic/boxes/ubuntu2004) (focal): - ```shell script - $ printf "deb [arch=amd64] http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/apt focal main" | sudo tee -a /etc/apt/sources.list.d/newrelic-infra.list - ``` -- For [Ubuntu 18.04](https://app.vagrantup.com/generic/boxes/ubuntu1804) (bionic): - ```shell script - $ printf "deb [arch=amd64] http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/apt bionic main" | sudo tee -a /etc/apt/sources.list.d/newrelic-infra.list - ``` -- For Ubuntu [16.04](https://app.vagrantup.com/generic/boxes/ubuntu1604) (xenial): - ```shell script - $ printf "deb [arch=amd64] http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/apt xenial main" | sudo tee -a /etc/apt/sources.list.d/newrelic-infra.list - ``` -- For Ubuntu [14.04](https://app.vagrantup.com/ubuntu/boxes/trusty64) (trusty): - ```shell script - $ printf "deb [arch=amd64] http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/apt trusty main" | sudo tee -a /etc/apt/sources.list.d/newrelic-infra.list - ``` -- For [Debian 10](https://app.vagrantup.com/generic/boxes/debian10) (buster): - ```shell script - $ printf "deb [arch=amd64] http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/apt buster main" | sudo tee -a /etc/apt/sources.list.d/newrelic-infra.list - ``` - -### Test suit -#### Scenario 1. Package naming follow dist convention -Lookup at apt repository folder and compare package under test with the previous package version name. \ -e.g.: [APT Repository folder](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/apt/pool/main/n/newrelic-infra/) \ -package under test version: 1.16.1, name: newrelic-infra_systemd_1.16.1_amd64.deb \ -previous package version: 1.16.0, name: newrelic-infra_systemd_1.16.0_amd64.deb - -note: Ubuntu 14 use upstart service manager therefore package name will be newrelic-infra_upstart_1.16.0_amd64.deb - -#### Scenario 2. Package contains all required files -Review file managed by package and compare with previous package version. -```shell script -$ sudo dpkg -L newrelic-infra -``` -e.g: Debian 10 output: -```shell script -/var/db/newrelic-infra -/var/db/newrelic-infra/custom-integrations -/var/db/newrelic-infra/integrations.d -/var/log/newrelic-infra -/var/run/newrelic-infra -/usr/bin/newrelic-infra-ctl -/usr/bin/newrelic-infra -/usr/bin/newrelic-infra-service -/var/db/newrelic-infra/LICENSE.txt -/etc/newrelic-infra -/etc/newrelic-infra/logging.d -/etc/newrelic-infra/logging.d/file.yml.example -/etc/newrelic-infra/logging.d/fluentbit.yml.example -/etc/newrelic-infra/logging.d/syslog.yml.example -/etc/newrelic-infra/logging.d/systemd.yml.example -/etc/newrelic-infra/logging.d/tcp.yml.example -/var/db/newrelic-infra/newrelic-integrations -/var/db/newrelic-infra/newrelic-integrations/logging -/var/db/newrelic-infra/newrelic-integrations/logging/parsers.conf -/etc/systemd/system/newrelic-infra.service -/var/db/newrelic-infra/newrelic-integrations/logging/out_newrelic.so -/var/db/newrelic-infra/newrelic-integrations/logging/fluent-bit -/etc/newrelic-infra/integrations.d -/etc/newrelic-infra/integrations.d/docker-config.yml -/var/db/newrelic-infra/newrelic-integrations/bin -/var/db/newrelic-infra/newrelic-integrations/bin/nri-docker -/var/db/newrelic-infra/newrelic-integrations/bin/nri-flex -/var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus -``` - -#### Scenario 3. Check agent version -Check if version number is well inform. -```shell script -$ newrelic-infra -version -``` -expected output: -```shell script -> New Relic Infrastructure Agent version: 1.16.1, GoVersion: go1.16.12, GitCommit: ... -``` - -#### Scenario 4. Service is working -Check if agent is running and sending metrics to NR. -```shell script -$ sudo systemctl show newrelic-infra --no-page|grep SubState=running -``` -expected output: -```shell script -SubState=running -``` - -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from SystemSample where displayName = '${DISPLAY_NAME}'" -``` -e.g. expected output: -```json -[ - { - "agentName": "Infrastructure", - "agentVersion": "1.16.0", - "displayName": "deb10-test", - "linuxDistribution": "Debian GNU/Linux 10 (buster)" - } -] -``` - -#### Scenario 5. Package metadata is valid -Review if basic metadata is in place. -```shell script -$ apt show newrelic-infra -``` -e.g: expected output: -```shell script -Package: newrelic-infra -Version: 1.16.1 -Priority: extra -Section: default -Maintainer: caos-team@newrelic.com -Installed-Size: 118 MB -Conflicts: newrelic-infra -Replaces: opspro-agent, opspro-agent-sysv -Homepage: https://docs.newrelic.com/docs/release-notes/infrastructure-release-notes/infrastructure-agent-release-notes -Vendor: New Relic, Inc. -Download-Size: 40.6 MB -APT-Manual-Installed: yes -APT-Sources: http://nr-downloads-ohai-testing.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/apt focal/main amd64 Packages -Description: New Relic Infrastructure provides flexible, dynamic server monitoring. With real-time data collection and a UI that scales from a handful of hosts to thousands, Infrastructure is designed for modern Operations teams with fast-changing systems. -``` -note: for Debian 10 You can use `aptitude show` - -#### Scenario 6. Package signature is valid -Review if pub GPG key is same as PROD. -```shell script -$ apt-key list | grep newrelic -n2 -``` -e.g.: expected output: -```shell script -pub rsa4096 2016-10-26 [SCEA] <> -uid [ unknown] infrastructure-eng -/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-archive.gpg -``` - -#### Scenario 7. Agent privileged mode is working -For this use case You should install the agent with privileged mode. -```shell script -$ sudo NRIA_MODE=PRIVILEGED apt install newrelic-infra -y -``` -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 8. Agent unprivileged mode is working -Similar to previous scenario You should install the agent with unprivileged mode. -```shell script -$ sudo NRIA_MODE=UNPRIVILEGED apt install newrelic-infra -y -``` -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 9. Package uninstall -```shell script -$ sudo apt remove newrelic-infra -y -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` -no data should be returned. - -#### Scenario 10. Package upgrade -With an old agent version install, install the latest. -```shell script -$ sudo apt install newrelic-infra=1.15.1 -y --allow-downgrades -$ newrelic-infra -version -$ sudo apt install newrelic-infra -y -$ newrelic-infra -version -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 11. Built in Flex integration is working -Add Flex example yml file and review data in NR. -```shell script -$ sudo curl -o /etc/newrelic-infra/integrations.d/flex-dig.yml https://raw.githubusercontent.com/newrelic/nri-flex/master/examples/linux/dig-example.yml - -$ sudo service newrelic-infra restart -``` -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT uniques(integrationVersion) from flexStatusSample where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "uniques.integrationVersion": [ - "1.4.0" - ] - } -] -``` - -#### Scenario 11. Built in Log-forwarded integration is working -Enable verbose mode in agent configuration file and review data in NR. -```shell script -$ sudo sed -i 's#verbose:.*#verbose: 3#g' /etc/newrelic-infra.yml -$ sudo systemctl restart newrelic-infra -``` -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from Log where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "count": 31 - } -] -``` - -#### Scenario 12. Built in Prometheus integration is working -Check if binary works. -```shell script -$ /var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus --help -``` -expected value: -```shell script -Usage of /var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus: - -config_path string - Path to the config file - -configfile string - Deprecated. --config_path takes precedence if both are set -``` diff --git a/test/manual/use-case/docker_validations.md b/test/manual/use-case/docker_validations.md deleted file mode 100644 index 1be700d2e..000000000 --- a/test/manual/use-case/docker_validations.md +++ /dev/null @@ -1,151 +0,0 @@ -### Scope -This document share which are the commands and validations needed to passed all acceptance criteria for the manual system acceptance test. -It covers support for use case of: -* Docker amd64, arm64 - -### Setup - - For [amd64](https://app.vagrantup.com/bento/boxes/ubuntu-20.04) - - For [arm64](https://aws.amazon.com/ec2/graviton/) - -Steps to install containerize agent [here](https://hub.docker.com/r/newrelic/infrastructure/) or using docker-compose: -```shell script -version: '3' -services: - agent-rc: - image: newrelic/infrastructure:1.16.1-rc - container_name: infra-agent-rc - cap_add: - - SYS_PTRACE - network_mode: host - pid: host - privileged: true - environment: - - NRIA_LICENSE_KEY=${NR_LICENSE_KEY} - - NRIA_STAGING=true - - NRIA_DISPLAY_NAME=${DISPLAY_NAME} - volumes: - - "/:/host:ro" - - "/var/run/docker.sock:/var/run/docker.sock" - restart: unless-stopped -``` -Then up the container: -```shell script -$ sudo -E docker-compose up -``` - -### Test suit -#### Scenario 1. Package naming follow dist convention -Lookup at docker hub and compare tag under test with the previous tag name. \ -e.g.: [Docker hub](https://hub.docker.com/r/newrelic/infrastructure/tags?page=1&ordering=last_updated) \ -Check arch: -```shell script -$ uname -a -``` -expected output: -```shell script -> Linux ip-172-31-19-77.eu-central-1.compute.internal 4.14.219-161.340.amzn2.aarch64 #1 SMP Thu Feb 4 05:54:27 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux -``` - -#### Scenario 2. Package contains all required files -Review file managed by package and compare with previous package version. -```shell script -$ du -a /var/db/newrelic-infra/ -``` -e.g: arm64 output: -```shell script -20160 /var/db/newrelic-infra/newrelic-integrations/bin/nri-flex -24832 /var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus -44992 /var/db/newrelic-infra/newrelic-integrations/bin -44992 /var/db/newrelic-infra/newrelic-integrations -``` - -#### Scenario 3. Check agent version -Check if version number is well inform. -```shell script -$ sudo docker exec -it <> /bin/bash -c 'newrelic-infra -version' -``` -expected output: -```shell script -> New Relic Infrastructure Agent version: 1.16.1, GoVersion: go1.16.12, GitCommit: ... -``` - -#### Scenario 4. Service is working -Check if agent is sending metrics to NR. -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from SystemSample where displayName = '${DISPLAY_NAME}'" -``` -e.g. expected output: -```json -[ - { - "agentName": "Infrastructure", - "agentVersion": "1.16.0", - "displayName": "docker-test", - "linuxDistribution": "Ubuntu Core 18" - } -] -``` - -#### Scenario 5. Package metadata is valid -NA - -#### Scenario 6. Package signature is valid -NA - -#### Scenario 7. Agent privileged mode is working -For this use case You should enable in docker-compose file the privileged mode. -```yaml - privileged: true -``` -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 8. Agent unprivileged mode is working -NA - -#### Scenario 9. Package uninstall -```shell script -$ sudo docker-compose down -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` -no data should be returned. - -#### Scenario 10. Package upgrade -NA - -#### Scenario 11. Built in Flex integration is working -Add Flex example yml file and review data in NR. -e.g.: Dockerfile: -```Dockerfile -FROM newrelic/infrastructure:1.16.1-rc-arm64 -ADD newrelic-infra.yml /etc/newrelic-infra.yml -RUN mkdir /etc/newrelic-infra -RUN mkdir /etc/newrelic-infra/integrations.d/ -RUN wget https://raw.githubusercontent.com/newrelic/nri-flex/master/examples/windows/windows-uptime.yml -P /etc/newrelic-infra/integrations.d/ -``` -Platform verification: -``` -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT uniques(integrationVersion) from flexStatusSample where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -``` -[ - { - "uniques.integrationVersion": [ - "1.4.0" - ] - } -] -``` - -#### Scenario 11. Built in Log-forwarded integration is working -Not supported yet. - -#### Scenario 12. Built in Prometheus integration is working -Not supported yet. diff --git a/test/manual/use-case/win_validations.md b/test/manual/use-case/win_validations.md deleted file mode 100644 index a3f3e1103..000000000 --- a/test/manual/use-case/win_validations.md +++ /dev/null @@ -1,202 +0,0 @@ -### Scope -This document share which are the commands and validations needed to pass all acceptance criteria for the manual system acceptance test. -It covers support for use case of: -* Windows 64-bits, 32-bits, -* Zip and msi installation - -### Setup -- [Windows 64-bits](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/windows/newrelic-infra.msi) -- [Windows 32-bits](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/windows/386/newrelic-infra-386.msi) -- [Zip installation 32-bits](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/binaries/windows/386/newrelic-infra-386.1.16.0.zip) -- [Zip installation 64-bits](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/binaries/windows/amd64/newrelic-infra-amd64.1.16.0.zip) - -### Test suit -#### Scenario 1. Package naming follow dist convention -Lookup at WIN repository folder and compare package under test with the previous package version name. -* [msi WIN Repository folder](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/windows/) \ -package under test version: 1.16.1, name: newrelic-infra.1.16.1.msi \ -previous package version: 1.16.0, name: newrelic-infra.1.16.0.msi - -* [zip WIN Repository folder](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/binaries/windows) \ -package under test version: 1.16.1, name: newrelic-infra-amd64.1.16.1.zip \ -previous package version: 1.16.0, name: newrelic-infra-amd64.1.16.0.zip - -#### Scenario 2. Package contains all required files -Review file managed by package and compare with previous package version. -```shell script -$ dir -r | % { if ($_.PsIsContainer) { $_.FullName + "\" } else { $_.FullName } } -``` -e.g: Win zip installation output: -```shell script -C:\Program Files\New Relic\newrelic-infra\custom-integrations\ -C:\Program Files\New Relic\newrelic-infra\integrations.d\ -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\ -C:\Program Files\New Relic\newrelic-infra\LICENSE.txt -C:\Program Files\New Relic\newrelic-infra\newrelic-infra-ctl.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-infra-service.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-infra.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-infra.log -C:\Program Files\New Relic\newrelic-infra\newrelic-infra.yml -C:\Program Files\New Relic\newrelic-infra\yamlgen.exe -``` -e.g: Win msi installation output: -``` -C:\Program Files\New Relic\newrelic-infra\custom-integrations\ -C:\Program Files\New Relic\newrelic-infra\integrations.d\ -C:\Program Files\New Relic\newrelic-infra\logging.d\ -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\ -C:\Program Files\New Relic\newrelic-infra\LICENSE.txt -C:\Program Files\New Relic\newrelic-infra\newrelic-infra-ctl.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-infra-service.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-infra.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-infra.log -C:\Program Files\New Relic\newrelic-infra\newrelic-infra.yml -C:\Program Files\New Relic\newrelic-infra\yamlgen.exe -C:\Program Files\New Relic\newrelic-infra\integrations.d\newrelic-infra-winpkg-config.yml -C:\Program Files\New Relic\newrelic-infra\logging.d\file.yml.example -C:\Program Files\New Relic\newrelic-infra\logging.d\fluentbit.yml.example -C:\Program Files\New Relic\newrelic-infra\logging.d\logs-test.yml -C:\Program Files\New Relic\newrelic-infra\logging.d\tcp.yml.example -C:\Program Files\New Relic\newrelic-infra\logging.d\winlog.yml.example -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\ -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\newrelic-infra-winpkg-definition.yml -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\nr-winpkg.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\nri-prometheus.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\fb.db -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\fb.db-shm -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\fb.db-wal -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\fluent-bit.dll -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\fluent-bit.exe -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\out_newrelic.dll -C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\logging\parsers.conf -``` - -#### Scenario 3. Check agent version -Check if version number is well inform. -```shell script -$ [System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Program Files\New Relic\newrelic-infra\newrelic-infra.exe").FileVersion -``` -expected output: -```shell script -> 1.16.1 -``` - -#### Scenario 4. Service is working -Check if agent is running and sending metrics to NR. -```shell script -$ Get-Service newrelic-infra | Where-Object {$_.Status -EQ "Running"} -``` -expected output: -```shell script -Status Name DisplayName ------- ---- ----------- -Running newrelic-infra New Relic Infrastructure Agent (x86) -``` - -Platform validation: -``` -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from SystemSample where displayName = '${DISPLAY_NAME}'" -``` -e.g. expected output: -``` -[ - { - "agentName": "Infrastructure", - "agentVersion": "1.16.1", - ... - "displayName": "win-test", - ... - "windowsPlatform": "Microsoft Windows 10 Pro", - } -] -``` - -#### Scenario 5. Package metadata is valid -Review if basic metadata is in place. These steps are completely manual or through a [custom function](https://devblogs.microsoft.com/scripting/list-music-file-metadata-in-a-csv-and-open-in-excel-with-powershell/) - -#### Scenario 6. Package signature is valid -Review if PDX key is same as PROD and valid. -```shell script -$ Get-AuthenticodeSignature -FilePath "C:\Program Files\New Relic\newrelic-infra\newrelic-infra.exe" - -$ (Get-AuthenticodeSignature "newrelic-infra.exe").Status -eq 'Valid' -``` -e.g.: expected output: -```shell script -Directory: C:\Program Files\New Relic\newrelic-infra -SignerCertificate Status Path ------------------ ------ ---- -<> Valid newrelic-infra.exe -``` - -#### Scenario 7. Agent privileged mode is working -Not supported - -#### Scenario 8. Agent unprivileged mode is working -Not supported - -#### Scenario 9. Package uninstall -```shell script -$ root > sc delete newrelic-infra -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` -no data should be returned. - -#### Scenario 10. Package upgrade -With an old agent version install, install the latest. -```shell script -$ msiexec.exe /qn /i PATH\TO\newrelic-infra.msi - -$ net start newrelic-infra -``` -Platform verification: -``` -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 11. Built in Flex integration is working -Add Flex example yml file and review data in NR. -```shell script -$ (New-Object System.Net.WebClient).DownloadFile("https://raw.githubusercontent.com/newrelic/nri-flex/master/examples/windows/windows-uptime.yml", "C:\Program Files\New Relic\newrelic-infra\integrations.d\flex-uptime.yml"); -``` - -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT uniques(integrationVersion) from flexStatusSample where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "uniques.integrationVersion": [ - "1.4.0" - ] - } -] -``` -note: not supported for zip installation - -#### Scenario 11. Built in Log-forwarded integration is working -Enable verbose mode in agent configuration file and review data in NR. -```shell script -$ Rename-Item -Path " C:\Program Files\New Relic\newrelic-infra\logging.d\winlog.yml.example" -NewName "C:\Program Files\New Relic\newrelic-infra\logging.d\winlog.yml" -``` -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from Log where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "count": 31 - } -] -``` -note: not supported for zip installation - -#### Scenario 12. Built in Prometheus integration is working -Not applicable diff --git a/test/manual/use-case/yum_validations.md b/test/manual/use-case/yum_validations.md deleted file mode 100644 index a43069de9..000000000 --- a/test/manual/use-case/yum_validations.md +++ /dev/null @@ -1,222 +0,0 @@ -### Scope -This document share which are the commands and validations needed to passed all acceptance criteria for the manual system acceptance test. -It covers support for use case of: -* Red Hat Enterprise Linux version 8, 7 and 6 -* CentOs version 8, 7, 6 - -### Setup -- For [RHEL 6](https://app.vagrantup.com/generic/boxes/rhel6) or [CentOs 6](https://app.vagrantup.com/generic/boxes/centos6): - ```shell script - $ sudo curl -o /etc/yum.repos.d/newrelic-infra.repo http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/yum/el/6/x86_64/newrelic-infra.repo - ``` -- For [RHEL 7](https://app.vagrantup.com/generic/boxes/rhel7) or [CentOs 7](https://app.vagrantup.com/generic/boxes/centos7): - ```shell script - $ sudo curl -o /etc/yum.repos.d/newrelic-infra.repo http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/yum/el/7/x86_64/newrelic-infra.repo - ``` -- For [RHEL 8](https://app.vagrantup.com/generic/boxes/rhel8) or [CentOs 8](https://app.vagrantup.com/generic/boxes/centos8): - ```shell script - $ sudo curl -o /etc/yum.repos.d/newrelic-infra.repo http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/yum/el/8/x86_64/newrelic-infra.repo - ``` - -### Test suit -#### Scenario 1. Package naming follow dist convention -Lookup at apt repository folder and compare package under test with the previous package version name. \ -e.g.: [YUM v8 Repository folder](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/yum/el/8/x86_64/) \ -package under test version: 1.16.1, name: newrelic-infra-1.16.1-1.el8.x86_64.rpm \ -previous package version: 1.16.0, name: newrelic-infra-1.16.0-1.el8.x86_64.rpm - -#### Scenario 2. Package contains all required files -Review file managed by package and compare with previous package version. -```shell script -$ repoquery -l newrelic-infra --installed -``` -e.g: RHEL 6 output: -```shell script -/etc/init.d/newrelic-infra -/etc/init/newrelic-infra.conf -/etc/newrelic-infra/integrations.d/docker-config.yml -/usr/bin/newrelic-infra -/usr/bin/newrelic-infra-ctl -/usr/bin/newrelic-infra-service -/var/db/newrelic-infra/LICENSE.txt -/var/db/newrelic-infra/custom-integrations -/var/db/newrelic-infra/integrations.d -/var/db/newrelic-infra/newrelic-integrations/bin/nri-docker -/var/db/newrelic-infra/newrelic-integrations/bin/nri-flex -/var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus -/var/log/newrelic-infra -/var/run/newrelic-infra -``` - -#### Scenario 3. Check agent version -Check if version number is well inform. -```shell script -$ newrelic-infra -version -``` -expected output: -```shell script -> New Relic Infrastructure Agent version: 1.16.1, GoVersion: go1.16.12, GitCommit: ... -``` - -#### Scenario 4. Service is working -Check if agent is running and sending metrics to NR. -```shell script -$ sudo systemctl show newrelic-infra --no-page|grep SubState=running -``` -expected output: -```shell script -> SubState=running -``` - -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from SystemSample where displayName = '${DISPLAY_NAME}'" -``` -e.g. expected output: -```json -[ - { - "agentName": "Infrastructure", - "agentVersion": "1.16.1", - "displayName": "centos7-test", - "linuxDistribution": "CentOS Linux 7 (Core)" - } -] -``` -note: CentOS6 and RHEL6 run `$ sudo initctl status newrelic-infra` instate. - -#### Scenario 5. Package metadata is valid -Review if basic metadata is in place. -```shell script -$ sudo yum info newrelic-infra -q -``` -e.g: expected output: -```shell script -Installed Packages -Name : newrelic-infra -Arch : x86_64 -Version : 1.16.11 -Release : 1.el7 -Size : 112 M -Repo : installed -From repo : newrelic-infra -Summary : New Relic Infrastructure Agent -URL : https://docs.newrelic.com/docs/release-notes/infrastructure-release-notes/infrastructure-agent-release-notes -License : Copyright (c) 2008-2021 New Relic, Inc. All rights reserved. -Description : New Relic Infrastructure provides flexible, dynamic server monitoring. With real-time data collection and - : a UI that scales from a handful of hosts to thousands, Infrastructure is designed for modern Operations - : teams with fast-changing systems. -``` - -#### Scenario 6. Package signature is valid -Adding a repository in YUM is a manual operation, which consists in creating a file with the .repo extension under the folder /etc/yum.repos.d. -The file must contain all the information about the custom repository that we are connecting to, therefore in setup step we already downloading the GPG. -What You can check if it was correctly added to rpm: -```shell script -$ rpm -qa gpg-pubkey --qf "%{summary}\n" | grep infrastructure-eng -``` -expected output: -```shell script -gpg(infrastructure-eng ) -``` - -#### Scenario 7. Agent privileged mode is working -For this use case You should install the agent with privileged mode. -```shell script -$ sudo NRIA_MODE=PRIVILEGED yum install newrelic-infra -y -``` -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 8. Agent unprivileged mode is working -Similar to previous scenario You should install the agent with unprivileged mode. -```shell script -$ sudo NRIA_MODE=UNPRIVILEGED yum install newrelic-infra -y -``` -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 9. Package uninstall -```shell script -$ sudo yum remove newrelic-infra -y -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` -no data should be returned. - -#### Scenario 10. Package upgrade -With an old agent version install, install the latest. -```shell script -$ sudo yum install newrelic-infra=1.15.1 -y --allow-downgrades -$ newrelic-infra -version -$ sudo yum install newrelic-infra -y -$ newrelic-infra -version -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 11. Built in Flex integration is working -Add Flex example yml file and review data in NR. -```shell script -$ sudo curl -o /etc/newrelic-infra/integrations.d/flex-dig.yml https://raw.githubusercontent.com/newrelic/nri-flex/master/examples/linux/dig-example.yml - -$ sudo systemctl newrelic-infra restart -``` -notes: for version 6 use `sudo initctl newrelic-infra restart` - -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT uniques(integrationVersion) from flexStatusSample where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "uniques.integrationVersion": [ - "1.4.0" - ] - } -] -``` - -#### Scenario 11. Built in Log-forwarded integration is working -Enable verbose mode in agent configuration file and review data in NR. -```shell script -$ sudo sed -i 's#verbose:.*#verbose: 3#g' /etc/newrelic-infra.yml -$ sudo systemctl restart newrelic-infra -``` -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from Log where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "count": 31 - } -] -``` -notes: not supported in version 6. - -#### Scenario 12. Built in Prometheus integration is working -Check if binary works. -```shell script -$ /var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus --help -``` -expected value: -```shell script -Usage of /var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus: - -config_path string - Path to the config file - -configfile string - Deprecated. --config_path takes precedence if both are set -``` diff --git a/test/manual/use-case/zypper_validations.md b/test/manual/use-case/zypper_validations.md deleted file mode 100644 index 9205abb70..000000000 --- a/test/manual/use-case/zypper_validations.md +++ /dev/null @@ -1,228 +0,0 @@ -### Scope -This document share which are the commands and validations needed to passed all acceptance criteria for the manual system acceptance test. -It covers support for use case of: -* SLES 12.04 - -### Setup -- For [SLES 12.04](https://app.vagrantup.com/wandisco/boxes/sles-12.4-64) -```shell script -$ sudo curl -o /etc/zypp/repos.d/newrelic-infra.repo http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/zypp/sles/12.4/x86_64/newrelic-infra.repo -``` - -### Test suit -#### Scenario 1. Package naming follow dist convention -Lookup at apt repository folder and compare package under test with the previous package version name. \ -e.g.: [ZYPP Repository folder](http://nr-downloads-ohai-staging.s3-website-us-east-1.amazonaws.com/infrastructure_agent/linux/zypp/sles/12.4/x86_64/) \ -package under test version: 1.16.1, name: newrelic-infra-1.16.1-1.sles12.4.x86_64.rpm \ -previous package version: 1.16.0, name: newrelic-infra-1.16.0-1.sles12.4.x86_64.rpm - -#### Scenario 2. Package contains all required files -Review file managed by package and compare with previous package version. -```shell script -$ sudo rpm -ql newrelic-infra -``` -e.g: expected output: -```shell script -/etc/newrelic-infra/integrations.d/docker-config.yml -/etc/newrelic-infra/logging.d/file.yml.example -/etc/newrelic-infra/logging.d/fluentbit.yml.example -/etc/newrelic-infra/logging.d/syslog.yml.example -/etc/newrelic-infra/logging.d/systemd.yml.example -/etc/newrelic-infra/logging.d/tcp.yml.example -/etc/systemd/system/newrelic-infra.service -/usr/bin/newrelic-infra -/usr/bin/newrelic-infra-ctl -/usr/bin/newrelic-infra-service -/usr/lib/.build-id -/usr/lib/.build-id/5c -/usr/lib/.build-id/5c/c36c6cb0850c532449ab2940aba99a9ae4dacf -/var/db/newrelic-infra/LICENSE.txt -/var/db/newrelic-infra/custom-integrations -/var/db/newrelic-infra/integrations.d -/var/db/newrelic-infra/newrelic-integrations/bin/nri-docker -/var/db/newrelic-infra/newrelic-integrations/bin/nri-flex -/var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus -/var/db/newrelic-infra/newrelic-integrations/logging/fluent-bit -/var/db/newrelic-infra/newrelic-integrations/logging/out_newrelic.so -/var/db/newrelic-infra/newrelic-integrations/logging/parsers.conf -/var/log/newrelic-infra -/var/run/newrelic-infra -``` - -#### Scenario 3. Check agent version -Check if version number is well inform. -```shell script -$ newrelic-infra -version -``` -expected output: -```shell script -> New Relic Infrastructure Agent version: 1.16.1, GoVersion: go1.16.12, GitCommit: ... -``` - -#### Scenario 4. Service is working -Check if agent is running and sending metrics to NR. -```shell script -$ sudo systemctl show newrelic-infra --no-page|grep SubState=running -``` -expected output: -```shell script -> SubState=running -``` - -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from SystemSample where displayName = '${DISPLAY_NAME}'" -``` -e.g. expected output: -```json -[ - { - "agentName": "Infrastructure", - "agentVersion": "1.16.1", - "displayName": "test-script", - "linuxDistribution": "SUSE Linux Enterprise Server 12 SP2" - } -] -``` - -#### Scenario 5. Package metadata is valid -Review if basic metadata is in place. -```shell script -$ rpm -qi newrelic-infra -``` -e.g: expected output: -```shell script -Name : newrelic-infra -Epoch : 0 -Version : 1.16.1 -Release : 1.sles12.4 -Architecture: x86_64 -Install Date: Tue 09 Mar 2021 04:46:17 PM UTC -Group : default -Size : 135872380 -License : Copyright (c) 2008-2021 New Relic, Inc. All rights reserved. -Signature : RSA/SHA1, Wed 17 Feb 2021 02:55:10 PM UTC, Key ID bb29ee038ecce87c -Source RPM : newrelic-infra-1.15.2-1.sles12.4.src.rpm -Build Date : Wed 10 Feb 2021 10:13:03 AM UTC -Build Host : 7c9aed865529 -Relocations : / -Packager : caos-team@newrelic.com -Vendor : New Relic, Inc. -URL : https://docs.newrelic.com/docs/release-notes/infrastructure-release-notes/infrastructure-agent-release-notes/new-relic-infrastructure-agent-1152 -Summary : New Relic Infrastructure Agent -Description : -New Relic Infrastructure provides flexible, dynamic server monitoring. With real-time data collection and a UI that scales from a handful of hosts to thousands, Infrastructure is designed for modern Operations teams with fast-changing systems. -Distribution: (none) -``` - -#### Scenario 6. Package signature is valid -Review if pub GPG key is same as PROD. -```shell script -$ gpg --keyid-format short --list-keys -``` -e.g.: expected output: -```shell script -pub rsa4096 2016-10-26 [SCEA] <> -uid [ unknown] infrastructure-eng -``` - -#### Scenario 7. Agent privileged mode is working -For this use case You should install the agent with privileged mode. -```shell script -$ sudo NRIA_MODE=PRIVILEGED zypper -n install newrelic-infra -``` - -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 8. Agent unprivileged mode is working -Similar to previous scenario You should install the agent with unprivileged mode. -```shell script -$ sudo NRIA_MODE=UNPRIVILEGED yum install newrelic-infra -y -``` - -Platform validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 9. Package uninstall -```shell script -$ sudo zypper remove newrelic-infra -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` -no data should be returned. - -#### Scenario 10. Package upgrade -With an old agent version install, install the latest. -```shell script -$ sudo zypper -n install newrelic-infra-1.15.1 -$ newrelic-infra -version -$ sudo zypper -n install newrelic-infra -$ newrelic-infra -version -``` -Platform Validation: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT * from SystemSample where displayName = '${DISPLAY_NAME}' limit 1" -``` - -#### Scenario 11. Built in Flex integration is working -Add Flex example yml file and review data in NR. -```shell script -$ sudo curl -o /etc/newrelic-infra/integrations.d/flex-dig.yml https://raw.githubusercontent.com/newrelic/nri-flex/master/examples/linux/dig-example.yml - -$ sudo systemctl restart newrelic-infra -``` - -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT uniques(integrationVersion) from flexStatusSample where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "uniques.integrationVersion": [ - "1.4.0" - ] - } -] -``` - -#### Scenario 11. Built in Log-forwarded integration is working -Enable verbose mode in agent configuration file and review data in NR. -```shell script -$ sudo sed -i 's#verbose:.*#verbose: 3#g' /etc/newrelic-infra.yml -$ sudo systemctl restart newrelic-infra -``` -Platform verification: -```shell script -$ newrelic nrql query -a ${NR_ACCOUNT_ID} -q "SELECT count(*) from Log where displayName = '${DISPLAY_NAME}'" -``` -e.g: expected output: -```json -[ - { - "count": 31 - } -] -``` - -#### Scenario 12. Built in Prometheus integration is working -Check if binary works. -```shell script -$ /var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus --help -``` -expected value: -```shell script -Usage of /var/db/newrelic-infra/newrelic-integrations/bin/nri-prometheus: - -config_path string - Path to the config file - -configfile string - Deprecated. --config_path takes precedence if both are set -``` \ No newline at end of file