Releases: zebrunner/community-edition
1.5 (2021-02-16)
Enhancements
- [#405] Added information about templates updates for AWS S3 storage use-case
- [#397] Added the default minio setup if user disable AWS S3 integration
Fixes
- [#406][#392] Shutdown procedure should remove all ".disabled" files
- [#403] Mail service doesn't work. Failed to process message MailDataMessage
Known Issues
- [#2243][#409] AWS S3 Templates sync up needed
Apply workarounds manually for AWS S3 templates based on the tickets
Components Updates
- reporting 1.19 -> 1.20
- mcloud 1.3 -> 1.4 (mcloud-grid 1.0 -> 1.1)
Migration Steps
Note: It is possible to migrate from 1.0+ versions automatically!
Current steps describe migration from 1.4 release.
- it is recommended to make a "golden" backup running backup task and making a copy of zebrunner folder
./zebrunner.sh backup
cp -r ../zebrunner ../zebrunner.bak
- start services and make sure everything is up&running!
./zebrunner.sh start
- pull recursively latest changes and verify that sub-modules are valid
git pull --recurse-submodules
- Important! Verify that submodules have below versions:
git submodule status
... jenkins (1.4-4-g74f61db)
... mcloud (1.4)
... reporting (1.20)
... selenoid (1.2-2-g6d64b13)
... sonarqube (1.2-2-gcd35598)
There is no sense to continue if any sub-module has older version! Ask for the assistance in telegram
- run "upgrade" task
./zebrunner.sh upgrade
// Answer "y" to start an upgrade and restart services.
- [Optional] If you use your AWS S3 bucket, please, upload manually new minio test_run_results.ftl template into the s3://your-bucket/templates/
cd zebrunner
aws s3 cp reporting/minio-storage/configuration/minio/data/templates/test_run_results.ftl s3://your-bucket/templates/
1.4 (2021-01-25)
Enhancements
- [#381][#353] Simplified setup process removing 8 questions and making reporting an obligatory component
- [#382] Enabled an easy way to manage Live VNC sessions including custom ones
- [#378] Configured external S3 storage for mcloud and selenoid if reporting is disabled
- [#352][#268] Improved default README documentation steps
- [#371] Limited nginx logs by size (512Mb)
- [#351] Disabled 3rd party integration with browserstack/saucelabs, etc.
- [#329] Setup tools order in case of disabled reporting
- [#291] provided sample skeleton and default folders for SSL setup on NGiNX
Fixes
- [#383] Don't put to NOTICE.txt information about empty integrations
- [#376] Reviewed and updated references to selenium-hub container name using mcloud-grid
- [#268] Zebrunner doc contained out-of-date image of structure
Known Issues
- #2243 AWS S3: Providers not available in Test Launchers
Apply workaround manually for AWS S3 templates based on the ticket
Components Updates
- reporting 1.16 -> 1.19
- jenkins-master 1.3 -> 1.4
- mcloud 1.2 -> 1.3 (mcloud-android 1.0, mcloud-ios 1.0, SmartTestFarm 1.0)
- selenoid 1.1 -> 1.2
Migration Steps
Note: It is possible to migrate from 1.0+ versions automatically!
Current steps describe migration from 1.3 release.
- it is recommended to make a "golden" backup running backup task and making a copy of zebrunner folder
./zebrunner.sh backup
cp -r ../zebrunner ../zebrunner.bak
- start services and make sure everything is up&running!
./zebrunner.sh start
- pull recursively latest changes and verify that sub-modules are valid
git pull --recurse-submodules
- Important! Verify that submodules have below versions:
git submodule status
... jenkins (1.4)
... mcloud (1.3)
... reporting (1.19)
... selenoid (1.2)
... sonarqube (1.2)
There is no sense to continue if any sub-module has older version! Ask for the assistance in telegram
- run "upgrade" task
./zebrunner.sh upgrade
// Answer "y" to start an upgrade and restart services.
For MCloud upgrade one extra question should appear:
Do you want to remove MCloud FTP data volume? You might lose old video recordings!
-> Answer "n" to keep FTP artifacts.
- Login to Jenkins, edit <your_organization>/RegisterRepository job(s) updating pipeline library version:
@Library('[email protected]')
->
@Library('[email protected]')
- Reregister all your test repositories using updated above RegisterRepository job(s).
Important! To be in sync with reporting changes minimal carina-core versions are 6.5.47 and 7.0.0
with 6.5.46 and lower versions video recording will be corrupted
- [Optional] Remove
aws-jacoco-token
jenkins creds
1.3 (2020-12-04)
Enhancements
- Bump to new jenkins-master and slaves.
- Produced jnpl jenkins-slave images for ECS/Fargate integration
- Bump to 1.16 reporting
- Bump up to the 1.3 pipeline release
- [#366] Removed reporting/db-migration-tool
- [#359] Replaced several system jobs during the upgrade
Fixes
- [#365] Mail service did not work. Failed to process message MailDataMessage
- [#362] Repo was empty after scanning it via Launchers
- [#358] jenkins/variables.env contained changes and were regenerated during the upgrade
Known Issues
- #2243 AWS S3: Providers not available in Test Launchers
Apply workaround manually for AWS S3 templates based on the ticket
Components Updates
- reporting 1.14 -> 1.16
- jenkins-master 1.2 -> 1.3
Migration Steps
Note: It is possible to migrate from 1.0, 1.1 and 1.2 versions automatically!
Current steps describe migration from 1.2 release.
- it is recommended to make a "golden" backup running backup task and making a copy of zebrunner folder
./zebrunner.sh backup
cp -r ../zebrunner ../zebrunner.bak
- start services and make sure everything is up&running!
./zebrunner.sh start
- pull recursively latest changes and verify that sub-modules are valid
git pull --recurse-submodules
- Important! Verify that submodules have below versions:
git submodule status
... jenkins (1.3)
... mcloud (1.2)
... reporting (1.16)
... selenoid (1.1-2-g8a510df)
... sonarqube (1.2)
There is no sense to continue if any sub-module has older version! Ask for the assistance in telegram
- run "upgrade" task
./zebrunner.sh upgrade
// Answer "y" to start an upgrade and restart services.
1.2 (2020-11-09)
Enhancements
- Bump up Java 8 -> 11 on jenkins-slave
- [#276][#234] Certified integration with github/gitlab and bitbucket repositories
- [#324] Selenium video recording pushed to S3 compatible storage (by default into embedded minio)
- [#319] Automated jenkins upgrade tasks
- [#335][#334][#322][#316] Improved warn/info messages and guide for the next steps like start/restart etc
- [#314][#280] Added support for special symbols except quotas and ambiguous characters
Fixes
- [#336] Backup/restore should not be started for disabled components
- [#331] Fixed the issue when it was impossible to create a new widget
- [#326] Public filters did not work in the Zebrunner UI tool
- [#321] Automatic services startup are not started
- [#264] Updated User Guide .xml file configuration parameters list
Known Issues
- [#347] reporting can't be started after upgrade to 1.2: function update_modified_at() does not exist
- #2243 AWS S3: Providers not available in Test Launchers
Apply workaround manually for AWS S3 templates based on the ticket - Some conflicts might appear for manually installed jenkins plugins. It is recommended to upgrade all compatible to the latest versions in such case.
[COMPONENTS UPDATES]
- reporting 1.12 -> 1.14
- jenkins-master 1.1 -> 1.2
- sonarqube 1.1 -> 1.2
- mcloud 3.3 -> 1.2 (3.4 in qps)
[MIGRATION STEPS]
Note: It is possible to migrate from 1.0 and/or 1.1 versions automatically!
- it is recommended to make a "golden" backup running backup task and making a copy of zebrunner folder
./zebrunner.sh backup
cp -r ../zebrunner ../zebrunner.bak
- start services and make sure everything is up&running!
./zebrunner.sh start
- pull recursively latest changes and verify that sub-modules are valid
git pull --recurse-submodules
- Important! Verify that submodules have below versions:
git submodule status
... jenkins (1.2-4-gd9ea800)
... mcloud (1.2)
... reporting (1.14)
... selenoid (1.1-2-g8a510df)
... sonarqube (1.2)
There is no sense to continue if any sub-module has older version! Ask for the assistance in telegram
- run "upgrade" task
./zebrunner.sh upgrade
// Answer "y" to start an upgrade, provide "n" replies if you don't use custom pipeline libraries and global choices and restart services.
- Login to Jenkins
- Re-register again Organization via Management_Jobs/RegisterOrganization job
Note: hopefully last time when organization should be re-registered :) - Re-register again all your repositories using <ORG_NAME>/RegisterRepository jobs
Note: make sure to provide scmType and repoUrl in new way using https or ssh url! - Obligatory remove in Manage Jenkins -> Configure System GITHUB_* global variables:
- GITHUB_HOST
- GITHUB_API_URL
- GITHUB_ORGANIZATION
- GITHUB_HTML_URL
- GITHUB_OAUTH_TOKEN
- GITHUB_SSH_URL
- [Optional] Remove in Manage Jenkins -> Configure System deprecated global variables:
- QPS_PIPELINE_GIT_BRANCH
- QPS_PIPELINE_GIT_URL
- QPS_PIPELINE_LOG_LEVEL
- In case of using AWS S3 as a storage manually update ./selenoid/.env parameters using valid S3 data for
S3_ENDPOINT=https://s3.us-east-1.amazonaws.com:443
S3_ACCESS_KEY_ID=<access_key>
S3_SECRET=<secret_key>
S3_REGION=us-east-1
S3_BUCKET=mybucket
1.1 (2020-10-09)
Enhancements
- [#305] Automated the upgrade scenario for all sub-components using
./zebrunner.sh upgrade
For details, take a look at Migration Steps - [#308][#311] Organized automatic services startup after the setup process
- [#302] Simplified integrations settings configuration
- [#273] Added a warning and extra confirmation for the shutdown operation
- [#281] Redirected NGiNX stdout/stderr logs into the container stdout/etderr
- [#182] Provided the possibility to run headless web tests from scratch
Fixes
- [#312] A warning is displayed for any Zebrunner command if the setup wasn't executed
- [#309] The basic zebrunner folder title is shown before the shutdown starts
- [#307] Updated the confirmation method to show the valid "y/n" variant
- [#288][#285] Increased proxy_connect_timeout for all selenium/appium hubs from 10 to 60 by default
- [#212] "Reporting service" contained additional spaces
- [#310][#304][#303][#282] Set of regressions during new features development
Known Issues
- #2243 AWS S3: Providers not available in Test Launchers
Apply workaround manually for AWS S3 templates based on the ticket - Some conflicts might appear for manually installed jenkins plugins. So analyze and upgrade such plugins to make them compatible with latest changes
[COMPONENTS UPDATES]
- reporting 1.9 -> 1.12
- jenkins-master 1.0 -> 1.1
- sonarqube 1.0 -> 1.1
- mcloud 3.2 -> 3.3
[MIGRATION STEPS]
- it is recommended to make a "golden" backup running backup task and making a copy of zebrunner folder
./zebrunner.sh backup
cp -r ../zebrunner ../zebrunner.bak
- start services and make sure everything is up&running
./zebrunner.sh start
- pull recursively latest changes and verify that sub-modules are valid
git pull --recurse-submodules
- Important! Verify that submodules have below versions:
git submodule status
... jenkins (1.1)
... mcloud (3.3)
... reporting (1.12.1)
... selenoid (remotes/origin/master)
... sonarqube (1.1)
There is no sense to continue if any sub-module has older version!
- run "upgrade" task
./zebrunner.sh upgrade
// answer "y" to make an upgrade and restart services
- remove orphan container manually
docker rm -f stf-auth-mock
- Login to jenkins -> Manage Jenkins -> Configure System
Update QPS_PIPELINE_GIT_BRANCH to 5.3
1.0 (2020-08-24); 5.2-qps
Enhancements
- Moved qaprosoft/qps-infra into the new organization zebrunner/zebrunner.
Starting from now it is called "Zebrunner Community Edition".
- Moved each sub-module into the Zebrunner organization as well
- Upgraded Reporting Tool from 1.3 to 1.9
- Reset release numbering of all sub-components starting with 1.0 (5.2 in term of qaprosoft org).
Next 1-2 releases we are going to release both versions for each organization
- Introduced single zebrunner.sh script to manage deployed components
- Delivered backup/restore feature for all sub-components
- Default jenkins-master are delivered with jdk11:
- zebrunner/jenkins-master:1.0
- qaprosoft/jenkins-master:5.2
- jdk8 images:
- zebrunner/jenkins-master:1.0-jdk8
- qaprosoft/jenkins-master:5.2-jdk8
- [#246][#245][#244][#240][#235][#227] registered reporting, jenkins, sonarqube, mcloud and selenoid as independent sub-modules
- [#262][#261] Introduced interactive bash installer to configure sub-modules and their properties
- [#162] Secured https protocol is configurable using interactive installer (actual WebServer or ELB https configuration should be done by IT)
- [#228] Integrated local s3 compatible minio storage for screenshots etc with ability to setup minio or AWS S3 during setup
- [#233] provided easy way to enable/disable sub-module for the infrastructure
- [#237][#230] Updated documentation based on recent changes
- [#249][#241] Standardized messaging and services manipulation using zebrunner.sh shell script
./zebrunner setup
./zebrunner.sh start
./zebrunner.sh stop
./zebrunner.sh restart
./zebrunner.sh down
./zebrunner.sh shutdown
Fixes
- [#267] Integration with AWS S3 did not work right after the setup
- [#255] Fixed the ability to see real-time VNC sessions for web tests
- [#254] Sonarqube value was not set up for Sonar URL in Jenkins after the default setup
- [#253] Selenoid submodule contained untracked files
- [#252] Content was not displayed correctly on UI for http://demo:demo@hostname/mcloud/grid/console
- [#251] Fixed documentation deployment process
- [#247] Changed the wrong branches for sub-modules in recursive cloning of Community Edition
- [#232] Fixed Register Organization issues
- [#211] After job scanning, the Console log contained errors
- [#195] The old URL was shown on adding GitHub enterprise via DB insert query when running via a test launcher scan
- [#151] Zafira logo in Safari and Firefox were displayed incorrectly on the pin tab menu
Known Issues
- [#249] Scanner is not working properly in Zafira
[COMPONENTS UPDATES]
- reporting 1.3 -> 1.9
- jenkins-master 5.1 -> 1.0 (5.2)
- sonarqube 2.2 -> 1.0 (2.3)
[MIGRATION STEPS]
That's a valid steps for upgrading from 5.1 and 5.2 releases! Please, ask asistance if you wanna to upgrade from 4.x version
- Login to jenkins, goto Management_Jobs and remove RegisterOrganization job
- Delete content of Jenkins init.groovy.d folder in console
docker exec -it jenkins-master /bin/bash
rm -rf ./var/jenkins_home/init.groovy.d/*
ls -la ./var/jenkins_home/init.groovy.d
edit .env file with new tag
- Stop everything using ./stop.sh command
- clone everything into the new location and execute setup script
git clone --recurse-submodule https://github.com/zebrunner/zebrunner.git
cd zebrunner
./zebrunner.sh setup
- Provide valid values for protocol, host, port; enabled/disabled components and credentials
make sure to provide the same credentials you have in your previous environment
- Start everything using ./zebrunner.sh start command
- Login to Jenkins -> Goto Manage Jenkins -> Configure System
- update "QPS_PIPELINE_GIT_BRANCH=5.2"
- Apply widget templates and views changes executing vs pgdata postgres DB such queries:
5.1 (2020-07-13)
Enhancements
- [#122][#172] Integrated MCloud into default infra package
- Upgraded embedded SonarQube from 7.1 to 7.9.3 to support Java 11
- [#189][#204] Automatic update selenoid images (chrome, firefox and opera browsers)
cd qps-infra/selenoid
./update.sh
// two latest versions of chrome, firefox and opera browsers should be downloaded
// restart infra
- [#202][#203] Improved performance for timestamps analysis for views/widgets
- [#159][#187] Updated installation, configuration and user guides
Fixes
- [#194] Invalid folder share for mcloud/ftp container
- [#191] Doc: obsolete screenshots were replaced
- [#170] Default Z icon was not displayed for the active tab
- [#133] Fixed UI issues in the User Guides
[COMPONENTS UPDATES]
- jenkins-master 5.0 -> 5.1
- sonarqube 7.1 -> 7.9.3
- mcloud 3.2
[MIGRATION STEPS]
- Goto Management_Jobs and remove RegisterOrganization job
- Stop everything using ./stop.sh command
- Pull changes from qps-infra (master branch)
- Update your .env file putting latest versions for
TAG_JENKINS_MASTER=5.1
TAG_SONAR=2.2
TAG_STF=3.2
- Start everything using ./start.sh command
- Login to Jenkins -> Goto Manage Jenkins -> Configure System
- update "QPS_PIPELINE_GIT_BRANCH=5.1"
- declare new global choice variables:
name: gc_PIPELINE_LIBRARY value: QPS-Pipeline name: gc_RUNNER_CLASS values: com.qaprosoft.jenkins.pipeline.runner.maven.TestNG com.qaprosoft.jenkins.pipeline.runner.maven.Runner com.qaprosoft.jenkins.pipeline.runner.gradle.Runner
- Reregister all organizations and repositories to re-create all jobs
- Setup SonarQube integration according to Configuration Guide
5.0 (05-05-2020)
Enhancements
- Redesigned components structure making it more modular.
- Each module/component has their own docker-composer.yml with ability to enable/disable much easier.
Note: exact steps to enable/disable going to be released in 5.1 - Unified docker containers naming to have nginx, jenkins-master and similar stable naming for each micro-service. That's should significantly simplify troubleshooting articles creation and regular containers manipulation
- [#139] Switched to the Zebrunner Insights v 1.3 (former Zafira)
- [#132][#39] Organized main NGiNX webserver fully independent from disabled components, i.e. no needs anymore to update default.conf if any component is disabled
- Upgraded to 2.237 Jenkins. Delivered jenkins-master and slave based on jdk11. All plugins upgraded and tested using latest versions.
- Prepared and switched to Alpine based jenkins-slave. It is optimized for carina tests.
- [#164][#65] Upgraded embedded sonar to 7.1 and simplified default integration for java code analysis (up to jdk8 so far). Configuration steps published.
- [#142] Reused docker-composer 3.7 for all sub-components
- [#88] Organized browsers images upgrade and quota regeneration shell script for Linux. Enhancement for Mac submitted [#189]
- [#147] setup.sh is now prohibited for "localhost" or "127.0.0.1" hostname :)
- [#149][#122] Started integration of MCloud components into the infra
Note: For valid integration extra configuration steps needed and additional domain name like mcloud-myserver.domain.com. Ask about assistance in telegram channel
Fixes
- [#165] Default slave labels were renamed
- [#163][#161] root_url was removed from variables.env for Jenkins docker-composer
- [#155] Fixed the ability to set custom AUTH_TOKEN_SECRET and CRYPTO_SALT
- [#154] Zafira test launcher was not active
- [#153] An error during clear.sh was fixed
- [#150] The POSTGRES_PASSWORD variable was not set
- [#146] Found a way to use valid global var for sub-modules
- [#145] Most of the browser containers could not be stopped
- [#144] carina + zafira-client: [L000N] indexes were lost
- [#140] Restored the default integration in infra
- [#138] Verified that the network exists during start.sh
- [#137] QPS_HOST was renamed to INFRA_HOST
- [#127] Logs did not work, there was no redirection to Zafira
[COMPONENTS UPDATES]
- zafira 4.1.90 -> Zebrunner Insights 1.3
- jenkins-master 4.10.2 -> 5.0.3
- sonarqube 5.6.7 -> 7.1
[MIGRATION STEPS]
Due to the full folders redesign it is recommended to clone to absolutely fresh folder and move historical data from previous version based on steps below:
-
Deploy a new infra tool set from scratch according to the steps and start. Do a sanity verification that all enabled components are up&running
-
Migrate historical jenkins data:
- docker rm -f jenkins-master
- Link historical jenkins folder using volume mount in legacy way, for volume and comment modern share
volumes: - "/home/ubuntu/old-qps-infra/jenkins:/var/jenkins_home" # - "data-volume:/var/jenkins_home:rw"
- Start jenkins-master using ./start.sh
- Analyze logs by docker logs -f jenkins-master
- Try to login using old users/credentials. All historical content should be available
- Goto Manage Jenkins (some warnings about broken plugins could be displayed)
- Upgrade all compatible plugins manually and restart jenkins (repeat step until all plugins are ok)
- Goto Management_Jobs and remove all existing jobs, restart jenkins-master
- Goto Manage jenkins -> Configure System and update "QPS_PIPELINE_GIT_BRANCH=5.0.4", rename QPS_HOST to INFRA_HOST
- [Optional] re-register organization and repositories...
- Run any job/cron to make sure everything is ok
-
Migrate historical Zafira data:
- Stop everything using ./stop.sh command
- Link historical postgres folder using volume mount in legacy way, for volume and comment modern share
volumes: - "/home/ubuntu/old-qps-infra/pgdata:/var/lib/postgresql/data" # - "pgdata-volume:/var/lib/postgresql/data" - "./database/:/docker-entrypoint-initdb.d"
- Link historical ElasticSearch data folder using volume mount in legacy way, for volume and comment modern share
volumes: - "/home/ubuntu/old-qps-infra/pgdata:/usr/share/elasticsearch/data" # - "esdata-volume:/usr/share/elasticsearch/data" - "./configuration/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro"
- Restart everything
- Run any job/cron to make sure everything is ok
4.10 (2020-04-14)
Enhancements
- [#130] bump up to zafira:4.1.90 and zafira-ui:1.38
- [#129] Switched to lightweight jenkins-slave:latest-alpine in the default setup
- [#103] ghprb-token setup steps were added to the documentation
- [#102] Initial documentation on qps-infra was created
- [#68] all infra images are UTF-8 compatible
- [#2] Together with carina-core 6.4.33 we can sniff all requests/response to MCloud devices
Fixes
- [#128] Description for zafiraServiceURL and zafiraAccessToken fields was added to the Create organization page
- [#117] Volume sharing for FTP container was added
- [#126][#125] Video files could not be loaded completely
[COMPONENTS UPDATES]
zafira 4.1.73->4.1.90
zafira-ui 1.25->1.38
jenkins-master 4.9.1->4.10.2
qps-pipeline 4.9->4.10.2
[MIGRATION STEPS]
- Make sure to rename/remove set of Management_Jobs according to the steps
- Stop everything
./stop.sh
- Generate a backup of the jenkins and pgdata content
cd qps-infra
tar -czvf jenkins.tar.gz ./jenkins
sudo tar -czvf pgdata.tar.gz ./pgdata
- Remove existing qaprosoft/pure-ftpd:latest image
docker rmi <id>
- Add valid volume share for pure-ftpd container, example:
volumes:
- $PWD/selenoid/video:/usr/share/ftp:rw
-
Bump up images in docker-compose.yml
- jenkins-master to 4.10.2
- zafira to 4.1.90
- zafira-ui to 1.38
-
Start everything
./start.sh
- Go to Jenkins -> Manage Jenkins
- Bump up QPS_PIPELINE_GIT_BRANCH to 4.10.2
4.9 (2020-03-04)
Enhancements
- Published Getting Started steps
- [#114] Automatically cleaned the container during the stop.sh except use case when "-k" or "--keep-containers" were provided
// stop infra and keep containers
./stop.sh -k
- [#112][#94] Set JENKINS_JOB_URL_VISIBILITY to True
- [#92] Added docker into the jenkins-slave image
- [#86] Refactored the setup.sh script to minimize usage of public address
Fixes
- [#109] The number of slaves for running a full regression job was reduced
- [#108] No live video sessions (VNC) for Tests via Zafira were seen if 4445 port was not opened
- [#107] Opera was added to the supported browsers
- [#100] Removed Jenkins warning "You have data stored in an older format and/or unreadable data."
[COMPONENTS UPDATES]
zafira 4.1.73->4.1.89
zafira-ui 1.25->1.35
jenkins-master 4.8->4.9.1
qps-pipeline 4.8->4.9.1
[MIGRATION STEPS]
- Make sure to rename/remove management_Jobs/RegisterZafiraCredentials according to the steps
- Stop everything
./stop.sh
- Generate a backup of the jenkins and pgdata content
cd qps-infra
tar -czvf jenkins.tar.gz ./jenkins
sudo tar -czvf pgdata.tar.gz ./pgdata
- Bump up images in docker-compose.yml
- jenkins-master to 4.9.1
- zafira to 4.1.89
- zafira-ui to 1.35
- Start everything
./start.sh
- Follow Zafira Migration Steps to regenerate db-views and apply widget updates
- Go to Jenkins -> Manage Jenkins
- Bump up QPS_PIPELINE_GIT_BRANCH to 4.9.1