Automated testing for rendered visualization differences #958
Comments
|
Status update: Ideally, we want these visual regression tests to run in CI/CD. One challenge is that the the screenshots taken in in the CI environment are different from those taken in my local environment (rendering is slightly different so screenshots don't match between CI and local). This means that either you don't be able to run visual regression testing locally, or we'll have to have two copies of the screenshots: one from local env and one from the test runner env. I think we'll probably opt for the former. Here is the Playwright guide for running tests in CI: https://playwright.dev/docs/ci |
Background:
Sometimes, Gosling code changes will break a visualization, but will not cause any tests to fail. For example, #938 #957 were patches that had to be made after a release because automatic tests did not alert us of anything wrong.
Potential solution:
@sehilyi: To avoid future instances of this, we could set up tests that compare the actual images of visualizations that the previous and updated displayed. It looks like puppeteer, which is a library that helps to capture screenshots, has the ability to add <script> using a local file (https://pptr.dev/api/puppeteer.page.addscripttag), so I imagine that we can bundle the gosling.js and then use in puppeteer to create Gosling image files based on a updated Gosling.js.
An example of puppeteer with Gosling: https://github.com/hms-dbmi/chromoscope/blob/master/src/script/gosling-screenshot.js
The text was updated successfully, but these errors were encountered: