Skip to content

Latest commit

 

History

History
149 lines (90 loc) · 6.55 KB

JavascriptTesting.md

File metadata and controls

149 lines (90 loc) · 6.55 KB
Raw

JavaScript Testing

This documentation doesn't discuss much about testing. You can check Protractor and Jasmine documentation for more details. This documentation adds both karma/jasmine unit test and protractor end-to-end testing support for AngularJS-JavaScript report.

These tools are configured for specific conventions described below.

It is unwise and rarely possible to run the the unit tests, and the e2e tests at the same time. We recommend that you shut down one before starting another.

Writing Test Cases

To create tests, add it() (or test()) blocks with the name of the test and its code. You may optionally wrap them in describe() blocks for logical grouping but this is not required.

Jasmine provides a built-in expect() global function for making assertions. A basic test could look like this:

describe("true", function () {
    let a;
    it("Should be true", function () {
        a = true;
        expect(a).toBe(true);
    });
});

All expect() matchers supported by Jasmine/Karma for unit testing are extensively documented here.
End-to-end testing also uses jasmine framework, so these matchers can also be used in end-to-end testing.

Unit Testing

JavaScript unit-tests are located in the src/org/sosy_lab/cpachecker/core/counterexample/test folder. Their filenames must end in .js.

The above directory is a root for testing JavaScript for AngularJS report application.

Look for the example src/org/sosy_lab/cpachecker/core/counterexample/test/test.js. Add more .js files as you wish in the above folder; we have configured karma to find them.

Installation

1. karma.conf.js:

This is the configuration file for your Karma/Jasmine and it houses information such as the file format that Jasmine should look for when running test cases.

2. 'test' directory:

There is a directory titled test in the testing directory.

It houses all test case files for Unit testing.

Configuration

At first the javascript loads all the scripts in the the report.html to the testreport.html and then we start testing our functions using Jasmine/Karma.

We will use all available browsers out of Google Chrome, Chromium and Firefox that are installed locally in headless mode for Unit testing application. Make sure that at least one of these is installed on your machine.

For more information regarding configuration you can browse karma.conf.js.

Running tests

Run the below commands in your terminal for Unit testing:

  1. npm install At first you have to install all dependencies and third party libraries to run the test cases. This command is not needed every time you run the tests, it is required for the first time.

  2. npm run build Next, you will need to bundle all necessary .js and .css of our source code and all third party libraries into single files. The tests assume that these bundles files are available and will not work with only the raw files. This command is not needed for every test run either, but should be used every time any changes to the JS-related source files are made.

  3. npm test This command first compiles the application, then simultaneously re-compiles and runs the karma test-runner. Test-runner output appears in the terminal window. We can update our app and our tests in real-time, keeping a weather eye on the console for broken tests during development by setting up karma.conf.js file ( Set variable autoWatch : true and singleRun : false ).

Unit tests Report

The test reports for Unit testing is generated by karma-html-reporter plugin. You can get test reports in the folder src/org/sosy_lab/cpachecker/core/counterexample/unit_testing_report after running tests.

End-to-End testing

Setup

First we generate our AngularJS report application for an example program and then we use this report for end-to-end testing. We are using Chromium browser and Selenium web driver for end-to-end testing application.

The tests are written for Protractor and are located in the src/org/sosy_lab/cpachecker/core/counterexample/e2e-tests folder. Their filenames must end in _spec.js. An example test is in src/org/sosy_lab/cpachecker/core/counterexample/e2e-tests/test_spec.js. Add more _spec.js files as you wish in the above folder; we have configured Protractor to find them.

The configuration for Protractor is present in e2e-conf.js.

Running tests

Run the below commands in your terminal for End-to-end testing:

  1. Check if you have Java >= 8, Node >= 8, NPM, and Chrome or Chromium Browser installed. If not install it for running tests. On Ubuntu 18.04, these can be installed from the package repository. Alternatively, use our docker image: docker run -it -v $(pwd):$(pwd) -w $(pwd) registry.gitlab.com/sosy-lab/software/cpachecker/test:java-node /bin/bash

  2. Generate Counterexample report for a specific program (test/programs/simple/SSAMap-bug.c) in the root of this application using scripts/cpa.sh -predicateAnalysis test/programs/simple/SSAMap-bug.c (for Unix based OS) and scripts\cpa.bat -predicateAnalysis-linear -setprop analysis.checkCounterexamples=false -setprop solver.solver=smtinterpol test\programs\simple\SSAMap-bug.c (for Windows based OS).

  3. Navigate to src/org/sosy_lab/cpachecker/core/counterexample and run npm install and ./node_modules/protractor/node_modules/webdriver-manager/bin/webdriver-manager update to install all dependencies and third party libraries. This is not needed only once.

  4. npm run e2e-test
    This command first compiles the application, then simultaneously re-compiles and runs the Protractor test-runner. Test-runner output appears in the terminal window.

End-to-end Test Report

The test report for End-to-end testing is generated by protractor-jasmine2-screenshot-reporter plugin. It also captures the screenshots after each test case. You can get test report and screenshots in the folder src/org/sosy_lab/cpachecker/core/counterexample/e2e-tests_report/screenshots/ after running tests.

Continous Integration with Gitlab

All End-to-end tests and unit tests are configured to run in GitLab CI. The configuration file is .gitlab-ci.yml in the root folder of this repository.