This is not an officially supported Google product.

OAS Adherence Test Suite

OAS Adherence Test Suite will allow a user to test whether an API is correctly implemented as per a defined OpenAPI Specification. The intention is to call an implemented API and validate basic assumptions and requirements such as

• Resources
• Data Types
• Enumerations
• Required / Optional fields
• Error Responses

We will add more as we build and develop this project.

Note: This is not meant to be a functional test suite. It cannot make semantically correct calls and cannot also chain calls. It's only purpose is to give the API Spec creator or consumer the confidence that the API is implemented correctly as per the spec and is ready to consume without fear of mismatched expectations of request / response.

The aim is that this project will reduce manual reviews of OpenAPI Spec implementations and faster development as well as time to integration.

This template uses the Apache license, as is Google's default. See the documentation for instructions on using alternate license.

Basic Requirements

• OAS Document should be in JSON format.
• Version of the OAS Document should be v3.0 (Latest version of OpenApi Spec).

Visit Swagger Editor to convert your OpenApi Spec to the required format.

Commands

Note: App can serve the users through interative cli as well. Users will be asked/prompted for necessary data if not provided through command options. Also, users can just type ats <command> to start a complete interactive cli flow.

Generate testsuite

Generates testsuite containing testcases for all the api endpoints present in the OAS 3.0 document.

ats generate [--oaspath <oaspath>] [--testsuitepath <testsuitepath>] [--overridespath <overridespath>] [--verbose]

Options

• --oaspath <oaspath>: Path of OAS 3.0 document.
• --testsuitepath <testsuitepath>: Path where the generated testsuite is saved.
• --overridespath <overridespath>: Path of Overrides file.
• --verbose: Provides more information about events that occur through logs.

Examples

• --oaspath="/foldername/petstore.json"
• --testsuitepath="/foldername/petstore_1.0.5_testsuite.json"
• --overridespath="/ats/overrides.json"
• --verbose

Validate API Endpoints

Validates the API Endpoints against the OpenAPI Specification.

ats validate [--testsuitepath <testsuitepath>] [--oaspath <oaspath>] [--overridespath <overridespath>]
[--baseURL <baseURL>] [--apiendpoints <apiendpoints>] [--apikeys <apikeys>] [--basicauth <basicauth>] 
[--saveconfigto <configpath>] [--uploadconfigfrom <configpath>] [--timeout <timeout>] [--verbose]

Options

• --testsuitepath <testsuitepath>: Path of testsuite. (App runs testcases present in the testsuite)
• --oaspath <oaspath>: Path of OAS 3.0 document. (App runs testcases which are generated against the provided OAS 3.0 document)
• --overridespath <overridespath>: Path of Overrides file.
• --baseURL <baseURL>: BaseURL
• --apiendpoints <apiendpoints>: API Endpoints that needs to be validated.
• --apikeys <apikeys>: API Keys used for Authentication/Authorisation.
• --basicauth <basicauth>: Basic Authentication Credentials (username, password).
• --saveconfigto <configpath>: Updates/Creates a config file with config object in configpath.
• --uploadconfigfrom <configpath>: Reads the config object from the config file present in the configpath.
• --timeout <timeout>: Specifies the number of milliseconds before the request times out. (Default: 5000 ms)
• --verbose: Provides more descriptive test results and information about events that occur through logs.

Examples

• --testsuitepath="/foldername/petstore_1.0.5_testsuite.json"
• --oaspath="/foldername/petstore.json"
• --overridespath="/ats/overrides.json"
• --baseURL="http://www.ats.com"
• --apiendpoints='[{"path": "/pet", "httpMethod": "post"} , {"path": "/store", "httpMethod": "post"}]'
• --apikeys='[{"name": "X-API-KEY", "value":"foo"}, {"name": "X-API-KEY_DUP", "value": "bar"}]'
• --basicauth='{"username": "sundar", "password": "sundar@123"}'
• --saveconfigto="/foldername/petstore_config.json"
• --uploadconfigfrom="/foldername/petstore_config.json"
• --timeout=9000
• --verbose

Additional Information

Overrides

Testcases generated by the app contains random values complying with the schema for request body and request headers. In order to have a reserved/fixed value for a field, the user can take help of the overrides option provided by the app.

File Structure of Overrides

{
  "{{path}}": {
    "{{httpMethod}}": {
      "requestBody": {
        "{{fieldName}}": "{{value}}"
      },
      "responseHeaders": {
        "{{headerName}}": "{{value}}"
      }
    }
  }
}

Example

{
  "/v1/petowner": {
    "post": {
      "requestBody": {
        "name": {
          "firstname": "foo",
          "lastname": "bar"
        },
        "requestId": "abcd123",
        "requestTimestamp": {
          "epochMillis": 1594940819161
        }
      }
    },
    "requestHeaders": {
      "X-CSRF-Token": "IuwgtXZjlQ"
    }
  }
}

Config

Active users of the app can store the command-options's values in the config file. Therefore, the users don't need to provide all the options everytime they want to run a command, they can simply upload the config file.

Command Options

• --saveconfigto <configpath>
• --uploadconfigfrom <configpath>

Options that can be stored in the config file are:

• testsuitepath
• apiendpoints
• baseURL
• apikeys
• basicauth
• timeout

Developer Guide

Run Unit Tests

  npm test

Note: On running the command, along with running the tests it will generate the code-coverage report as well.

Generate Documentation for the Source Code

  npm run generate-docs

Highly recommended for new contributors of the app to generate the doc and get an overview of all modules present.

License

This is not an officially supported Google product.

Copyright 2020 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.