Skip to content

5. Messages and Result Status

Jump to bottom
Elsa Perelli edited this page Oct 23, 2024 · 6 revisions

Goal

Inferno's testing DSL enables tests to provide feedback to users beyond simple pass and fail statuses.

Statuses:

  • Pass
  • Fail
  • Omit (rolls up to a 'pass')
  • Skip (rolls up to a 'fail')
  • Error (rolls up to a 'fail' -- indicative of a problem with the test that should be fixed)

Messages:

  • Info
  • Warning

When writing tests, oftentimes there are cases where a system displays behavior that does not violate the specification, but does not behave in a way that enables the test to verify that the system supports the necessary behavior. For example, if a test is designed to verify support for a search, it needs to see a search that successfully returns valid resources. If the query performed does not return any resources, it doesn't necessarily mean that the system is incorrect, and therefore a 'fail' status isn't quite an appropriate outcome. In these cases, testers can use the skip status to indicate that the system isn't necessarily non-conformant, but hasn't provided enough information to verify conformance to the test under evaluation.

In this exercise, update the search query to skip the test if the system did not return any values.

Adding an info message for how many entries are in each search bundle would be helpful as well.

Resources

Solution

Add the following lines after asserting the resource type within the tests:

  # skip_if is a shortcut to wrapping `skip` statement in an `if` block
  # it is a good idea to use the safe navigation operator `&.` to avoid runtime errors on nil
  skip_if resource.entry&.empty?, 'No entries in bundle response.'

  info "Bundle contains #{resource.entry&.count} resources."

link to the diff between step-5 and step-5-solution branches: https://github.com/inferno-training/inferno-tutorial/compare/step-5...step-5-solution

Note: When running this branch with the Inferno Reference Server and Patient 85, Test 2.02 will fail. This is an expected failure since Patient 85 in the reference server no longer conforms to USCore 3.1.1. Additionally, tests 3.01, 3.05, and 3.09 should be skipped.

Clone this wiki locally