Injecting Behaviors into Mimic

Behaviors are a way to provide deterministic error injection into Mimic functionality. For instance, mimic will always successfully create a server, immediately. But perhaps you want to see how your service or client handles a server build timing out. Or a 400 response.

Contents:

Goals - what the behaviors API was designed to do
Concepts - terminology and ideas used
Behavior storage - object models used to store behaviors - a little bit of code
Implementation guide - how to implement injected behaviors and CRUD endpoints to manipulate them

Goals

Provide an out-of-band method to specify behaviors, so the actual payload to the system under test does not need to change based on whether it is being run against Mimic or a production system. In other words, your test harness should be able to inject behaviors, and your application does not need to know.
Support different behaviors for different functionality at the same time. For instance, we want to be able to cause server creation 500 failures at the same time as identity authentication 403 failures.
Support different behaviors for the same functionality at the same time. For instance, we want to be able to cause server creation 500 failures for some servers, and also server creation 400 failures for some other servers, and also default 202 success for other servers, depending on certain criteria.
Provide a consistent REST interface for injecting behaviors, so that the many different types of behavior which may have to be injected for many different types of events will follow a similar pattern, and developers can pick up new ones quickly.

Concepts

These explain some terminology and how behaviors are modeled in mimic. Some of these also correspond to objects (in the Python sense) in Mimic, and some only partially correspond to objects in Mimic.

Note that this does not actually cover all the behavior-related objects (in the Python sense) in Mimic. Please see the behavior storage section for that.

Event

An event is the Mimic functionality that we want to declare behaviors for. Examples are: server creation, server deletion, authentication against identity, etc. (Only server creation and authentication against identity are implemented).

This corresponds what to mimic.model.behaviors.EventDescription. An event would be a global instance of EventDescription - it'd be declared as: server_creation = EventDescription().

An instance of EventDescription has a default behavior, which is the behavior that gets used when there are no injected behaviors specified. This would be successfully creating a server, for instance, in the case of server creation.

It also contains the bijection (a mapping and its reverse) between criteria and behaviors.

Criterion

A pair of (attribute, predicate) to match against to determine whether or not to apply a behavior to an event. By predicate we just mean a callable that returns True or False, given the value for the attribute. For example:

The server name in the JSON passed to the create server endpoint - maybe we only want to apply the behavior if the server name matches "fail_test.*". The criterion would be (server_name, predicate that regexp-matches against "fail_test.*").
The metadata in the JSON passed to the create server endpoint. Maybe we only want to apply the behavior if certain metadata is passed in. So the criterion would be (metadata, predicate that matches the metadata dictionary against a predefined dictionary).

mimic.model.behaviors.Criterion

Criterion takes a name and a predicate. This API is currently in flux - see this issue for more details.

Criteria

A set of criterion to match against to determine whether or not to apply a behavior to an event. All of them have to apply in order for a behavior to apply. Mimic actually always applies criteria (as opposed to a single criterion)

In Mimic, this corresponds to a list of criterion, and no other separate object.

Behavior

A way in which Mimic behaves. This is just a function that does something. Injected behaviors are functions that do something else other than the normal thing.

In Mimic, the normal thing looks like:

@event.declare_default_behavior
def do_my_normal_thing(*args, **kwargs):
    ...

(see the implementation guide for default behaviors). The injected behavior looks like:

@event.declare_behavior_creator("other_behavior_name")
def create_other_behavior_callable(parameters):
    ...
    def do_other_behavior(*args, **kwargs):
        # do something with both the parameters and the args and kwargs
    return do_other_behavior

Note that the *args and **kwargs in the default behavior and in the callable returned by the injected behavior code, while arbitrary, should match.

(see the implementation guide for injected behaviors for more information)

Behavior storage

This section describes the behavior storage mechanism in Mimic.

`BehaviorRegistryCollection`

This is the is the top-level store, and contains behaviors for multiple events. It is up to the implementer of the control plane where an instance of this should be stored.

When retrieving a BehaviorRegistry from a BehaviorRegistryCollection, if one does not exist for a particular event, one is created and then returned.

`BehaviorRegistry`

This contains a instance of a mimic.model.behaviors.EventDescription, which if you'll remember from the Event section contains a bijection of criteria to behaviors.

That bijection is ordered, with criteria looking like:

[Criterion(name="criterion1", predicate=lambda ...),
 Criterion(name="criterion2", predicate=lambda ...),
 Criterion(name="criterion3", predicate=lambda ...)]

When BehaviorRegistry.behavior_for_attributes is called, it takes a dictionary whose keys are some subset of ("criterion1", "criterion2", "criterion3"), and whose values should be evaluatable by the predicates.

It iterates through that mapping/bijection, and the first behavior whose criteria matches the attribute dictionary will be returned.

Uniqueness is not enforced. There could be two behaviors with the same set of criteria, or even the same behavior with the same set of criteria twice. But the first one is always the one that gets returned.

Implementation Guide

For this guide, we will be using Nova server creation as an example. The exact code is in mimic.model.nova_objects, and the code provided here will be more pseudocode and somewhat handwavy.

Let's first describe the spec: how we want behavior injection to work:

The user to make a POST request to a behavior injection endpoint with the following JSON:
```
{
    "criteria": [
        {"server_name": "my_failure_name.*"},
        {"imageRef": "abcd.*"}
    ],
    "name": "fail",
    "parameters": {
        "code": 404,
        "message": "Stuff is broken, what"
    }
}
```
This means that they want the behavior named "fail" to be applied to any server created with a server name that matches the regex expression "my_failure_name.*" and also an image ID that matches "abcd.*".

The "fail" behavior apparently takes the parameter attributes "code" and "message".

Once they post, a 201 response will be returned containing the ID of the behavior that was just registered. It looks like:
```
{
    "id": "this-is-a-uuid-here"
}
```
Now, when the user creates a server, with the name "my_failure_name" and the image ID "abcdef", the creation will fail with a 404 status code and response body: "Stuff is broken, what".
The user issues a DELETE request to the same endpoint, plus the behavior ID, and to remove said behavior.
Now, when the user creates a server, even with the name "my_failure_name" and the image ID "abcdef", the creation will succeed instead of returning a 404.

This guide will accomplish the above in multiple steps:

The default behavior for a single event
Injected behaviors for the same event.
The REST endpoints and behavior storage objects
Adding additional events

We suggest implementing the default behavior (1) in one PR, some injected behavior(s) and the REST endpoints in another PR (2-3), and additional events (4) in other PRs, whether you are adding a control plane to an existing plugin or writing a new plugin.

Assuming that you are modifying an existing plugin:

The default behavior wouldn't change any existing behavior, so no additional tests are needed.
When adding new behaviors, tests for those new behaviors can be added at the same time. The REST endpoint and tests are already templated and provided for you, and do not require very much code. And the templated tests require that there be at least 1 behavior in addition to the default behavior.

If you are providing a new plugin at the same time as the control plane, we'd suggest either:

Implementing the the plugin first, or at least the part you want to provide injection behavior for first, and then adding the control plane.
Implementing the default behavior from teh start, and only for one event you want to provide a control plane for. Provide tests for the default behavior for that single event. Implement the rest endpoints and additional behaviors in another PR, and then other events in later PRs.

Default Behavior

First, we define what happens normally, without any behavior injection.

Declare an event for which behaviors apply:
```
server_creation = EventDescription()
```
Define a default behavior for the event. This is just a decorated callable (the decorator makes use of the previously declared server_creation) which take arbitrary parameters - the parameters it requires is defined by the code which uses it (see the next step).
```
@server_creation.declare_default_behavior
def default_create_behavior(*args, **kwargs):
    # create server code
    # set response code to 202
    # return server JSON
```
The pseudocode may not be the actual behavior - maybe it just creates the server and returns a server, and the handler sets the response code and returns the server JSON isntead. But this is the general idea.
Create a BehaviorRegistryCollection somewhere in the plugin's code.

We recommend placing this in your regional session store (bottom level of the session store diagram here), so that there is one set of behaviors per plugin per region. You can also place it on your global plugin session store (second-to-last level of bottom level of the session store diagram here), if you want the behaviors to apply to all regions. Or if your plugin does not support regions.

The identity behavior collection, for instance, is stored on the global resource object the behavior being injected affects everything).
In the code that normally handles the event (such as creating a server), call the behavior function.
```
def handle_server_creation(..., behavior_registry_collection, ...):
    server_creation_behavior_registry = (
        behavior_registry_collection.registry_by_event(server_creation))
    behavior_to_apply = (
        server_creation_behavior_registry.behavior_for_attributes({}))
    return behavior_to_apply(*args, **kwargs)
```
This function accepts as a parameter the BehaviorRegistryCollection created in the previous step. It knows that the event it handles is server_creation. From that, it obtains a behavior to apply.

In this case, we pass behavior_for_attributes an empty attribute dictionary, because we have not defined any criteria yet or any other behaviors - this will get is the default behavior for now. This will change in the next section.

Note that the above implementation is just a suggestion. For example, if handle_server_creation is an instance method, as it is in Nova, behavior_registry_collection might be an instance attribute instead. Alternately, the function could accept or refer to just a BehaviorRegistry instead of a whole BehaviorRegistryCollection.

If you are modifying an existing plugin, this function probably previously looked like:
```
def handle_server_creation(...):
    # create server code
    # set response code to 202
    # return server JSON
```
Note that this looks exactly like the previous step's default_create_behavior. That's because the easiest way to create a default behavior is probably to move all the existing code to that function, unless you want to factor some code out for injected error behavior

Injected Behaviors

Note that this builds on the code from the previous section, particularly the event (server_creation) that was declared.

Declare one or more criterion for this event. Note that this API will probably change slightly, but right now, this looks something like:
```
@server_creation.declare_criterion("server_name")
def server_name_criterion(value):
    return Criterion(
        name="server_name",
        predicate=lambda name: re.compile(value).match(name))

@server_creation.declare_criterion("image_id")
def image_id_criterion(value):
    return Criterion(
        name="image_id",
        predicate=lambda _id: re.compile(value).match(_id))
```
This function, when given a value, will create a criterion that specifies that a server name should be equal that value.

The value and predicate can be anything your plugin specifies, so long as it's documented. For example, the value can be just a string, and the predicate checks for equality. The value can be a number, and the predicate can check the length of whatever gets passed in.
- what happens behind the scenes (e.g. not something you have to implement):
  
  Let's look at the the criteria part of the REST request body in the implementation spec above:
```
{
    "criteria": [
        {"server_name": "my_failure_name.*"},
        {"image_id": "abcd.*"}
    ],
    ...
}
```
  This will all be handled by the behavior models and REST endpoint template code (e.g. code in mimic.models.behaviors), but what will happen when that request is made, at least in regards to criteria, is that the criteria JSON will be turned into a list of 2 Criterion.
  
  mimic.models.behaviors will look up the declared functions by name, and call those functions to return Criterions. So it will find:
  - server_name_criterion because of "server_name", and call it with "my_failure_name.*", returning a
```
Criterion(
    name="server_name",
    predicate=(
        lambda name: re.compile("my_server_name.*").match(name)))
```
  - image_id_criterion because of "image_id", and call it with "abcd.*", returning a
```
Criterion(name="image_id",
          predicate=lambda _id: re.compile("abcd.*").match(_id))
```
  And mimic.models.behaviors will map the set of these two criteria to a behavior that will be created in the next step.
write stuff here about declaring named behaviors
write stuff here about testing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly