Handling null values and omissions in API requests

[igboyes]

The Virtool API still inconsistently handles null values and omissions.

What does null mean in a request body?

The client wants to set the value to null.

If a user send null for a field that is not nullable, a validation error (400) will be returned. The response will include a description of the problem.

A null value in a request does not mean the value is being omitted from the request.

What does omitting a field mean in a request body?

When a client omits a field from the request body, it means they don't want to modify it.

Make sure than an omitted value doesn't get converted to None in API code. This is a potential footgun because of how Pydantic request validation works.

Fields

Make sure omission results in no change to the omitted field.

What does an empty string mean in a request body?

Use empty strings when a user-defined string value has not been filled.

For example, a description field that a user types out and adds to a resource should be "" when not populated.

Use null if a string value is one of several options the user can select and the field is optional.

For example, the group field on a sample can be one of the user groups defined for the instance. Users cannot enter their own arbitrary string value. The group field is optional, so it can be set to null when no group is selected (not "").

Pattern

Use the request model (eg. UpdateUserSchema) to pass data from the request handler to the data layer. Your data layer method signature should look like:

async def update(self, user_id: str, data: UpdateUserSchema) -> User:
    ...

This avoids:

• Passing data as a dict, which loses typing information.
• Passing individual arguments. There is no way to provide typing and omission with this strategy.
• Calling dict.get() and accidentally setting a value to None.

In the data layer method, use exclude_unset to get a dict that doesn't include fields omitted in the request:

# Call the `dict()` method on a Pydantic data object.
data_dict = data.dict(exclude_unset=True)

You can than use the in membership test to decide whether to update the field in database:

update = {}

if "name" in data_dict:
    update["name"] = name

await db.groups.update_one({"_id": group_id}, {"$set": update})

There is probably opportunity to reduce repetion by implementing functions to unpack model objects and create MongoDB update dictionaries.