Unable to easily re-ecode fauna query results to JSON due to special objects being returned

[ryancausey]

In certain instances, the FQLv10 Python driver will de-serialize a result into specific models it defines, such as Document or Module. This makes it difficult to easily serialize the result to JSON for storage or transport to other locations outside of fauna.

What I was hoping to be able to accomplish would be something like the following for a simple illustrative example:

result = fauna_client.query(
    fql(
        """
        Key.create({
            role: ${role},
            data: {name: ${name}, token: ${token}},
        })
        """,
        role=role,
        name=name,
        token=token,
    )
).data
service_client = boto3.client("secretsmanager")
service_client.put_secret_value(
    SecretId=arn,
    ClientRequestToken=token,
    SecretString=json.dumps(result),
    VersionStages=["AWSPENDING"],
)

This fails because the Document type is not inherently JSON serializable. In digging through the repo, I find that there are encoders and decoders, but they don't seem to fully serialize or de-serialize the document into a plain dictionary like I would hope. I'm looking for a result like the following after calling json.dumps:

{
  "id": "380934557415768132",
  "coll": "key",
  "ts": "2023-11-09T16:12:33.665000+00:00",
  "secret": "asdf",
  "data": {
    "name": "test",
    "token": "5918c24f-0449-4577-bbae-9f675fbaae7d"
  }
}

I think it would be a nice feature for this library to assist with being able to easily de-serialize an object to JSON by providing any of the following:

• provide a flag on the query level or client level or both to return raw, JSON serializable dictionaries instead of any of the special object types.
• provide a subclass of json.JSONEncoder to use with json.dump, json.dumps, json.load, and json.loads that knows how to serialize and de-serialize the special objects that the Python driver may return.
• provide a way on each model type, such as Document, Module, etc to get a JSON serializable dictionary

[ryancausey]

I found a workaround for now where I can take advantage of the x-format header to force the responses to be untagged results.

https://docs.fauna.com/fauna/current/reference/http/reference/query/post#query-post-request

client = Client(secret=secret, additional_headers={"x-format": "simple"})

This feels a bit dirty though, and I assume I'm losing some functionality beyond the TS being automatically made into a datetime.datetime object. It would still be nice to have some method to easily serialize all fauna-python models to JSON.

[pnwpedro]

@ryancausey Thanks for the thoughtful feedback and apologies we haven't engaged sooner.

First, I agree 100% that we need to expose a de/serialization API in this library.

For context, the motivation for Document was round-tripping and typing such that users can know confidently that they're interacting with a Fauna document. That said, this sharp edge has me wondering if we did something somewhat unpythonic. Certainly I expected python to play nicely with the Mapping type, but the default json lib doesn't know what to do with it. :/

Regardless, for your specific issue, I think there's another way you can work around this in the absence of configurability and still take advantage of the rest of the decoder. In FQL, you can project the values you care about and return them. In projection, a simple object is returned on the wire instead of a document, so the decoder will put them into a dict for you.

Here are two examples of what that can look like. The first keeps data nested, the second projects to the top level.

"""
  Key.create({
    role: 'admin',
    data: {name: 'foo', token: 'token'}
  }) { id, ts, role, data }
"""

"""
  Key.create({
    role: 'admin',
    data: {name: 'foo', token: 'token'}
  }) { id, ts, role, name: .data.name, token: .data.token }
"""

Thanks again for the feedback, and we'll definitely be taking this into consideration as we build on this library.