[Bug] Sending a query that is not registered in the workflow results in an "Encoded failure" when encode_common_attributes=True

[antmendoza]

What are you really trying to do?

List workflow queries using the UI (Queries tab)

Describe the bug

The UI shows "Encoded failure" but this is an error thrown by the worker when the UI tries to get the list of queries the workflow exposes with queryType: "@@temporal-internal__list

I get the same error when I use the client to send a query that is not registered in the workflow, and the client configuration has the data_converter which failure_converter_class has encode_common_attributes=True

Minimal Reproduction

Full reproduction is here https://github.com/antmendoza/my-temporal-pocs/tree/main/python/_8156

clone https://github.com/temporalio/samples-python/tree/main/encryption

Add the following modifications:

• Create a new DefaultFailureConverter with encode_common_attributes=True

class FailureConverterWithDecodedAttributes(temporalio.DefaultFailureConverter):
    def __init__(self) -> None:
        super().__init__(encode_common_attributes=True)

• Add failure_converter_class=FailureConverterWithDecodedAttributes to the client.

    client = await Client.connect(
        "localhost:7233",
        # Use the default converter, but change the codec
        data_converter=dataclasses.replace(
            temporalio.converter.default(),
            payload_codec=EncryptionCodec(),
            failure_converter_class=FailureConverterWithDecodedAttributes,
        ),
    )

After running a workflow, try to send a query that does not exist for the workflow.

        await client.get_workflow_handle("encryption-workflow-id").query("non-existing-query")

The query will fail with "Encoded failure"

[cretz]

Hrmm, so this is a general Temporal Core SDK issue where query failures use the Failure structure but the server doesn't support that for query failures. I will confer with the team on this.

[cretz]

I believe we have confirmed that this is due to the UI using query failures to populate the query list but encrypting these failures is incompatible with that. We have filed an issue with the UI team to support failure decryption.

[cretz]

Note, there is a server side issue to fix query failure encoding: temporalio/temporal#6845. If this becomes high enough priority, we can add SDK-side workarounds to keep query failures in plain text temporarily.