add S2 Message Union type

[victorgarcia98]

Recently, I had the need to wrap the S2 messages in an envelope to simulate accelerated interactions between a CEM and a RM.

Nonetheless, for this to work, the message_type had to be a Literal. This PR refactors the code to use Literals instead of regular fields.

Example usage:

from pydantic import BaseModel, Field
from typing import Union, Annotated,Type, Literal
from datetime import datetime

from s2python.message import S2Message

class MetaData(BaseModel):
    dt : datetime

class S2Wrapper(BaseModel):
    message : S2Message = Field(discriminator='message_type')
    metadata : MetaData

[lfse-slafleur]

@victorgarcia98 Apologies for a late response. I do not work at TNO anymore and it took some time to get settled in my new situation again. Reviewing now!

[lfse-slafleur]

We already have a version of this file in generate_s2_message_type_to_class.py to generate a mapping from message_type literals into the appropriate class. You are using it to generate a list of all message types into an alternative S2Message type which only contains messages.

I think we can combine the 2. My proposal would be to move TYPE_TO_MESSAGE_CLASS to the new src/s2python/message.py file and generate this entire file with a single script. The result would be that we generate both the TYPE_TO_MESSAGE_CLASS constant as well as the S2Message type.

[Flix6x]

Is it correct that this suggestion is purely a refactoring request?

Since I'll be taking over this PR on behalf of @victorgarcia98, and I'm still a little uncertain as to increase the scope of this PR's code changes, would it be okay to tackle this refactoring in a separate PR?

[lfse-slafleur]

That is not an issue at all, lets do it in a seperate PR. It is an idea purely for refactoring/code deduplication. If you are busy, I can submit the PR for review to you.

[lfse-slafleur]

This name S2Message is already used here:

s2-python/src/s2python/validate_values_mixin.py

Line 122 in 5f978a3

class S2Message(Generic[C], ValidateValuesMixin[C], BaseModel):

My proposal is to change the original usage from S2Message to something like S2MessageComponent as it is more descriptive and keep S2Message as you introduce it here. Especially since the members of this Union type are all the messages and not parts of a message.

[Flix6x]

Good idea, will rename.

[lfse-slafleur]

Thanks! 😄 Will keep this open so you can optionally resolve it. (I usually keep the conversations open until the commit is made to keep an overview, but feel free to use it however you want of course)

[lfse-slafleur]

No issues with moving from a pydantic const to the Literal type. However, datamodel-codegen does not support to use it for pydantic v1 output. Prioritising updating to pydantic v2 so it is automatically generated. #10

[lfse-slafleur]

PR is ready that has taken care of generating Literals instead of constants by moving to pydantic v2: #40

[Flix6x]

HomeAssistant is not yet compatible with Pydantic 2. We are tracking home-assistant/core#99218 to see when it is, but until then I'm not sure of the implications with regards to compatibility if the s2-python package would already migrate to Pydantic 2. Is the use of a Literal problematic at this point, or do you only regard it as a less preferred option over moving to Pydantic first.

[lfse-slafleur]

HI @Flix6x ! gen_s2.py is the output of the datamodel-codegen code generator. As such, we can move to literals but in its current shape it would be modifying a generated code file manually or as a post-processing step.

Perhaps we should split the discussion. We could discuss 'Why and when to move to pydantic v2' here: #10

Then we can keep the discussion in this comment thread to how to close this PR. Moving to pydantic v2 solves this automatically as datamodel-codegen uses the Literal type in its output for pydantic-v2 output. So I was expecting that moving to pydantic v2 would solve this issue without any extra work. However, if we want to move to Literal without moving to pydantic v2, I would like to propose an alternative. Manually altering generated files is error prone. We could move the Literal lines from the generated code that are introduced by this PR to the non-generated classes, essentially overwriting the fields as we have done with ID's. The referenced line of this comment refers to InstructionStatusUpdate. An example on where to move it (see the last line):

File: https://github.com/flexiblepower/s2-python/blob/main/src/s2python/common/instruction_status_update.py

@catch_and_convert_exceptions
class InstructionStatusUpdate(
    GenInstructionStatusUpdate, S2Message["InstructionStatusUpdate"]
):
    class Config(GenInstructionStatusUpdate.Config):
        validate_assignment = True

    message_id: uuid.UUID = GenInstructionStatusUpdate.__fields__["message_id"].field_info  # type: ignore[assignment]
    instruction_id: uuid.UUID = GenInstructionStatusUpdate.__fields__[
        "instruction_id"
    ].field_info  # type: ignore[assignment]
    message_type: Literal["InstructionStatusUpdate"] = Field(default="InstructionStatusUpdate")  # <-- Line which is added

What do you think of this idea? In case you like it, perhaps its a good idea to first test with a small case to see if it has the intended effect.