[Feature Request] Handle edge case of recursive exceptions in failure converter

[yunmanger1]

Is your feature request related to a problem? Please describe.

It is kind of an easy mistake to do in python:

try:
    raise ValueError("test")
except Exception as e:
    raise e from e

The re-raised exception will have e.__cause__ referencing itself resulting in failure to serialize error and python sdk won't report anything resulting in retries and startToClose timeouts.

Describe the solution you'd like

Handle a case when e.__cause__ == e in DefaultFailureConverter. Created a dummy pull request #696

Additional context

https://temporalio.slack.com/archives/CTT84RS0P/p1719632201314119

[cretz]

python sdk won't report anything

I think this is the primary bug. There are situations where recursion and stack overflow can occur in serialization, and the linked PR naively only helps the case of the cause being the same as itself (but if it was two causes deep that recursed, the issue would still appear). We just need to make it clear when it happens (or any other failure serialization happens).

If we are indeed not reporting anything, we need to fix that. #685 is related when failing to build a failure.

[yunmanger1]

You are right that it doesn't solve all cycles. Not sure if your case happens in practice. What needs to happen to have an exception cycled after 2 or 3 steps?

Maybe the solution could be just have a max depth of 10 let's say. During serialization you truncate too deep stacks anyway.

[cretz]

I think we just need to log the failure of serializing this (assume there's a stack depth/overflow exception but is not visible due to #685)