Discussion: Can UDF expression cause inconsistency when UDF server fails?

[kwannoel]

When UDF server fails, we insert null values.
Consider that this is updatable stream, in that case when running updates / deletes, the null value can lead to panics due to inconsistent state:

insert udf(1) -> null (fails initially)
delete udf(1) -> 3 (succeeds later) this can cause panic due to inconsistent state, since no such value (3) exists.

Not sure how to deal with this case yet.

We also need a reproduction of it first. I think can just create a UDF server, kill it to force null values being emitted, restart it, update some values.

[kwannoel]

Seems we have to drop and recreate mv in that scenario.

[BugenZhao]

This may apply to all expressions that are impure, whether deliberate or accidental (like NULL padding on temporary failures). There's an unimplemented RFC for this: risingwavelabs/rfcs#24

[kwannoel]

Can we add a config to always retry on UDF server error? I.e. block the entire stream graph, until UDF server is back online.

[kwannoel]

Added the config #15303