fix memory leak caused by ThreadLocal in PipelineOptionFactory

[jrthe42]

This pr aims to fix a classloader leak caused by ThreadLocal in PipelineOptionFactory. The threadlocal for DESERIALIZATION_CONTEXT cannot be removed and will eventually cause meatspace oom in Flink Jobmanager, which has been reported by several users #29890, #25510.

We can remove the ThreadLocal and use DefaultDeserializationContext.copy instead.

[github-actions]

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

[Abacn]

Thanks for fixing this long standing issue. CC: @jinyangzhen @ashishdeole would you mind taking a look?

[Abacn]

In #16680 there were DESERIALIZATION_CONTEXT.get() call in 4 places, here it changed to .copy() in two places, is this suffice? Have you been able to test the memory leak is fixed with this PR?

[jrthe42]

@Abacn I've tested this with my local environment, and yes, no memory leak on Flink's ChildFirstClassloader using this fix. As for your first question, DESERIALIZATION_CONTEXT is currently only used in 2 places in PipelineOptionsFactory.

It also works fine with multithreaded contexts.

[jinyangzhen]

@Abacn - confirm this PR resolve this issue.
If this PR is merged, how long do we expect to see a new version in maven repo?

[Abacn]

PipelineOptionFactory is used in pipeline construction time and is not performance sensitive. .copy() should be fine and it avoids racing condition

[Abacn]

@Abacn - confirm this PR resolve this issue. If this PR is merged, how long do we expect to see a new version in maven repo?

It will ship with Beam 2.54.0 scheduled in late Feburary (over 6 week interval).