[Bug]: Cannot use Json type with BigQueryIO and write method FILE_LOAD, avro

[takaaki7]

What happened?

BigQueryIO document says FILE_LOAD not support JSON type.
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1841

And when specifying schema to use avro, validation logic causes error.

beam/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java

Line 3078 in 948af30

checkArgument(

But currently, It seems JSON type works well with load api.

I think now we can remove this validation and modify document.

Issue Priority

Priority: 3

Issue Component

Component: extensions-java-gcp

[Abacn]

I tried to remove the validation and use run BigQueryIOJsonIT. There is a caveat that for a TableRow, if I set it to be

.set("country_code", some_json_of_string_type) // some_json_of_string_type = "{\"a\":\"b\"}"

The written json is a single escaped string ( "{"a":"b"}" ) using FILE_LOAD method. But the string is unescaped using STORAGE_WRITE_API or STREAMING_INSERT method ( {"a":"b"} ).

[Abacn]

Checked that this feature was added in late 2022 (internal tracker cl/457809718). So it is safe to lift this checkArgument. Opened #29923