[Feature Request]: Update Python JDBC write transforms to support configuring the batch size

[chamikaramj]

What would you like to happen?

Currently batch size [1] is not one of the configurable parameters available to cross-language Python JDBC write transform [2][3]. Should be just a matter of wiring this similar to other properties there and adding a test.

Might also be covered by recent work related to auto-generating Python x-lang wrappers. cc: @ahmedabu98

[1]

beam/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java

Line 1725 in e845caa

public Write<T> withBatchSize(long batchSize) {

[2]

beam/sdks/python/apache_beam/io/jdbc.py

Line 174 in e845caa

def __init__(

[3]

beam/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcSchemaIOProvider.java

Line 57 in e845caa

public Schema configurationSchema() {

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[mravi]

@chamikaramj Can I take this issue ?

[chamikaramj]

Absolutely. Thank you for looking into this.

[mravi]

.take-issue

[mravi]

@chamikaramj ptal at the PR ! A qq on if I should extend the configuration for batch size to JDBC here that seem to be used by the YAML based pipeline ?

[ahmedabu98]

I can take a look @mravi. Yes adding it to the SchemaTransform would be a good bonus!

[mravi]

I can take a look @mravi. Yes adding it to the SchemaTransform would be a good bonus!

Ack ! Will send out an updated PR today

[chamikaramj]

Thanks Ravi and Ahmed.