[yaml] add mysql, oracle, postgres and sql server providers (#33124)

* [yaml] add JDBC docs Signed-off-by: Jeffrey Kinard <[email protected]> * Address comments Signed-off-by: Jeffrey Kinard <[email protected]> * remove baseDescription Signed-off-by: Jeffrey Kinard <[email protected]> * fix spotless Signed-off-by: Jeffrey Kinard <[email protected]> * fix failing tests Signed-off-by: Jeffrey Kinard <[email protected]> * remove hikari testImplementation for JDBCIO Signed-off-by: Jeffrey Kinard <[email protected]> --------- Signed-off-by: Jeffrey Kinard <[email protected]>
apache · Dec 29, 2024 · 7ca424d · 7ca424d
1 parent 0dc2c24
commit 7ca424d
Show file tree

Hide file tree

Showing 18 changed files with 854 additions and 161 deletions.
diff --git a/.../src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerReadSchemaTransformProvider.java b/.../src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerReadSchemaTransformProvider.java
@@ -76,40 +76,34 @@ public String description() {
         + "\n"
         + "Example configuration for performing a read using a SQL query: ::\n"
         + "\n"
-        + "    pipeline:\n"
-        + "      transforms:\n"
-        + "        - type: ReadFromSpanner\n"
-        + "          config:\n"
-        + "            instance_id: 'my-instance-id'\n"
-        + "            database_id: 'my-database'\n"
-        + "            query: 'SELECT * FROM table'\n"
+        + "    - type: ReadFromSpanner\n"
+        + "      config:\n"
+        + "        instance_id: 'my-instance-id'\n"
+        + "        database_id: 'my-database'\n"
+        + "        query: 'SELECT * FROM table'\n"
         + "\n"
         + "It is also possible to read a table by specifying a table name and a list of columns. For "
         + "example, the following configuration will perform a read on an entire table: ::\n"
         + "\n"
-        + "    pipeline:\n"
-        + "      transforms:\n"
-        + "        - type: ReadFromSpanner\n"
-        + "          config:\n"
-        + "            instance_id: 'my-instance-id'\n"
-        + "            database_id: 'my-database'\n"
-        + "            table: 'my-table'\n"
-        + "            columns: ['col1', 'col2']\n"
+        + "    - type: ReadFromSpanner\n"
+        + "      config:\n"
+        + "        instance_id: 'my-instance-id'\n"
+        + "        database_id: 'my-database'\n"
+        + "        table: 'my-table'\n"
+        + "        columns: ['col1', 'col2']\n"
         + "\n"
         + "Additionally, to read using a <a href=\"https://cloud.google.com/spanner/docs/secondary-indexes\">"
         + "Secondary Index</a>, specify the index name: ::"
         + "\n"
-        + "    pipeline:\n"
-        + "      transforms:\n"
-        + "        - type: ReadFromSpanner\n"
-        + "          config:\n"
-        + "            instance_id: 'my-instance-id'\n"
-        + "            database_id: 'my-database'\n"
-        + "            table: 'my-table'\n"
-        + "            index: 'my-index'\n"
-        + "            columns: ['col1', 'col2']\n"
+        + "    - type: ReadFromSpanner\n"
+        + "      config:\n"
+        + "        instance_id: 'my-instance-id'\n"
+        + "        database_id: 'my-database'\n"
+        + "        table: 'my-table'\n"
+        + "        index: 'my-index'\n"
+        + "        columns: ['col1', 'col2']\n"
         + "\n"
-        + "### Advanced Usage\n"
+        + "#### Advanced Usage\n"
         + "\n"
         + "Reads by default use the <a href=\"https://cloud.google.com/spanner/docs/reads#read_data_in_parallel\">"
         + "PartitionQuery API</a> which enforces some limitations on the type of queries that can be used so that "
@@ -118,12 +112,10 @@ public String description() {
         + "\n"
         + "For example: ::"
         + "\n"
-        + "    pipeline:\n"
-        + "      transforms:\n"
-        + "        - type: ReadFromSpanner\n"
-        + "          config:\n"
-        + "            batching: false\n"
-        + "            ...\n"
+        + "    - type: ReadFromSpanner\n"
+        + "      config:\n"
+        + "        batching: false\n"
+        + "        ...\n"
         + "\n"
         + "Note: See <a href=\""
         + "https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.html\">"

diff --git a/...src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerWriteSchemaTransformProvider.java b/...src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerWriteSchemaTransformProvider.java
@@ -84,14 +84,12 @@ public String description() {
         + "\n"
         + "Example configuration for performing a write to a single table: ::\n"
         + "\n"
-        + "    pipeline:\n"
-        + "      transforms:\n"
-        + "        - type: ReadFromSpanner\n"
-        + "          config:\n"
-        + "            project_id: 'my-project-id'\n"
-        + "            instance_id: 'my-instance-id'\n"
-        + "            database_id: 'my-database'\n"
-        + "            table: 'my-table'\n"
+        + "    - type: ReadFromSpanner\n"
+        + "      config:\n"
+        + "        project_id: 'my-project-id'\n"
+        + "        instance_id: 'my-instance-id'\n"
+        + "        database_id: 'my-database'\n"
+        + "        table: 'my-table'\n"
         + "\n"
         + "Note: See <a href=\""
         + "https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.html\">"