| Configuration | Description |
|---|---|
| topic.prefix | Unique name for connector. |
| database.hostname | Source Database HostName |
| database.port | Source Database Port number |
| database.user | Source Database Username(user needs to have replication permission, Refer https://debezium.io/documentation/reference/stable/connectors/mysql.html) GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON . TO 'user' IDENTIFIED BY 'password'; |
| database.password | Source Database Password |
| database.include.list | List of databases to be included in replication. |
| table.include.list | List of tables to be included in replication. MySQL(db_name.table_name), PostgreSQL(schema_name.table_name) |
| clickhouse.server.url | ClickHouse URL, For TLS(use https and set port to 8443) |
| clickhouse.server.user | ClickHouse username |
| clickhouse.server.password | ClickHouse password |
| clickhouse.server.port | ClickHouse port, For TLS(use the correct port 8443 or 443 |
| snapshot.mode | "initial" -> Data that already exists in source database will be replicated. "schema_only" -> Replicate data that is added/modified after the connector is started.<br/> MySQL: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-snapshot-mode \ PostgreSQL: https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-snapshot-mode MongoDB: initial, never. https://debezium.io/documentation/reference/stable/connectors/mongodb.html |
| connector.class | MySQL -> "io.debezium.connector.mysql.MySqlConnector" PostgreSQL -> Mongo -> |
| offset.storage.file.filename | Offset storage file(This stores the offsets of the source database) MySQL: mysql binlog file and position, gtid set. Make sure this file is durable and its not persisted in temp directories. |
| database.history.file.filename | Database History: Make sure this file is durable and its not persisted in temp directories. |
| schema.history.internal.file.filename | Schema History: Make sure this file is durable and its not persisted in temp directories. |
| disable.ddl | Optional, Default: false, if DDL execution needs to be disabled |
| enable.ddl.snapshot | Optional, Default: false, If set to true, the DDL that is passed as part of snapshot process will be executed. Default behavior is DROP/TRUNCATE as part of snapshot is disabled. |
| database.allowPublicKeyRetrieval | Optional, MySQL specific: true/false |
| auto.create.tables | When True, connector will create tables(transformed DDL from source) |
| persist.raw.bytes | Debezium.BYTES data(usually UUID) is persisted as raw bytes(CH String) if set to true. |
| database.connectionTimeZone | Example: "US/Samoa, Specify MySQL timezone for DATETIME conversions.https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-temporal-types |
| enable.snapshot.ddl | When true, pre-existing DDL statements from source(MySQL) will be executed. Warning: This might run DROP TABLE commands. |
| clickhouse.datetime.timezone | Override timezone for DateTime columns in ClickHouse server. |
| skip_replica_start | If set to true, replication is not started, the user is expected to start replication with the sink-connector-client program. |
| restart.event.loop | If set to true, replication will be re-started based on the restart.event.loop.timeout.period parameter(which is defined in seconds) |
| restart.event.loop.timeout.period.secs | If the last change record(CDC) received from source database exceeds this threshold period defined in seconds, then replication is restarted. |
| batch.max.records | Size of the batch that is persisted to ClickHouse.(Default 100000) |
| sink.connector.max.queue.size | Size of the Queue(in Memory) that holds the CDC records(Use a lower number for Out of Memory exceptions) |
| schema.history.internal.store.only.captured.tables.ddl | Works with table.include.list/table.exclude.list , set this to true to avoid debezium capturing schemas of all tables. |
| schema.history.internal.store.only.captured.databases.ddl | Works with table.include.list/table.exclude.list , set this to true to avoid debezium capturing schemas of all databases. |
| single.threaded | False(Default), This mode skips the entire sink connector thread pool/queue and inserts records in batches on a single thread(Refer doc/production_setup.md) |