diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md new file mode 100644 index 0000000000000..20d1d1759b01b --- /dev/null +++ b/TOC-tidb-cloud.md @@ -0,0 +1,357 @@ + + + +- [Docs Home](https://docs.pingcap.com/) +- About TiDB Cloud + - [Why TiDB Cloud](/tidb-cloud/tidb-cloud-intro.md) + - [Architecture](/tidb-cloud/tidb-cloud-intro.md#architecture) + - [High Availability](/tidb-cloud/high-availability-with-multi-az.md) + - [MySQL Compatibility](/mysql-compatibility.md) +- Get Started + - [Try Out TiDB Cloud](/tidb-cloud/tidb-cloud-quickstart.md) + - [Try Out HTAP](/tidb-cloud/tidb-cloud-htap-quickstart.md) + - [Perform a PoC](/tidb-cloud/tidb-cloud-poc.md) +- Manage Cluster + - Plan Your Cluster + - [Select Your Cluster Tier](/tidb-cloud/select-cluster-tier.md) + - [Determine Your TiDB Size](/tidb-cloud/size-your-cluster.md) + - [Create a TiDB Cluster](/tidb-cloud/create-tidb-cluster.md) + - Connect to Your TiDB Cluster + - [Connect via a SQL Client](/tidb-cloud/connect-to-tidb-cluster.md) + - [Connect via SQL Shell](/tidb-cloud/connect-to-tidb-cluster.md#connect-via-sql-shell) + - [Set Up VPC Peering Connections](/tidb-cloud/set-up-vpc-peering-connections.md) + - Use an HTAP Cluster with TiFlash + - [TiFlash Overview](/tiflash/tiflash-overview.md) + - [Create TiFlash Replicas](/tiflash/create-tiflash-replicas.md) + - [Read Data from TiFlash](/tiflash/use-tidb-to-read-tiflash.md) + - [Use MPP Mode](/tiflash/use-tiflash-mpp-mode.md) + - [Supported Push-down Calculations](/tiflash/tiflash-supported-pushdown-calculations.md) + - [Compatibility](/tiflash/tiflash-compatibility.md) + - [Scale a TiDB Cluster](/tidb-cloud/scale-tidb-cluster.md) + - [Upgrade a TiDB Cluster](/tidb-cloud/upgrade-tidb-cluster.md) + - [Delete a TiDB Cluster](/tidb-cloud/delete-tidb-cluster.md) +- Migrate Data + - [Import Sample Data](/tidb-cloud/import-sample-data.md) + - Migrate Data into TiDB + - [Configure Amazon S3 Access and GCS Access](/tidb-cloud/config-s3-and-gcs-access.md) + - [Migrate from MySQL-Compatible Databases](/tidb-cloud/migrate-data-into-tidb.md) + - [Migrate Incremental Data from MySQL-Compatible Databases](/tidb-cloud/migrate-incremental-data-from-mysql.md) + - [Migrate from Amazon Aurora MySQL in Bulk](/tidb-cloud/migrate-from-aurora-bulk-import.md) + - [Import or Migrate from Amazon S3 or GCS to TiDB Cloud](/tidb-cloud/migrate-from-amazon-s3-or-gcs.md) + - [Import CSV Files from Amazon S3 or GCS into TiDB Cloud](/tidb-cloud/import-csv-files.md) + - [Import Apache Parquet Files from Amazon S3 or GCS into TiDB Cloud](/tidb-cloud/import-parquet-files.md) + - [Troubleshoot Access Denied Errors during Data Import from Amazon S3](/tidb-cloud/troubleshoot-import-access-denied-error.md) + - [Export Data from TiDB](/tidb-cloud/export-data-from-tidb-cloud.md) +- Back Up and Restore + - [Automatic Backup](/tidb-cloud/backup-and-restore.md) + - [Manual Backup](/tidb-cloud/backup-and-restore.md#manual-backup) + - [Restore](/tidb-cloud/backup-and-restore.md#restore) +- Monitor and Alert + - [Overview](/tidb-cloud/monitor-tidb-cluster.md) + - [Built-in Monitoring](/tidb-cloud/built-in-monitoring.md) + - [Built-in Alerting](/tidb-cloud/monitor-built-in-alerting.md) + - Third-Party Monitoring Integrations + - [Datadog Integration](/tidb-cloud/monitor-datadog-integration.md) + - [Prometheus and Grafana Integration](/tidb-cloud/monitor-prometheus-and-grafana-integration.md) +- Tune Performance + - Analyze Performance + - [Statement Analysis](/tidb-cloud/tune-performance.md) + - [Key Visualizer](/tidb-cloud/tune-performance.md#key-visualizer) + - [Statement Summary Tables](/statement-summary-tables.md) + - SQL Tuning + - Understanding the Query Execution Plan + - [Overview](/explain-overview.md) + - [`EXPLAIN` Walkthrough](/explain-walkthrough.md) + - [Indexes](/explain-indexes.md) + - [Joins](/explain-joins.md) + - [MPP Queries](/explain-mpp.md) + - [Subqueries](/explain-subqueries.md) + - [Aggregation](/explain-aggregation.md) + - [Views](/explain-views.md) + - [Partitions](/explain-partitions.md) + - SQL Optimization Process + - [Overview](/sql-optimization-concepts.md) + - Logic Optimization + - [Overview](/sql-logical-optimization.md) + - [Subquery Related Optimizations](/subquery-optimization.md) + - [Column Pruning](/column-pruning.md) + - [Decorrelation of Correlated Subquery](/correlated-subquery-optimization.md) + - [Eliminate Max/Min](/max-min-eliminate.md) + - [Predicates Push Down](/predicate-push-down.md) + - [Partition Pruning](/partition-pruning.md) + - [TopN and Limit Push Down](/topn-limit-push-down.md) + - [Join Reorder](/join-reorder.md) + - Physical Optimization + - [Overview](/sql-physical-optimization.md) + - [Index Selection](/choose-index.md) + - [Statistics](/statistics.md) + - [Wrong Index Solution](/wrong-index-solution.md) + - [Distinct Optimization](/agg-distinct-optimization.md) + - [Cost Model](/cost-model.md) + - [Prepare Execution Plan Cache](/sql-prepared-plan-cache.md) + - Control Execution Plans + - [Overview](/control-execution-plan.md) + - [Optimizer Hints](/optimizer-hints.md) + - [SQL Plan Management](/sql-plan-management.md) + - [The Blocklist of Optimization Rules and Expression Pushdown](/blocklist-control-plan.md) + - [TiKV Follower Read](/follower-read.md) + - [Coprocessor Cache](/coprocessor-cache.md) + - Garbage Collection (GC) + - [Overview](/garbage-collection-overview.md) + - [Configuration](/garbage-collection-configuration.md) + - [Tune TiFlash performance](/tiflash/tune-tiflash-performance.md) +- Manage User Access + - [Manage Console User Access](/tidb-cloud/manage-user-access.md) + - [Configure Cluster Security Settings](/tidb-cloud/configure-security-settings.md) +- Billing + - [Node Cost](/tidb-cloud/tidb-cloud-billing.md) + - [Backup Storage Cost](/tidb-cloud/tidb-cloud-billing.md#backup-storage-cost) + - [Data Transfer Cost](/tidb-cloud/tidb-cloud-billing.md#data-transfer-cost) + - [Invoices](/tidb-cloud/tidb-cloud-billing.md#invoices) + - [Billing Details](/tidb-cloud/tidb-cloud-billing.md#billing-details) + - [Trial Points](/tidb-cloud/tidb-cloud-billing.md#trial-points) + - [Payment Method Setting](/tidb-cloud/tidb-cloud-billing.md#payment-method) +- Reference + - TiDB Cluster Architecture + - [Overview](/tidb-architecture.md) + - [Storage](/tidb-storage.md) + - [Computing](/tidb-computing.md) + - [Scheduling](/tidb-scheduling.md) + - [TiDB Cloud Cluster Limits and Quotas](/tidb-cloud/limitations-and-quotas.md) + - [TiDB Limitations](/tidb-limitations.md) + - SQL + - [Explore SQL with TiDB](/basic-sql-operations.md) + - SQL Language Structure and Syntax + - Attributes + - [AUTO_INCREMENT](/auto-increment.md) + - [AUTO_RANDOM](/auto-random.md) + - [SHARD_ROW_ID_BITS](/shard-row-id-bits.md) + - [Literal Values](/literal-values.md) + - [Schema Object Names](/schema-object-names.md) + - [Keywords and Reserved Words](/keywords.md) + - [User-Defined Variables](/user-defined-variables.md) + - [Expression Syntax](/expression-syntax.md) + - [Comment Syntax](/comment-syntax.md) + - SQL Statements + - [`ADD COLUMN`](/sql-statements/sql-statement-add-column.md) + - [`ADD INDEX`](/sql-statements/sql-statement-add-index.md) + - [`ADMIN`](/sql-statements/sql-statement-admin.md) + - [`ADMIN CANCEL DDL`](/sql-statements/sql-statement-admin-cancel-ddl.md) + - [`ADMIN CHECKSUM TABLE`](/sql-statements/sql-statement-admin-checksum-table.md) + - [`ADMIN CHECK [TABLE|INDEX]`](/sql-statements/sql-statement-admin-check-table-index.md) + - [`ADMIN SHOW DDL [JOBS|QUERIES]`](/sql-statements/sql-statement-admin-show-ddl.md) + - [`ALTER DATABASE`](/sql-statements/sql-statement-alter-database.md) + - [`ALTER INDEX`](/sql-statements/sql-statement-alter-index.md) + - [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md) + - [`ALTER TABLE COMPACT`](/sql-statements/sql-statement-alter-table-compact.md) + - [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) + - [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) + - [`BATCH`](/sql-statements/sql-statement-batch.md) + - [`BEGIN`](/sql-statements/sql-statement-begin.md) + - [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md) + - [`COMMIT`](/sql-statements/sql-statement-commit.md) + - [`CHANGE DRAINER`](/sql-statements/sql-statement-change-drainer.md) + - [`CHANGE PUMP`](/sql-statements/sql-statement-change-pump.md) + - [`CREATE [GLOBAL|SESSION] BINDING`](/sql-statements/sql-statement-create-binding.md) + - [`CREATE DATABASE`](/sql-statements/sql-statement-create-database.md) + - [`CREATE INDEX`](/sql-statements/sql-statement-create-index.md) + - [`CREATE ROLE`](/sql-statements/sql-statement-create-role.md) + - [`CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) + - [`CREATE TABLE LIKE`](/sql-statements/sql-statement-create-table-like.md) + - [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) + - [`CREATE USER`](/sql-statements/sql-statement-create-user.md) + - [`CREATE VIEW`](/sql-statements/sql-statement-create-view.md) + - [`DEALLOCATE`](/sql-statements/sql-statement-deallocate.md) + - [`DELETE`](/sql-statements/sql-statement-delete.md) + - [`DESC`](/sql-statements/sql-statement-desc.md) + - [`DESCRIBE`](/sql-statements/sql-statement-describe.md) + - [`DO`](/sql-statements/sql-statement-do.md) + - [`DROP [GLOBAL|SESSION] BINDING`](/sql-statements/sql-statement-drop-binding.md) + - [`DROP COLUMN`](/sql-statements/sql-statement-drop-column.md) + - [`DROP DATABASE`](/sql-statements/sql-statement-drop-database.md) + - [`DROP INDEX`](/sql-statements/sql-statement-drop-index.md) + - [`DROP ROLE`](/sql-statements/sql-statement-drop-role.md) + - [`DROP SEQUENCE`](/sql-statements/sql-statement-drop-sequence.md) + - [`DROP STATS`](/sql-statements/sql-statement-drop-stats.md) + - [`DROP TABLE`](/sql-statements/sql-statement-drop-table.md) + - [`DROP USER`](/sql-statements/sql-statement-drop-user.md) + - [`DROP VIEW`](/sql-statements/sql-statement-drop-view.md) + - [`EXECUTE`](/sql-statements/sql-statement-execute.md) + - [`EXPLAIN ANALYZE`](/sql-statements/sql-statement-explain-analyze.md) + - [`EXPLAIN`](/sql-statements/sql-statement-explain.md) + - [`FLASHBACK TABLE`](/sql-statements/sql-statement-flashback-table.md) + - [`FLUSH PRIVILEGES`](/sql-statements/sql-statement-flush-privileges.md) + - [`FLUSH STATUS`](/sql-statements/sql-statement-flush-status.md) + - [`FLUSH TABLES`](/sql-statements/sql-statement-flush-tables.md) + - [`GRANT `](/sql-statements/sql-statement-grant-privileges.md) + - [`GRANT `](/sql-statements/sql-statement-grant-role.md) + - [`INSERT`](/sql-statements/sql-statement-insert.md) + - [`KILL [TIDB]`](/sql-statements/sql-statement-kill.md) + - [`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) + - [`PREPARE`](/sql-statements/sql-statement-prepare.md) + - [`RECOVER TABLE`](/sql-statements/sql-statement-recover-table.md) + - [`RENAME INDEX`](/sql-statements/sql-statement-rename-index.md) + - [`RENAME TABLE`](/sql-statements/sql-statement-rename-table.md) + - [`REPLACE`](/sql-statements/sql-statement-replace.md) + - [`REVOKE `](/sql-statements/sql-statement-revoke-privileges.md) + - [`REVOKE `](/sql-statements/sql-statement-revoke-role.md) + - [`ROLLBACK`](/sql-statements/sql-statement-rollback.md) + - [`SAVEPOINT`](/sql-statements/sql-statement-savepoint.md) + - [`SELECT`](/sql-statements/sql-statement-select.md) + - [`SET DEFAULT ROLE`](/sql-statements/sql-statement-set-default-role.md) + - [`SET [NAMES|CHARACTER SET]`](/sql-statements/sql-statement-set-names.md) + - [`SET PASSWORD`](/sql-statements/sql-statement-set-password.md) + - [`SET ROLE`](/sql-statements/sql-statement-set-role.md) + - [`SET TRANSACTION`](/sql-statements/sql-statement-set-transaction.md) + - [`SET [GLOBAL|SESSION] `](/sql-statements/sql-statement-set-variable.md) + - [`SHOW ANALYZE STATUS`](/sql-statements/sql-statement-show-analyze-status.md) + - [`SHOW [GLOBAL|SESSION] BINDINGS`](/sql-statements/sql-statement-show-bindings.md) + - [`SHOW BUILTINS`](/sql-statements/sql-statement-show-builtins.md) + - [`SHOW CHARACTER SET`](/sql-statements/sql-statement-show-character-set.md) + - [`SHOW COLLATION`](/sql-statements/sql-statement-show-collation.md) + - [`SHOW [FULL] COLUMNS FROM`](/sql-statements/sql-statement-show-columns-from.md) + - [`SHOW CREATE SEQUENCE`](/sql-statements/sql-statement-show-create-sequence.md) + - [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md) + - [`SHOW CREATE USER`](/sql-statements/sql-statement-show-create-user.md) + - [`SHOW DATABASES`](/sql-statements/sql-statement-show-databases.md) + - [`SHOW DRAINER STATUS`](/sql-statements/sql-statement-show-drainer-status.md) + - [`SHOW ENGINES`](/sql-statements/sql-statement-show-engines.md) + - [`SHOW ERRORS`](/sql-statements/sql-statement-show-errors.md) + - [`SHOW [FULL] FIELDS FROM`](/sql-statements/sql-statement-show-fields-from.md) + - [`SHOW GRANTS`](/sql-statements/sql-statement-show-grants.md) + - [`SHOW INDEX [FROM|IN]`](/sql-statements/sql-statement-show-index.md) + - [`SHOW INDEXES [FROM|IN]`](/sql-statements/sql-statement-show-indexes.md) + - [`SHOW KEYS [FROM|IN]`](/sql-statements/sql-statement-show-keys.md) + - [`SHOW MASTER STATUS`](/sql-statements/sql-statement-show-master-status.md) + - [`SHOW PLUGINS`](/sql-statements/sql-statement-show-plugins.md) + - [`SHOW PRIVILEGES`](/sql-statements/sql-statement-show-privileges.md) + - [`SHOW [FULL] PROCESSSLIST`](/sql-statements/sql-statement-show-processlist.md) + - [`SHOW PROFILES`](/sql-statements/sql-statement-show-profiles.md) + - [`SHOW PUMP STATUS`](/sql-statements/sql-statement-show-pump-status.md) + - [`SHOW SCHEMAS`](/sql-statements/sql-statement-show-schemas.md) + - [`SHOW STATS_HEALTHY`](/sql-statements/sql-statement-show-stats-healthy.md) + - [`SHOW STATS_HISTOGRAMS`](/sql-statements/sql-statement-show-histograms.md) + - [`SHOW STATS_META`](/sql-statements/sql-statement-show-stats-meta.md) + - [`SHOW STATUS`](/sql-statements/sql-statement-show-status.md) + - [`SHOW TABLE NEXT_ROW_ID`](/sql-statements/sql-statement-show-table-next-rowid.md) + - [`SHOW TABLE REGIONS`](/sql-statements/sql-statement-show-table-regions.md) + - [`SHOW TABLE STATUS`](/sql-statements/sql-statement-show-table-status.md) + - [`SHOW [FULL] TABLES`](/sql-statements/sql-statement-show-tables.md) + - [`SHOW [GLOBAL|SESSION] VARIABLES`](/sql-statements/sql-statement-show-variables.md) + - [`SHOW WARNINGS`](/sql-statements/sql-statement-show-warnings.md) + - [`SHUTDOWN`](/sql-statements/sql-statement-shutdown.md) + - [`SPLIT REGION`](/sql-statements/sql-statement-split-region.md) + - [`START TRANSACTION`](/sql-statements/sql-statement-start-transaction.md) + - [`TABLE`](/sql-statements/sql-statement-table.md) + - [`TRACE`](/sql-statements/sql-statement-trace.md) + - [`TRUNCATE`](/sql-statements/sql-statement-truncate.md) + - [`UPDATE`](/sql-statements/sql-statement-update.md) + - [`USE`](/sql-statements/sql-statement-use.md) + - [`WITH`](/sql-statements/sql-statement-with.md) + - Data Types + - [Overview](/data-type-overview.md) + - [Default Values](/data-type-default-values.md) + - [Numeric Types](/data-type-numeric.md) + - [Date and Time Types](/data-type-date-and-time.md) + - [String Types](/data-type-string.md) + - [JSON Type](/data-type-json.md) + - Functions and Operators + - [Overview](/functions-and-operators/functions-and-operators-overview.md) + - [Type Conversion in Expression Evaluation](/functions-and-operators/type-conversion-in-expression-evaluation.md) + - [Operators](/functions-and-operators/operators.md) + - [Control Flow Functions](/functions-and-operators/control-flow-functions.md) + - [String Functions](/functions-and-operators/string-functions.md) + - [Numeric Functions and Operators](/functions-and-operators/numeric-functions-and-operators.md) + - [Date and Time Functions](/functions-and-operators/date-and-time-functions.md) + - [Bit Functions and Operators](/functions-and-operators/bit-functions-and-operators.md) + - [Cast Functions and Operators](/functions-and-operators/cast-functions-and-operators.md) + - [Encryption and Compression Functions](/functions-and-operators/encryption-and-compression-functions.md) + - [Locking Functions](/functions-and-operators/locking-functions.md) + - [Information Functions](/functions-and-operators/information-functions.md) + - [JSON Functions](/functions-and-operators/json-functions.md) + - [Aggregate (GROUP BY) Functions](/functions-and-operators/aggregate-group-by-functions.md) + - [Window Functions](/functions-and-operators/window-functions.md) + - [Miscellaneous Functions](/functions-and-operators/miscellaneous-functions.md) + - [Precision Math](/functions-and-operators/precision-math.md) + - [Set Operations](/functions-and-operators/set-operators.md) + - [List of Expressions for Pushdown](/functions-and-operators/expressions-pushed-down.md) + - [TiDB Specific Functions](/functions-and-operators/tidb-functions.md) + - [Clustered Indexes](/clustered-indexes.md) + - [Constraints](/constraints.md) + - [Generated Columns](/generated-columns.md) + - [SQL Mode](/sql-mode.md) + - [Table Attributes](/table-attributes.md) + - Transactions + - [Overview](/transaction-overview.md) + - [Isolation Levels](/transaction-isolation-levels.md) + - [Optimistic Transactions](/optimistic-transaction.md) + - [Pessimistic Transactions](/pessimistic-transaction.md) + - [Non-Transactional DML Statements](/non-transactional-dml.md) + - [Views](/views.md) + - [Partitioning](/partitioned-table.md) + - [Temporary Tables](/temporary-tables.md) + - [Cached Tables](/cached-tables.md) + - Character Set and Collation + - [Overview](/character-set-and-collation.md) + - [GBK](/character-set-gbk.md) + - Read Historical Data + - Use Stale Read (Recommended) + - [Usage Scenarios of Stale Read](/stale-read.md) + - [Perform Stale Read Using `As OF TIMESTAMP`](/as-of-timestamp.md) + - [Perform Stale Read Using `tidb_read_staleness`](/tidb-read-staleness.md) + - [Use the `tidb_snapshot` System Variable](/read-historical-data.md) + - System Tables + - [`mysql`](/mysql-schema.md) + - INFORMATION_SCHEMA + - [Overview](/information-schema/information-schema.md) + - [`ANALYZE_STATUS`](/information-schema/information-schema-analyze-status.md) + - [`CLIENT_ERRORS_SUMMARY_BY_HOST`](/information-schema/client-errors-summary-by-host.md) + - [`CLIENT_ERRORS_SUMMARY_BY_USER`](/information-schema/client-errors-summary-by-user.md) + - [`CLIENT_ERRORS_SUMMARY_GLOBAL`](/information-schema/client-errors-summary-global.md) + - [`CHARACTER_SETS`](/information-schema/information-schema-character-sets.md) + - [`CLUSTER_INFO`](/information-schema/information-schema-cluster-info.md) + - [`COLLATIONS`](/information-schema/information-schema-collations.md) + - [`COLLATION_CHARACTER_SET_APPLICABILITY`](/information-schema/information-schema-collation-character-set-applicability.md) + - [`COLUMNS`](/information-schema/information-schema-columns.md) + - [`DATA_LOCK_WAITS`](/information-schema/information-schema-data-lock-waits.md) + - [`DDL_JOBS`](/information-schema/information-schema-ddl-jobs.md) + - [`DEADLOCKS`](/information-schema/information-schema-deadlocks.md) + - [`ENGINES`](/information-schema/information-schema-engines.md) + - [`KEY_COLUMN_USAGE`](/information-schema/information-schema-key-column-usage.md) + - [`PARTITIONS`](/information-schema/information-schema-partitions.md) + - [`PROCESSLIST`](/information-schema/information-schema-processlist.md) + - [`REFERENTIAL_CONSTRAINTS`](/information-schema/information-schema-referential-constraints.md) + - [`SCHEMATA`](/information-schema/information-schema-schemata.md) + - [`SEQUENCES`](/information-schema/information-schema-sequences.md) + - [`SESSION_VARIABLES`](/information-schema/information-schema-session-variables.md) + - [`SLOW_QUERY`](/information-schema/information-schema-slow-query.md) + - [`STATISTICS`](/information-schema/information-schema-statistics.md) + - [`TABLES`](/information-schema/information-schema-tables.md) + - [`TABLE_CONSTRAINTS`](/information-schema/information-schema-table-constraints.md) + - [`TABLE_STORAGE_STATS`](/information-schema/information-schema-table-storage-stats.md) + - [`TIDB_HOT_REGIONS_HISTORY`](/information-schema/information-schema-tidb-hot-regions-history.md) + - [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md) + - [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md) + - [`TIDB_TRX`](/information-schema/information-schema-tidb-trx.md) + - [`TIFLASH_REPLICA`](/information-schema/information-schema-tiflash-replica.md) + - [`TIKV_REGION_PEERS`](/information-schema/information-schema-tikv-region-peers.md) + - [`TIKV_REGION_STATUS`](/information-schema/information-schema-tikv-region-status.md) + - [`TIKV_STORE_STATUS`](/information-schema/information-schema-tikv-store-status.md) + - [`USER_PRIVILEGES`](/information-schema/information-schema-user-privileges.md) + - [`VARIABLES_INFO`](/information-schema/information-schema-variables-info.md) + - [`VIEWS`](/information-schema/information-schema-views.md) + - [System Variables](/system-variables.md) + - Storage Engines + - TiKV + - [TiKV Overview](/tikv-overview.md) + - [RocksDB Overview](/storage-engine/rocksdb-overview.md) + - TiFlash + - [TiFlash Overview](/tiflash/tiflash-overview.md) + - [Troubleshoot Inconsistency Between Data and Indexes](/troubleshoot-data-inconsistency-errors.md) +- [FAQs](/tidb-cloud/tidb-cloud-faq.md) +- Release Notes + - [2022](/tidb-cloud/release-notes-2022.md) + - [2021](/tidb-cloud/release-notes-2021.md) + - [2020](/tidb-cloud/release-notes-2020.md) +- [Support](/tidb-cloud/tidb-cloud-support.md) +- [Glossary](/tidb-cloud/tidb-cloud-glossary.md) diff --git a/agg-distinct-optimization.md b/agg-distinct-optimization.md index b6b9186771836..8af03607b242c 100644 --- a/agg-distinct-optimization.md +++ b/agg-distinct-optimization.md @@ -27,9 +27,19 @@ mysql> explain SELECT DISTINCT a from t; Usually, aggregate functions with the `DISTINCT` option is executed in the TiDB layer in a single-threaded execution model. -The [`tidb_opt_distinct_agg_push_down`](/system-variables.md#tidb_opt_distinct_agg_push_down) system variable or the [`distinct-agg-push-down`](/tidb-configuration-file.md#distinct-agg-push-down) configuration item in TiDB controls whether to rewrite the distinct aggregate queries and push them to the TiKV/TiFlash Coprocessor. + -Take the following queries as an example of this optimization. `tidb_opt_distinct_agg_push_down` is disabled by default, which means the aggregate functions are executed in the TiDB layer. After enabling this optimization by setting its value to `1`, the `distinct a` part of `count(distinct a)` is pushed to TiKV/TiFlash Coprocessor: there is a HashAgg_5 to remove the duplicated values on column a in the TiKV Coprocessor. It might reduce the computation overhead of `HashAgg_8` in the TiDB layer. +The [`tidb_opt_distinct_agg_push_down`](/system-variables.md#tidb_opt_distinct_agg_push_down) system variable or the [`distinct-agg-push-down`](/tidb-configuration-file.md#distinct-agg-push-down) configuration item in TiDB controls whether to rewrite the distinct aggregate queries and push them to the TiKV or TiFlash Coprocessor. + + + + + +The [`tidb_opt_distinct_agg_push_down`](/system-variables.md#tidb_opt_distinct_agg_push_down) system variable in TiDB controls whether to rewrite the distinct aggregate queries and push them to the TiKV or TiFlash Coprocessor. + + + +Take the following queries as an example of this optimization. `tidb_opt_distinct_agg_push_down` is disabled by default, which means the aggregate functions are executed in the TiDB layer. After enabling this optimization by setting its value to `1`, the `distinct a` part of `count(distinct a)` is pushed to TiKV or TiFlash Coprocessor: there is a HashAgg_5 to remove the duplicated values on column a in the TiKV Coprocessor. It might reduce the computation overhead of `HashAgg_8` in the TiDB layer. ```sql mysql> desc select count(distinct a) from test.t; diff --git a/auto-increment.md b/auto-increment.md index 6e6e4595506f5..72ee5f9b666f5 100644 --- a/auto-increment.md +++ b/auto-increment.md @@ -8,10 +8,22 @@ aliases: ['/docs/dev/auto-increment/'] This document introduces the `AUTO_INCREMENT` column attribute, including its concept, implementation principles, auto-increment related features, and restrictions. + + > **Note:** > > The `AUTO_INCREMENT` attribute might cause hotspot in production environments. See [Troubleshoot HotSpot Issues](/troubleshoot-hot-spot-issues.md) for details. It is recommended to use [`AUTO_RANDOM`](/auto-random.md) instead. + + + + +> **Note:** +> +> The `AUTO_INCREMENT` attribute might cause hotspot in production environments. See [Troubleshoot HotSpot Issues](https://docs.pingcap.com/tidb/stable/troubleshoot-hot-spot-issues#handle-auto-increment-primary-key-hotspot-tables-using-auto_random) for details. It is recommended to use [`AUTO_RANDOM`](/auto-random.md) instead. + + + ## Concept `AUTO_INCREMENT` is a column attribute that is used to automatically fill in default column values. When the `INSERT` statement does not specify values for the `AUTO_INCREMENT` column, the system automatically assigns values to this column. diff --git a/auto-random.md b/auto-random.md index 7027779911eaf..27ff4bd4bb40c 100644 --- a/auto-random.md +++ b/auto-random.md @@ -8,11 +8,17 @@ aliases: ['/docs/dev/auto-random/','/docs/dev/reference/sql/attributes/auto-rand > **Note:** > -> `AUTO_RANDOM` was marked as stable in v4.0.3. +> `AUTO_RANDOM` has been generally available since v4.0.3. ## User scenario -When you write data intensively into TiDB and TiDB has the table with a primary key of the auto-increment integer type, hotspot issue might occur. To solve the hotspot issue, you can use the `AUTO_RANDOM` attribute. Refer to [Highly Concurrent Write Best Practices](/best-practices/high-concurrency-best-practices.md#complex-hotspot-problems) for details. +When you write data intensively into TiDB and TiDB has a table with the primary key of the auto-increment integer type, hotspot issue might occur. To solve the hotspot issue, you can use the `AUTO_RANDOM` attribute. + + + +For more information, see [Highly Concurrent Write Best Practices](/best-practices/high-concurrency-best-practices.md#complex-hotspot-problems). + + Take the following created table as an example: diff --git a/basic-sql-operations.md b/basic-sql-operations.md index 112774b2b5c25..ff9c60cd3575b 100644 --- a/basic-sql-operations.md +++ b/basic-sql-operations.md @@ -6,18 +6,8 @@ aliases: ['/docs/dev/basic-sql-operations/','/docs/dev/how-to/get-started/explor # Explore SQL with TiDB - - TiDB is compatible with MySQL, you can use MySQL statements directly in most of the cases. For unsupported features, see [Compatibility with MySQL](/mysql-compatibility.md#unsupported-features). - - - - -TiDB is compatible with MySQL, you can use MySQL statements directly in most of the cases. For unsupported features, see [Compatibility with MySQL](https://docs.pingcap.com/tidb/stable/mysql-compatibility#unsupported-features). - - - To experiment with SQL and test out TiDB compatibility with MySQL queries, you can [run TiDB directly in your web browser without installing it](https://tour.tidb.io/). You can also first deploy a TiDB cluster and then run SQL statements in it. diff --git a/certificate-authentication.md b/certificate-authentication.md index 875451e8c3996..f260e39b7e664 100644 --- a/certificate-authentication.md +++ b/certificate-authentication.md @@ -19,7 +19,17 @@ The rest of the document introduces in detail how to perform these operations. ## Create security keys and certificates -It is recommended that you use [OpenSSL](https://www.openssl.org/) to create keys and certificates. The certificate generation process is similar to the process described in [Enable TLS Between TiDB Clients and Servers](/enable-tls-between-clients-and-servers.md). The following paragraphs demonstrate on how to configure more attribute fields that need to be verified in the certificate. + + +It is recommended that you use [OpenSSL](https://www.openssl.org/) to create keys and certificates. The certificate generation process is similar to the process described in [Enable TLS Between TiDB Clients and Servers](/enable-tls-between-clients-and-servers.md). The following paragraphs demonstrate how to configure more attribute fields that need to be verified in the certificate. + + + + + +It is recommended that you use [OpenSSL](https://www.openssl.org/) to create keys and certificates. The certificate generation process is similar to the process described in [Enable TLS Between TiDB Clients and Servers](https://docs.pingcap.com/tidb/stable/enable-tls-between-clients-and-servers). The following paragraphs demonstrate how to configure more attribute fields that need to be verified in the certificate. + + ### Generate CA key and certificate @@ -278,7 +288,7 @@ The user certificate information can be specified by `require subject`, `require openssl x509 -noout -subject -in ca-cert.pem | sed 's/.\{8\}//' | sed 's/, /\//g' | sed 's/ = /=/g' | sed 's/^/\//' ``` -+ `require san`: Specifies the `Subject Alternative Name` information of the CA certificate that issues the user certificate. The information to be specified is consistent with the [`alt_names` of the `openssl.cnf` configuration file](/generate-self-signed-certificates.md) used to generate the client certificate. ++ `require san`: Specifies the `Subject Alternative Name` information of the CA certificate that issues the user certificate. The information to be specified is consistent with the [`alt_names` of the `openssl.cnf` configuration file](https://docs.pingcap.com/tidb/stable/generate-self-signed-certificates) used to generate the client certificate. + Execute the following command to get the information of the `require san` item in the generated certificate: diff --git a/character-set-and-collation.md b/character-set-and-collation.md index dab2286467eee..5bee561c6a5f0 100644 --- a/character-set-and-collation.md +++ b/character-set-and-collation.md @@ -382,8 +382,12 @@ To disable this error reporting, use `set @@tidb_skip_utf8_check=1;` to skip the ## Collation support framework + + The syntax support and semantic support for the collation are influenced by the [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap) configuration item. The syntax support and semantic support are different. The former indicates that TiDB can parse and set collations. The latter indicates that TiDB can correctly use collations when comparing strings. + + Before v4.0, TiDB provides only the [old framework for collations](#old-framework-for-collations). In this framework, TiDB supports syntactically parsing most of the MySQL collations but semantically takes all collations as binary collations. Since v4.0, TiDB supports a [new framework for collations](#new-framework-for-collations). In this framework, TiDB semantically parses different collations and strictly follows the collations when comparing strings. @@ -407,7 +411,11 @@ Query OK, 1 row affected # In TiDB, it is successfully executed. In MySQL, becau ### New framework for collations -In TiDB 4.0, a complete framework for collations is introduced. This new framework supports semantically parsing collations and introduces the `new_collations_enabled_on_first_bootstrap` configuration item to decide whether to enable the new framework when a cluster is first initialized. To enable the new framework, set `new_collations_enabled_on_first_bootstrap` to `true`. For details, see [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap). If you initialize the cluster after the configuration item is enabled, you can check whether the new collation is enabled through the `new_collation_enabled` variable in the `mysql`.`tidb` table: +Since TiDB v4.0, a complete framework for collations is introduced. + + + +This new framework supports semantically parsing collations and introduces the `new_collations_enabled_on_first_bootstrap` configuration item to decide whether to enable the new framework when a cluster is first initialized. To enable the new framework, set `new_collations_enabled_on_first_bootstrap` to `true`. For details, see [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap). If you initialize the cluster after the configuration item is enabled, you can check whether the new collation is enabled through the `new_collation_enabled` variable in the `mysql`.`tidb` table: {{< copyable "sql" >}} @@ -424,6 +432,14 @@ SELECT VARIABLE_VALUE FROM mysql.tidb WHERE VARIABLE_NAME='new_collation_enabled 1 row in set (0.00 sec) ``` + + + + +This new framework supports semantically parsing collations. TiDB enables the new framework by default when a cluster is first initialized. + + + Under the new framework, TiDB supports the `utf8_general_ci`, `utf8mb4_general_ci`, `utf8_unicode_ci`, `utf8mb4_unicode_ci`, `gbk_chinese_ci`, and `gbk_bin` collations, which is compatible with MySQL. When one of `utf8_general_ci`, `utf8mb4_general_ci`, `utf8_unicode_ci`, `utf8mb4_unicode_ci`, and `gbk_chinese_ci` is used, the string comparison is case-insensitive and accent-insensitive. At the same time, TiDB also corrects the collation's `PADDING` behavior: diff --git a/character-set-gbk.md b/character-set-gbk.md index 5247b3f589fe4..7060f99294ff3 100644 --- a/character-set-gbk.md +++ b/character-set-gbk.md @@ -33,8 +33,18 @@ This section provides the compatibility information between MySQL and TiDB. The default collation of the GBK character set in MySQL is `gbk_chinese_ci`. Unlike MySQL, the default collation of the GBK character set in TiDB is `gbk_bin`. Additionally, because TiDB converts GBK to UTF8MB4 and then uses a binary collation, the `gbk_bin` collation in TiDB is not the same as the `gbk_bin` collation in MySQL. + + To make TiDB compatible with the collations of MySQL GBK character set, when you first initialize the TiDB cluster, you need to set the TiDB option [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap) to `true` to enable the [new framework for collations](/character-set-and-collation.md#new-framework-for-collations). + + + + +To make TiDB compatible with the collations of MySQL GBK character set, when you first initialize the TiDB cluster, TiDB Cloud enables the [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) by default. + + + After enabling the new framework for collations, if you check the collations corresponding to the GBK character set, you can see that the TiDB GBK default collation is changed to `gbk_chinese_ci`. ```sql diff --git a/clustered-indexes.md b/clustered-indexes.md index 18e701246a3cb..a75cf3650f95f 100644 --- a/clustered-indexes.md +++ b/clustered-indexes.md @@ -148,7 +148,7 @@ Currently, there are several different types of limitations for the clustered in - Situations that are not supported yet but in the support plan: - Adding, dropping, and altering clustered indexes using `ALTER TABLE` statements are not supported. - Limitations for specific versions: - - In v5.0, using the clustered index feature together with TiDB Binlog is not supported. After TiDB Binlog is enabled, TiDB only allows creating a single integer column as the clustered index of a primary key. TiDB Binlog does not replicate data changes (such as insertion, deletion, and update) on existing tables with clustered indexes to the downstream. If you need to replicate tables with clustered indexes to the downstream, upgrade your cluster to v5.1 or use [TiCDC](/ticdc/ticdc-overview.md) for replication instead. + - In v5.0, using the clustered index feature together with TiDB Binlog is not supported. After TiDB Binlog is enabled, TiDB only allows creating a single integer column as the clustered index of a primary key. TiDB Binlog does not replicate data changes (such as insertion, deletion, and update) on existing tables with clustered indexes to the downstream. If you need to replicate tables with clustered indexes to the downstream, upgrade your cluster to v5.1 or use [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) for replication instead. After TiDB Binlog is enabled, if the clustered index you create is not a single integer primary key, TiDB returns the following error: diff --git a/coprocessor-cache.md b/coprocessor-cache.md index 2aa9de69baf05..8c995ed0648c1 100644 --- a/coprocessor-cache.md +++ b/coprocessor-cache.md @@ -10,8 +10,18 @@ Starting from v4.0, the TiDB instance supports caching the results of the calcul ## Configuration + + You can configure Coprocessor Cache via the `tikv-client.copr-cache` configuration items in the TiDB configuration file. For details about how to enable and configure Coprocessor Cache, see [TiDB Configuration File](/tidb-configuration-file.md#tikv-clientcopr-cache-new-in-v400). + + + + +The Coprocessor Cache feature is enabled by default. The maximum size of the data that can be cached is 1000 MB. + + + ## Feature description + When a SQL statement is executed on a single TiDB instance for the first time, the execution result is not cached. diff --git a/enable-tls-between-components.md b/enable-tls-between-components.md index 5d540f80de4fd..45b5263e32878 100644 --- a/enable-tls-between-components.md +++ b/enable-tls-between-components.md @@ -23,8 +23,18 @@ Currently, it is not supported to only enable encrypted transmission of some spe You can use tools like `openssl`, `easy-rsa` and `cfssl` to generate self-signed certificates. + + If you choose `openssl`, you can refer to [generating self-signed certificates](/generate-self-signed-certificates.md). + + + + + If you choose `openssl`, you can refer to [generating self-signed certificates](https://docs.pingcap.com/tidb/stable/generate-self-signed-certificates). + + + 2. Configure certificates. To enable mutual authentication among TiDB components, configure the certificates of TiDB, TiKV, and PD as follows. diff --git a/explain-views.md b/explain-views.md index 80dc990ba4145..5b9eb243eb3ac 100644 --- a/explain-views.md +++ b/explain-views.md @@ -7,8 +7,18 @@ summary: Learn about the execution plan information returned by the `EXPLAIN` st `EXPLAIN` displays the tables and indexes that a [view](/views.md) references, not the name of the view itself. This is because views are only virtual tables and do not store any data themselves. The definition of the view and the rest of the statement are merged together during SQL optimization. + + From the [bikeshare example database](/import-example-data.md), you can see that the following two queries are executed in a similar manner: + + + + +From the [bikeshare example database](/tidb-cloud/import-sample-data.md), you can see that the following two queries are executed in a similar manner: + + + {{< copyable "sql" >}} ```sql diff --git a/explain-walkthrough.md b/explain-walkthrough.md index 9350b919ac422..eaf48b72473eb 100644 --- a/explain-walkthrough.md +++ b/explain-walkthrough.md @@ -7,7 +7,17 @@ summary: Learn how to use EXPLAIN by walking through an example statement Because SQL is a declarative language, you cannot automatically tell whether a query is executed efficiently. You must first use the [`EXPLAIN`](/sql-statements/sql-statement-explain.md) statement to learn the current execution plan. -The following statement from the [bikeshare example database](/import-example-data.md) counts how many trips were taken on the July 1, 2017: + + +The following statement from the [bikeshare example database](/import-example-data.md) counts how many trips were taken on July 1, 2017: + + + + + +The following statement from the [bikeshare example database](/tidb-cloud/import-sample-data.md) counts how many trips were taken on July 1, 2017: + + {{< copyable "sql" >}} diff --git a/information-schema/information-schema-deadlocks.md b/information-schema/information-schema-deadlocks.md index 67c377ff82ae5..4e9f6363fad99 100644 --- a/information-schema/information-schema-deadlocks.md +++ b/information-schema/information-schema-deadlocks.md @@ -44,8 +44,18 @@ The meaning of each column field in the `DEADLOCKS` table is as follows: * `KEY_INFO`: The detailed information of `KEY`. See the [KEY_INFO](#key_info) section. * `TRX_HOLDING_LOCK`: The ID of the transaction that currently holds the lock on the key and causes blocking. This ID is also the `start_ts` of the transaction. + + To adjust the maximum number of deadlock events that can be recorded in the `DEADLOCKS` table, adjust the [`pessimistic-txn.deadlock-history-capacity`](/tidb-configuration-file.md#deadlock-history-capacity) configuration in the TiDB configuration file. By default, the information of the recent 10 deadlock events is recorded in the table. + + + + +The information of the recent 10 deadlock events is recorded in the `DEADLOCKS` table. + + + > **Warning:** > > * Only users with the [PROCESS](https://dev.mysql.com/doc/refman/8.0/en/privileges-provided.html#priv_process) privilege can query this table. @@ -78,10 +88,22 @@ In the above fields, if the information of a field is not applicable or currentl ## Retryable deadlock errors + + +> **Note:** +> +> This section is not applicable to TiDB Cloud. + + + + + > **Note:** > > The `DEADLOCKS` table does not collect the information of retryable deadlock errors by default. If you want the table to collect the retryable deadlock error information, you can adjust the value of [`pessimistic-txn.deadlock-history-collect-retryable`](/tidb-configuration-file.md#deadlock-history-collect-retryable) in the TiDB configuration file. + + When transaction A is blocked by a lock already held by transaction B, and transaction B is directly or indirectly blocked by the lock held by the current transaction A, a deadlock error will occur. In this deadlock, there might be two cases: + Case 1: Transaction B might be (directly or indirectly) blocked by a lock generated by a statement that has been executed after transaction A starts and before transaction A gets blocked. diff --git a/information-schema/information-schema-slow-query.md b/information-schema/information-schema-slow-query.md index ab40bf4d92524..a0faa422b318f 100644 --- a/information-schema/information-schema-slow-query.md +++ b/information-schema/information-schema-slow-query.md @@ -5,7 +5,13 @@ summary: Learn the `SLOW_QUERY` information_schema table. # SLOW_QUERY -The `SLOW_QUERY` table provides the slow query information of the current node, which is the parsing result of the TiDB slow log file. The column names in the table are corresponding to the field names in the slow log. For how to use this table to identify problematic statements and improve query performance, see [Slow Query Log Document](/identify-slow-queries.md). +The `SLOW_QUERY` table provides the slow query information of the current node, which is the parsing result of the TiDB slow log file. The column names in the table are corresponding to the field names in the slow log. + + + +For how to use this table to identify problematic statements and improve query performance, see [Slow Query Log Document](/identify-slow-queries.md). + + {{< copyable "sql" >}} @@ -97,7 +103,13 @@ DESC slow_query; ## CLUSTER_SLOW_QUERY table -The `CLUSTER_SLOW_QUERY` table provides the slow query information of all nodes in the cluster, which is the parsing result of the TiDB slow log files. You can use the `CLUSTER_SLOW_QUERY` table the way you do with `SLOW_QUERY`. The table schema of the `CLUSTER_SLOW_QUERY` table differs from that of the `SLOW_QUERY` table in that an `INSTANCE` column is added to `CLUSTER_SLOW_QUERY`. The `INSTANCE` column represents the TiDB node address of the row information on the slow query. For how to use this table to identify problematic statements and improve query performance, see [Slow Query Log Document](/identify-slow-queries.md). +The `CLUSTER_SLOW_QUERY` table provides the slow query information of all nodes in the cluster, which is the parsing result of the TiDB slow log files. You can use the `CLUSTER_SLOW_QUERY` table the way you do with `SLOW_QUERY`. The table schema of the `CLUSTER_SLOW_QUERY` table differs from that of the `SLOW_QUERY` table in that an `INSTANCE` column is added to `CLUSTER_SLOW_QUERY`. The `INSTANCE` column represents the TiDB node address of the row information on the slow query. + + + +For how to use this table to identify problematic statements and improve query performance, see [Slow Query Log Document](/identify-slow-queries.md). + + {{< copyable "sql" >}} diff --git a/information-schema/information-schema-tidb-hot-regions-history.md b/information-schema/information-schema-tidb-hot-regions-history.md index 3d43e0d08afa0..36b38bbe373fb 100644 --- a/information-schema/information-schema-tidb-hot-regions-history.md +++ b/information-schema/information-schema-tidb-hot-regions-history.md @@ -5,7 +5,19 @@ summary: Learn the `TIDB_HOT_REGIONS_HISTORY` information_schema table. # TIDB_HOT_REGIONS_HISTORY -The `TIDB_HOT_REGIONS_HISTORY` table provides information about history hot Regions that are periodically recorded locally by PD. You can specify the record interval by configuring [`hot-regions-write-interval`](/pd-configuration-file.md#hot-regions-write-interval-new-in-v540). The default value is 10 minutes. You can specify the period for reserving history information about hot Regions by configuring [`hot-regions-reserved-days`](/pd-configuration-file.md#hot-regions-reserved-days-new-in-v540). The default value is 7 days. See [PD configuration file description](/pd-configuration-file.md#hot-regions-write-interval-new-in-v540) for details. +The `TIDB_HOT_REGIONS_HISTORY` table provides information about history hot Regions that are periodically recorded locally by PD. + + + +You can specify the record interval by configuring [`hot-regions-write-interval`](/pd-configuration-file.md#hot-regions-write-interval-new-in-v540). The default value is 10 minutes. You can specify the period for reserving history information about hot Regions by configuring [`hot-regions-reserved-days`](/pd-configuration-file.md#hot-regions-reserved-days-new-in-v540). The default value is 7 days. See [PD configuration file description](/pd-configuration-file.md#hot-regions-write-interval-new-in-v540) for details. + + + + + +By default, the record interval is 10 minutes, and the period for reserving history information about hot Regions is 7 days. + + {{< copyable "sql" >}} diff --git a/information-schema/information-schema.md b/information-schema/information-schema.md index 472fd04ddb065..891ada720d75d 100644 --- a/information-schema/information-schema.md +++ b/information-schema/information-schema.md @@ -56,34 +56,34 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | [`CLIENT_ERRORS_SUMMARY_BY_HOST`](/information-schema/client-errors-summary-by-host.md) | Provides a summary of errors and warnings generated by client requests and returned to clients. | | [`CLIENT_ERRORS_SUMMARY_BY_USER`](/information-schema/client-errors-summary-by-user.md) | Provides a summary of errors and warnings generated by clients. | | [`CLIENT_ERRORS_SUMMARY_GLOBAL`](/information-schema/client-errors-summary-global.md) | Provides a summary of errors and warnings generated by clients. | -| [`CLUSTER_CONFIG`](/information-schema/information-schema-cluster-config.md) | Provides details about configuration settings for the entire TiDB cluster. | +| [`CLUSTER_CONFIG`](https://docs.pingcap.com/tidb/stable/information-schema-cluster-config) | Provides details about configuration settings for the entire TiDB cluster. This table is not applicable to TiDB Cloud. | | `CLUSTER_DEADLOCKS` | Provides a cluster-level view of the `DEADLOCKS` table. | -| [`CLUSTER_HARDWARE`](/information-schema/information-schema-cluster-hardware.md) | Provides details on the underlying physical hardware discovered on each TiDB component. | +| [`CLUSTER_HARDWARE`](https://docs.pingcap.com/tidb/stable/information-schema-cluster-hardware) | Provides details on the underlying physical hardware discovered on each TiDB component. This table is not applicable to TiDB Cloud. | | [`CLUSTER_INFO`](/information-schema/information-schema-cluster-info.md) | Provides details on the current cluster topology. | -| [`CLUSTER_LOAD`](/information-schema/information-schema-cluster-load.md) | Provides current load information for TiDB servers in the cluster. | -| [`CLUSTER_LOG`](/information-schema/information-schema-cluster-log.md) | Provides a log for the entire TiDB cluster | +| [`CLUSTER_LOAD`](https://docs.pingcap.com/tidb/stable/information-schema-cluster-load) | Provides current load information for TiDB servers in the cluster. This table is not applicable to TiDB Cloud. | +| [`CLUSTER_LOG`](https://docs.pingcap.com/tidb/stable/information-schema-cluster-log) | Provides a log for the entire TiDB cluster. This table is not applicable to TiDB Cloud. | | `CLUSTER_PROCESSLIST` | Provides a cluster-level view of the `PROCESSLIST` table. | | `CLUSTER_SLOW_QUERY` | Provides a cluster-level view of the `SLOW_QUERY` table. | | `CLUSTER_STATEMENTS_SUMMARY` | Provides a cluster-level view of the `STATEMENTS_SUMMARY` table. | | `CLUSTER_STATEMENTS_SUMMARY_HISTORY` | Provides a cluster-level view of the `STATEMENTS_SUMMARY_HISTORY` table. | | `CLUSTER_TIDB_TRX` | Provides a cluster-level view of the `TIDB_TRX` table. | -| [`CLUSTER_SYSTEMINFO`](/information-schema/information-schema-cluster-systeminfo.md) | Provides details about kernel parameter configuration for servers in the cluster. | +| [`CLUSTER_SYSTEMINFO`](https://docs.pingcap.com/tidb/stable/information-schema-cluster-systeminfo) | Provides details about kernel parameter configuration for servers in the cluster. This table is not applicable to TiDB Cloud. | | [`DATA_LOCK_WAITS`](/information-schema/information-schema-data-lock-waits.md) | Provides the lock-waiting information on the TiKV server. | | [`DDL_JOBS`](/information-schema/information-schema-ddl-jobs.md) | Provides similar output to `ADMIN SHOW DDL JOBS` | | [`DEADLOCKS`](/information-schema/information-schema-deadlocks.md) | Provides the information of several deadlock errors that have recently occurred. | -| [`INSPECTION_RESULT`](/information-schema/information-schema-inspection-result.md) | Triggers internal diagnostics checks. | -| [`INSPECTION_RULES`](/information-schema/information-schema-inspection-rules.md) | A list of internal diagnostic checks performed. | -| [`INSPECTION_SUMMARY`](/information-schema/information-schema-inspection-summary.md) | A summarized report of important monitoring metrics. | -| [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md) | A summary of metrics extracted from Prometheus. | +| [`INSPECTION_RESULT`](https://docs.pingcap.com/tidb/stable/information-schema-inspection-result) | Triggers internal diagnostics checks. This table is not applicable to TiDB Cloud. | +| [`INSPECTION_RULES`](https://docs.pingcap.com/tidb/stable/information-schema-inspection-rules) | A list of internal diagnostic checks performed. This table is not applicable to TiDB Cloud. | +| [`INSPECTION_SUMMARY`](https://docs.pingcap.com/tidb/stable/information-schema-inspection-summary) | A summarized report of important monitoring metrics. This table is not applicable to TiDB Cloud. | +| [`METRICS_SUMMARY`](https://docs.pingcap.com/tidb/stable/information-schema-metrics-summary) | A summary of metrics extracted from Prometheus. This table is not applicable to TiDB Cloud. | | `METRICS_SUMMARY_BY_LABEL` | See `METRICS_SUMMARY` table. | -| [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md) | Provides the PromQL definitions for tables in `METRICS_SCHEMA`. | -| [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) | Provides information on all placement policies. | +| [`METRICS_TABLES`](https://docs.pingcap.com/tidb/stable/information-schema-metrics-tables) | Provides the PromQL definitions for tables in `METRICS_SCHEMA`. This table is not applicable to TiDB Cloud. | +| [`PLACEMENT_POLICIES`](https://docs.pingcap.com/tidb/stable/information-schema-placement-policies) | Provides information on all placement policies. This table is not applicable to TiDB Cloud. | | [`SEQUENCES`](/information-schema/information-schema-sequences.md) | The TiDB implementation of sequences is based on MariaDB. | | [`SLOW_QUERY`](/information-schema/information-schema-slow-query.md) | Provides information on slow queries on the current TiDB server. | | [`STATEMENTS_SUMMARY`](/statement-summary-tables.md) | Similar to performance_schema statement summary in MySQL. | | [`STATEMENTS_SUMMARY_HISTORY`](/statement-summary-tables.md) | Similar to performance_schema statement summary history in MySQL. | | [`TABLE_STORAGE_STATS`](/information-schema/information-schema-table-storage-stats.md) | Provides details about table sizes in storage. | -| [`TIDB_HOT_REGIONS`](/information-schema/information-schema-tidb-hot-regions.md) | Provides statistics about which regions are hot. | +| [`TIDB_HOT_REGIONS`](https://docs.pingcap.com/tidb/stable/information-schema-tidb-hot-regions) | Provides statistics about which regions are hot. This table is not applicable to TiDB Cloud. | | [`TIDB_HOT_REGIONS_HISTORY`](/information-schema/information-schema-tidb-hot-regions-history.md) | Provides history statistics about which Regions are hot. | | [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md) | Provides index information about TiDB tables. | | [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md) | Provides a list of TiDB servers (namely, tidb-server component) | diff --git a/media/tidb-cloud/vpc-peering/vpc-peering-creating-infos.png b/media/tidb-cloud/vpc-peering/vpc-peering-creating-infos.png index e4f5240549cbd..218dd601f7a80 100644 Binary files a/media/tidb-cloud/vpc-peering/vpc-peering-creating-infos.png and b/media/tidb-cloud/vpc-peering/vpc-peering-creating-infos.png differ diff --git a/mysql-compatibility.md b/mysql-compatibility.md index d5615798d4fe7..f8479b8ef95f5 100644 --- a/mysql-compatibility.md +++ b/mysql-compatibility.md @@ -10,14 +10,30 @@ TiDB is highly compatible with the MySQL 5.7 protocol and the common features an However, some features of MySQL are not supported. This could be because there is now a better way to solve the problem (such as XML functions superseded by JSON), or a lack of current demand versus effort required (such as stored procedures and functions). Some features might also be difficult to implement as a distributed system. + + - In addition, TiDB does not support the MySQL replication protocol, but provides specific tools to replicate data with MySQL. - Replicate data from MySQL: [TiDB Data Migration (DM)](/dm/dm-overview.md) is a tool that supports the full data migration and the incremental data replication from MySQL/MariaDB into TiDB. - Replicate data to MySQL: [TiCDC](/ticdc/ticdc-overview.md) is a tool for replicating the incremental data of TiDB by pulling TiKV change logs. TiCDC uses the [MySQL sink](/ticdc/ticdc-overview.md#sink-support) to replicate the incremental data of TiDB to MySQL. + + + + > **Note:** > > This page refers to general differences between MySQL and TiDB. Refer to the dedicated pages for [Security](/security-compatibility-with-mysql.md) and [Pessimistic Transaction Mode](/pessimistic-transaction.md#difference-with-mysql-innodb) compatibility. + + + + +> **Note:** +> +> For information about transaction differences between MySQL and TiDB, see [Pessimistic Transaction Mode](/pessimistic-transaction.md#difference-with-mysql-innodb). + + + ## Unsupported features + Stored procedures and functions @@ -77,14 +93,36 @@ mysql> SELECT _tidb_rowid, id FROM t; 3 rows in set (0.01 sec) ``` + + > **Note:** > > The `AUTO_INCREMENT` attribute might cause hotspot in production environments. See [Troubleshoot HotSpot Issues](/troubleshoot-hot-spot-issues.md) for details. It is recommended to use [`AUTO_RANDOM`](/auto-random.md) instead. + + + + +> **Note:** +> +> The `AUTO_INCREMENT` attribute might cause hotspot in production environments. See [Troubleshoot HotSpot Issues](https://docs.pingcap.com/tidb/stable/troubleshoot-hot-spot-issues#handle-auto-increment-primary-key-hotspot-tables-using-auto_random) for details. It is recommended to use [`AUTO_RANDOM`](/auto-random.md) instead. + + + ### Performance schema + + TiDB uses a combination of [Prometheus and Grafana](/tidb-monitoring-api.md) to store and query the performance monitoring metrics. Performance schema tables return empty results in TiDB. + + + + +To check performance metrics in TiDB Cloud, you can either check the cluster overview page on the TiDB Cloud console or use [third-party monitoring integrations](/tidb-cloud/monitor-tidb-cluster.md#third-party-integrations). Performance schema tables return empty results in TiDB. + + + ### Query Execution Plan The output format, output content, and the privilege setting of Query Execution Plan (`EXPLAIN`/`EXPLAIN FOR`) in TiDB is greatly different from those in MySQL. @@ -155,8 +193,12 @@ For details, see [Compatibility between TiDB local temporary tables and MySQL te For compatibility reasons, TiDB supports the syntax to create tables with alternative storage engines. In implementation, TiDB describes the metadata as the InnoDB storage engine. + + TiDB supports storage engine abstraction similar to MySQL, but you need to specify the storage engine using the [`--store`](/command-line-flags-for-tidb-configuration.md#--store) option when you start the TiDB server. + + ### SQL modes TiDB supports most [SQL modes](/sql-mode.md): diff --git a/mysql-schema.md b/mysql-schema.md index 9cbe3dfc2c670..197f6c9b844b9 100644 --- a/mysql-schema.md +++ b/mysql-schema.md @@ -35,4 +35,9 @@ Currently, the `help_topic` is NULL. ## Miscellaneous system tables - `GLOBAL_VARIABLES`: global system variable table + + + - `tidb`: to record the version information when TiDB executes `bootstrap` + + diff --git a/non-transactional-dml.md b/non-transactional-dml.md index fd3f48645885e..a75fcbec6cbf8 100644 --- a/non-transactional-dml.md +++ b/non-transactional-dml.md @@ -226,8 +226,19 @@ The following are hard restrictions on non-transactional DML statements. If thes - Cannot be used with the `prepare` statement. - `ENUM`, `BIT`, `SET`, `JSON` types are not supported as the dividing columns. - Not supported for [temporary tables](/temporary-tables.md). + + + - [Common Table Expression](/develop/dev-guide-use-common-table-expression.md) is not supported. + + + + +- [Common Table Expression](https://docs.pingcap.com/tidb/stable/dev-guide-use-common-table-expression) is not supported. + + + ## Control batch execution failure Non-transactional DML statements do not satisfy atomicity. Some batches might succeed and some might fail. The system variable [`tidb_nontransactional_ignore_error`](/system-variables.md#tidb_nontransactional_ignore_error-new-in-v610) controls how the non-transactional DML statements handle errors. diff --git a/optimizer-hints.md b/optimizer-hints.md index 809d70182c6cd..d0a4e96c156cb 100644 --- a/optimizer-hints.md +++ b/optimizer-hints.md @@ -366,7 +366,7 @@ WITH CTE1 AS (SELECT * FROM t1), CTE2 AS (WITH CTE3 AS (SELECT /*+ MERGE() */ * > > `MERGE()` is only applicable to simple CTE queries. It is not applicable in the following situations: > -> - [Recursive CTE](/develop/dev-guide-use-common-table-expression.md#recursive-cte) +> - [Recursive CTE](https://docs.pingcap.com/tidb/stable/dev-guide-use-common-table-expression#recursive-cte) > - Subqueries with inlines that cannot be expanded, such as aggregate operators, window functions, and `DISTINCT`. > > When the number of CTE references is too high, the query performance might be lower than the default materialization behavior. diff --git a/pessimistic-transaction.md b/pessimistic-transaction.md index 4735fca3da41d..10b1b2efe5b8e 100644 --- a/pessimistic-transaction.md +++ b/pessimistic-transaction.md @@ -149,6 +149,8 @@ To reduce the overhead of locking, TiKV implements the pipelined locking process * There is a low probability that the transaction commit fails, but it does not affect the correctness of the transactions. + + If the application logic relies on the locking or lock waiting mechanisms, or if you want to guarantee as much as possible the success rate of transaction commits even in the case of TiKV cluster anomalies, you should disable the pipelined locking feature. ![Pipelined pessimistic lock](/media/pessimistic-transaction-pipelining.png) @@ -168,6 +170,14 @@ If the TiKV cluster is v4.0.9 or later, you can also dynamically disable this fe set config tikv pessimistic-txn.pipelined='false'; ``` + + + + +If the application logic relies on the locking or lock waiting mechanisms, or if you want to guarantee as much as possible the success rate of transaction commits even in the case of TiKV cluster anomalies, you can contact PingCAP Support to disable the pipelined locking feature. + + + ## In-memory pessimistic lock In v6.0.0, TiKV introduces the feature of in-memory pessimistic lock. When this feature is enabled, pessimistic locks are usually stored in the memory of the Region leader only, and are not persisted to disk or replicated through Raft to other replicas. This feature can greatly reduce the overhead of acquiring pessimistic locks and improve the throughput of pessimistic transactions. diff --git a/read-historical-data.md b/read-historical-data.md index 340deaf2b28a1..6fbe413b6656c 100644 --- a/read-historical-data.md +++ b/read-historical-data.md @@ -175,4 +175,14 @@ To restore data from an older version, you can use one of the following methods: - For simple cases, use `SELECT` after setting the `tidb_snapshot` variable and copy-paste the output, or use `SELECT ... INTO LOCAL OUTFLE` and use `LOAD DATA` to import the data later on. + + - Use [Dumpling](/dumpling-overview.md#export-historical-data-snapshot-of-tidb) to export a historical snapshot. Dumpling performs well in exporting larger sets of data. + + + + + +- Use [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview) to export a historical snapshot. Dumpling performs well in exporting larger sets of data. + + \ No newline at end of file diff --git a/security-compatibility-with-mysql.md b/security-compatibility-with-mysql.md index dfd0d5ea23e27..b069d90e8e973 100644 --- a/security-compatibility-with-mysql.md +++ b/security-compatibility-with-mysql.md @@ -19,8 +19,18 @@ TiDB supports multiple authentication methods. These methods can be specified on You can use one of the following supported authentication methods in the table. To specify a default method that the server advertises when the client-server connection is being established, set the [`default_authentication_plugin`](/system-variables.md#default_authentication_plugin) variable. + + The support for TLS authentication is configured differently. For detailed information, see [Enable TLS between TiDB Clients and Servers](/enable-tls-between-clients-and-servers.md). + + + + +The support for TLS authentication is configured differently. For detailed information, see [Enable TLS between TiDB Clients and Servers](https://docs.pingcap.com/tidb/stable/enable-tls-between-clients-and-servers). + + + | Authentication Method | Supported | | :------------------------| :--------------- | | `mysql_native_password` | Yes | @@ -33,5 +43,3 @@ The support for TLS authentication is configured differently. For detailed infor | ed25519 (MariaDB) | No | | GSSAPI (MariaDB) | No | | FIDO | No | - -[TLS Certificates]: /enable-tls-between-clients-and-servers.md diff --git a/sql-plan-management.md b/sql-plan-management.md index 700dc254e8858..5cfeb53f14605 100644 --- a/sql-plan-management.md +++ b/sql-plan-management.md @@ -359,7 +359,7 @@ Insert filtering conditions into the system table `mysql.capture_plan_baselines_ | **Dimension name** | **Description** | Remarks | | :----------- | :----------------------------------------------------------- | ------------------------------------------------------------ | -| table | Filter by table name. Each filtering rule is in the `db.table` format. The supported filtering syntax includes [Plain table names](/table-filter.md#plain-table-names) and [Wildcards](/table-filter.md#wildcards). | Case insensitive. If a table name contains illegal characters, the log returns a warning message `[sql-bind] failed to load mysql.capture_plan_baselines_blacklist`. | +| table | Filter by table name. Each filtering rule is in the `db.table` format. The supported filtering syntax includes [Plain table names](https://docs.pingcap.com/tidb/stable/table-filter#plain-table-names) and [Wildcards](https://docs.pingcap.com/tidb/stable/table-filter#wildcards). | Case insensitive. If a table name contains illegal characters, the log returns a warning message `[sql-bind] failed to load mysql.capture_plan_baselines_blacklist`. | | frequency | Filter by frequency. SQL statements executed more than once are captured by default. You can set a high frequency to capture statements that are frequently executed. | Setting frequency to a value smaller than 1 is considered illegal, and the log returns a warning message `[sql-bind] frequency threshold is less than 1, ignore it`. If multiple frequency filter rules are inserted, the value with the highest frequency prevails. | | user | Filter by user name. Statements executed by blocklisted users are not captured. | If multiple users execute the same statement and their user names are all in the blocklist, this statement is not captured. | diff --git a/sql-prepared-plan-cache.md b/sql-prepared-plan-cache.md index e993fbf8d39df..17939ca46e240 100644 --- a/sql-prepared-plan-cache.md +++ b/sql-prepared-plan-cache.md @@ -53,7 +53,7 @@ There are several points worth noting about execution plan caching and query per - Considering that the parameters of `Execute` are different, the execution plan cache prohibits some aggressive query optimization methods that are closely related to specific parameter values to ensure adaptability. This causes that the query plan may not be optimal for certain parameter values. For example, the filter condition of the query is `where a > ? And a < ?`, the parameters of the first `Execute` statement are `2` and `1` respectively. Considering that these two parameters maybe be `1` and `2` in the next execution time, the optimizer does not generate the optimal `TableDual` execution plan that is specific to current parameter values; - If cache invalidation and elimination are not considered, an execution plan cache is applied to various parameter values, which in theory also results in non-optimal execution plans for certain values. For example, if the filter condition is `where a < ?` and the parameter value used for the first execution is `1`, then the optimizer generates the optimal `IndexScan` execution plan and puts it into the cache. In the subsequent executions, if the value becomes `10000`, the `TableScan` plan might be the better one. But due to the execution plan cache, the previously generated `IndexScan` is used for execution. Therefore, the execution plan cache is more suitable for application scenarios where the query is simple (the ratio of compilation is high) and the execution plan is relatively fixed. -Since v6.1.0 the execution plan cache is enabled by default. You can control prepared plan cache via the system variable [`tidb_enable_prepared_plan_cache`](/system-variables.md#tidb_enable_prepared_plan_cache-new-in-v610). +Since v6.1.0, the execution plan cache is enabled by default. You can control prepared plan cache via the system variable [`tidb_enable_prepared_plan_cache`](/system-variables.md#tidb_enable_prepared_plan_cache-new-in-v610). > **Note:** > @@ -138,8 +138,18 @@ When the unused memory of the TiDB server is less than a certain threshold, the You can control the threshold by configuring the system variable `tidb_prepared_plan_cache_memory_guard_ratio`. The threshold is 0.1 by default, which means when the unused memory of the TiDB server is less than 10% of the total memory (90% of the memory is used), the memory protection mechanism is triggered. + + Due to memory limit, plan cache might be missed sometimes. You can check the status by viewing the [`Plan Cache Miss OPS` metric](/grafana-tidb-dashboard.md) in the Grafana dashboard. + + + + +Due to memory limit, plan cache might be missed sometimes. + + + ## Clear execution plan cache You can clear execution plan cache by executing the `ADMIN FLUSH [SESSION | INSTANCE] PLAN_CACHE` statement. diff --git a/sql-statements/sql-statement-admin.md b/sql-statements/sql-statement-admin.md index 16f7702ffd701..8c471c304c31f 100644 --- a/sql-statements/sql-statement-admin.md +++ b/sql-statements/sql-statement-admin.md @@ -89,6 +89,14 @@ The above statement is used to reload SQL Plan binding information. ## `ADMIN REPAIR` statement + + +> **Note:** +> +> This TiDB statement is not applicable to TiDB Cloud. + + + To overwrite the metadata of the stored table in an untrusted way in extreme cases, use `ADMIN REPAIR TABLE`: {{< copyable "sql" >}} @@ -97,8 +105,12 @@ To overwrite the metadata of the stored table in an untrusted way in extreme cas ADMIN REPAIR TABLE tbl_name CREATE TABLE STATEMENT; ``` + + Here "untrusted" means that you need to manually ensure that the metadata of the original table can be covered by the `CREATE TABLE STATEMENT` operation. To use this `REPAIR` statement, enable the [`repair-mode`](/tidb-configuration-file.md#repair-mode) configuration item, and make sure that the tables to be repaired are listed in the [`repair-table-list`](/tidb-configuration-file.md#repair-table-list). + + ## `ADMIN SHOW SLOW` statement {{< copyable "sql" >}} @@ -113,8 +125,12 @@ ADMIN SHOW SLOW RECENT N; ADMIN SHOW SLOW TOP [INTERNAL | ALL] N; ``` + + For details, refer to [admin show slow statement](/identify-slow-queries.md#admin-show-slow-command) + + ## Synopsis ```ebnf+diagram diff --git a/sql-statements/sql-statement-alter-table-compact.md b/sql-statements/sql-statement-alter-table-compact.md index d8348e45846d9..37e50146e4d32 100644 --- a/sql-statements/sql-statement-alter-table-compact.md +++ b/sql-statements/sql-statement-alter-table-compact.md @@ -57,8 +57,12 @@ The `ALTER TABLE ... COMPACT` statement compacts all replicas in a table simulta To avoid a significant impact on online business, each TiFlash instance only compacts data in one table at a time by default (except for the compaction triggered in the background). This means that if you execute the `ALTER TABLE ... COMPACT` statement on multiple tables at the same time, their executions will be queued on the same TiFlash instance, rather than being executed simultaneously. + + To obtain greater table-level concurrency with higher resource usage, you can modify the TiFlash configuration [`manual_compact_pool_size`](/tiflash/tiflash-configuration.md). For example, when `manual_compact_pool_size` is set to 2, compaction for 2 tables can be processed simultaneously. + + ## MySQL compatibility The `ALTER TABLE ... COMPACT` syntax is TiDB specific, which is an extension to the standard SQL syntax. Although there is no equivalent MySQL syntax, you can still execute this statement by using MySQL clients or various database drivers that comply with the MySQL protocol. diff --git a/sql-statements/sql-statement-alter-user.md b/sql-statements/sql-statement-alter-user.md index 293680864cc3f..c8522eaf1d82d 100644 --- a/sql-statements/sql-statement-alter-user.md +++ b/sql-statements/sql-statement-alter-user.md @@ -59,7 +59,12 @@ mysql> SHOW CREATE USER 'newuser'; ## See also + + * [Security Compatibility with MySQL](/security-compatibility-with-mysql.md) + + + * [CREATE USER](/sql-statements/sql-statement-create-user.md) * [DROP USER](/sql-statements/sql-statement-drop-user.md) * [SHOW CREATE USER](/sql-statements/sql-statement-show-create-user.md) diff --git a/sql-statements/sql-statement-backup.md b/sql-statements/sql-statement-backup.md index b71a9dcfb431f..a0382e397610e 100644 --- a/sql-statements/sql-statement-backup.md +++ b/sql-statements/sql-statement-backup.md @@ -8,7 +8,7 @@ aliases: ['/docs/dev/sql-statements/sql-statement-backup/'] This statement is used to perform a distributed backup of the TiDB cluster. -The `BACKUP` statement uses the same engine as the [BR tool](/br/backup-and-restore-use-cases.md) does, except that the backup process is driven by TiDB itself rather than a separate BR tool. All benefits and warnings of BR also apply in this statement. +The `BACKUP` statement uses the same engine as the [BR tool](/br/backup-and-restore-overview.md) does, except that the backup process is driven by TiDB itself rather than a separate BR tool. All benefits and warnings of BR also apply in this statement. Executing `BACKUP` requires either the `BACKUP_ADMIN` or `SUPER` privilege. Additionally, both the TiDB node executing the backup and all TiKV nodes in the cluster must have read or write permission to the destination. Local storage (storage paths starting with `local://`) is not permitted when [Security Enhanced Mode](/system-variables.md#tidb_enable_enhanced_security) is enabled. diff --git a/sql-statements/sql-statement-create-index.md b/sql-statements/sql-statement-create-index.md index b22761c4a2765..092e38d9b29ce 100644 --- a/sql-statements/sql-statement-create-index.md +++ b/sql-statements/sql-statement-create-index.md @@ -130,9 +130,9 @@ You can also specify the expression index when you create the table: CREATE TABLE t1(col1 char(10), col2 char(10), index((lower(col1)))); ``` -> **Note** +> **Note:** > -> The expression in an expression index must be surrounded by '(' and ')'. Otherwise, a syntax error is reported. +> The expression in an expression index must be surrounded by `(` and `)`. Otherwise, a syntax error is reported. You can drop an expression index in the same way as dropping an ordinary index: @@ -142,30 +142,36 @@ You can drop an expression index in the same way as dropping an ordinary index: DROP INDEX idx1 ON t1; ``` +Expression index involves various kinds of expressions. To ensure correctness, only some fully tested functions are allowed for creating an expression index. This means that only these functions are allowed in expressions in a production environment. You can get these functions by querying `tidb_allow_function_for_expression_index` variable. + +{{< copyable "sql" >}} + +```sql +mysql> select @@tidb_allow_function_for_expression_index; ++--------------------------------------------+ +| @@tidb_allow_function_for_expression_index | ++--------------------------------------------+ +| lower, md5, reverse, upper, vitess_hash | ++--------------------------------------------+ +1 row in set (0.00 sec) +``` + +For the functions that are not included in the returned result above, those functions are not fully tested and not recommended for a production environment, which can be seen as experimental. Other expressions such as operators, `cast`, and `case when` are also seen as experimental and not recommended for production. + + + +If you still want to use those expressions, you can make the following configuration in the [TiDB configuration file](/tidb-configuration-file.md#allow-expression-index-new-in-v400): + +{{< copyable "sql" >}} + +```sql +allow-expression-index = true +``` + + + > **Note:** > -> Expression index involves various kinds of expressions. To ensure correctness, only some fully tested functions are allowed for creating an expression index. This means that only these functions are allowed in expressions in a production environment. You can get these functions by querying `tidb_allow_function_for_expression_index` variable. In future versions, more functions might be added to the list. -> -> {{< copyable "sql" >}} -> -> ```sql -> mysql> select @@tidb_allow_function_for_expression_index; -> +--------------------------------------------+ -> | @@tidb_allow_function_for_expression_index | -> +--------------------------------------------+ -> | lower, md5, reverse, upper, vitess_hash | -> +--------------------------------------------+ -> 1 row in set (0.00 sec) -> ``` -> -> For the functions that are not included in the returned result above, those functions are not fully tested and not recommended for a production environment, which can be seen as experimental. Other expressions such as operators, `cast`, and `case when` are also seen as experimental and not recommended for production. However, if you still want to use those expressions, you can make the following configuration in the [TiDB configuration file](/tidb-configuration-file.md#allow-expression-index-new-in-v400): -> -> {{< copyable "sql" >}} -> -> ```sql -> allow-expression-index = true -> ``` -> > An expression index cannot be created on a primary key. > > The expression in an expression index cannot contain the following content: diff --git a/sql-statements/sql-statement-create-role.md b/sql-statements/sql-statement-create-role.md index 4f8f5899fefdf..985dcf4c3b916 100644 --- a/sql-statements/sql-statement-create-role.md +++ b/sql-statements/sql-statement-create-role.md @@ -164,4 +164,9 @@ This statement is understood to be fully compatible with roles, which are a feat * [`REVOKE `](/sql-statements/sql-statement-revoke-role.md) * [SET ROLE](/sql-statements/sql-statement-set-role.md) * [SET DEFAULT ROLE](/sql-statements/sql-statement-set-default-role.md) + + + * [Role-Based Access Control](/role-based-access-control.md) + + diff --git a/sql-statements/sql-statement-create-table.md b/sql-statements/sql-statement-create-table.md index 0ba2d42c4bd34..1018418ad26d4 100644 --- a/sql-statements/sql-statement-create-table.md +++ b/sql-statements/sql-statement-create-table.md @@ -100,10 +100,22 @@ The following *table_options* are supported. Other options such as `AVG_ROW_LENG | `CHARACTER SET` | To specify the [character set](/character-set-and-collation.md) for the table | `CHARACTER SET` = 'utf8mb4' | | `COMMENT` | The comment information | `COMMENT` = 'comment info' | + + > **Note:** > > The `split-table` configuration option is enabled by default. When it is enabled, a separate Region is created for each newly created table. For details, see [TiDB configuration file](/tidb-configuration-file.md). + + + + +> **Note:** +> +> TiDB creates a separate Region for each newly created table. + + + ## Examples Creating a simple table and inserting one row: @@ -190,7 +202,19 @@ mysql> DESC t1; * All of the data types except spatial types are supported. * `FULLTEXT`, `HASH` and `SPATIAL` indexes are not supported. + + + * For compatibility, the `index_col_name` attribute supports the length option with a maximum length limit of 3072 bytes by default. The length limit can be changed through the `max-index-length` configuration option. For details, see [TiDB configuration file](/tidb-configuration-file.md#max-index-length). + + + + + +* For compatibility, the `index_col_name` attribute supports the length option with a maximum length limit of 3072 bytes. + + + * The `[ASC | DESC]` in `index_col_name` is currently parsed but ignored (MySQL 5.7 compatible behavior). * The `COMMENT` attribute does not support the `WITH PARSER` option. * TiDB supports at most 512 columns in a single table. The corresponding number limit in InnoDB is 1017, and the hard limit in MySQL is 4096. For details, see [TiDB Limitations](/tidb-limitations.md). diff --git a/sql-statements/sql-statement-create-user.md b/sql-statements/sql-statement-create-user.md index 465177d6dc56a..e0d6ab6066918 100644 --- a/sql-statements/sql-statement-create-user.md +++ b/sql-statements/sql-statement-create-user.md @@ -72,8 +72,13 @@ The following `CREATE USER` options are not yet supported by TiDB, and will be p ## See also + + * [Security Compatibility with MySQL](/security-compatibility-with-mysql.md) +* [Privilege Management](/privilege-management.md) + + + * [DROP USER](/sql-statements/sql-statement-drop-user.md) * [SHOW CREATE USER](/sql-statements/sql-statement-show-create-user.md) * [ALTER USER](/sql-statements/sql-statement-alter-user.md) -* [Privilege Management](/privilege-management.md) diff --git a/sql-statements/sql-statement-drop-role.md b/sql-statements/sql-statement-drop-role.md index e93dc37894358..ccd1032f3a4f0 100644 --- a/sql-statements/sql-statement-drop-role.md +++ b/sql-statements/sql-statement-drop-role.md @@ -209,4 +209,9 @@ This statement is understood to be fully compatible with roles, which are a feat * [`REVOKE `](/sql-statements/sql-statement-revoke-role.md) * [SET ROLE](/sql-statements/sql-statement-set-role.md) * [SET DEFAULT ROLE](/sql-statements/sql-statement-set-default-role.md) + + + * [Role-Based Access Control](/role-based-access-control.md) + + diff --git a/sql-statements/sql-statement-drop-user.md b/sql-statements/sql-statement-drop-user.md index dd9b4ae9c024b..e87ae43ac2324 100644 --- a/sql-statements/sql-statement-drop-user.md +++ b/sql-statements/sql-statement-drop-user.md @@ -69,4 +69,9 @@ ERROR 1141 (42000): There is no such grant defined for user 'newuser' on host '% * [CREATE USER](/sql-statements/sql-statement-create-user.md) * [ALTER USER](/sql-statements/sql-statement-alter-user.md) * [SHOW CREATE USER](/sql-statements/sql-statement-show-create-user.md) + + + * [Privilege Management](/privilege-management.md) + + diff --git a/sql-statements/sql-statement-explain-analyze.md b/sql-statements/sql-statement-explain-analyze.md index ad11cef8e5445..c1acd8ffd26fe 100644 --- a/sql-statements/sql-statement-explain-analyze.md +++ b/sql-statements/sql-statement-explain-analyze.md @@ -128,7 +128,7 @@ cop_task: {num: 6, max: 1.07587ms, min: 844.312µs, avg: 919.601µs, p95: 1.0758 - `max`, `min`, `avg`, `p95`: The maximum, minimum, average, and P95 values of the execution time consumed for executing cop tasks. - `max_proc_keys` and `p95_proc_keys`: The maximum and P95 key-values scanned by TiKV in all cop tasks. If the difference between the maximum value and the P95 value is large, the data distribution might be imbalanced. - `rpc_num`, `rpc_time`: The total number and total time consumed for `Cop` RPC requests sent to TiKV. - - `copr_cache_hit_ratio`: The hit rate of Coprocessor Cache for `cop` task requests. See [Coprocessor Cache Configuration](/tidb-configuration-file.md) for details. + - `copr_cache_hit_ratio`: The hit rate of Coprocessor Cache for `cop` task requests. - `backoff`: Contains different types of backoff and the total waiting time of backoff. ### Insert diff --git a/sql-statements/sql-statement-flush-privileges.md b/sql-statements/sql-statement-flush-privileges.md index 6bad67d7acd4f..aa1ece3d1117f 100644 --- a/sql-statements/sql-statement-flush-privileges.md +++ b/sql-statements/sql-statement-flush-privileges.md @@ -39,4 +39,10 @@ This statement is understood to be fully compatible with MySQL. Any compatibilit ## See also +* [SHOW GRANTS](/sql-statements/sql-statement-show-grants.md) + + + * [Privilege Management](/privilege-management.md) + + diff --git a/sql-statements/sql-statement-grant-privileges.md b/sql-statements/sql-statement-grant-privileges.md index 98ad080f6eef4..2c97a947e47b5 100644 --- a/sql-statements/sql-statement-grant-privileges.md +++ b/sql-statements/sql-statement-grant-privileges.md @@ -87,4 +87,9 @@ mysql> SHOW GRANTS FOR 'newuser'; * [`GRANT `](/sql-statements/sql-statement-grant-role.md) * [`REVOKE `](/sql-statements/sql-statement-revoke-privileges.md) * [SHOW GRANTS](/sql-statements/sql-statement-show-grants.md) + + + * [Privilege Management](/privilege-management.md) + + diff --git a/sql-statements/sql-statement-grant-role.md b/sql-statements/sql-statement-grant-role.md index a2cac8ae67dc4..5f1dadbe948ab 100644 --- a/sql-statements/sql-statement-grant-role.md +++ b/sql-statements/sql-statement-grant-role.md @@ -165,4 +165,9 @@ This statement is understood to be fully compatible with roles, which are a feat * [`REVOKE `](/sql-statements/sql-statement-revoke-role.md) * [SET ROLE](/sql-statements/sql-statement-set-role.md) * [SET DEFAULT ROLE](/sql-statements/sql-statement-set-default-role.md) -* [Role-Based Access Control](/role-based-access-control.md) \ No newline at end of file + + + +* [Role-Based Access Control](/role-based-access-control.md) + + diff --git a/sql-statements/sql-statement-kill.md b/sql-statements/sql-statement-kill.md index 3aa775d0b034f..a1858613ea1c3 100644 --- a/sql-statements/sql-statement-kill.md +++ b/sql-statements/sql-statement-kill.md @@ -51,12 +51,30 @@ Query OK, 0 rows affected (0.00 sec) ## Behavior change descriptions -Starting from v6.1.0, TiDB supports the Global Kill feature, which is enabled by default and controlled by the [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) configuration. When the Global Kill feature is enabled, both `KILL` and `KILL TIDB` statements can terminate queries or connections across instances so you do not need to worry about erroneously terminating queries or connections. When you use a client to connect to any TiDB instance and execute the `KILL` or `KILL TIDB` statement, the statement will be forwarded to the target TiDB instance. If there is a proxy between the client and the TiDB cluster, the `KILL` and `KILL TIDB` statements will also be forwarded to the target TiDB instance for execution. + + +Starting from v6.1.0, TiDB supports the Global Kill feature, which is enabled by default and controlled by the [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) configuration. + + + + + +Starting from v6.1.0, TiDB supports the Global Kill feature, which is enabled by default. + + + +When the Global Kill feature is enabled, both `KILL` and `KILL TIDB` statements can terminate queries or connections across instances so you do not need to worry about erroneously terminating queries or connections. When you use a client to connect to any TiDB instance and execute the `KILL` or `KILL TIDB` statement, the statement will be forwarded to the target TiDB instance. If there is a proxy between the client and the TiDB cluster, the `KILL` and `KILL TIDB` statements will also be forwarded to the target TiDB instance for execution. If the Global Kill feature is not enabled or you are using a TiDB version earlier than v6.1.0, note the following: - By default, `KILL` is not compatible with MySQL. This helps prevent against a case of a connection being terminated by a wrong TiDB server, because it is common to place multiple TiDB servers behind a load balancer. To terminate other connections on the currently connected TiDB instance, you need to add the `TIDB` suffix explicitly by executing the `KILL TIDB` statement. + + + - It is **STRONGLY NOT RECOMMENDED** to set [`compatible-kill-query = true`](/tidb-configuration-file.md#compatible-kill-query) in your configuration file UNLESS you are certain that clients will be always connected to the same TiDB instance. This is because pressing ctrl+c in the default MySQL client opens a new connection in which `KILL` is executed. If there is a proxy between the client and the TiDB cluster, the new connection might be routed to a different TiDB instance, which possibly kills a different session by mistake. + + + - The `KILL TIDB` statement is a TiDB extension. The feature of this statement is similar to the MySQL `KILL [CONNECTION|QUERY]` command and the MySQL command line ctrl+c. It is safe to use `KILL TIDB` on the same TiDB instance. ## See also diff --git a/sql-statements/sql-statement-recover-table.md b/sql-statements/sql-statement-recover-table.md index 3069a7d4152ea..94714dd83b3d6 100644 --- a/sql-statements/sql-statement-recover-table.md +++ b/sql-statements/sql-statement-recover-table.md @@ -48,7 +48,9 @@ NUM ::= intLit > - TiDB 3.0 is used both in the upstream cluster and the downstream cluster. > - The GC life time of the secondary cluster must be longer than that of the primary cluster. However, as latency occurs during data replication between upstream and downstream databases, data recovery might fail in the downstream. -### Troubleshoot errors during TiDB Binlog replication + + +**Troubleshoot errors during TiDB Binlog replication** When you use `RECOVER TABLE` in the upstream TiDB during TiDB Binlog replication, TiDB Binlog might be interrupted in the following three situations: @@ -60,6 +62,8 @@ When you use `RECOVER TABLE` in the upstream TiDB during TiDB Binlog replication For the above three situations, you can resume data replication from TiDB Binlog with a [full import of the deleted table](/ecosystem-tool-user-guide.md#backup-and-restore). + + ## Examples + Recover the deleted table according to the table name. diff --git a/sql-statements/sql-statement-restore.md b/sql-statements/sql-statement-restore.md index ae00fdae4c3b9..6b44946915bac 100644 --- a/sql-statements/sql-statement-restore.md +++ b/sql-statements/sql-statement-restore.md @@ -8,7 +8,7 @@ aliases: ['/docs/dev/sql-statements/sql-statement-restore/'] This statement performs a distributed restore from a backup archive previously produced by a [`BACKUP` statement](/sql-statements/sql-statement-backup.md). -The `RESTORE` statement uses the same engine as the [BR tool](/br/backup-and-restore-use-cases.md), except that the restore process is driven by TiDB itself rather than a separate BR tool. All benefits and caveats of BR also apply here. In particular, **`RESTORE` is currently not ACID-compliant**. Before running `RESTORE`, ensure that the following requirements are met: +The `RESTORE` statement uses the same engine as the [BR tool](/br/backup-and-restore-overview.md), except that the restore process is driven by TiDB itself rather than a separate BR tool. All benefits and caveats of BR also apply here. In particular, **`RESTORE` is currently not ACID-compliant**. Before running `RESTORE`, ensure that the following requirements are met: * The cluster is "offline", and the current TiDB session is the only active SQL connection to access all tables being restored. * When a full restore is being performed, the tables being restored should not already exist, because existing data might be overridden and causes inconsistency between the data and indices. diff --git a/sql-statements/sql-statement-revoke-privileges.md b/sql-statements/sql-statement-revoke-privileges.md index 4bd176a5aad42..3b57100c089dd 100644 --- a/sql-statements/sql-statement-revoke-privileges.md +++ b/sql-statements/sql-statement-revoke-privileges.md @@ -101,4 +101,9 @@ This statement is understood to be fully compatible with MySQL. Any compatibilit * [`GRANT `](/sql-statements/sql-statement-grant-privileges.md) * [SHOW GRANTS](/sql-statements/sql-statement-show-grants.md) + + + * [Privilege Management](/privilege-management.md) + + diff --git a/sql-statements/sql-statement-revoke-role.md b/sql-statements/sql-statement-revoke-role.md index 75878257422ed..12264ae44a4fd 100644 --- a/sql-statements/sql-statement-revoke-role.md +++ b/sql-statements/sql-statement-revoke-role.md @@ -207,4 +207,9 @@ This statement is understood to be fully compatible with roles, which are a feat * [`GRANT `](/sql-statements/sql-statement-grant-role.md) * [SET ROLE](/sql-statements/sql-statement-set-role.md) * [SET DEFAULT ROLE](/sql-statements/sql-statement-set-default-role.md) + + + * [Role-Based Access Control](/role-based-access-control.md) + + diff --git a/sql-statements/sql-statement-select.md b/sql-statements/sql-statement-select.md index e5fb943e88bfe..68195290ec12b 100644 --- a/sql-statements/sql-statement-select.md +++ b/sql-statements/sql-statement-select.md @@ -14,6 +14,14 @@ The `SELECT` statement is used to read data from TiDB. ![SelectStmt](/media/sqlgram/SelectStmt.png) + + +> **Note:** +> +> The `SELECT ... INTO OUTFILE` statement is not supported by TiDB Cloud. + + + **FromDual:** ![FromDual](/media/sqlgram/FromDual.png) diff --git a/sql-statements/sql-statement-set-default-role.md b/sql-statements/sql-statement-set-default-role.md index 2c3af1b48d3c6..ba56a6bb7af90 100644 --- a/sql-statements/sql-statement-set-default-role.md +++ b/sql-statements/sql-statement-set-default-role.md @@ -176,4 +176,9 @@ This statement is understood to be fully compatible with roles, which are a feat * [`GRANT `](/sql-statements/sql-statement-grant-role.md) * [`REVOKE `](/sql-statements/sql-statement-revoke-role.md) * [SET ROLE](/sql-statements/sql-statement-set-role.md) + + + * [Role-Based Access Control](/role-based-access-control.md) + + diff --git a/sql-statements/sql-statement-set-password.md b/sql-statements/sql-statement-set-password.md index 54762391515d4..08904c5240eb5 100644 --- a/sql-statements/sql-statement-set-password.md +++ b/sql-statements/sql-statement-set-password.md @@ -61,4 +61,9 @@ This statement is understood to be fully compatible with MySQL. Any compatibilit ## See also * [CREATE USER](/sql-statements/sql-statement-create-user.md) + + + * [Privilege Management](/privilege-management.md) + + diff --git a/sql-statements/sql-statement-set-role.md b/sql-statements/sql-statement-set-role.md index 0711182781ccd..dee5238082196 100644 --- a/sql-statements/sql-statement-set-role.md +++ b/sql-statements/sql-statement-set-role.md @@ -118,4 +118,9 @@ This statement is understood to be fully compatible with roles, which are a feat * [`GRANT `](/sql-statements/sql-statement-grant-role.md) * [`REVOKE `](/sql-statements/sql-statement-revoke-role.md) * [SET DEFAULT ROLE](/sql-statements/sql-statement-set-default-role.md) + + + * [Role-Based Access Control](/role-based-access-control.md) + + diff --git a/sql-statements/sql-statement-show-table-regions.md b/sql-statements/sql-statement-show-table-regions.md index b1c939f251a9d..263da309bb93c 100644 --- a/sql-statements/sql-statement-show-table-regions.md +++ b/sql-statements/sql-statement-show-table-regions.md @@ -50,7 +50,19 @@ Executing `SHOW TABLE REGIONS` returns the following columns: * `READ_BYTES`: The estimated amount of data read from the Region within one heartbeat cycle. The unit is byte. * `APPROXIMATE_SIZE(MB)`: The estimated amount of data in the Region. The unit is megabytes (MB). * `APPROXIMATE_KEYS`: The estimated number of Keys in the Region. -* `SCHEDULING_CONSTRAINTS`: The [placement policy settings](/placement-rules-in-sql.md) associated with the table or partition to which a Region belongs. + + + +* `SCHEDULING_CONSTRAINTS`: The [placement policy settings](/placement-rules-in-sql.md) associated with the table or partition to which a Region belongs. + + + + + +* `SCHEDULING_CONSTRAINTS`: The placement policy settings associated with the table or partition to which a Region belongs. + + + * `SCHEDULING_STATE`: The scheduling state of the Region which has a placement policy. > **Note:** diff --git a/sql-statements/sql-statement-split-region.md b/sql-statements/sql-statement-split-region.md index 28accb9dfa76f..5e59e509bd6d5 100644 --- a/sql-statements/sql-statement-split-region.md +++ b/sql-statements/sql-statement-split-region.md @@ -429,9 +429,13 @@ region3: [ 2<<61 , 3<<61 ) region4: [ 3<<61 , +inf ) ``` -## Notes + -The Region split by the Split Region statement is controlled by the [Region merge](/best-practices/pd-scheduling-best-practices.md#region-merge) scheduler in PD. To avoid PD re-merging the newly split Region soon after, you need to [dynamically modify](/pd-control.md) configuration items related to the Region merge feature. +> **Note:** +> +> The Region split by the Split Region statement is controlled by the [Region merge](/best-practices/pd-scheduling-best-practices.md#region-merge) scheduler in PD. To avoid PD re-merging the newly split Region soon after, you need to [dynamically modify](/pd-control.md) configuration items related to the Region merge feature. + + ## MySQL compatibility diff --git a/stale-read.md b/stale-read.md index 2b938e3f3701c..500cc563c9df6 100644 --- a/stale-read.md +++ b/stale-read.md @@ -11,10 +11,20 @@ When you are using Stale Read, TiDB will randomly select a replica for data read ## Scenario examples + + + Scenario one: If a transaction only involves read operations and is tolerant of data staleness to some extent, you can use Stale Read to get historical data. Using Stale Read, TiDB makes the query requests sent to any replica at the expense of some real-time performance, and thus increases the throughput of query executions. Especially in some scenarios where small tables are queried, if strongly consistent reads are used, leader might be concentrated on a certain storage node, causing the query pressure to be concentrated on that node as well. Therefore, that node might become a bottleneck for the whole query. Stale Read, however, can improve the overall query throughput and significantly improve the query performance. + Scenario two: In some scenarios of geo-distributed deployment, if strongly consistent follower reads are used, to make sure that the data read from the Followers is consistent with that stored in the Leader, TiDB requests `Readindex` from different data centers for verification, which increases the access latency for the whole query process. With Stale Read, TiDB accesses the replica in the current data center to read the corresponding data at the expense of some real-time performance, which avoids network latency brought by cross-center connection and reduces the access latency for the entire query. For more information, see [Local Read under Three Data Centers Deployment](/best-practices/three-dc-local-read.md). + + + + +If a transaction only involves read operations and is tolerant of data staleness to some extent, you can use Stale Read to get historical data. Using Stale Read, TiDB makes the query requests sent to any replica at the expense of some real-time performance, and thus increases the throughput of query executions. Especially in some scenarios where small tables are queried, if strongly consistent reads are used, leader might be concentrated on a certain storage node, causing the query pressure to be concentrated on that node as well. Therefore, that node might become a bottleneck for the whole query. Stale Read, however, can improve the overall query throughput and significantly improve the query performance. + + + ## Usages TiDB provides the methods of performing Stale Read at the statement level and the session level as follows: diff --git a/statistics.md b/statistics.md index 6d87aed47bec3..875a81a1aa786 100644 --- a/statistics.md +++ b/statistics.md @@ -6,7 +6,19 @@ aliases: ['/docs/dev/statistics/','/docs/dev/reference/performance/statistics/'] # Introduction to Statistics -TiDB uses statistics to decide [which index to choose](/choose-index.md). The `tidb_analyze_version` variable controls the statistics collected by TiDB. Currently, two versions of statistics are supported: `tidb_analyze_version = 1` and `tidb_analyze_version = 2`. In versions before v5.1.0, the default value of this variable is `1`. In v5.3.0 and later versions, the default value of this variable is `2`. If your cluster is upgraded from a version earlier than v5.3.0 to v5.3.0 or later, the default value of `tidb_analyze_version` does not change. +TiDB uses statistics to decide [which index to choose](/choose-index.md). The `tidb_analyze_version` variable controls the statistics collected by TiDB. Currently, two versions of statistics are supported: `tidb_analyze_version = 1` and `tidb_analyze_version = 2`. + + + +In versions before v5.1.0, the default value of this variable is `1`. In v5.3.0 and later versions, the default value of this variable is `2`. If your cluster is upgraded from a version earlier than v5.3.0 to v5.3.0 or later, the default value of `tidb_analyze_version` does not change. + + + + + +For TiDB Cloud, the default value of this variable is `1`. + + > **Note:** > @@ -113,11 +125,23 @@ You can perform full collection using the following syntax. Before v5.3.0, TiDB uses the reservoir sampling method to collect statistics. Since v5.3.0, the TiDB Version 2 statistics uses the Bernoulli sampling method to collect statistics by default. To re-use the reservoir sampling method, you can use the `WITH NUM SAMPLES` statement. +The current sampling rate is calculated based on an adaptive algorithm. When you can observe the number of rows in a table using [`SHOW STATS_META`](/sql-statements/sql-statement-show-stats-meta.md), you can use this number of rows to calculate the sampling rate corresponding to 100,000 rows. If you cannot observe this number, you can use the `TABLE_KEYS` column in the [`TABLE_STORAGE_STATS`](/information-schema/information-schema-table-storage-stats.md) table as another reference to calculate the sampling rate. + + + > **Note:** > -> The current sampling rate is calculated based on an adaptive algorithm. When you can observe the number of rows in a table using [`SHOW STATS_META`](/sql-statements/sql-statement-show-stats-meta.md), you can use this number of rows to calculate the sampling rate corresponding to 100,000 rows. If you cannot observe this number, you can use the `TABLE_KEYS` column in the [`TABLE_STORAGE_STATS`](/information-schema/information-schema-table-storage-stats.md) table as another reference to calculate the sampling rate. +> Normally, `STATS_META` is more credible than `TABLE_KEYS`. However, after importing data through the methods like [TiDB Lightning](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview), the result of `STATS_META` is `0`. To handle this situation, you can use `TABLE_KEYS` to calculate the sampling rate when the result of `STATS_META` is much smaller than the result of `TABLE_KEYS`. + + + + + +> **Note:** > -> Normally, `STATS_META` is more credible than `TABLE_KEYS`. However, after importing data through the methods like [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md), the result of `STATS_META` is `0`. To handle this situation, you can use `TABLE_KEYS` to calculate the sampling rate when the result of `STATS_META` is much smaller than the result of `TABLE_KEYS`. +> Normally, `STATS_META` is more credible than `TABLE_KEYS`. However, after importing data through TiDB Cloud console (see [Import Sample Data](/tidb-cloud/import-sample-data.md)), the result of `STATS_META` is `0`. To handle this situation, you can use `TABLE_KEYS` to calculate the sampling rate when the result of `STATS_META` is much smaller than the result of `TABLE_KEYS`. + + ##### Collect statistics on some columns @@ -151,8 +175,18 @@ If a table has many columns, collecting statistics on all the columns can cause 1. Set the value of the [`tidb_enable_column_tracking`](/system-variables.md#tidb_enable_column_tracking-new-in-v540) system variable to `ON` to enable TiDB to collect `PREDICATE COLUMNS`. + + After the setting, TiDB writes the `PREDICATE COLUMNS` information to the `mysql.column_stats_usage` system table every 100 * [`stats-lease`](/tidb-configuration-file.md#stats-lease). + + + + + After the setting, TiDB writes the `PREDICATE COLUMNS` information to the `mysql.column_stats_usage` system table every 300 seconds. + + + 2. After the query pattern of your business is relatively stable, collect statistics on `PREDICATE COLUMNS` by using the following syntax: {{< copyable "sql" >}} @@ -360,8 +394,18 @@ Since TiDB v6.0, TiDB supports using the `KILL` statement to terminate an `ANALY 2. Terminate the `ANALYZE` task that is running in the background. - - If [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is `true` (`true` by default), you can execute the `KILL TIDB ${id};` statement directly, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. - - If `enable-global-kill` is `false`, you need to use a client to connect to the TiDB instance that is executing the backend `ANALYZE` task, and then execute the `KILL TIDB ${id};` statement. If you use a client to connect to another TiDB instance, or if there is a proxy between the client and the TiDB cluster, the `KILL` statement cannot terminate the background `ANALYZE` task. + + + - If [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is `true` (`true` by default), you can execute the `KILL TIDB ${id};` statement directly, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. + - If `enable-global-kill` is `false`, you need to use a client to connect to the TiDB instance that is executing the backend `ANALYZE` task, and then execute the `KILL TIDB ${id};` statement. If you use a client to connect to another TiDB instance, or if there is a proxy between the client and the TiDB cluster, the `KILL` statement cannot terminate the background `ANALYZE` task. + + + + + + To terminate the `ANALYZE` task, you can execute the `KILL TIDB ${id};` statement, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. + + For more information on the `KILL` statement, see [`KILL`](/sql-statements/sql-statement-kill.md). @@ -398,7 +442,19 @@ The following are the `ANALYZE` configurations that support persistence: #### Enable ANALYZE configuration persistence -The `ANALYZE` configuration persistence feature is enabled by default (the system variable `tidb_analyze_version` is `2` and `tidb_persist_analyze_options` is `ON` by default). You can use this feature to record the persistence configurations specified in the `ANALYZE` statement when executing the statement manually. Once recorded, the next time TiDB automatically updates statistics or you manually collect statistics without specifying these configuration, TiDB will collect statistics according to the recorded configurations. + + +The `ANALYZE` configuration persistence feature is enabled by default (the system variable `tidb_analyze_version` is `2` and `tidb_persist_analyze_options` is `ON` by default). + + + + + +The `ANALYZE` configuration persistence feature is disabled by default. To enable the feature, ensure that the system variable `tidb_persist_analyze_options` is `ON` and set the system variable `tidb_analyze_version` to `2`. + + + +You can use this feature to record the persistence configurations specified in the `ANALYZE` statement when executing the statement manually. Once recorded, the next time TiDB automatically updates statistics or you manually collect statistics without specifying these configuration, TiDB will collect statistics according to the recorded configurations. When you manually execute the `ANALYZE` statement multiple times with persistence configurations specified, TiDB overwrites the previously recorded persistent configuration using the new configurations specified by the latest `ANALYZE` statement. @@ -648,6 +704,14 @@ The preceding statement only deletes GlobalStats generated in dynamic pruning mo ## Load statistics + + +> **Note:** +> +> This section is not applicable to TiDB Cloud. + + + By default, depending on the size of column statistics, TiDB loads statistics differently as follows: - For statistics that consume small space (such as count, distinctCount, and nullCount), as long as the column data is updated, TiDB automatically loads the corresponding statistics into memory for use in the SQL optimization stage. @@ -661,14 +725,26 @@ Since v5.4.0, TiDB introduces the synchronously loading statistics feature. This The synchronously loading statistics feature is disabled by default. To enable this feature, set the value of the [`tidb_stats_load_sync_wait`](/system-variables.md#tidb_stats_load_sync_wait-new-in-v540) system variable to a timeout (in milliseconds) that SQL optimization can wait for at most to synchronously load complete column statistics. The default value of this variable is `0`, indicating that the feature is disabled. + + After enabling the synchronously loading statistics feature, you can further configure the feature as follows: - To control how TiDB behaves when the waiting time of SQL optimization reaches the timeout, modify the value of the [`tidb_stats_load_pseudo_timeout`](/system-variables.md#tidb_stats_load_pseudo_timeout-new-in-v540) system variable. The default value of this variable is `OFF`, indicating that the SQL execution fails after the timeout. If you set this variable to `ON`, after the timeout, the SQL optimization process does not use any histogram, TopN, or CMSketch statistics on any columns, but gets back to using pseudo statistics. - To specify the maximum number of columns that the synchronously loading statistics feature can process concurrently, modify the value of the [`stats-load-concurrency`](/tidb-configuration-file.md#stats-load-concurrency-new-in-v540) option in the TiDB configuration file. The default value is `5`. - To specify the maximum number of column requests that the synchronously loading statistics feature can cache, modify the value of the [`stats-load-queue-size`](/tidb-configuration-file.md#stats-load-queue-size-new-in-v540) option in the TiDB configuration file. The default value is `1000`. + + ## Import and export statistics + + +> **Note:** +> +> This section is not applicable to TiDB Cloud. + + + ### Export statistics The interface to export statistics is as follows: @@ -717,5 +793,15 @@ LOAD STATS 'file_name' ## See also + + * [LOAD STATS](/sql-statements/sql-statement-load-stats.md) * [DROP STATS](/sql-statements/sql-statement-drop-stats.md) + + + + + +* [SQL Prepare Execution Plan Cache](/sql-prepared-plan-cache.md) + + diff --git a/storage-engine/rocksdb-overview.md b/storage-engine/rocksdb-overview.md index 91d8c8da17d10..8ada230088a7d 100644 --- a/storage-engine/rocksdb-overview.md +++ b/storage-engine/rocksdb-overview.md @@ -47,6 +47,14 @@ Generally speaking, users do not need to change this configuration. If the user ## WriteStall + + +> **Note:** +> +> This section is for TiDB and not applicable to TiDB Cloud. + + + The L0 of RocksDB is different from other levels. The SSTs of L0 are arranged in the order of generation. The key ranges between the SSTs can overlap. Therefore, each SST in L0 must be queried in turn when a query is performed. In order not to affect query performance, WriteStall is triggered to block writing when there are too many files in L0. When encountering a sudden sharp increase in write delay, you can first check the **WriteStall Reason** metric on the Grafana RocksDB KV panel. If it is a WriteStall caused by too many L0 files, you can adjust the following configurations to 64. diff --git a/system-variables.md b/system-variables.md index 385483bc3e254..82485bac76ec2 100644 --- a/system-variables.md +++ b/system-variables.md @@ -186,6 +186,14 @@ mysql> SELECT * FROM t1; ### ddl_slow_threshold + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Default value: `300` @@ -198,9 +206,14 @@ mysql> SELECT * FROM t1; - Persists to cluster: Yes - Type: Enumeration - Default value: `mysql_native_password` -- Possible values: `mysql_native_password`, `caching_sha2_password` -- This variable sets the authentication method that the server advertises when the server-client connection is being established. Possible values for this variable are documented in [Authentication plugin status](/security-compatibility-with-mysql.md#authentication-plugin-status). -- Value options: `mysql_native_password` and `caching_sha2_password`. For more details, see [Authentication plugin status](/security-compatibility-with-mysql.md#authentication-plugin-status). +- Possible values: `mysql_native_password` and `caching_sha2_password` +- This variable sets the authentication method that the server advertises when the server-client connection is being established. + + + +For more possible values of this variable, see [Authentication plugin status](/security-compatibility-with-mysql.md#authentication-plugin-status). + + ### default_week_format @@ -309,7 +322,7 @@ This variable is an alias for `last_insert_id`. - Scope: NONE - Type: Boolean - Default value: `OFF` -- This variable indicates whether [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) is used. +- This variable indicates whether [TiDB Binlog](https://docs.pingcap.com/tidb/stable/tidb-binlog-overview) is used. ### max_allowed_packet @@ -413,7 +426,19 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'max_prepared_stmt_count'; - Persists to cluster: Yes - Type: Boolean - Default value: `OFF` + + + - This variable ensures that all connections to TiDB are either on a local socket, or using TLS. See [Enable TLS between TiDB Clients and Servers](/enable-tls-between-clients-and-servers.md) for additional details. + + + + + +- This variable ensures that all connections to TiDB are either on a local socket, or using TLS. + + + - Setting this variable to `ON` requires you to connect to TiDB from a session that has TLS enabled. This helps prevent lock-out scenarios when TLS is not configured correctly. - This setting was previously a `tidb.toml` option (`security.require-secure-transport`), but changed to a system variable starting from TiDB v6.1.0. @@ -448,7 +473,7 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'max_prepared_stmt_count'; - Persists to cluster: Yes - Type: Boolean - Default value: `ON` -- Indicates whether to write changes to [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) or not. +- Indicates whether to write changes to [TiDB Binlog](https://docs.pingcap.com/tidb/stable/tidb-binlog-overview) or not. > **Note:** > @@ -535,6 +560,14 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a ### tidb_allow_remove_auto_inc New in v2.1.18 and v3.0.4 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION - Type: Boolean - Default value: `OFF` @@ -545,11 +578,34 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer + + + - Default value: `2` + + + + + +- Default value: `1` + + + - Range: `[1, 2]` - Controls how TiDB collects statistics. + + + - In v5.3.0 and later versions, the default value of this variable is `2`. If your cluster is upgraded from a version earlier than v5.3.0 to v5.3.0 or later, the default value of `tidb_analyze_version` does not change. For detailed introduction, see [Introduction to Statistics](/statistics.md). + + + + +- For detailed introduction of this variable, see [Introduction to Statistics](/statistics.md). + + + ### tidb_auto_analyze_end_time - Scope: GLOBAL @@ -658,12 +714,20 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a ### tidb_check_mb4_value_in_utf8 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Boolean - Default value: `ON` - This variable is used to enforce that the `utf8` character set only stores values from the [Basic Multilingual Plane (BMP)](https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane). To store characters outside the BMP, it is recommended to use the `utf8mb4` character set. -- You might need to disable this option when upgrading your cluster from an earlier version of TiDB where the `utf8` checking was more relaxed. For details, see [FAQs After Upgrade](/faq/upgrade-faq.md). +- You might need to disable this option when upgrading your cluster from an earlier version of TiDB where the `utf8` checking was more relaxed. For details, see [FAQs After Upgrade](https://docs.pingcap.com/tidb/stable/upgrade-faq). ### tidb_checksum_table_concurrency @@ -686,6 +750,14 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a ### tidb_config + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION - Default value: "" - This variable is read-only. It is used to obtain the configuration information of the current TiDB server. @@ -761,7 +833,7 @@ Constraint checking is always performed in place for pessimistic transactions (d - Unit: Rows - This variable is used to set the batch size during the `re-organize` phase of the DDL operation. For example, when TiDB executes the `ADD INDEX` operation, the index data needs to backfilled by `tidb_ddl_reorg_worker_cnt` (the number) concurrent workers. Each worker backfills the index data in batches. - If many updating operations such as `UPDATE` and `REPLACE` exist during the `ADD INDEX` operation, a larger batch size indicates a larger probability of transaction conflicts. In this case, you need to adjust the batch size to a smaller value. The minimum value is 32. - - If the transaction conflict does not exist, you can set the batch size to a large value (consider the worker count. See [Interaction Test on Online Workloads and `ADD INDEX` Operations](/benchmark/online-workloads-and-add-index-operations.md) for reference). This can increase the speed of the backfilling data, but the write pressure on TiKV also becomes higher. + - If the transaction conflict does not exist, you can set the batch size to a large value (consider the worker count. See [Interaction Test on Online Workloads and `ADD INDEX` Operations](https://docs.pingcap.com/tidb/stable/online-workloads-and-add-index-operations) for reference). This can increase the speed of the backfilling data, but the write pressure on TiKV also becomes higher. ### tidb_ddl_reorg_priority @@ -806,8 +878,18 @@ Constraint checking is always performed in place for pessimistic transactions (d For more details, see [limits of retry](/optimistic-transaction.md#limits-of-retry). + + This variable only applies to optimistic transactions, not to pessimistic transactions. The number of retries for pessimistic transactions is controlled by [`max_retry_count`](/tidb-configuration-file.md#max-retry-count). + + + + + This variable only applies to optimistic transactions, not to pessimistic transactions. The number of retries for pessimistic transactions is 256. + + + ### tidb_distsql_scan_concurrency - Scope: SESSION | GLOBAL @@ -843,7 +925,7 @@ Constraint checking is always performed in place for pessimistic transactions (d > **Note:** > > - The default value of `ON` only applies to new clusters. if your cluster was upgraded from an earlier version of TiDB, the value `OFF` will be used instead. -> - If you have enabled TiDB Binlog, enabling this variable cannot improve the performance. To improve the performance, it is recommended to use [TiCDC](/ticdc/ticdc-overview.md) instead. +> - If you have enabled TiDB Binlog, enabling this variable cannot improve the performance. To improve the performance, it is recommended to use [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) instead. > - Enabling this parameter only means that one-phase commit becomes an optional mode of transaction commit. In fact, the most suitable mode of transaction commit is determined by TiDB. ### tidb_enable_amend_pessimistic_txn New in v4.0.7 @@ -886,7 +968,7 @@ Constraint checking is always performed in place for pessimistic transactions (d > **Note:** > > - The default value of `ON` only applies to new clusters. if your cluster was upgraded from an earlier version of TiDB, the value `OFF` will be used instead. -> - If you have enabled TiDB Binlog, enabling this variable cannot improve the performance. To improve the performance, it is recommended to use [TiCDC](/ticdc/ticdc-overview.md) instead. +> - If you have enabled TiDB Binlog, enabling this variable cannot improve the performance. To improve the performance, it is recommended to use [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) instead. > - Enabling this parameter only means that Async Commit becomes an optional mode of transaction commit. In fact, the most suitable mode of transaction commit is determined by TiDB. ### tidb_enable_auto_analyze New in v6.1.0 @@ -939,6 +1021,14 @@ Constraint checking is always performed in place for pessimistic transactions (d ### tidb_enable_collect_execution_info + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Boolean @@ -972,8 +1062,21 @@ Constraint checking is always performed in place for pessimistic transactions (d - Scope: NONE - Type: Boolean + + + - Default value: `OFF` - This variable indicates whether the TiDB server you are connected to has the Security Enhanced Mode (SEM) enabled. To change its value, you need to modify the value of `enable-sem` in your TiDB server configuration file and restart the TiDB server. + + + + + +- Default value: `ON` +- This variable is read-only. For TiDB Cloud, the Security Enhanced Mode (SEM) is enabled by default. + + + - SEM is inspired by the design of systems such as [Security-Enhanced Linux](https://en.wikipedia.org/wiki/Security-Enhanced_Linux). It reduces the abilities of users with the MySQL `SUPER` privilege and instead requires `RESTRICTED` fine-grained privileges to be granted as a replacement. These fine-grained privileges include: - `RESTRICTED_TABLES_ADMIN`: The ability to write data to system tables in the `mysql` schema and to see sensitive columns on `information_schema` tables. - `RESTRICTED_STATUS_ADMIN`: The ability to see sensitive variables in the command `SHOW STATUS`. @@ -1032,7 +1135,7 @@ Constraint checking is always performed in place for pessimistic transactions (d - Persists to cluster: Yes - Type: Boolean - Default value: `ON` -- This variable is used to control whether to enable TiDB mutation checker, which is a tool used to check consistency between data and indexes during the execution of DML statements. If the checker returns an error for a statement, TiDB rolls back the execution of the statement. Enabling this variable causes a slight increase in CPU usage. For more information, see [Troubleshoot Inconsistency Between Data and Indexes](/troubleshoot-data-inconsistency-errors.md ). +- This variable is used to control whether to enable TiDB mutation checker, which is a tool used to check consistency between data and indexes during the execution of DML statements. If the checker returns an error for a statement, TiDB rolls back the execution of the statement. Enabling this variable causes a slight increase in CPU usage. For more information, see [Troubleshoot Inconsistency Between Data and Indexes](/troubleshoot-data-inconsistency-errors.md). - For new clusters of v6.0.0 or later versions, the default value is `ON`. For existing clusters that upgrade from versions earlier than v6.0.0, the default value is `OFF`. ### tidb_enable_new_cost_interface New in v6.2.0 @@ -1135,7 +1238,19 @@ Constraint checking is always performed in place for pessimistic transactions (d - Type: Boolean - Default value: `ON` - This variable controls the behavior of the optimizer on using statistics of a table when the statistics are outdated. + + + - The optimizer determines whether the statistics of a table is outdated in this way: since the last time `ANALYZE` is executed on a table to get the statistics, if 80% of the table rows are modified (the modified row count divided by the total row count), the optimizer determines that the statistics of this table is outdated. You can change this ratio using the [`pseudo-estimate-ratio`](/tidb-configuration-file.md#pseudo-estimate-ratio) configuration. + + + + + +- The optimizer determines whether the statistics of a table is outdated in this way: since the last time `ANALYZE` is executed on a table to get the statistics, if 80% of the table rows are modified (the modified row count divided by the total row count), the optimizer determines that the statistics of this table is outdated. + + + - By default (with the variable value `ON`), when the statistics of a table is outdated, the optimizer determines that the statistics of the table is no longer reliable except for the total row count. Then, the optimizer uses the pseudo statistics. If you set the variable value to `OFF`, even if the statistics of a table are outdated, the optimizer still keeps using the statistics. - If the data on a table is frequently modified without executing `ANALYZE` on this table in time, to keep the execution plan stable, you can set the variable value to `OFF`. @@ -1146,10 +1261,29 @@ Constraint checking is always performed in place for pessimistic transactions (d - Type: Boolean - Default value: `ON` - This variable controls whether to enable the dynamic memory control feature for the operator that reads data. By default, this operator enables the maximum number of threads that [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) allows to read data. When the memory usage of a single SQL statement exceeds [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) each time, the operator that reads data stops one thread. + + + - When the operator that reads data has only one thread left and the memory usage of a single SQL statement continues to exceed [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query), this SQL statement triggers other memory control behaviors, such as [spilling data to disk](/tidb-configuration-file.md#oom-use-tmp-storage). + + + + +- When the operator that reads data has only one thread left and the memory usage of a single SQL statement continues to exceed [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query), this SQL statement triggers other memory control behaviors, such as spilling data to disk. + + + ### tidb_enable_slow_log + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Boolean @@ -1204,14 +1338,41 @@ Query OK, 0 rows affected (0.09 sec) ### tidb_enable_telemetry New in v4.0.2 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: Yes - Type: Boolean - Default value: `ON` + + + - This variable is used to dynamically control whether the telemetry collection in TiDB is enabled. By setting the value to `OFF`, the telemetry collection is disabled. If the [`enable-telemetry`](/tidb-configuration-file.md#enable-telemetry-new-in-v402) TiDB configuration item is set to `false` on all TiDB instances, the telemetry collection is always disabled and this system variable will not take effect. See [Telemetry](/telemetry.md) for details. + + + + +- This variable is used to dynamically control whether the telemetry collection in TiDB is enabled. + + + ### tidb_enable_top_sql New in v5.4.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + > **Warning:** > > Currently, Top SQL is an experimental feature. It is not recommended that you use it for production environments. @@ -1220,8 +1381,19 @@ Query OK, 0 rows affected (0.09 sec) - Persists to cluster: Yes - Type: Boolean - Default value: `OFF` + + + - This variable is used to control whether to enable the [Top SQL](/dashboard/top-sql.md) feature. + + + + +- This variable is used to control whether to enable the [Top SQL](https://docs.pingcap.com/tidb/stable/top-sql) feature. + + + ### tidb_enable_tso_follower_proxy New in v5.3.0 - Scope: GLOBAL @@ -1258,7 +1430,13 @@ Query OK, 0 rows affected (0.09 sec) - Scope: SESSION - Type: Boolean - Default value: `OFF` + + + - To change this default value, modify the [`performance.enforce-mpp`](/tidb-configuration-file.md#enforce-mpp) configuration value. + + + - Controls whether to ignore the optimizer's cost estimation and to forcibly use TiFlash's MPP mode for query execution. The value options are as follows: - `0` or `OFF`, which means that the MPP mode is not forcibly used (by default). - `1` or `ON`, which means that the cost estimation is ignored and the MPP mode is forcibly used. Note that this setting only takes effect when `tidb_allow_mpp=true`. @@ -1336,6 +1514,14 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_expensive_query_time_threshold + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Integer @@ -1348,6 +1534,14 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_force_priority + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Default value: `NO_PRIORITY` @@ -1428,15 +1622,44 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Possible values: `PHYSICAL`, `LEGACY` - `LEGACY`: Uses the old way of scanning, that is, disable Green GC. - `PHYSICAL`: Uses the physical scanning method, that is, enable Green GC. + + + - This variable specifies the way of scanning locks in the Resolve Locks step of GC. When the variable value is set to `LEGACY`, TiDB scans locks by Regions. When the value `PHYSICAL` is used, it enables each TiKV node to bypass the Raft layer and directly scan data, which can effectively mitigate the impact of GC wakening up all Regions when the [Hibernate Region](/tikv-configuration-file.md#hibernate-regions) feature is enabled, thus improving the execution speed in the Resolve Locks step. + + + + +- This variable specifies the way of scanning locks in the Resolve Locks step of GC. When the variable value is set to `LEGACY`, TiDB scans locks by Regions. When the value `PHYSICAL` is used, it enables each TiKV node to bypass the Raft layer and directly scan data, which can effectively mitigate the impact of GC wakening up all Regions, thus improving the execution speed in the Resolve Locks step. + + + ### tidb_general_log + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Boolean - Default value: `OFF` + + + +- This variable is used to set whether to record all SQL statements in the log. This feature is disabled by default. If you need to trace all SQL statements when locating issues, enable this feature. + + + + + - This variable is used to set whether to record all SQL statements in the [log](/tidb-configuration-file.md#logfile). This feature is disabled by default. If maintenance personnel needs to trace all SQL statements when locating issues, they can enable this feature. + - To see all records of this feature in the log, query the `"GENERAL_LOG"` string. The following information is recorded: - `conn`: The ID of the current session. - `user`: The current session user. @@ -1448,6 +1671,8 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - `txn_mode`: The transactional mode. Value options are `OPTIMISTIC` and `PESSIMISTIC`. - `sql`: The SQL statement corresponding to the current query. + + ### tidb_guarantee_linearizability New in v5.0 - Scope: SESSION | GLOBAL @@ -1612,7 +1837,18 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Type: Integer - Default value: `0` - Range: `[0, 2147483647]` -- This variable is used to adjust the maximum days of logger on the current TiDB instance. Its value defaults to the value of the [`max-days`](/tidb-configuration-file.md#max-days) configuration in the configuration file. Changing the variable value only affects the current TiDB instance. After TiDB is restarted, the variable value is reset and the configuration value is not affected. + + + +- This variable is used to set the maximum days that the log is retained on the current TiDB instance. Its value defaults to the value of the [`max-days`](/tidb-configuration-file.md#max-days) configuration in the configuration file. Changing the variable value only affects the current TiDB instance. After TiDB is restarted, the variable value is reset and the configuration value is not affected. + + + + + +- This variable is used to set the maximum days that the log is retained on the current TiDB instance. + + ### tidb_low_resolution_tso @@ -1658,7 +1894,19 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Type: Enumeration - Default value: `CANCEL` - Possible values: `CANCEL`, `LOG` + + + - Specifies what operation TiDB performs when a single SQL statement exceeds the memory quota specified by `tidb_mem_quota_query` and cannot be spilled over to disk. See [TiDB Memory Control](/configure-memory-usage.md) for details. + + + + + +- Specifies what operation TiDB performs when a single SQL statement exceeds the memory quota specified by [`tidb_mem_quota_query`](#tidb_mem_quota_query) and cannot be spilled over to disk. + + + - The default value is `CANCEL`, but in TiDB v4.0.2 and earlier versions, the default value is `LOG`. - This setting was previously a `tidb.toml` option (`oom-action`), but changed to a system variable starting from TiDB v6.1.0. @@ -1710,20 +1958,58 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Unit: Bytes - This variable is used to set the threshold value of memory quota for a query. - If the memory quota of a query during execution exceeds the threshold value, TiDB performs the operation designated by `tidb_mem_oom_action`. -- This setting was previously session scoped and used the value of `mem-quota-query` from `tidb.toml` as an initial value. Starting from v6.1.0 `tidb_mem_quota_query` is now a `SESSION | GLOBAL` scoped variable. + + + +- For versions earlier than TiDB v6.1.0, this is a session scope variable and uses the value of `mem-quota-query` from `tidb.toml` as an initial value. Starting from v6.1.0, `tidb_mem_quota_query` is a `SESSION | GLOBAL` scope variable. + + + + + +- For versions earlier than TiDB v6.1.0, this is a session scope variable. Starting from v6.1.0, `tidb_mem_quota_query` is a `SESSION | GLOBAL` scope variable. + + ### tidb_memory_usage_alarm_ratio + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Float - Default value: `0.8` - Range: `[0, 1]` + + + - TiDB triggers an alarm when the percentage of the memory it takes exceeds a certain threshold. For the detailed usage description of this feature, see [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio-new-in-v409). - You can set the initial value of this variable by configuring [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio-new-in-v409). + + + + +- TiDB triggers an alarm when the percentage of the memory it takes exceeds a certain threshold. For the detailed usage description of this feature, see [`memory-usage-alarm-ratio`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file#memory-usage-alarm-ratio-new-in-v409). + + + ### tidb_metric_query_range_duration New in v4.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION - Type: Integer - Default value: `60` @@ -1733,6 +2019,14 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_metric_query_step New in v4.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION - Type: Integer - Default value: `60` @@ -2050,6 +2344,14 @@ explain select * from t where age=5; ### tidb_opt_write_row_id + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION - Default value: `OFF` - This variable is used to control whether to allow `INSERT`, `REPLACE`, and `UPDATE` statements to operate on the `_tidb_rowid` column. This variable can be used only when you import data using TiDB tools. @@ -2071,16 +2373,44 @@ explain select * from t where age=5; ### tidb_placement_mode New in v6.0.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Enumeration - Default value: `STRICT` - Possible values: `STRICT`, `IGNORE` + + + - This variable controls whether DDL statements ignore the [placement rules specified in SQL](/placement-rules-in-sql.md). When the variable value is `IGNORE`, all placement rule options are ignored. + + + + + +- This variable controls whether DDL statements ignore the [placement rules specified in SQL](https://docs.pingcap.com/tidb/stable/placement-rules-in-sql). When the variable value is `IGNORE`, all placement rule options are ignored. + + + - It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if invalid placement rules are assigned. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. ### tidb_pprof_sql_cpu New in v4.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Integer @@ -2168,6 +2498,14 @@ explain select * from t where age=5; ### tidb_record_plan_in_slow_log + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Boolean @@ -2176,6 +2514,14 @@ explain select * from t where age=5; ### tidb_redact_log + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Boolean @@ -2215,6 +2561,14 @@ explain select * from t where age=5; ### tidb_restricted_read_only New in v5.2.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: Yes - Type: Boolean @@ -2228,7 +2582,7 @@ explain select * from t where age=5; - For DBaaS providers of TiDB, if a TiDB cluster is a downstream database of another database, to make the TiDB cluster read-only, you might need to use `tidb_restricted_read_only` with [Security Enhanced Mode](#tidb_enable_enhanced_security) enabled, which prevents your customers from using [`tidb_super_read_only`](#tidb_super_read_only-new-in-v531) to make the cluster writable. To achieve this, you need to enable [Security Enhanced Mode](#tidb_enable_enhanced_security), use an admin user with the `SYSTEM_VARIABLES_ADMIN` and `RESTRICTED_VARIABLES_ADMIN` privileges to control `tidb_restricted_read_only`, and let your database users use the root user with the `SUPER` privilege to control [`tidb_super_read_only`](#tidb_super_read_only-new-in-v531) only. - This variable controls the read-only status of the entire cluster. When the variable is `ON`, all TiDB servers in the entire cluster are in the read-only mode. In this case, TiDB only executes the statements that do not modify data, such as `SELECT`, `USE`, and `SHOW`. For other statements such as `INSERT` and `UPDATE`, TiDB rejects executing those statements in the read-only mode. - Enabling the read-only mode using this variable only ensures that the entire cluster finally enters the read-only status. If you have changed the value of this variable in a TiDB cluster but the change has not yet propagated to other TiDB servers, the un-updated TiDB servers are still **not** in the read-only mode. -- TiDB checks the read-only flag before SQL statements are executed. Since v6.2.0, the flag is also checked before SQL statements are committed. This helps prevent the case where long-running [auto commit](/transaction-overview.md#autocommit) statements might modify data after the server has been placed in read-only mode. +- When this variable is enabled, the SQL statements being executed are not affected. TiDB only performs the read-only check for the SQL statements **to be** executed. - When this variable is enabled, TiDB handles the uncommitted transactions in the following ways: - For uncommitted read-only transactions, you can commit the transactions normally. - For uncommitted transactions that are not read-only, SQL statements that perform write operations in these transactions are rejected. @@ -2246,6 +2600,14 @@ explain select * from t where age=5; ### tidb_row_format_version + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: Yes - Type: Integer @@ -2306,6 +2668,14 @@ Query OK, 0 rows affected, 1 warning (0.00 sec) ### tidb_slow_log_threshold + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to. - Type: Integer @@ -2324,9 +2694,23 @@ Query OK, 0 rows affected, 1 warning (0.00 sec) ### tidb_slow_query_file + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: SESSION - Default value: "" -- When `INFORMATION_SCHEMA.SLOW_QUERY` is queried, only the slow query log name set by `slow-query-file` in the configuration file is parsed. The default slow query log name is "tidb-slow.log". To parse other logs, set the `tidb_slow_query_file` session variable to a specific file path, and then query `INFORMATION_SCHEMA.SLOW_QUERY` to parse the slow query log based on the set file path. For details, see [Identify Slow Queries](/identify-slow-queries.md). +- When `INFORMATION_SCHEMA.SLOW_QUERY` is queried, only the slow query log name set by `slow-query-file` in the configuration file is parsed. The default slow query log name is "tidb-slow.log". To parse other logs, set the `tidb_slow_query_file` session variable to a specific file path, and then query `INFORMATION_SCHEMA.SLOW_QUERY` to parse the slow query log based on the set file path. + + + +For details, see [Identify Slow Queries](/identify-slow-queries.md). + + ### tidb_snapshot @@ -2475,15 +2859,42 @@ Query OK, 0 rows affected, 1 warning (0.00 sec) ### tidb_top_sql_max_meta_count New in v6.0.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + - Scope: GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `5000` - Range: `[1, 10000]` + + + - This variable is used to control the maximum number of SQL statement types collected by [Top SQL](/dashboard/top-sql.md) per minute. + + + + +- This variable is used to control the maximum number of SQL statement types collected by [Top SQL](https://docs.pingcap.com/tidb/stable/top-sql) per minute. + + + ### tidb_top_sql_max_time_series_count New in v6.0.0 + + +> **Note:** +> +> This TiDB variable is not applicable to TiDB Cloud. + + + > **Note:** > > Currently, the Top SQL page in TiDB Dashboard only displays the top 5 types of SQL queries that contribute the most to the load, which is irrelevant with the configuration of `tidb_top_sql_max_time_series_count`. @@ -2493,8 +2904,19 @@ Query OK, 0 rows affected, 1 warning (0.00 sec) - Type: Integer - Default value: `100` - Range: `[1, 5000]` + + + - This variable is used to control how many SQL statements that contribute the most to the load (that is, top N) can be recorded by [Top SQL](/dashboard/top-sql.md) per minute. + + + + +- This variable is used to control how many SQL statements that contribute the most to the load (that is, top N) can be recorded by [Top SQL](https://docs.pingcap.com/tidb/stable/top-sql) per minute. + + + ### tidb_track_aggregate_memory_usage - Scope: SESSION ｜ GLOBAL @@ -2535,7 +2957,7 @@ Query OK, 0 rows affected, 1 warning (0.00 sec) - Type: Enumeration - Default value: `FAST` - Possible values: `OFF`, `FAST`, `STRICT` -- This variable is used to control the assertion level. Assertion is a consistency check between data and indexes, which checks whether a key being written exists in the transaction commit process. For more information, see [Troubleshoot Inconsistency Between Data and Indexes](/troubleshoot-data-inconsistency-errors.md ). +- This variable is used to control the assertion level. Assertion is a consistency check between data and indexes, which checks whether a key being written exists in the transaction commit process. For more information, see [Troubleshoot Inconsistency Between Data and Indexes](/troubleshoot-data-inconsistency-errors.md). - `OFF`: Disable this check. - `FAST`: Enable most of the check items, with almost no impact on performance. diff --git a/table-attributes.md b/table-attributes.md index 9eba37e214a6e..371a9fce4e9b7 100644 --- a/table-attributes.md +++ b/table-attributes.md @@ -7,9 +7,20 @@ summary: Learn how to use the table attribute feature of TiDB. The Table Attributes feature is introduced in TiDB v5.3.0. Using this feature, you can add specific attributes to a table or partition to perform the operations corresponding to the attributes. For example, you can use table attributes to control the Region merge behavior. + + +Currently, TiDB only supports adding the `merge_option` attribute to a table or partition to control the Region merge behavior. The `merge_option` attribute is only part of how to deal with hotspots. For more information, refer to [Troubleshoot Hotspot Issues](/troubleshoot-hot-spot-issues.md). + + + + + +Currently, TiDB only supports adding the `merge_option` attribute to a table or partition to control the Region merge behavior. The `merge_option` attribute is only part of how to deal with hotspots. + + + > **Note:** > -> - Currently, TiDB only supports adding the `merge_option` attribute to a table or partition to control the Region merge behavior. The `merge_option` attribute is only part of how to deal with hotspots. For more information, refer to [Troubleshoot Hotspot Issues](/troubleshoot-hot-spot-issues.md). > - When you use TiDB Binlog or TiCDC to perform replication or use BR to perform incremental backup, the replication or backup operations skip the DDL statement that sets table attributes. To use table attributes in the downstream or in the backup cluster, you need to manually execute the DDL statement in the downstream or in the backup cluster. ## Usage @@ -118,7 +129,20 @@ ALTER TABLE t PARTITION p ATTRIBUTES 'merge_option=allow'; When the above two attributes are configured at the same time, the Regions belonging to the partition `p` can actually be merged. When the attribute of the partition is reset, the partition `p` inherits the attribute from the table `t`, and the Regions cannot be merged. + + > **Note:** > -> - When there is now only an attribute of a partition, even if the `merge_option=allow` attribute is configured, the partition is still split into multiple Regions by default according to the actual number of partitions. To merge all Regions, you need to [reset the attribute of the table](#usage). +> - For a table with partitions, if the `merge_option` attribute is configured at the table level only, even if `merge_option=allow`, the table is still split into multiple Regions by default according to the actual number of partitions. To merge all Regions, you need to [reset the attribute of the table](#usage). > - When using the `merge_option` attribute, you need to pay attention to the PD configuration parameter [`split-merge-interval`](/pd-configuration-file.md#split-merge-interval). Suppose that the `merge_option` attribute is not configured. In this case, if Regions meet conditions, Regions can be merged after the interval specified by `split-merge-interval`. If the `merge_option` attribute is configured, PD decides whether to merge Regions after the specified interval according to the `merge_option` configuration. + + + + + +> **Note:** +> +> - For a table with partitions, if the `merge_option` attribute is configured at the table level only, even if `merge_option=allow`, the table is still split into multiple Regions by default according to the actual number of partitions. To merge all Regions, you need to [reset the attribute of the table](#usage). +> - Suppose that the `merge_option` attribute is not configured. In this case, if Regions meet conditions, Regions can be merged after one hour. If the `merge_option` attribute is configured, PD decides whether to merge Regions after one hour according to the `merge_option` configuration. + + \ No newline at end of file diff --git a/tidb-architecture.md b/tidb-architecture.md index 4a6695930803b..415f8de3709c1 100644 --- a/tidb-architecture.md +++ b/tidb-architecture.md @@ -12,8 +12,13 @@ Compared with the traditional standalone databases, TiDB has the following advan * Fully compatible with the MySQL 5.7 protocol, common features and syntax of MySQL. To migrate your applications to TiDB, you do not need to change a single line of code in many cases. * Supports high availability with automatic failover when a minority of replicas fail; transparent to applications. * Supports ACID transactions, suitable for scenarios requiring strong consistency such as bank transfer. + + + * Provides a rich series of [data migration tools](/migration-overview.md) for migrating, replicating, or backing up data. + + As a distributed database, TiDB is designed to consist of multiple components. These components communicate with each other and form a complete TiDB system. The architecture is as follows: ![TiDB Architecture](/media/tidb-architecture-v3.1.png) @@ -30,7 +35,21 @@ The PD server is the metadata managing component of the entire cluster. It store ### TiKV server -The TiKV server is responsible for storing data. TiKV is a distributed transactional key-value storage engine. [Region](/glossary.md#regionpeerraft-group) is the basic unit to store data. Each Region stores the data for a particular Key Range which is a left-closed and right-open interval from StartKey to EndKey. Multiple Regions exist in each TiKV node. TiKV APIs provide native support to distributed transactions at the key-value pair level and supports the Snapshot Isolation level isolation by default. This is the core of how TiDB supports distributed transactions at the SQL level. After processing SQL statements, the TiDB server converts the SQL execution plan to an actual call to the TiKV API. Therefore, data is stored in TiKV. All the data in TiKV is automatically maintained in multiple replicas (three replicas by default), so TiKV has native high availability and supports automatic failover. +The TiKV server is responsible for storing data. TiKV is a distributed transactional key-value storage engine. + + + +[Region](/glossary.md#regionpeerraft-group) is the basic unit to store data. Each Region stores the data for a particular Key Range which is a left-closed and right-open interval from StartKey to EndKey. + + + + + +[Region](/tidb-cloud/tidb-cloud-glossary.md#region) is the basic unit to store data. Each Region stores the data for a particular Key Range which is a left-closed and right-open interval from StartKey to EndKey. + + + +Multiple Regions exist in each TiKV node. TiKV APIs provide native support to distributed transactions at the key-value pair level and supports the Snapshot Isolation level isolation by default. This is the core of how TiDB supports distributed transactions at the SQL level. After processing SQL statements, the TiDB server converts the SQL execution plan to an actual call to the TiKV API. Therefore, data is stored in TiKV. All the data in TiKV is automatically maintained in multiple replicas (three replicas by default), so TiKV has native high availability and supports automatic failover. ### TiFlash server diff --git a/tidb-cloud/_index.md b/tidb-cloud/_index.md index ca69403b60952..2a4c4825928e0 100644 --- a/tidb-cloud/_index.md +++ b/tidb-cloud/_index.md @@ -27,6 +27,16 @@ hide_commit: true + + +[Developer Guide Overview](https://docs.pingcap.com/tidbcloud/dev-guide-overview) + +[Quick Start](https://docs.pingcap.com/tidbcloud/dev-guide-build-cluster-in-cloud) + +[Example Application](https://docs.pingcap.com/tidbcloud/dev-guide-sample-application-spring-boot) + + + [Create a Cluster](https://docs.pingcap.com/tidbcloud/create-tidb-cluster) diff --git a/tidb-cloud/backup-and-restore.md b/tidb-cloud/backup-and-restore.md index 11160c3c010af..3c4ddca55a65f 100644 --- a/tidb-cloud/backup-and-restore.md +++ b/tidb-cloud/backup-and-restore.md @@ -32,20 +32,20 @@ By the automatic backup, you can back up the cluster data every day at the backu If you do not specify a preferred backup time, TiDB Cloud assigns a default backup time based on each region. The following table lists the default backup time for each region: -| Cloud provider | Region name | Region | Default backup time | -|----------------|--------------------------|-----------------|---------------------| -| AWS | US East (N. Virginia) | us-east-1 | 07:00 UTC | -| AWS | US West (Oregon) | us-west-2 | 10:00 UTC | -| AWS | Asia Pacific (Tokyo) | ap-northeast-1 | 17:00 UTC | -| AWS | Asia Pacific (Seoul) | ap-northeast-2 | 17:00 UTC | -| AWS | Asia Pacific (Singapore) | ap-southeast-1 | 18:00 UTC | -| AWS | Asia Pacific (Mumbai) | ap-south-1 | 20:30 UTC | -| AWS | Europe (Frankfurt) | eu-central-1 | 03:00 UTC | -| GCP | Iowa | us-central1 | 08:00 UTC | -| GCP | Oregon | us-west1 | 10:00 UTC | -| GCP | Tokyo | asia-northeast1 | 17:00 UTC | -| GCP | Singapore | asia-southeast1 | 18:00 UTC | -| GCP | Taiwan | asia-east1 | 18:00 UTC | +| Cloud provider | Region | Default backup time | +|----------------|-----------------------------|---------------------| +| AWS | Oregon (us-west-2) | 10:00 UTC | +| AWS | N. Virginia (us-east-1) | 07:00 UTC | +| AWS | Mumbai (ap-south-1) | 20:30 UTC | +| AWS | Singapore (ap-southeast-1) | 18:00 UTC | +| AWS | Tokyo (ap-northeast-1) | 17:00 UTC | +| AWS | Frankfurt (eu-central-1) | 03:00 UTC | +| GCP | Oregon (us-west1) | 10:00 UTC | +| GCP | Iowa (us-central1) | 08:00 UTC | +| GCP | Singapore (asia-southeast1) | 18:00 UTC | +| GCP | Taiwan (asia-east1) | 18:00 UTC | +| GCP | Tokyo (asia-northeast1) | 17:00 UTC | +| GCP | Osaka (asia-northeast2) | 17:00 UTC | Note that you can not disable automatic backup. @@ -107,9 +107,9 @@ To restore your TiDB cluster data from a backup to a new cluster, take the follo 4. Click **Confirm**. - The cluster restore process starts and the **Security Quick Start** dialog box is displayed. + The cluster restore process starts and the **Security Settings** dialog box is displayed. -5. In the **Security Quick Start** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. +5. In the **Security Settings** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. ### Restore a deleted cluster @@ -125,6 +125,6 @@ To restore a deleted cluster from recycle bin, take the following steps: 5. Click **Confirm**. - The cluster restore process starts and the **Security Quick Start** dialog box is displayed. + The cluster restore process starts and the **Security Settings** dialog box is displayed. -6. In the **Security Quick Start** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. +6. In the **Security Settings** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. diff --git a/tidb-cloud/built-in-monitoring.md b/tidb-cloud/built-in-monitoring.md new file mode 100644 index 0000000000000..7a4e9339c3afd --- /dev/null +++ b/tidb-cloud/built-in-monitoring.md @@ -0,0 +1,109 @@ +--- +title: TiDB Cloud Built-in Monitoring +summary: Learn how to view TiDB Cloud built-in monitoring metrics and understand the meanings of these metrics. +--- + +# TiDB Cloud Built-in Monitoring + +TiDB Cloud collects and displays a full set of standard metrics of your cluster on the Monitoring page. By viewing these metrics, you can easily identify performance issues and determine whether your current database deployment meets your requirements. + +> **Note:** +> +> Currently, the Monitoring page is unavailable for [Developer Tier clusters](/tidb-cloud/select-cluster-tier.md#developer-tier). + +## View the Monitoring page + +To view the metrics on the Monitoring page, take the following steps: + +1. Navigate to the **Diagnosis** tab of a cluster. + +2. Click the **Monitoring** tab. + +## Monitoring metrics + +The following sections illustrate the metrics on the Monitoring page. + +### Database Time + +| Metric name | Labels | Description | +| :------------| :------| :-------------------------------------------- | +| Database Time by SQL types | database time, {SQL type} | database time: total database time per second.
{SQL type}: database time consumed by SQL statements per second, which are collected by SQL types, such as `SELECT`, `INSERT`, and `UPDATE`. | +| Database Time by SQL Phase | database time, get token, parse, compile, execute | Database time consumed in four SQL processing phases: get token, parse, compile, and execute. The SQL execution phase is in green and other phases are in red on general. If non-green areas take a large proportion, it means most database time is consumed by other phases than the execution phase and further cause analysis is required. | +| SQL Execute Time Overview | tso_wait, Get, Cop, Commit, etc. | Green metrics stand for common KV write requests (such as prewrite and commit), blue metrics stand for common read requests, and metrics in other colors stand for unexpected situations which you need to pay attention to. For example, pessimistic lock KV requests are marked red and TSO waiting is marked dark brown. If non-blue or non-green areas take a large proportion, it means there is a bottleneck during SQL execution. For example, if serious lock conflicts occur, the red area will take a large proportion. If excessive time is consumed in waiting TSO, the dark brown area will take a large proportion. | + +### Application Connection + +| Metric name | Labels | Description | +| :------------| :------| :-------------------------------------------- | +| Connection Count | Total, active connection | Total: the number of connections to all TiDB instances.
Active connections: the number of active connections to all TiDB instances. | +| Disconnection | Instances | The number of clients disconnected to each TiDB instance. | + +### SQL Count + +| Metric name | Labels | Description | +| :------------| :------| :-------------------------------------------- | +| Query Per Second | {SQL type} | The number of SQL statements executed per second in all TiDB instances, which are collected by SQL types, such as `SELECT`, `INSERT`, and `UPDATE`. | +| Failed Queries | Error types | The statistics of error types (such as syntax errors and primary key conflicts) according to the SQL statement execution errors per minute on each TiDB instance. It contains the module in which an error occurs and the error code. | +| Command Per Second | Query, StmtExecute, StmtPrepare, etc. | The number of commands processed by all TiDB instances per second based on command types. | +| Queries Using Plan Cache OPS | hit, miss | hit: the number of queries using plan cache per second in all TiDB instances.
miss: the number of queries missing plan cache per second in all TiDB instances. | + +### Latency Break Down + +| Metric name | Labels | Description | +| :------------| :------| :-------------------------------------------- | +| Query Duration | avg-{SQL Types}, 99-{SQL Types} | The duration from receiving a request from the client to TiDB till TiDB executing the request and returning the result to the client. In general, client requests are sent in the form of SQL statements; however, this duration can include the execution time of commands such as `COM_PING`, `COM_SLEEP`, `COM_STMT_FETCH`, and `COM_SEND_LONG_DATA`. TiDB supports Multi-Query, which means the client can send multiple SQL statements at one time, such as `select 1; select 1; select 1;`. In this case, the total execution time of this query includes the execution time of all SQL statements. | +| Average Idle Connection Duration | avg-in-txn, avg-not-in-txn | The connection idle duration indicates the duration of a connection being idle.
avg-in-txn: The average connection idle duration when a connection is within a transaction.
avg-not-in-txn: The average connection idle duration when a connection is not within a transaction. | +| Get Token Duration | avg, 99 | The average time or P99 duration consumed in getting tokens of SQL statements. | +| Parse Duration | avg, 99 | The average time or P99 duration consumed in parsing SQL statements. | +| Compile Duration | avg, 99 | The average time or P99 duration consumed in compiling the parsed SQL AST to execution plans. | +| Execute Duration | avg, 99 | The average time or P99 duration consumed in executing execution plans of SQL statements. | + +### Transaction + +| Metric name | Labels | Description | +| :------------| :------| :-------------------------------------------- | +| Transaction Per Second | {types}-{transaction model} | The number of transactions executed per second. | +| Transaction Duration | avg-{transaction model}, 99-{transaction model} | The execution duration of a transaction. | + +### Core Path Duration + +| Metric name | Labels | Description | +| :------------| :------| :-------------------------------------------- | +| Avg TiDB KV Request Duration | Get, Prewirite, Commit, PessimisticLock, etc. | The average time consumed in executing KV requests in all TiDB instances based on request types, including `Get`, `Prewrite`, and `Commit`. | +| Avg TiKV GRPC Duration | kv_get, kv_prewirite, kv_commit, kv_pessimisticLock, etc. | The average time consumed in executing gRPC requests in all TiKV instances based request types, including `kv_get`, `kv_prewrite`, and `kv_commit`. | +| Average / P99 PD TSO Wait/RPC Duration | wait-avg/99, rpc-avg/99 | Wait: the average time or P99 duration in waiting for PD to return TSO in all TiDB instances.
RPC: the average time or P99 duration from sending TSO requests to PD to receiving TSO in all TiDB instances. | +| Average / P99 Storage Async Write Duration | avg, 99 | The average time or P99 duration consumed in asynchronous writing. Average storage async write duration = Average store duration + Average apply duration. | +| Average / P99 Store Duration | avg, 99 | The average time or P99 duration consumed in storing loop during asynchronously writing. | +| Average / P99 Apply Duration | avg, 99 | The average time or P99 duration consumed in applying loop during asynchronously writing. | +| Average / P99 Append Log Duration | avg, 99 | The average time or P99 duration consumed by Raft to append logs. | +| Average / P99 Commit Log Duration | avg, 99 | The average time or P99 duration consumed by Raft to commit logs. | +| Average / P99 Apply Log Duration | avg, 99 | The average time or P99 duration consumed by Raft to apply logs. | + +### Server + +| Metric name | Labels | Description | +| :------------| :------| :-------------------------------------------- | +| TiDB Uptime | instances | The runtime of each TiDB instance since last restart. | +| TiDB CPU Usage | instances | The statistics of CPU usage of each TiDB instance. | +| TiDB Memory Usage | instances | The memory usage statistics of each TiDB instance. | +| TiKV Uptime | instances | The runtime of each TiKV instance since last restart. | +| TiKV CPU Usage | instances | The statistics of CPU usage of each TiKV instance. | +| TiKV Memory Usage | instances | The memory usage statistics of each TiKV instance. | +| TiKV IO MBps | instances-write, instances-read | The total bytes of read and write in each TiKV instance. | +| TiKV Storage Usage | instances | The storage size per TiKV instance. | +| TiFlash Uptime | instances | The runtime of each TiFlash instance since last restart. | +| TiFlash CPU Usage | instances | The statistics of CPU usage of each TiFlash instance. | +| TiFlash Memory | instances | The memory usage statistics of each TiFlash instance. | +| TiFlash IO MBps | instances-write, instances-read | The total bytes of read and write in each TiFlash instance. | +| TiFlash Storage Usage | instances | The storage size per TiFlash instance. | + +## FAQ + +**1. Why are some panes empty on this page?** + +If a pane does not provide any metrics, the possible reasons are as follows: + +- The workload of the corresponding cluster does not trigger this metric. For example, the failed query metric is always empty in the case of no failed queries. +- The cluster version is low. You need to upgrade it to the latest version of TiDB to see these metrics. + +If all these reasons are excluded, you can contact the [PingCAP support team](/tidb-cloud/tidb-cloud-support.md) for troubleshooting. diff --git a/tidb-cloud/changefeed-sink-to-mysql.md b/tidb-cloud/changefeed-sink-to-mysql.md index 9c8ef4b189a8f..c60015cf13c5f 100644 --- a/tidb-cloud/changefeed-sink-to-mysql.md +++ b/tidb-cloud/changefeed-sink-to-mysql.md @@ -54,9 +54,9 @@ The **Sink to MySQL** connector can only sink incremental data from your TiDB cl SET GLOBAL tidb_gc_life_time = '720h'; ``` -2. Use [Dumpling](/dumpling-overview.md) to export data from your TiDB cluster, then use community tools such as [mydumper/myloader](https://centminmod.com/mydumper.html) to load data to the MySQL service. +2. Use [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview) to export data from your TiDB cluster, then use community tools such as [mydumper/myloader](https://centminmod.com/mydumper.html) to load data to the MySQL service. -3. From the [exported files of Dumpling](/dumpling-overview.md#format-of-exported-files), get the TSO from the metadata file: +3. From the [exported files of Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview#format-of-exported-files), get the TSO from the metadata file: {{< copyable "shell-regular" >}} diff --git a/tidb-cloud/config-s3-and-gcs-access.md b/tidb-cloud/config-s3-and-gcs-access.md new file mode 100644 index 0000000000000..90e81b3527a22 --- /dev/null +++ b/tidb-cloud/config-s3-and-gcs-access.md @@ -0,0 +1,137 @@ +--- +title: Configure Amazon S3 Access and GCS Access +summary: Learn how to configure Amazon Simple Storage Service (Amazon S3) access and Google Cloud Storage (GCS) access. +--- + +# Configure Amazon S3 Access and GCS Access + +If your source data is stored in Amazon S3 or GCS buckets, before importing or migrating the data to TiDB Cloud, you need to configure cross-account access to the buckets. This document describes how to do this. + +- [Configure Amazon S3 access](#configure-amazon-s3-access) +- [Configure GCS access](#configure-gcs-access) + +## Configure Amazon S3 access + +To allow TiDB Cloud to access the source data in your Amazon S3 bucket, take the following steps to configure the bucket access for TiDB Cloud and get the Role-ARN. Once the configuration is done for one TiDB cluster in a project, all TiDB clusters in that project can use the same Role-ARN to access your Amazon S3 bucket. + +1. In the TiDB Cloud Console, get the TiDB Cloud account ID and external ID of the target TiDB cluster. + + 1. In the TiDB Cloud Console, choose your target project, and navigate to the **Active Clusters** page. + 2. Find the area of your target cluster and click **Import Data** in the upper-right corner of the area. The **Data Import Task** page is displayed. + + > **Tip:** + > + > Alternatively, you can also click the name of your cluster on the **Active Clusters** page and click **Import Data** in the upper-right corner. + + 3. On the **Data Import Task** page, click **Show AWS IAM policy settings** to get the TiDB Cloud Account ID and TiDB Cloud External ID. Take a note of these IDs for later use. + +2. In the AWS Management Console, create a managed policy for your Amazon S3 bucket. + + 1. Sign in to the AWS Management Console and open the Amazon S3 console at . + 2. In the **Buckets** list, choose the name of your bucket with the source data, and then click **Copy ARN** to get your S3 bucket ARN (for example, `arn:aws:s3:::tidb-cloud-source-data`). Take a note of the bucket ARN for later use. + + ![Copy bucket ARN](/media/tidb-cloud/copy-bucket-arn.png) + + 3. Open the IAM console at , click **Policies** in the navigation pane on the left, and then click **Create Policy**. + + ![Create a policy](/media/tidb-cloud/aws-create-policy.png) + + 4. On the **Create policy** page, click the **JSON** tab. + 5. Copy the following access policy template and paste it to the policy text field. + + ``` + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "VisualEditor0", + "Effect": "Allow", + "Action": [ + "s3:GetObject", + "s3:GetObjectVersion" + ], + "Resource": "//*" + }, + { + "Sid": "VisualEditor1", + "Effect": "Allow", + "Action": [ + "s3:ListBucket", + "s3:GetBucketLocation" + ], + "Resource": "" + } + ] + } + ``` + + In the policy text field, update the following configurations to your own values. + + - `"Resource": "//*"` + + For example, if your source data is stored in the root directory of the `tidb-cloud-source-data` bucket, use `"Resource": "arn:aws:s3:::tidb-cloud-source-data/*"`. If your source data is stored in the `mydata` directory of the bucket, use `"Resource": "arn:aws:s3:::tidb-cloud-source-data/mydata/*"`. Make sure that `/*` is added to the end of the directory so TiDB Cloud can access all files in this directory. + + - `"Resource": ""` + + For example, `"Resource": "arn:aws:s3:::tidb-cloud-source-data"`. + + 6. Click **Next: Tags**, add a tag of the policy (optional), and then click **Next:Review**. + + 7. Set a policy name, and then click **Create policy**. + +3. In the AWS Management Console, create an access role for TiDB Cloud and get the role ARN. + + 1. In the IAM console at , click **Roles** in the navigation pane on the left, and then click **Create role**. + + ![Create a role](/media/tidb-cloud/aws-create-role.png) + + 2. To create a role, fill in the following information: + + - Under **Trusted entity type**, select **AWS account**. + - Under **An AWS account**, select **Another AWS account**, and then paste the TiDB Cloud account ID to the **Account ID** field. + - Under **Options**, click **Require external ID (Best practice when a third party will assume this role)**, and then paste the TiDB Cloud External ID to the **External ID** field. + + 3. Click **Next** to open the policy list, choose the policy you just created, and then click **Next**. + 4. Under **Role details**, set a name for the role, and then click **Create role** in the lower-right corner. After the role is created, the list of roles is displayed. + 5. In the list of roles, click the name of the role that you just created to go to its summary page, and then copy the role ARN. + + ![Copy AWS role ARN](/media/tidb-cloud/aws-role-arn.png) + +4. In the TiDB Cloud console, go to the **Data Import Task** page where you get the TiDB Cloud account ID and external ID, and then paste the role ARN to the **Role ARN** field. + +## Configure GCS access + +To allow TiDB cloud to access the source data in your GCS bucket, you need to configure the GCS access for each TiDB Cloud as a service on the GCP project and GCS bucket pair. Once the configuration is done for one cluster in a project, all database clusters in that project can access the GCS bucket. + +1. Get the Google Cloud Service Account ID of the target TiDB cluster. + + 1. In the TiDB Cloud Admin console, choose a target project and a target cluster deployed on the Google Cloud Platform, and then click **Import Data** in the upper-right corner. + 2. Click **Show Google Cloud Service Account ID**, and then copy the Service Account ID. + +2. In the Google Cloud Platform (GCP) Management Console, go to **IAM & Admin** > **Roles**, and then check whether a role with the following read-only permissions of the storage container exists. + + - storage.buckets.get + - storage.objects.get + - storage.objects.list + + If yes, you can use the matched role for the target TiDB cluster in the following steps. If not, go to **IAM & Admin** > **Roles** > **CREATE ROLE** to define a role for the target TiDB cluster. + +3. Go to **Cloud Storage** > **Browser**, select the GCS bucket you want TiDB Cloud to access, and then click **SHOW INFO PANEL**. + + The panel is displayed. + +4. In the panel, click **ADD PRINCIPAL**. + + The dialog box for adding principals is displayed. + +5. In the dialog box, perform the following steps: + + 1. In the **New Principals** field, paste the Google Cloud Service Account ID of the target TiDB cluster. + 2. In the **Role** drop-down list, choose the role of the target TiDB cluster. + 3. Click **SAVE**. + +Your TiDB Cloud cluster can now access the GCS bucket. + +> **Note:** +> +> To remove the access to TiDB Cloud, you can simply delete the principal that you added. \ No newline at end of file diff --git a/tidb-cloud/configure-security-settings.md b/tidb-cloud/configure-security-settings.md index de19c40c77e6a..76694cd513370 100644 --- a/tidb-cloud/configure-security-settings.md +++ b/tidb-cloud/configure-security-settings.md @@ -9,14 +9,14 @@ When you create a cluster, the TiDB Cloud console will automatically guide you t If you do not configure the root password or allowed IP addresses at that time, or if you want to modify the cluster security settings, take the following steps: -1. In the TiDB Cloud console, navigate to the **Active Clusters** page of your project, and then locate the pane of your cluster. -2. Click **Security Quick Start** in the pane. The **Security Quick Start** dialog is displayed. +1. In the TiDB Cloud console, navigate to the **Active Clusters** page of your project, and then find the area of your cluster. +2. Click **Security Settings** in the upper-right corner of the area. The **Security Settings** dialog is displayed. 3. In the dialog, configure the root password and allowed IP addresses. - To allow your cluster to be accessible by any IP addresses, click **Allow Access From Anywhere**. + To allow your cluster to be accessible by any IP addresses, click **Allow Access from Anywhere**. 4. Click **Apply**. > **Tip:** > -> If you are viewing the overview page of your cluster, you can click the **Quick Start** tab on the page and configure these settings, too. \ No newline at end of file +> If you are viewing the overview page of your cluster, you can click the **Security Settings** in the upper-right corner of the page and configure these settings, too. diff --git a/tidb-cloud/connect-to-tidb-cluster.md b/tidb-cloud/connect-to-tidb-cluster.md index 77d2b3a6358b4..f75062f835b29 100644 --- a/tidb-cloud/connect-to-tidb-cluster.md +++ b/tidb-cloud/connect-to-tidb-cluster.md @@ -18,9 +18,13 @@ After your TiDB cluster is created on TiDB Cloud, you can use one of the followi To connect to your TiDB cluster via standard connection, perform the following steps: -1. Navigate to the **Active Clusters** page and click the name of your newly created cluster. +1. Navigate to the **Active Clusters** page. -2. Click **Connect**. The **Connect to TiDB** dialog box is displayed. +2. Find the area of your cluster, and click **Connect** in the upper-right corner of the area. A connection dialog box is displayed. + + > **Tip:** + > + > Alternatively, you can also click the name of your cluster on the **Active Clusters** page and click **Connect** in the upper-right corner. 3. Create the traffic filter for the cluster. Traffic filter is a list of IPs and CIDR addresses that are allowed to access TiDB Cloud via a SQL client. @@ -49,9 +53,13 @@ To connect to your TiDB cluster via standard connection, perform the following s To connect to your TiDB cluster via VPC peering, perform the following steps: -1. Navigate to the **Active Clusters** page and click the name of your newly created cluster. +1. Navigate to the **Active Clusters** page. + +2. Find the area of your cluster, click **Connect** in the upper-right corner of the area, and select the **VPC Peering** tab in the connection dialog. -2. Click **Connect**, and select the **VPC Peering** tab at the **Connect to TiDB** dialog. + > **Tip:** + > + > Alternatively, you can also click the name of your cluster on the **Active Clusters** page and click **Connect** in the upper-right corner. 3. Set up VPC peering. See [Set up VPC Peering](/tidb-cloud/set-up-vpc-peering-connections.md) for details. @@ -69,9 +77,13 @@ To connect to your TiDB cluster via VPC peering, perform the following steps: To connect to your TiDB cluster using SQL shell, perform the following steps: -1. Navigate to the **Active Clusters** page and click the name of your newly created cluster. +1. Navigate to the **Active Clusters** page. + +2. Find the area of your cluster, click **Connect** in the upper-right corner of the area, and select the **Web SQL Shell** tab in the connection dialog. -2. Click **Connect**, and select the **Web SQL Shell** tab at the **Connect to TiDB** dialog. + > **Tip:** + > + > Alternatively, you can also click the name of your cluster on the **Active Clusters** page and click **Connect** in the upper-right corner. 3. Click **Open SQL Shell**. diff --git a/tidb-cloud/create-tidb-cluster.md b/tidb-cloud/create-tidb-cluster.md index 40e5d162e3f58..b5c7af696b5f9 100644 --- a/tidb-cloud/create-tidb-cluster.md +++ b/tidb-cloud/create-tidb-cluster.md @@ -64,17 +64,17 @@ If you are a project member, you can access only the specific projects to which To create a Developer Tier cluster, take the following steps: -1. On the TiDB Cloud cluster page, click **Create Cluster**, and then click **Developer Tier**. +1. On the **Active Clusters** page, click **Create Cluster**. -2. On the **Create a Cluster** page, update the default cluster name if necessary. +2. On the **Create Cluster** page, update the default cluster name if necessary. 3. Note that the cloud provider of Developer Tier is AWS, and then select the region where you want to create your cluster. 4. View the cluster size of the Developer Tier, and then click **Create**. - The cluster creation process starts and the **Security Quick Start** dialog box is displayed. + The cluster creation process starts and the **Security Settings** dialog box is displayed. -5. In the **Security Quick Start** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. +5. In the **Security Settings** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. Your TiDB Cloud cluster will be created in approximately 5 to 15 minutes. @@ -84,9 +84,9 @@ To create a Developer Tier cluster, take the following steps: To create a Dedicated Tier cluster, take the following steps: -1. On the TiDB Cloud cluster page, click **Create Cluster**, and then click **Dedicated Tier**. +1. On the **Active Clusters** page, click **Create Cluster**. -2. On the **Create a Cluster** page, update the default cluster name and port number if necessary, choose a cloud provider and a region, and then click **Next**. +2. On the **Create Cluster** page, select **Dedicated Tier**, update the default cluster name and port number if necessary, choose a cloud provider and a region, and then click **Next**. > **Note:** > @@ -100,9 +100,9 @@ To create a Dedicated Tier cluster, take the following steps: 4. Configure the [cluster size](/tidb-cloud/size-your-cluster.md) for TiDB, TiKV, and TiFlash (optional) respectively, and then click **Next**. -5. Confirm the cluster information in the middle area and the billing information in the right pane. +5. Confirm the cluster information on the page and the billing information in the lower-left corner. -6. If you have not added a payment method, click **Add Credit Card** in the right pane. +6. If you have not added a payment method, click **Add Credit Card** in the lower-right corner. > **Note:** > @@ -110,9 +110,9 @@ To create a Dedicated Tier cluster, take the following steps: 7. Click **Create**. - The cluster creation process starts and the **Security Quick Start** dialog box is displayed. + The cluster creation process starts and the **Security Settings** dialog box is displayed. -8. In the **Security Quick Start** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. +8. In the **Security Settings** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. Your TiDB Cloud cluster will be created in approximately 5 to 15 minutes. diff --git a/tidb-cloud/delete-tidb-cluster.md b/tidb-cloud/delete-tidb-cluster.md index 859bb99b90723..87a2f20e2c753 100644 --- a/tidb-cloud/delete-tidb-cluster.md +++ b/tidb-cloud/delete-tidb-cluster.md @@ -9,8 +9,13 @@ This document describes how to delete a TiDB cluster on TiDB Cloud. You can delete a cluster at any time by performing the following steps: -1. Navigate to the TiDB Clusters page and click the name of a cluster that you want to delete. The overview page of the cluster is displayed. -2. In the cluster information pane on the left, click **Setting**. +1. Navigate to the **Active Clusters** page. +2. Find the area of the cluster that you want to delete, and click **...** in the upper-right corner of the area. + + > **Tip:** + > + > Alternatively, you can also click the name of the cluster that you want to delete on the **Active Clusters** page and click **...** in the upper-right corner. + 3. Click **Delete** in the drop-down menu. 4. In the cluster deleting window, enter the cluster name. diff --git a/tidb-cloud/import-csv-files.md b/tidb-cloud/import-csv-files.md index 28687415ebdf9..471daa77eccdc 100644 --- a/tidb-cloud/import-csv-files.md +++ b/tidb-cloud/import-csv-files.md @@ -14,61 +14,83 @@ This document describes how to import uncompressed CSV files from Amazon Simple ## Step 1. Prepare the CSV files -1. If a CSV file is larger than 256 MB, consider splitting the file into smaller files, each with a size around 256 MB. +1. If a CSV file is larger than 256 MB, consider splitting it into smaller files, each with a size around 256 MB. - TiDB Cloud supports importing very large CSV files but performs best with multiple input files around 256 MB in size. This is because TiDB Cloud can process multiple files in parallel which can greatly improve the import speed. + TiDB Cloud supports importing very large CSV files but performs best with multiple input files around 256 MB in size. This is because TiDB Cloud can process multiple files in parallel, which can greatly improve the import speed. -2. According to the naming convention of existing objects in your bucket, identify a text pattern that matches the names of the CSV files to be imported. +2. Name the CSV files as follows: - For example, to import all data files in a bucket, you can use the wildcard symbol `*` or `*.csv` as a pattern. Similarly, to import the subset of data files in partition `station=402260`, you can use `*station=402260*` as a pattern. Make a note of this pattern as you will need to provide it to TiDB Cloud in [Step 4](#step-4-import-csv-files-to-tidb-cloud). + - If a CSV file contains all data of an entire table, name the file in the `${db_name}.${table_name}.csv` format, which maps to the `${db_name}.${table_name}` table when you import the data. + - If the data of one table is separated into multiple CSV files, append a numeric suffix to these CSV files. For example, `${db_name}.${table_name}.000001.csv` and `${db_name}.${table_name}.000002.csv`. The numeric suffixes can be inconsecutive but must be in ascending order. You also need to add extra zeros before the number to ensure all the suffixes are in the same length. -## Step 2. Create the target table schema + > **Note:** + > + > If you cannot update the CSV filenames according to the preceding rules in some cases (for example, the CSV file links are also used by your other programs), you can keep the filenames unchanged and use the **Custom Pattern** in [Step 4](#step-4-import-csv-files-to-tidb-cloud) to import your source data to a single target table. -Before importing CSV files into TiDB Cloud, you need to create the target database and table. Alternatively, TiDB Cloud can create these objects for you as part of the import process if you provide the target database and table schema as follows: +## Step 2. Create the target table schemas -1. In the Amazon S3 or GCS directory where the CSV files are located, create a `${db_name}-schema-create.sql` file that contains the `CREATE DATABASE` DDL statement. +Because CSV files do not contain schema information, before importing data from CSV files into TiDB Cloud, you need to create the table schemas using either of the following methods: - For example, you can create a `mydb-scehma-create.sql` file that contains the following statement: +- Method 1: In TiDB Cloud, create the target databases and tables for your source data. - {{< copyable "sql" >}} +- Method 2: In the Amazon S3 or GCS directory where the CSV files are located, create the target table schema files for your source data as follows: - ```sql - CREATE DATABASE mydb; - ``` + 1. Create database schema files for your source data. -2. In the Amazon S3 or GCS directory where the CSV files are located, create a `${db_name}.${table_name}-schema.sql` file that contains the `CREATE TABLE` DDL statement. + If your CSV files follow the naming rules in [Step 1](#step-1-prepare-the-csv-files), the database schema files are optional for the data import. Otherwise, the database schema files are mandatory. - For example, you can create a `mydb.mytable-schema.sql` file that contains the following statement: + Each database schema file must be in the `${db_name}-schema-create.sql` format and contain a `CREATE DATABASE` DDL statement. With this file, TiDB Cloud will create the `${db_name}` database to store your data when you import the data. - {{< copyable "sql" >}} + For example, if you create a `mydb-scehma-create.sql` file that contains the following statement, TiDB Cloud will create the `mydb` database when you import the data. - ```sql - CREATE TABLE mytable ( - ID INT, - REGION VARCHAR(20), - COUNT INT ); - ``` + {{< copyable "sql" >}} - > **Note:** - > - > The `${db_name}.${table_name}-schema.sql` file should only contain a single DDL statement. If the file contains multiple DDL statements, only the first statement takes effect. + ```sql + CREATE DATABASE mydb; + ``` + + 2. Create table schema files for your source data. + + If you do not include the table schema files in the Amazon S3 or GCS directory where the CSV files are located, TiDB Cloud will not create the corresponding tables for you when you import the data. + + Each table schema file must be in the `${db_name}.${table_name}-schema.sql` format and contain a `CREATE TABLE` DDL statement. With this file, TiDB Cloud will create the `${db_table}` table in the `${db_name}` database when you import the data. + + For example, if you create a `mydb.mytable-schema.sql` file that contains the following statement, TiDB Cloud will create the `mytable` table in the `mydb` database when you import the data. + + {{< copyable "sql" >}} + + ```sql + CREATE TABLE mytable ( + ID INT, + REGION VARCHAR(20), + COUNT INT ); + ``` + + > **Note:** + > + > Each `${db_name}.${table_name}-schema.sql` file should only contain a single DDL statement. If the file contains multiple DDL statements, only the first one takes effect. ## Step 3. Configure cross-account access To allow TiDB Cloud to access the CSV files in the Amazon S3 or GCS bucket, do one of the following: -- If your CSV files are located in Amazon S3, [configure cross-account access to Amazon S3](/tidb-cloud/migrate-from-amazon-s3-or-gcs.md#step-2-configure-amazon-s3-access). +- If your CSV files are located in Amazon S3, [configure cross-account access to Amazon S3](/tidb-cloud/config-s3-and-gcs-access.md#configure-amazon-s3-access). Once finished, make a note of the Role ARN value as you will need it in [Step 4](#step-4-import-csv-files-to-tidb-cloud). -- If your CSV files are located in GCS, [configure cross-account access to GCS](/tidb-cloud/migrate-from-amazon-s3-or-gcs.md#step-2-configure-gcs-access). +- If your CSV files are located in GCS, [configure cross-account access to GCS](/tidb-cloud/config-s3-and-gcs-access.md#configure-gcs-access). ## Step 4. Import CSV files to TiDB Cloud To import the CSV files to TiDB Cloud, take the following steps: -1. Navigate to the TiDB Clusters page and click the name of your target cluster. The overview page of your target cluster is displayed. -2. In the cluster information pane on the left, click **Import**. The **Data Import Task** page is displayed. +1. Navigate to the **Active Clusters** page. +2. Find the area of your target cluster and click **Import Data** in the upper-right corner of the area. The **Data Import Task** page is displayed. + + > **Tip:** + > + > Alternatively, you can also click the name of your target cluster on the **Active Clusters** page and click **Import Data** in the upper-right corner. + 3. On the **Data Import Task** page, provide the following information. - **Data Source Type**: select the type of the data source. @@ -81,10 +103,34 @@ To import the CSV files to TiDB Cloud, take the following steps: > > For the configurations of separator, delimiter, and null, you can use both alphanumeric characters and certain special characters. The supported special characters include `\t`, `\b`, `\n`, `\r`, `\f`, and `\u0001`. - - **Target Database**: fill in the **Username** and **Password** fields. - - **DB/Tables Filter**: if necessary, you can specify a [table filter](/table-filter.md#syntax). If you want to configure multiple filter rules, use `,` to separate the rules. - - **Object Name Pattern**: enter a pattern that matches the names of the CSV files to be imported. For example,`my-data.csv`. - - **Target Table Name**: enter the name of the target table. For example, `mydb.mytable`. + - **Target Cluster**: fill in the **Username** and **Password** fields. + - **DB/Tables Filter**: if you want to filter which tables to be imported, you can specify one or more table filters in this field, separated by `,`. + + For example: + + - `db01.*`: all tables in the `db01` database will be imported. + - `db01.table01*,db01.table02*`: all tables starting with `table01` and `table02` in the `db01` database will be imported. + - `!db02.*`: except the tables in the `db02` database, all other tables will be imported. `!` is used to exclude tables that do not need to be imported. + - `*.*` : all tables will be imported. + + For more information, see [table filter snytax](https://docs.pingcap.com/tidb/stable/table-filter#syntax). + + - **Custom Pattern**: enable the **Custom Pattern** feature if you want to import CSV files whose filenames match a certain pattern to a single target table. + + > **Note:** + > + > After enabling this feature, one import task can only import data to a single table at a time. If you want to use this feature to import data into different tables, you need to import several times, each time specifying a different target table. + + When **Custom Pattern** is enabled, you are required to specify a custom mapping rule between CSV files and a single target table in the following fields: + + - **Object Name Pattern**: enter a pattern that matches the names of the CSV files to be imported. If you have one CSV file only, you can enter the filename here directly. + + For example: + + - `my-data?.csv`: all CSV files starting with `my-data` and one character (such as `my-data1.csv` and `my-data2.csv`) will be imported into the same target table. + - `my-data*.csv`: all CSV files starting with `my-data` will be imported into the same target table. + + - **Target Table Name**: enter the name of the target table in TiDB Cloud, which must be in the `${db_name}.${table_name}` format. For example, `mydb.mytable`. Note that this field only accepts one specific table name, so wildcards are not supported. 4. Click **Import**. diff --git a/tidb-cloud/import-parquet-files.md b/tidb-cloud/import-parquet-files.md index 03c0b4f9b1209..4dc3768637188 100644 --- a/tidb-cloud/import-parquet-files.md +++ b/tidb-cloud/import-parquet-files.md @@ -23,71 +23,117 @@ You can import both uncompressed and Snappy compressed [Apache Parquet](https:// > - `ARRAY` > - `MAP` -1. If a Parquet file is larger than 256 MB, consider splitting the file into smaller files, each with a size around 256 MB. +1. If a Parquet file is larger than 256 MB, consider splitting it into smaller files, each with a size around 256 MB. - TiDB Cloud supports importing very large Parquet files but performs best with multiple input files around 256 MB in size. This is because TiDB Cloud can process multiple files in parallel which can greatly improve the import speed. + TiDB Cloud supports importing very large Parquet files but performs best with multiple input files around 256 MB in size. This is because TiDB Cloud can process multiple files in parallel, which can greatly improve the import speed. -2. According to the naming convention of existing objects in your bucket, identify a text pattern that matches the names of the parquet files to be imported. +2. Name the Parquet files as follows: - For example, to import all data files in a bucket, you can use the wildcard symbol `*` or `*.parquet` as a pattern. Similarly, to import the subset of data files in partition `station=402260`, you can use `*station=402260*` as a pattern. Make a note of this pattern as you will need to provide it to TiDB Cloud in [Step 4](#step-4-import-parquet-files-to-tidb-cloud). + - If a Parquet file contains all data of an entire table, name the file in the `${db_name}.${table_name}.parquet` format, which maps to the `${db_name}.${table_name}` table when you import the data. + - If the data of one table is separated into multiple Parquet files, append a numeric suffix to these Parquet files. For example, `${db_name}.${table_name}.000001.parquet` and `${db_name}.${table_name}.000002.parquet`. The numeric suffixes can be inconsecutive but must be in ascending order. You also need to add extra zeros before the number to ensure all the suffixes are in the same length. -## Step 2. Create the target database and table schema + > **Note:** + > + > If you cannot update the Parquet filenames according to the preceding rules in some cases (for example, the Parquet file links are also used by your other programs), you can keep the filenames unchanged and use the **Custom Pattern** in [Step 4](#step-4-import-parquet-files-to-tidb-cloud) to import your source data to a single target table. -Before importing Parquet files into TiDB Cloud, you need to create the target database and table. Alternatively, TiDB Cloud can create these objects for you as part of the import process if you provide the target database and table schema as follows: +## Step 2. Create the target table schemas -1. In the Amazon S3 or GCS directory where the parquet files are located, create a `${db_name}-schema-create.sql` file that contains the `CREATE DATABASE` DDL statement. +Because Parquet files do not contain schema information, before importing data from Parquet files into TiDB Cloud, you need to create the table schemas using either of the following methods: - For example, you can create a `mydb-scehma-create.sql` file that contains the following statement: +- Method 1: In TiDB Cloud, create the target databases and tables for your source data. - {{< copyable "sql" >}} +- Method 2: In the Amazon S3 or GCS directory where the Parquet files are located, create the target table schema files for your source data as follows: - ```sql - CREATE DATABASE mydb; - ``` + 1. Create database schema files for your source data. -2. In the Amazon S3 or GCS directory where the parquet files are located, create a `${db_name}.${table_name}-schema.sql` file that contains the `CREATE TABLE` DDL statement. + If your Parquet files follow the naming rules in [Step 1](#step-1-prepare-the-parquet-files), the database schema files are optional for the data import. Otherwise, the database schema files are mandatory. - For example, you can create a `mydb.mytable-schema.sql` file that contains the following statement: + Each database schema file must be in the `${db_name}-schema-create.sql` format and contain a `CREATE DATABASE` DDL statement. With this file, TiDB Cloud will create the `${db_name}` database to store your data when you import the data. - {{< copyable "sql" >}} + For example, if you create a `mydb-scehma-create.sql` file that contains the following statement, TiDB Cloud will create the `mydb` database when you import the data. - ```sql - CREATE TABLE mytable ( - ID INT, - REGION VARCHAR(20), - COUNT INT ); - ``` + {{< copyable "sql" >}} - > **Note:** - > - > The `${db_name}.${table_name}-schema.sql` file should only contain a single DDL statement. If the file contains multiple DDL statements, only the first statement takes effect. + ```sql + CREATE DATABASE mydb; + ``` + + 2. Create table schema files for your source data. + + If you do not include the table schema files in the Amazon S3 or GCS directory where the Parquet files are located, TiDB Cloud will not create the corresponding tables for you when you import the data. + + Each table schema file must be in the `${db_name}.${table_name}-schema.sql` format and contain a `CREATE TABLE` DDL statement. With this file, TiDB Cloud will create the `${db_table}` table in the `${db_name}` database when you import the data. + + For example, if you create a `mydb.mytable-schema.sql` file that contains the following statement, TiDB Cloud will create the `mytable` table in the `mydb` database when you import the data. + + {{< copyable "sql" >}} + + ```sql + CREATE TABLE mytable ( + ID INT, + REGION VARCHAR(20), + COUNT INT ); + ``` + + > **Note:** + > + > Each `${db_name}.${table_name}-schema.sql` file should only contain a single DDL statement. If the file contains multiple DDL statements, only the first one takes effect. ## Step 3. Configure cross-account access To allow TiDB Cloud to access the Parquet files in the Amazon S3 or GCS bucket, do one of the following: -- If your Parquet files are located in Amazon S3, [configure cross-account access to Amazon S3](/tidb-cloud/migrate-from-amazon-s3-or-gcs.md#step-2-configure-amazon-s3-access). +- If your Parquet files are located in Amazon S3, [configure cross-account access to Amazon S3](/tidb-cloud/config-s3-and-gcs-access.md#configure-amazon-s3-access). Once finished, make a note of the Role ARN value as you will need it in [Step 4](#step-4-import-parquet-files-to-tidb-cloud). -- If your Parquet files are located in GCS, [configure cross-account access to GCS](/tidb-cloud/migrate-from-amazon-s3-or-gcs.md#step-2-configure-gcs-access). +- If your Parquet files are located in GCS, [configure cross-account access to GCS](/tidb-cloud/config-s3-and-gcs-access.md#configure-gcs-access). ## Step 4. Import Parquet files to TiDB Cloud To import the Parquet files to TiDB Cloud, take the following steps: -1. Navigate to the TiDB Clusters page and click the name of your target cluster. The overview page of your target cluster is displayed. -2. In the cluster information pane on the left, click **Import**. The **Data Import Task** page is displayed. +1. Navigate to the **Active Clusters** page. +2. Find the area of your target cluster and click **Import Data** in the upper-right corner of the area. The **Data Import Task** page is displayed. + + > **Tip:** + > + > Alternatively, you can also click the name of your target cluster on the **Active Clusters** page and click **Import Data** in the upper-right corner. + 3. On the **Data Import Task** page, provide the following information. - **Data Source Type**: select the type of the data source. - **Bucket URL**: select the bucket URL where your Parquet files are located. - **Data Format**: select **Parquet**. - **Setup Credentials** (This field is visible only for AWS S3): enter the Role ARN value for **Role-ARN**. - - **Target Database**: fill in the **Username** and **Password** fields. - - **DB/Tables Filter**: if necessary, you can specify a [table filter](/table-filter.md#syntax). If you want to configure multiple filter rules, use `,` to separate the rules. - - **Object Name Pattern**: enter a pattern that matches the names of the Parquet files to be imported. For example,`my-data.parquet`. - - **Target Table Name**: enter the name of the target table. For example, `mydb.mytable`. + - **Target Cluster**: fill in the **Username** and **Password** fields. + - **DB/Tables Filter**: if you want to filter which tables to be imported, you can specify one or more table filters in this field, separated by `,`. + + For example: + + - `db01.*`: all tables in the `db01` database will be imported. + - `db01.table01*,db01.table02*`: all tables starting with `table01` and `table02` in the `db01` database will be imported. + - `!db02.*`: except the tables in the `db02` database, all other tables will be imported. `!` is used to exclude tables that do not need to be imported. + - `*.*` : all tables will be imported. + + For more information, see [table filter snytax](https://docs.pingcap.com/tidb/stable/table-filter#syntax). + + - **Custom Pattern**: enable the **Custom Pattern** feature if you want to import Parquet files whose filenames match a certain pattern to a single target table. + + > **Note:** + > + > After enabling this feature, one import task can only import data to a single table at a time. If you want to use this feature to import data into different tables, you need to import several times, each time specifying a different target table. + + When **Custom Pattern** is enabled, you are required to specify a custom mapping rule between Parquet files and a single target table in the following fields: + + - **Object Name Pattern**: enter a pattern that matches the names of the Parquet files to be imported. If you have one Parquet file only, you can enter the filename here directly. + + For example: + + - `my-data?.parquet`: all Parquet files starting with `my-data` and one character (such as `my-data1.parquet` and `my-data2.parquet`) will be imported into the same target table. + - `my-data*.parquet`: all Parquet files starting with `my-data` will be imported into the same target table. + + - **Target Table Name**: enter the name of the target table in TiDB Cloud, which must be in the `${db_name}.${table_name}` format. For example, `mydb.mytable`. Note that this field only accepts one specific table name, so wildcards are not supported. 4. Click **Import**. diff --git a/tidb-cloud/import-sample-data.md b/tidb-cloud/import-sample-data.md index 98899d0462666..a8f011877e9d0 100644 --- a/tidb-cloud/import-sample-data.md +++ b/tidb-cloud/import-sample-data.md @@ -7,9 +7,12 @@ summary: Learn how to import sample data into TiDB Cloud via UI. This document describes how to import sample data into TiDB Cloud via the UI. The sample data used is the system data from Capital Bikeshare, released under the Capital Bikeshare Data License Agreement. Before importing the sample data, you need to have one TiDB cluster. -1. Navigate to the **Active Clusters** page and click the name of your newly created cluster. The overview page of your cluster is displayed. +1. Navigate to the **Active Clusters** page. +2. Find the area of your cluster and click **Import Data** in the upper-right corner of the area. The **Data Import Task** page is displayed. -2. In the cluster information pane on the left, click **Import**. The **Data Import Task** page is displayed. + > **Tip:** + > + > Alternatively, you can also click the name of your cluster on the **Active Clusters** page and click **Import Data** in the upper-right corner. 3. Fill in the import parameters: @@ -22,9 +25,7 @@ This document describes how to import sample data into TiDB Cloud via the UI. Th - **Bucket URL**: enter the sample data URL `s3://tidbcloud-samples/data-ingestion/`. - **Data Format**: select **TiDB Dumpling**. - **Setup Credentials**: enter `arn:aws:iam::385595570414:role/import-sample-access` for Role-ARN. - - **Target Database**: - - **Username**: `root`. - - **Password**: enter your root password. + - **Target Cluster**: fill in the **Username** and **Password** fields. - **DB/Tables Filter**: leave this field blank. @@ -36,9 +37,7 @@ This document describes how to import sample data into TiDB Cloud via the UI. Th - **Data Source Type**: `Google Cloud Stroage`. - **Bucket URL**: enter the sample data URL `gcs://tidbcloud-samples-us-west1`. - **Data Format**: select **TiDB Dumpling**. - - **Target Database**: - - **Username**: `root`. - - **Password**: enter your root password. + - **Target Cluster**: fill in the **Username** and **Password** fields. - **DB/Tables Filter**: leave this field blank. diff --git a/tidb-cloud/migrate-data-into-tidb.md b/tidb-cloud/migrate-data-into-tidb.md index bef347761738b..4c877327ca31a 100644 --- a/tidb-cloud/migrate-data-into-tidb.md +++ b/tidb-cloud/migrate-data-into-tidb.md @@ -7,7 +7,7 @@ summary: Learn how to migrate data from MySQL-compatible databases to TiDB Cloud TiDB is highly compatible with MySQL. You can migrate data from any MySQL-compatible databases to TiDB smoothly, whether the data is from a self-hosted MySQL instance or RDS service provided by the public cloud. -This document describes how to use [Dumpling](/dumpling-overview.md) to export data from MySQL-compatible databases and use [TiDB Lightning](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview) logical import mode to import the data to TiDB Cloud. +This document describes how to use [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview) to export data from MySQL-compatible databases and use [TiDB Lightning](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview) logical import mode to import the data to TiDB Cloud. > **Note:** > @@ -55,7 +55,7 @@ TiUP is a package manager in the TiDB ecosystem, which can help you run any TiDB ## Step 2. Export data from MySQL-compatible databases -You can use several ways to dump data from MySQL, such as using `mysqldump` or `mydumper`. It is recommended to use [Dumpling](/dumpling-overview.md) for higher performance and compatibility with TiDB, which is also one of the open source tools created by PingCAP. +You can use several ways to dump data from MySQL, such as using `mysqldump` or `mydumper`. It is recommended to use [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview) for higher performance and compatibility with TiDB, which is also one of the open source tools created by PingCAP. 1. Install Dumpling: @@ -67,7 +67,7 @@ You can use several ways to dump data from MySQL, such as using `mysqldump` or ` 2. Export your MySQL database using Dumpling. - - To export your data to Amazon S3 cloud storage, see [Export data to Amazon S3 cloud storage](/dumpling-overview.md#export-data-to-amazon-s3-cloud-storage). + - To export your data to Amazon S3 cloud storage, see [Export data to Amazon S3 cloud storage](https://docs.pingcap.com/tidb/stable/dumpling-overview#export-data-to-amazon-s3-cloud-storage). - To export your data to local data files, use the following command: {{< copyable "shell-regular" >}} @@ -91,8 +91,8 @@ Depending on the location and size of your source data, the importing methods ar - If your source data is located in Amazon S3 cloud storage, take the following steps: - 1. Configure Amazon S3 access to allow TiDB cloud to access the source data in your Amazon S3 bucket. For more information, see [configure Amazon S3 access](/tidb-cloud/migrate-from-amazon-s3-or-gcs.md#step-2-configure-amazon-s3-access). - 2. From the TiDB Cloud console, navigate to the TiDB Clusters page, and then click the name of your target cluster to go to its own overview page. In the cluster information pane on the left, click **Import**, and then fill in the importing related information on the **Data Import Task** page. + 1. Configure Amazon S3 access to allow TiDB cloud to access the source data in your Amazon S3 bucket. For more information, see [configure Amazon S3 access](/tidb-cloud/config-s3-and-gcs-access.md#configure-amazon-s3-access). + 2. From the TiDB Cloud console, navigate to the TiDB Clusters page, and then click the name of your target cluster to go to its own overview page. In the upper-right corner, click **Import Data**, and then fill in the importing related information on the **Data Import Task** page. - If your source data is in local files, do one of the following: @@ -166,4 +166,8 @@ The following steps show how to import local data to TiDB Cloud using the logica After the importing task is started, you can view the importing progress in either of the following ways: - To get the progress using command lines, `grep` the keyword `progress` in logs, which is updated every 5 minutes by default. - - To get more monitoring metrics using the TiDB monitoring framework, see [TiDB Lightning Monitoring](https://docs.pingcap.com/tidb/stable/monitor-tidb-lightning). \ No newline at end of file + - To get more monitoring metrics using the TiDB monitoring framework, see [TiDB Lightning Monitoring](https://docs.pingcap.com/tidb/stable/monitor-tidb-lightning). + +## See also + +- [Migrate Incremental Data from MySQL-Compatible Databases](/tidb-cloud/migrate-incremental-data-from-mysql.md) diff --git a/tidb-cloud/migrate-from-amazon-s3-or-gcs.md b/tidb-cloud/migrate-from-amazon-s3-or-gcs.md index 9b401999673a6..f613117ac9beb 100644 --- a/tidb-cloud/migrate-from-amazon-s3-or-gcs.md +++ b/tidb-cloud/migrate-from-amazon-s3-or-gcs.md @@ -55,87 +55,9 @@ Before migrating data from Amazon S3 to TiDB Cloud, ensure you have administrato ### Step 2. Configure Amazon S3 access -To allow TiDB Cloud to access the source data in your Amazon S3 bucket, take the following steps to configure the bucket access for TiDB Cloud and get the Role-ARN. Once the configuration is done for one TiDB cluster in a project, all TiDB clusters in that project can use the same Role-ARN to access your Amazon S3 bucket. +To allow TiDB Cloud to access the source data in your Amazon S3 bucket, you need to configure the bucket access for TiDB Cloud and get the Role-ARN. Once the configuration is done for one TiDB cluster in a project, all TiDB clusters in that project can use the same Role-ARN to access your Amazon S3 bucket. -1. In the TiDB Cloud Console, get the TiDB Cloud account ID and external ID of the target TiDB cluster. - - 1. In the TiDB Cloud Console, choose your target project, and then click the name of your target cluster to go to its overview page. - 2. In the cluster overview pane on the left, click **Import**. The **Data Import Task** page is displayed. - 3. On the **Data Import Task** page, click **Show AWS IAM policy settings** to get the TiDB Cloud Account ID and TiDB Cloud External ID. Take a note of these IDs for later use. - -2. In the AWS Management Console, create a managed policy for your Amazon S3 bucket. - - 1. Sign in to the AWS Management Console and open the Amazon S3 console at . - 2. In the **Buckets** list, choose the name of your bucket with the source data, and then click **Copy ARN** to get your S3 bucket ARN (for example, `arn:aws:s3:::tidb-cloud-source-data`). Take a note of the bucket ARN for later use. - - ![Copy bucket ARN](/media/tidb-cloud/copy-bucket-arn.png) - - 3. Open the IAM console at , click **Policies** in the navigation pane on the left, and then click **Create Policy**. - - ![Create a policy](/media/tidb-cloud/aws-create-policy.png) - - 4. On the **Create policy** page, click the **JSON** tab. - 5. Copy the following access policy template and paste it to the policy text field. - - ``` - { - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "VisualEditor0", - "Effect": "Allow", - "Action": [ - "s3:GetObject", - "s3:GetObjectVersion" - ], - "Resource": "//*" - }, - { - "Sid": "VisualEditor1", - "Effect": "Allow", - "Action": [ - "s3:ListBucket", - "s3:GetBucketLocation" - ], - "Resource": "" - } - ] - } - ``` - - In the policy text field, update the following configurations to your own values. - - - `"Resource": "//*"` - - For example, if your source data is stored in the root directory of the `tidb-cloud-source-data` bucket, use `"Resource": "arn:aws:s3:::tidb-cloud-source-data/*"`. If your source data is stored in the `mydata` directory of the bucket, use `"Resource": "arn:aws:s3:::tidb-cloud-source-data/mydata/*"`. Make sure that `/*` is added to the end of the directory so TiDB Cloud can access all files in this directory. - - - `"Resource": ""` - - For example, `"Resource": "arn:aws:s3:::tidb-cloud-source-data"`. - - 6. Click **Next: Tags**, add a tag of the policy (optional), and then click **Next:Review**. - - 7. Set a policy name, and then click **Create policy**. - -3. In the AWS Management Console, create an access role for TiDB Cloud and get the role ARN. - - 1. In the IAM console at , click **Roles** in the navigation pane on the left, and then click **Create role**. - - ![Create a role](/media/tidb-cloud/aws-create-role.png) - - 2. To create a role, fill in the following information: - - - Under **Trusted entity type**, select **AWS account**. - - Under **An AWS account**, select **Another AWS account**, and then paste the TiDB Cloud account ID to the **Account ID** field. - - Under **Options**, click **Require external ID (Best practice when a third party will assume this role)**, and then paste the TiDB Cloud External ID to the **External ID** field. - - 3. Click **Next** to open the policy list, choose the policy you just created, and then click **Next**. - 4. Under **Role details**, set a name for the role, and then click **Create role** in the lower-right corner. After the role is created, the list of roles is displayed. - 5. In the list of roles, click the name of the role that you just created to go to its summary page, and then copy the role ARN. - - ![Copy AWS role ARN](/media/tidb-cloud/aws-role-arn.png) - -4. In the TiDB Cloud console, go to the **Data Import Task** page where you get the TiDB Cloud account ID and external ID, and then paste the role ARN to the **Role ARN** field. +For detailed steps, see [Configure Amazon S3 access](/tidb-cloud/config-s3-and-gcs-access.md#configure-amazon-s3-access). ### Step 3. Import data into TiDB Cloud @@ -145,7 +67,7 @@ To allow TiDB Cloud to access the source data in your Amazon S3 bucket, take the - **Bucket URL**: fill in the bucket URL of your source data. - **Data Format**: choose the format of your data. - **Target Cluster**: fill in the **Username** and **Password** fields. - - **DB/Tables Filter**: if necessary, you can specify a [table filter](/table-filter.md#syntax). If you want to configure multiple filter rules, use `,` to separate the rules. + - **DB/Tables Filter**: if necessary, you can specify a [table filter](https://docs.pingcap.com/tidb/stable/table-filter#syntax). If you want to configure multiple filter rules, use `,` to separate the rules. 2. Click **Import**. @@ -187,38 +109,7 @@ Before migrating data from GCS to TiDB Cloud, ensure the following: To allow TiDB cloud to access the source data in your GCS bucket, you need to configure the GCS access for each TiDB Cloud as a service on the GCP project and GCS bucket pair. Once the configuration is done for one cluster in a project, all database clusters in that project can access the GCS bucket. -1. Get the Google Cloud Service Account ID of the target TiDB cluster. - - 1. In the TiDB Cloud Admin console, choose a target project and a target cluster deployed on the Google Cloud Platform, and then click **Import**. - 2. Click **Show Google Cloud Service Account ID**, and then copy the Service Account ID. - -2. In the Google Cloud Platform (GCP) Management Console, go to **IAM & Admin** > **Roles**, and then check whether a role with the following read-only permissions of the storage container exists. - - - storage.buckets.get - - storage.objects.get - - storage.objects. list - - If yes, you can use the matched role for the target TiDB cluster in the following steps. If not, go to **IAM & Admin** > **Roles** > **CREATE ROLE** to define a role for the target TiDB cluster. - -3. Go to **Cloud Storage** > **Browser**, select the GCS bucket you want TiDB Cloud to access, and then click **SHOW INFO PANEL**. - - The panel is displayed. - -4. In the panel, click **ADD PRINCIPAL**. - - The dialog box for adding principals is displayed. - -5. In the dialog box, perform the following steps: - - 1. In the **New Principals** field, paste the Google Cloud Service Account ID of the target TiDB cluster. - 2. In the **Role** drop-down list, choose the role of the target TiDB cluster. - 3. Click **SAVE**. - -Your TiDB Cloud cluster can now access the GCS bucket. - -> **Note:** -> -> To remove the access to TiDB Cloud, you can simply delete the principal that you added. +For detailed steps, see [Configure GCS access](/tidb-cloud/config-s3-and-gcs-access.md#configure-gcs-access). ### Step 3. Copy source data files to GCS and import data into TiDB Cloud @@ -237,7 +128,7 @@ Your TiDB Cloud cluster can now access the GCS bucket. gsutil rsync -r ./tidbcloud-samples-us-west-2/ gs://target-url-in-gcs ``` -2. From the TiDB Cloud console, navigate to the TiDB Clusters page, and then click the name of your target cluster to go to its own overview page. In the cluster information pane on the left, click **Import**, and then fill in the importing related information on the **Data Import Task** page. +2. From the TiDB Cloud console, navigate to the TiDB Clusters page, and then click the name of your target cluster to go to its own overview page. In the upper-right corner, click **Import Data**, and then fill in the importing related information on the **Data Import Task** page. > **Note:** > diff --git a/tidb-cloud/migrate-from-aurora-bulk-import.md b/tidb-cloud/migrate-from-aurora-bulk-import.md index 56769fd46b603..293d65642535f 100644 --- a/tidb-cloud/migrate-from-aurora-bulk-import.md +++ b/tidb-cloud/migrate-from-aurora-bulk-import.md @@ -5,17 +5,22 @@ summary: Learn how to migrate data from Amazon Aurora MySQL to TiDB Cloud in bul # Migrate from Amazon Aurora MySQL to TiDB Cloud in Bulk -This document describes how to migrate data from Amazon Aurora MySQL to TiDB Cloud in bulk using the import tools on TiDB Cloud console. +This document describes how to migrate data from Amazon Aurora MySQL to TiDB Cloud in bulk using the import tools on TiDB Cloud console. ## Learn how to create an import task on the TiDB Cloud console To import data, perform the following steps: -1. Navigate to the TiDB Clusters page and click the name of your target cluster. The overview page of your target cluster is displayed. -2. In the cluster information pane on the left, click **Import**. The **Data Import Task** page is displayed. +1. Navigate to the **Active Clusters** page. +2. Find the area of your target cluster and click **Import Data** in the upper-right corner of the area. The **Data Import Task** page is displayed. + + > **Tip:** + > + > Alternatively, you can also click the name of your target cluster on the **Active Clusters** page and click **Import Data** in the upper-right corner. + 3. Prepare source data according to [Learn how to create an Amazon S3 Bucket and prepare source data files](#learn-how-to-create-an-amazon-s3-bucket-and-prepare-source-data-files). You can see the advantages and disadvantages of different **Data Format** in the preparing data part. 4. Fill in the **Data Source Type**, **Bucket URL**, and **Data Format** fields according to the specification of your source data. -5. Fill in the **Username** and **Password** fields of the **Target Database** according to the connection settings of your cluster. +5. Fill in the **Username** and **Password** fields of the **Target Cluster**. 6. Create the bucket policy and role for cross-account access according to [Learn how to configure cross-account access](#learn-how-to-configure-cross-account-access). 7. Click **Import**. @@ -35,7 +40,7 @@ To prepare data, you can select one from the following two options: - [Option 1: Prepare source data files using Dumpling](#option-1-prepare-source-data-files-using-dumpling) - You need to launch [Dumpling](/dumpling-overview.md) on your EC2, and export the data to Amazon S3. The data you export is the current latest data of your source database. This might affect the online service. Dumpling will lock the table when you export data. + You need to launch [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview) on your EC2, and export the data to Amazon S3. The data you export is the current latest data of your source database. This might affect the online service. Dumpling will lock the table when you export data. - [Option 2: Prepare source data files using Amazon Aurora snapshots](#option-2-prepare-source-data-files-using-amazon-aurora-snapshots) @@ -82,7 +87,7 @@ You need to prepare an EC2 to run the following data export task. It's better to ```bash curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh source ~/.bash_profile - tiup install dumpling + tiup install dumpling ``` In the above commands, you need to modify `~/.bash_profile` to the path of your profile file. @@ -92,7 +97,7 @@ You need to prepare an EC2 to run the following data export task. It's better to > **Note:** > > If you have assigned the IAM role to the EC2, you can skip configuring the access key and security key, and directly run Dumpling on this EC2. - + You can grant the write privilege using the access key and security key of your AWS account in the environment. Create a specific key pair for preparing data, and revoke the access key immediately after you finish the preparation. {{< copyable "shell-regular" >}} @@ -104,7 +109,7 @@ You need to prepare an EC2 to run the following data export task. It's better to 3. Back up the source database to S3. - Use Dumpling to export the data from Amazon Aurora. Based on your environment, replace the content in angle brackets (>), and then execute the following commands. If you want to use filter rules when exporting the data, refer to [Table Filter](/table-filter.md#syntax). + Use Dumpling to export the data from Amazon Aurora. Based on your environment, replace the content in angle brackets (>), and then execute the following commands. If you want to use filter rules when exporting the data, refer to [Table Filter](https://docs.pingcap.com/tidb/stable/table-filter#syntax). {{< copyable "shell-regular" >}} @@ -114,7 +119,7 @@ You need to prepare an EC2 to run the following data export task. It's better to export_endpoint="" # You will use the s3 url when you create importing task backup_dir="s3:///" - s3_bucket_region="" + s3_bucket_region="" # Use `tiup -- dumpling` instead if "flag needs an argument: 'h' in -h" is prompted for TiUP versions earlier than v1.8 tiup dumpling \ @@ -159,7 +164,7 @@ To migrate data from Aurora, you need to back up the schema of the database. mysqldump -h ${export_endpoint} -u ${export_username} -p --ssl-mode=DISABLED -d${export_database} >db.sql ``` -3. Import the schema of the database into TiDB Cloud. +3. Import the schema of the database into TiDB Cloud. {{< copyable "sql" >}} @@ -189,7 +194,7 @@ To migrate data from Aurora, you need to back up the schema of the database. 7. Choose the proper IAM role to grant write access to the S3 bucket. Make a note of this role as it will be used later when you import the snapshot to TiDB Cloud. - 8. Choose a proper AWS KMS Key and make sure the IAM role has already been added to the KMS Key Users. To add a role, you can select a KSM service, select the key, and then click **Add**. + 8. Choose a proper AWS KMS Key and make sure the IAM role has already been added to the KMS Key Users. To add a role, you can select a KSM service, select the key, and then click **Add**. 9. Click **Export Amazon S3**. You can see the progress in the task table. @@ -197,13 +202,13 @@ To migrate data from Aurora, you need to back up the schema of the database. ## Learn how to configure cross-account access -The TiDB Cloud cluster and the S3 bucket are in different AWS accounts. To allow the TiDB Cloud cluster to access the source data files in the S3 bucket, you need to configure the cross-account access to Amazon S3. For more information, see [Configure the Amazon S3 access](/tidb-cloud/migrate-from-amazon-s3-or-gcs.md#step-2-configure-amazon-s3-access). +The TiDB Cloud cluster and the S3 bucket are in different AWS accounts. To allow the TiDB Cloud cluster to access the source data files in the S3 bucket, you need to configure the cross-account access to Amazon S3. For more information, see [Configure Amazon S3 access](/tidb-cloud/config-s3-and-gcs-access.md#configure-amazon-s3-access). Once finished, you will have created a policy and role for cross-account. You can then continue with the configuration on the data import task panel of TiDB Cloud. ## Learn how to set up filter rules -Refer to the [Table Filter](/table-filter.md#syntax) document. +Refer to the [Table Filter](https://docs.pingcap.com/tidb/stable/table-filter#syntax) document. ## Learn how to clean up incomplete data diff --git a/tidb-cloud/migrate-incremental-data-from-mysql.md b/tidb-cloud/migrate-incremental-data-from-mysql.md new file mode 100644 index 0000000000000..5b0ffffcdd122 --- /dev/null +++ b/tidb-cloud/migrate-incremental-data-from-mysql.md @@ -0,0 +1,267 @@ +--- +title: Migrate Incremental Data from MySQL-Compatible Databases +summary: Learn how to migrate incremental data from MySQL-compatible databases to TiDB Cloud. +--- + +# Migrate Incremental Data from MySQL-Compatible Databases + +This document describes how to migrate incremental data from MySQL-compatible databases to TiDB Cloud. + +## Before you begin + +Before you perform incremental data migration, you should have finished full data migration from MySQL-compatible databases to TiDB Cloud. For more information, see [Migrate Data from MySQL-Compatible Databases](/tidb-cloud/migrate-data-into-tidb.md). + +## Step 1. Deploy a DM Cluster + +The TiDB Cloud console does not provide incremental data migration feature yet. You need to deploy [TiDB Data Migration](https://docs.pingcap.com/tidb/stable/dm-overview) (DM) manually to perform incremental migration to TiDB Cloud. For installation steps, see [Deploy a DM Cluster Using TiUP](https://docs.pingcap.com/tidb/stable/deploy-a-dm-cluster-using-tiup). + +## Step 2. Create a data source configuration file + +You need to create a data source configuration file first. The data source is a MySQL instance that you want to migrate data from. The following is an example of creating a data source configuration file. You need to replace the MySQL IP address, port, user name, and password values in the file with your own values. + +```shell +# Encrypt MySQL password +[root@localhost ~]# tiup dmctl encrypt {mysq-user-password} +mZMkdjbRztSag6qEgoh8UkDY6X13H48= + +[root@localhost ~]# cat dm-source1.yaml +``` + +```yaml +# MySQL Configuration. +source-id: "mysql-replica-01" + +# Configures whether DM-worker uses the global transaction identifier (GTID) to pull binlogs. +# To enable this mode, the upstream MySQL must also enable GTID. +# If the upstream MySQL service is configured to switch master between different nodes automatically, GTID mode is required. +enable-gtid: true + +from: + host: "192.168.10.101" + user: "user01" + password: "mZMkdjbRztSag6qEgoh8UkDY6X13H48=" + port: 3307 +``` + +Load the data source configuration to the DM cluster using `tiup dmctl` by running the following command: + +```shell +[root@localhost ~]# tiup dmctl --master-addr ${advertise-addr} operate-source create dm-source1.yaml +``` + +The parameters used in the command above are described as follows: + +|Parameter |Description | +|- |- | +|`--master-addr` |The `{advertise-addr}` of any DM-master node in the cluster where `dmctl` is to be connected. For example: 172.16.7.140:9261| +|`operate-source create`|Loads the data source to the DM cluster.| + +The following is an example output: + +``` +tiup is checking updates for component dmctl ... +Starting component `dmctl`: /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl --master-addr 192.168.11.110:9261 operate-source create dm-source1.yaml +{ + "result": true, + "msg": "", + "sources": [ + { + "result": true, + "msg": "", + "source": "mysql-replica-01", + "worker": "dm-192.168.11.120-9262" + } + ] +} +``` + +## Step 3. Create a migration task + +Create a `dm-task1.yaml` file for the migration. Configure the incremental migration mode and the starting point of the data source in the file. + +You can find the starting point in the metadata file exported by [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview). For example: + +```toml +# Get the contents of the metadata in the file exported by Dumpling +# Use it to configure the incremental migration starting point +# cat metadata +Started dump at: 2022-05-24 11:19:37 +SHOW MASTER STATUS: + Log: mysql-bin.000001 + Pos: 77092852 + GTID:b631bcad-bb10-11ec-9eee-fec83cf2b903:1-640 + +Finished dump at: 2022-05-24 11:19:53 +``` + +Based on the above starting point information, create a migration task as follows: + +```toml +## ********* Task Configuration ********* +name: test-task1 +# shard-mode: "pessimistic" +# Task mode. The "incremental" mode only performs incremental data migration. +task-mode: incremental +# timezone: "UTC" + +## ******** Data Source Configuration ********** +## (Optional) If you need to incrementally replicate data that has already been migrated in the full data migration, you need to enable the safe mode to avoid the incremental data migration error. +## This scenario is common in the following case: the full migration data does not belong to the data source's consistency snapshot, and after that, DM starts to replicate incremental data from a position earlier than the full migration. +syncers: # The running configurations of the sync processing unit. + global: # Configuration name. + safe-mode: false # If this field is set to true, DM changes INSERT of the data source to REPLACE for the target database, and changes UPDATE of the data source to DELETE and REPLACE for the target database. This is to ensure that when the table schema contains a primary key or unique index, DML statements can be imported repeatedly. In the first minute of starting or resuming an incremental migration task, DM automatically enables the safe mode. + +mysql-instances: + - source-id: "mysql-replica-01" + block-allow-list: "bw-rule-1" + route-rules: ["route-rule-1"] + filter-rules: ["tpcc-filter-rule"] + syncer-config-name: "global" # You can use the syncers incremental data configuration above. + meta: # When task-mode is "incremental" and the target database does not have a checkpoint, DM uses the binlog position as the starting point. If the target database has a checkpoint, DM uses the checkpoint as the starting point. + binlog-name: "mysql-bin.000001" + binlog-pos: 77092852 + binlog-gtid: "b631bcad-bb10-11ec-9eee-fec83cf2b903:1-640" + +## ******** Configuration of the target TiDB cluster on TiDB Cloud ********** +target-database: # The target TiDB cluster on TiDB Cloud + host: "tidb.70593805.b973b556.ap-northeast-1.prod.aws.tidbcloud.com" + port: 4000 + user: "root" + password: "oSWRLvR3F5GDIgm+l+9h3kB72VFWBUwzOw==" # If the password is not empty, it is recommended to use a dmctl-encrypted cipher. + +## ******** Function Configuration ********** +block-allow-list: + bw-rule-1: + do-dbs: ["~^tpcc.*"] + +routes: # Table renaming rules ('routes') from upstream to downstream tables, in order to support merging different tables into a single target table. + route-rule-1: # Rule name. + schema-pattern: "tpcc" # Rule for matching upstream schema names. It supports the wildcards "*" and "?". + target-schema: "tpdd" # Name of the target schema. + +filters: + tpcc-filter-rule: + schema-pattern: "tpcc" + events: ["drop database"] + action: Ignore + +## ******** Ignore check items ********** +ignore-checking-items: ["table_schema"] +``` + +For detailed task configurations, see [DM Task Configurations](https://docs.pingcap.com/tidb/stable/task-configuration-file-full). + +To run a data migration task smoothly, DM triggers a precheck automatically at the start of the task and returns the check results. DM starts the migration only after the precheck is passed. To trigger a precheck manually, run the `check-task` command: + +```shell +[root@localhost ~]# tiup dmctl --master-addr ${advertise-addr} check-task dm-task1.yaml +``` + +The following is an example output: + +``` +tiup is checking updates for component dmctl ... +Starting component `dmctl`: /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl --master-addr 192.168.11.110:9261 check-task dm-task1.yaml +{ + "result": true, + "msg": "check pass!!!" +} +``` + +## Step 4. Start the migration task + +Run the following command to start the migration task: + +```shell +[root@localhost ~]# tiup dmctl --master-addr ${advertise-addr} start-task dm-task1.yaml +``` + +The parameters used in the command above are described as follows: + +|Parameter |Description | +|- |- | +|`--master-addr` |The `{advertise-addr}` of any DM-master node in the cluster where `dmctl` is to be connected. For example: 172.16.7.140:9261| +|`start-task` |Starts the migration task.| + +The following is an example output: + +``` +tiup is checking updates for component dmctl ... +Starting component `dmctl`: /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl --master-addr 192.168.11.110:9261 start-task dm-task1.yaml +{ + "result": true, + "msg": "", + "sources": [ + { + "result": true, + "msg": "", + "source": "mysql-replica-01", + "worker": "dm-192.168.11.120-9262" + } + ], + "checkResult": "" +} +``` + +If the task fails to start, check the prompt message and fix the configuration. After that, you can re-run the command above to start the task. + +If you encounter any problem, refer to [DM error handling](https://docs.pingcap.com/tidb/stable/dm-error-handling) and [DM FAQ](https://docs.pingcap.com/tidb/stable/dm-faq). + +## Step 5. Check the migration task status + +To learn whether the DM cluster has an ongoing migration task and view the task status, run the `query-status` command using `tiup dmctl`: + +```shell +[root@localhost ~]# tiup dmctl --master-addr ${advertise-addr} query-status ${task-name} +``` + +The following is an example output: + +``` +tiup is checking updates for component dmctl ... +Starting component `dmctl`: /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl /root/.tiup/components/dmctl/v6.0.0/dmctl/dmctl --master-addr 192.168.11.110:9261 query-status test-task1 +{ + "result": true, + "msg": "", + "sources": [ + { + "result": true, + "msg": "", + "sourceStatus": { + "source": "mysql-replica-01", + "worker": "dm-192.168.11.120-9262", + "result": null, + "relayStatus": null + }, + "subTaskStatus": [ + { + "name": "test-task1", + "stage": "Running", + "unit": "Sync", + "result": null, + "unresolvedDDLLockID": "", + "sync": { + "totalEvents": "3", + "totalTps": "0", + "recentTps": "0", + "masterBinlog": "(mysql-bin.000001, 77093211)", + "masterBinlogGtid": "b631bcad-bb10-11ec-9eee-fec83cf2b903:1-641", + "syncerBinlog": "(mysql-bin.000001, 77093211)", + "syncerBinlogGtid": "b631bcad-bb10-11ec-9eee-fec83cf2b903:1-641", + "blockingDDLs": [ + ], + "unresolvedGroups": [ + ], + "synced": true, + "binlogType": "remote", + "secondsBehindMaster": "0", + "blockDDLOwner": "", + "conflictMsg": "" + } + } + ] + ] +} +``` + +For a detailed interpretation of the results, see [Query Status](https://docs.pingcap.com/tidb/stable/dm-query-status). diff --git a/tidb-cloud/monitor-tidb-cluster.md b/tidb-cloud/monitor-tidb-cluster.md index 2eb52ccca69e5..f6eca5ea70e56 100644 --- a/tidb-cloud/monitor-tidb-cluster.md +++ b/tidb-cloud/monitor-tidb-cluster.md @@ -45,13 +45,36 @@ You can see the current status of each running cluster on the cluster page. ## Monitoring metrics -On the cluster overview page, you can view the commonly used metrics of the cluster. +In TiDB Cloud, you can view the commonly used metrics of a cluster from the following pages: + +- Cluster overview page +- Cluster monitoring page + +### Metrics on the cluster overview page + +The cluster overview page provides general metrics of a cluster, including Total QPS, Latency, Connections, TiFlash Request QPS, TiFlash Request Duration, TiFlash Storage Size, TiKV Storage Size, TiDB CPU, TiKV CPU, TiKV IO Read, and TiKV IO Write. + +To view metrics on the cluster overview page, take the following steps: 1. Navigate to the **Active Clusters** page. -2. Click the name of a cluster to go to the cluster overview page. +2. Click the name of a cluster to go to its cluster overview page. + +### Metrics on the cluster monitoring page + +The cluster monitoring page provides a full set of standard metrics of a cluster. By viewing these metrics, you can easily identify performance issues and determine whether your current database deployment meets your requirements. + +> **Note:** +> +> Currently, the cluster monitoring page is unavailable for [Developer Tier clusters](/tidb-cloud/select-cluster-tier.md#developer-tier). + +To view metrics on the cluster monitoring page, take the following steps: + +1. Navigate to the **Diagnosis** tab of a cluster. + +2. Click the **Monitoring** tab. - Currently, the metrics include Total QPS, Latency, Connections, TiFlash Request QPS, TiFlash Request Duration, TiFlash Storage Size, TiKV Storage Size, TiDB CPU, TiKV CPU, TiKV IO Read, and TiKV IO Write. +For more information, see [Built-in Monitoring](/tidb-cloud/built-in-monitoring.md). ## Built-in alerting diff --git a/tidb-cloud/release-notes-2022.md b/tidb-cloud/release-notes-2022.md index 816f4625a4daf..64f4d85dc23d9 100644 --- a/tidb-cloud/release-notes-2022.md +++ b/tidb-cloud/release-notes-2022.md @@ -8,47 +8,83 @@ aliases: ['/tidbcloud/beta/supported-tidb-versions','/tidbcloud/release-notes'] This page lists the release notes of [TiDB Cloud](https://en.pingcap.com/tidb-cloud/) in 2022. +## August 9, 2022 + +* Add the support of the GCP region `Osaka` for [Dedicated Tier](/tidb-cloud/select-cluster-tier.md#dedicated-tier) cluster creation. + +## August 2, 2022 + +* The `4 vCPU, 16 GiB` node size of TiDB and TiKV is now in General Availability (GA). + + * For each `4 vCPU, 16 GiB` TiKV node, the storage size is between 200 GiB and 2 TiB. + * Suggested usage scenarios: + + * Low workload production environments for SMB + * PoC and staging environments + * Development environments + +* Add a [Monitoring page](/tidb-cloud/built-in-monitoring.md) to the **Diagnosis** tab for [Dedicated Tier clusters](/tidb-cloud/select-cluster-tier.md#dedicated-tier). + + The Monitoring page provides a system-level entry for overall performance diagnosis. According to the top-down performance analysis methodology, the Monitoring page organizes TiDB performance metrics based on database time breakdown and displays these metrics in different colors. By checking these colors, you can identify performance bottlenecks of the entire system at the first glance, which significantly reduces performance diagnosis time and simplifies performance analysis and diagnosis. + +* Add a switch to enable or disable **Custom Pattern** on the **Data Import** page for CSV and Parquet source files. + + The **Custom Pattern** feature is disabled by default. You can enable it when you are going to import CSV or Parquet files whose filenames match a certain pattern to a single target table. + + For more information, see [Import CSV Files](/tidb-cloud/import-csv-files.md) and [Import Apache Parquet Files](/tidb-cloud/import-parquet-files.md). + +* Add TiDB Cloud Support Plans (Basic, Standard, Enterprise, and Premium) to meet different support needs of customers' organizations. For more information, see [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md). + +* Optimize the UI of the [Active Clusters](https://tidbcloud.com/console/clusters) page and the cluster details page: + + * Add **Connect** and **Import data** buttons to the **Active Clusters** page. + * Move **Connect** and **Import data** buttons to the upper-right corner on the cluster details page. + +## July 28, 2022 + +* Add the **Allow Access from Anywhere** button to the **Security Quick Start** dialog, which allows your cluster to be accessible by any IP addresses. For more information, see [Configure Cluster Security Settings](/tidb-cloud/configure-security-settings.md). + ## July 26, 2022 -* Support automatic hibernation and resuming for new Developer Tier clusters. +* Support [automatic hibernation and resuming](/tidb-cloud/select-cluster-tier.md#automatic-hibernation-and-resuming) for new [Developer Tier clusters](/tidb-cloud/select-cluster-tier.md#developer-tier). A Developer Tier cluster will not be deleted after 7 days of inactivity so you can still use it at any time until the one-year free trial ends. After 24 hours of inactivity, the Developer Tier cluster will hibernate automatically. To resume the cluster, either send a new connection to the cluster or click the **Resume** button in the TiDB Cloud console. The cluster will be resumed within 50 seconds and back to service automatically. -* Add a user name prefix limitation for new Developer Tier clusters +* Add a user name prefix limitation for new [Developer Tier clusters](/tidb-cloud/select-cluster-tier.md#developer-tier). Whenever you use or set a database user name, you must include the prefix for your cluster in the user name. For more information, see [User name prefix](/tidb-cloud/select-cluster-tier.md#user-name-prefix). -* Disable the backup and restore feature for Developer Tier clusters. +* Disable the backup and restore feature for [Developer Tier clusters](/tidb-cloud/select-cluster-tier.md#developer-tier). The backup and restore feature (including both automatic backup and manual backup) is disabled for Developer Tier clusters. You can still use [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview) to export your data as a backup. -* Increase the storage size of a Developer Tier cluster from 500 MiB to 1 GiB. +* Increase the storage size of a [Developer Tier](/tidb-cloud/select-cluster-tier.md#developer-tier) cluster from 500 MiB to 1 GiB. * Add breadcrumbs to the TiDB Cloud console to improve the navigation experience. * Support configuring multiple filter rules when you import data into TiDB Cloud. * Remove the **Traffic Filters** page from **Project Settings**, and remove the **Add Rules from Default Set** button from the **Connect to TiDB** dialog. ## July 19, 2022 -* Provide a new option for TiKV node size: `8 vCPU, 32 GiB`. You can choose either `8 vCPU, 32 GiB` or `8 vCPU, 64 GiB` for an 8 vCPU TiKV node. -* Support syntax highlighting in sample code provided in the **Connect to TiDB** dialog to improve code readability. You can easily identify the parameters that you need to replace in the sample code. -* Support automatically validating whether TiDB Cloud can access your source data after you confirm the import task on the **Data Import Task** page. +* Provide a new option for [TiKV node size](/tidb-cloud/size-your-cluster.md#tikv-node-size): `8 vCPU, 32 GiB`. You can choose either `8 vCPU, 32 GiB` or `8 vCPU, 64 GiB` for an 8 vCPU TiKV node. +* Support syntax highlighting in sample code provided in the [**Connect to TiDB**](/tidb-cloud/connect-to-tidb-cluster.md#connect-via-standard-connection) dialog to improve code readability. You can easily identify the parameters that you need to replace in the sample code. +* Support automatically validating whether TiDB Cloud can access your source data after you confirm the import task on the [**Data Import Task**](/tidb-cloud/import-sample-data.md) page. * Change the theme color of the TiDB Cloud console to make it consistent with that of [PingCAP website](https://en.pingcap.com/). ## July 12, 2022 -* Add the **Validate** button to the **Data Import Task** page for Amazon S3, which helps you detect data access issues before the data import starts. -* Add **Billing Profile** under the **Payment Method** tab. By providing your tax registration number in **Billing Profile**, certain taxes might be exempted from your invoice. +* Add the **Validate** button to the [**Data Import Task**](/tidb-cloud/import-sample-data.md) page for Amazon S3, which helps you detect data access issues before the data import starts. +* Add **Billing Profile** under the [**Payment Method**](/tidb-cloud/tidb-cloud-billing.md#payment-method) tab. By providing your tax registration number in **Billing Profile**, certain taxes might be exempted from your invoice. For more information, see [Edit billing profile information](/tidb-cloud/tidb-cloud-billing.md#edit-billing-profile-information). ## July 05, 2022 -* The columnar storage TiFlash is now in General Availability (GA). +* The columnar storage [TiFlash](/tiflash/tiflash-overview.md) is now in General Availability (GA). - TiFlash makes TiDB essentially an Hybrid Transactional/Analytical Processing (HTAP) database. Your application data is first stored in TiKV and then replicated to TiFlash via the Raft consensus algorithm. So it is real time replication from the row storage to the columnar storage. - For tables with TiFlash replicas, the TiDB optimizer automatically determines whether to use either TiKV or TiFlash replicas based on the cost estimation. To experience the benefits brought by TiFlash, see [TiDB Cloud HTAP Quick Start Guide](/tidb-cloud/tidb-cloud-htap-quickstart.md). -* Support [increasing the storage size](/tidb-cloud/scale-tidb-cluster.md) of TiKV and TiFlash for a Dedicated Tier cluster. +* Support [increasing the storage size](/tidb-cloud/scale-tidb-cluster.md#increase-storage-size) of TiKV and TiFlash for a Dedicated Tier cluster. * Support showing the memory information in the node size field. ## June 28, 2022 @@ -57,21 +93,21 @@ This page lists the release notes of [TiDB Cloud](https://en.pingcap.com/tidb-cl ## June 23, 2022 -* Increase the maximum storage capacity of TiKV on TiDB Cloud. +* Increase the maximum [storage capacity of TiKV](/tidb-cloud/size-your-cluster.md#tikv-storage-size) on TiDB Cloud. * 8 vCPU or 16 vCPU TiKV: support up to 4 TiB storage capacity. * 4 vCPU TiKV: support up to 2 TiB storage capacity. ## June 21, 2022 -* Add the support of the GCP region `Taiwan` for Dedicated Tier cluster creation. +* Add the support of the GCP region `Taiwan` for [Dedicated Tier](/tidb-cloud/select-cluster-tier.md#dedicated-tier) cluster creation. * Support [updating user profiles](/tidb-cloud/manage-user-access.md#manage-user-profiles) on the TiDB Cloud console, including first name, last time, company name, country, and phone number. -* Provide the connection strings for MySQL, MyCLI, JDBC, Python, Go, and Node.js on the TiDB Cloud console so you can easily connect to your TiDB cluster. +* Provide the connection strings for MySQL, MyCLI, JDBC, Python, Go, and Node.js in the [**Connect to TiDB**](/tidb-cloud/connect-to-tidb-cluster.md#connect-via-standard-connection) dialog so you can easily connect to your TiDB cluster. * Support obtaining bucket regions from bucket URLs automatically during data import to save your effort to fill in such information. ## June 16, 2022 -* Simplify the cluster creation process. +* Simplify the [cluster creation process](/tidb-cloud/create-tidb-cluster.md). - When you create a cluster, TiDB Cloud provides a default cluster name. You can either use the default name or update it. - When you create a cluster, you do not need to set the password on the **Create a Cluster** page. @@ -87,15 +123,15 @@ This page lists the release notes of [TiDB Cloud](https://en.pingcap.com/tidb-cl * Add the [Try Free](https://tidbcloud.com/free-trial) registration page to quickly sign up for TiDB Cloud. * Remove the **Proof of Concept plan** option from the plan selection page. If you want to apply for a 14-day PoC trial for free, go to the [Apply for PoC](https://en.pingcap.com/apply-for-poc/) page. For more information, see [Perform a Proof of Concept (PoC) with TiDB Cloud](/tidb-cloud/tidb-cloud-poc.md). -* Improve the system security by prompting users who sign up for TiDB Cloud with emails and passwords to reset their passwords every 90 days. +* Improve the system security by prompting users who sign up for TiDB Cloud with emails and passwords to reset their passwords every 90 days. For more information, see [Manage user passwords](/tidb-cloud/manage-user-access.md#manage-user-passwords). ## May 24, 2022 -* Support customizing TiDB port number when you create or restore a Dedicated Tier cluster. +* Support customizing TiDB port number when you [create](/tidb-cloud/create-tidb-cluster.md) or [restore](/tidb-cloud/backup-and-restore.md#restore) a Dedicated Tier cluster. ## May 19, 2022 -* Add the support of the AWS region `Frankfurt` for Developer Tier cluster creation. +* Add the support of the AWS region `Frankfurt` for [Developer Tier](/tidb-cloud/select-cluster-tier.md#developer-tier) cluster creation. ## May 18, 2022 @@ -107,7 +143,7 @@ This page lists the release notes of [TiDB Cloud](https://en.pingcap.com/tidb-cl ## May 1, 2022 -* Support configuring vCPU size of TiDB, TiKV, and TiFlash when you create or restore a cluster. +* Support configuring vCPU size of TiDB, TiKV, and TiFlash when you [create](/tidb-cloud/create-tidb-cluster.md) or [restore](/tidb-cloud/backup-and-restore.md#restore) a [Dedicated Tier](/tidb-cloud/select-cluster-tier.md#dedicated-tier) cluster. * Add the support of the AWS region `Mumbai` for cluster creation. * Update the compute, storage, and data transfer cost for [TiDB Cloud billing](/tidb-cloud/tidb-cloud-billing.md). @@ -119,9 +155,9 @@ This page lists the release notes of [TiDB Cloud](https://en.pingcap.com/tidb-cl TiDB Cloud is now in General Availability. You can [sign up](https://tidbcloud.com/signup) and select one of the following options: -* Get started with Developer Tier for free. -* Apply for a 14-day PoC trial for free. -* Get full access with the Dedicated Tier. +* Get started with [Developer Tier](/tidb-cloud/select-cluster-tier.md#developer-tier) for free. +* Apply for [a 14-day PoC trial for free](https://en.pingcap.com/apply-for-poc/). +* Get full access with [Dedicated Tier](/tidb-cloud/select-cluster-tier.md#dedicated-tier). ## March 25, 2022 @@ -135,9 +171,9 @@ New feature: General changes: -* No cluster tier with the fixed cluster size any more. You can customize the cluster size of TiDB, TiKV, and TiFlash easily. -* Support adding TiFlash nodes for an existing cluster without TiFlash. -* Support specifying the storage size (500 to 2048 GiB) when creating a new cluster. The storage size cannot be changed after the cluster is created. +* No cluster tier with the fixed cluster size any more. You can customize the [cluster size](/tidb-cloud/size-your-cluster.md) of TiDB, TiKV, and TiFlash easily. +* Support adding [TiFlash](/tiflash/tiflash-overview.md) nodes for an existing cluster without TiFlash. +* Support specifying the storage size (500 to 2048 GiB) when [creating a new cluster](/tidb-cloud/create-tidb-cluster.md). The storage size cannot be changed after the cluster is created. * Introduce a new public region: `eu-central-1`. * Deprecate 8 vCPU TiFlash and provide 16 vCPU TiFlash. * Separate the price of CPU and storage (both have 30% public preview discount). @@ -179,7 +215,7 @@ General change: Improvement: -* Add a suggested option `--connect-timeout 15` to the MySQL client on the **Connect** page. +* Add a suggested option `--connect-timeout 15` to the MySQL client on the [**Connect**](/tidb-cloud/connect-to-tidb-cluster.md#connect-via-standard-connection) page. Bug fixes: diff --git a/tidb-cloud/scale-tidb-cluster.md b/tidb-cloud/scale-tidb-cluster.md index 812f2a6eebaa3..f54bdb0a6372a 100644 --- a/tidb-cloud/scale-tidb-cluster.md +++ b/tidb-cloud/scale-tidb-cluster.md @@ -19,6 +19,14 @@ You can scale a TiDB cluster in the following dimensions: For information about how to determine the size of your TiDB cluster, see [Determine Your TiDB Size](/tidb-cloud/size-your-cluster.md). +> **Note:** +> +> If the node size of TiDB or TiKV is set as **4 vCPU, 16 GiB**, note the following restrictions. To bypass these restrictions, you can [increase your node size](#increase-node-size) first. +> +> - The node quantity of TiDB can only be set to 1 or 2, and the node quantity of TiKV is fixed to 3. +> - 4 vCPU TiDB can only be used with 4 vCPU TiKV, and 4 vCPU TiKV can only be used with 4 vCPU TiDB. +> - TiFlash is unavailable. + ## Change node number You can change the number of TiDB, TiKV, or TiFlash nodes. @@ -27,8 +35,13 @@ You can change the number of TiDB, TiKV, or TiFlash nodes. To increase the number of TiDB, TiKV, or TiFlash nodes, take the following steps: -1. In the TiDB Cloud console, navigate to the **Active Clusters** page for your project, and then click the name of a cluster that you want to scale. The overview page of the cluster is displayed. -2. In the cluster information pane on the left, click **Setting**. +1. In the TiDB Cloud console, navigate to the **Active Clusters** page for your project. +2. Find the area of the cluster that you want to scale, and click **...** in the upper-right corner of the area. + + > **Tip:** + > + > Alternatively, you can also click the name of the cluster that you want to scale on the **Active Clusters** page and click **...** in the upper-right corner. + 3. Click **Scale** in the drop-down menu. The **Scale** window is displayed. 4. In the **Scale** window, increase the number of TiDB, TiKV, or TiFlash nodes. 5. Click **Confirm**. @@ -37,8 +50,13 @@ To increase the number of TiDB, TiKV, or TiFlash nodes, take the following steps To decrease the number of TiDB nodes, take the following steps: -1. In the TiDB Cloud console, navigate to the **Active Clusters** page for your project, and then click the name of a cluster that you want to scale. The overview page of the cluster is displayed. -2. In the cluster information pane on the left, click **Setting**. +1. In the TiDB Cloud console, navigate to the **Active Clusters** page for your project. +2. Find the area of the cluster that you want to scale, and click **...** in the upper-right corner of the area. + + > **Tip:** + > + > Alternatively, you can also click the name of the cluster that you want to scale on the **Active Clusters** page and click **...** in the upper-right corner. + 3. Click **Scale** in the drop-down menu. The **Scale** window is displayed. 4. In the **Scale** window, decrease the number of TiDB nodes. 5. Click **Confirm**. @@ -65,8 +83,13 @@ You can change the storage size of TiKV or TiFlash. To increase the storage size of TiKV or TiFlash, take the following steps: -1. In the TiDB Cloud console, navigate to the **Active Clusters** page for your project, and then click the name of a cluster that you want to scale. The overview page of the cluster is displayed. -2. In the cluster information pane on the left, click **Setting**. +1. In the TiDB Cloud console, navigate to the **Active Clusters** page for your project. +2. Find the area of the cluster that you want to scale, and click **...** in the upper-right corner of the area. + + > **Tip:** + > + > Alternatively, you can also click the name of the cluster that you want to scale on the **Active Clusters** page and click **...** in the upper-right corner. + 3. Click **Scale** in the drop-down menu. The **Scale** window is displayed. 4. In the **Scale** window, increase the storage size of TiKV or TiFlash. 5. Click **Confirm**. diff --git a/tidb-cloud/select-cluster-tier.md b/tidb-cloud/select-cluster-tier.md index 6c327a6d254da..b00e856b1bd02 100644 --- a/tidb-cloud/select-cluster-tier.md +++ b/tidb-cloud/select-cluster-tier.md @@ -51,8 +51,12 @@ Whenever you use or set a database user name, you must include the prefix in the To get the prefix for your cluster, take the following steps: -1. In the TiDB Cloud console, navigate to the **Active Clusters** page of your project and click the name of your cluster. -2. In the cluster information pane on the left, click **Connect**. The **Connect to TiDB** dialog is displayed. +1. Navigate to the **Active Clusters** page. +2. Click **Connect** in the upper-right corner of the area. A connection dialog box is displayed. + + > **Tip:** + > + > Alternatively, you can also click the name of your cluster on the **Active Clusters** page and click **Connect** in the upper-right corner. 3. In the dialog, locate **Step 2: Connect with a SQL client** and get the prefix. ### Automatic hibernation and resuming @@ -63,7 +67,7 @@ The hibernation does not affect your data stored in the cluster but only stops t During the hibernation, the status of the cluster is still displayed as **Normal**, and you can see a message about hibernation in the TiDB Cloud console. -Anytime you want to use your Developer Tier cluster again, just connect to your cluster using your MySQL client driver or ORM framework as you usually do. The cluster will be resumed within 50 seconds and back to service automatically. +Anytime you want to use your Developer Tier cluster again, just [connect to your cluster](/tidb-cloud/connect-to-tidb-cluster.md) using your MySQL client driver or ORM framework as you usually do. The cluster will be resumed within 50 seconds and back to service automatically. Alternatively, you can log in to the TiDB Cloud console, and then click **Resume** for the cluster on the **Active Clusters** page. @@ -77,7 +81,8 @@ Alternatively, you can log in to the TiDB Cloud console, and then click **Resume - You cannot create any changefeeds (Apache Kafka Sink and MySQL Sink) or use [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) to replicate incremental data. - You cannot use VPC Peering to connect to clusters. - You cannot scale clusters to larger storage, standard nodes, or increase the number of nodes. -- You cannot use a third-party monitoring service. +- You cannot view the [Monitoring page](/tidb-cloud/built-in-monitoring.md). +- You cannot use the third-party monitoring service. - You cannot customize the port number of a TiDB cluster. - The data transfer is limited to a total of 20 GiB in and out per week. If the 20 GiB limit is reached, the network traffic will be throttled to 10 KB/s. @@ -91,4 +96,4 @@ To create a Dedicated Tier cluster, you need to [add a payment method](/tidb-clo > **Note:** > -> You cannot decrease the cluster storage size after your cluster is created. \ No newline at end of file +> You cannot decrease the cluster storage size after your cluster is created. diff --git a/tidb-cloud/set-up-vpc-peering-connections.md b/tidb-cloud/set-up-vpc-peering-connections.md index 2e79c9b10ca1c..e105593bd7682 100644 --- a/tidb-cloud/set-up-vpc-peering-connections.md +++ b/tidb-cloud/set-up-vpc-peering-connections.md @@ -5,7 +5,7 @@ summary: Learn how to set up VPC peering connections. # Set Up VPC Peering Connections -To connect your application to TiDB Cloud, you need to set up [VPC peering](/tidb-cloud/tidb-cloud-glossary.md#vpc-peering) with TiDB Cloud. It's one step of [Connect to Your TiDB Cluster](/tidb-cloud/connect-to-tidb-cluster.md). This document walks you through setting up VPC peering connections [on AWS](#on-aws) and [on GCP](#on-gcp). +To connect your application to TiDB Cloud, you need to set up [VPC peering](/tidb-cloud/tidb-cloud-glossary.md#vpc-peering) with TiDB Cloud. It is one step of [Connect to Your TiDB Cluster](/tidb-cloud/connect-to-tidb-cluster.md). This document walks you through setting up VPC peering connections [on AWS](#on-aws) and [on GCP](#on-gcp). VPC peering connection is a networking connection between two VPCs that enables you to route traffic between them using private IP addresses. Instances in either VPC can communicate with each other as if they are within the same network. @@ -27,7 +27,7 @@ You can set the project CIDR when creating the first Dedicated Tier of your proj > **Note:** > - > When setting the project CIDR, avoid any conflicts with the CIDR of the VPC where your application is located. + > To avoid any conflicts with the CIDR of the VPC where your application is located, you need to set a different project CIDR in this field. - 10.250.0.0/16 - 10.250.0.0/17 @@ -59,7 +59,7 @@ You can set the project CIDR when creating the first Dedicated Tier of your proj - VPC ID - VPC CIDR - You could get these information from your VPC details on the VPC dashboard. + You can get these information from your VPC details on the VPC dashboard. ![VPC peering](/media/tidb-cloud/vpc-peering/vpc-peering-creating-infos.png) @@ -107,14 +107,14 @@ Use either of the following two options to approve and configure the VPC peering For example: ``` - # Set up the related variables + # Sets up the related variables pcx_tidb_to_app_id="pcx-069f41efddcff66c8" app_region="us-west-2" app_vpc_id="vpc-0039fb90bb5cf8698" tidbcloud_project_cidr="10.250.0.0/16" ``` -4. Execute the following commands. +4. Run the following commands. {{< copyable "shell-regular" >}} @@ -127,13 +127,17 @@ Use either of the following two options to approve and configure the VPC peering ```bash # Creates route table rules. - aws ec2 describe-route-tables --region "$app_region" --filters Name=vpc-id,Values="$app_vpc_id" --query 'RouteTables[*].RouteTableId' --output text | xargs -n 1 | while read row + aws ec2 describe-route-tables --region "$app_region" --filters Name=vpc-id,Values="$app_vpc_id" --query 'RouteTables[*].RouteTableId' --output text | tr "\t" "\n" | while read row do app_route_table_id="$row" aws ec2 create-route --route-table-id "$app_route_table_id" --destination-cidr-block "$tidbcloud_project_cidr" --vpc-peering-connection-id "$pcx_tidb_to_app_id" done ``` + > **Note:** + > + > Sometimes, even if the route table rules are successfully created, you might still get the `An error occurred (MissingParameter) when calling the CreateRoute operation: The request must contain the parameter routeTableId` error. In this case, you can check the created rules and ignore the error. + {{< copyable "shell-regular" >}} ```bash @@ -156,7 +160,7 @@ You can also use the AWS dashboard to configure the VPC peering connection. 2. From the left navigation bar, open the **Peering Connections** page. On the **Create Peering Connection** tab, a peering connection is in the **Pending Acceptance** status. - 3. Confirm the requester owner is TiDB Cloud (`380838443567`). Right click on the peering connection and click **Accept Request** to accept the request. + 3. Confirm the requester owner is TiDB Cloud (`380838443567`). Right-click the peering connection and select **Accept Request** to accept the request in the **Accept VPC peering connection request** dialog. ![AWS VPC peering requests](/media/tidb-cloud/vpc-peering/aws-vpc-guide-3.png) @@ -168,7 +172,7 @@ You can also use the AWS dashboard to configure the VPC peering connection. ![Search all route tables related to VPC](/media/tidb-cloud/vpc-peering/aws-vpc-guide-4.png) - 3. Edit each route table to add a route with destination to the Project CIDR, and select your peering ID on the **Target** column. + 3. Right-click each route table and select **Edit routes**. On the edit page, add a route with a destination to the Project CIDR (by checking the **VPC Peering** configuration page in the TiDB Cloud console) and fill in your peering connection ID in the **Target** column. ![Edit all route tables](/media/tidb-cloud/vpc-peering/aws-vpc-guide-5.png) @@ -186,9 +190,12 @@ You can also use the AWS dashboard to configure the VPC peering connection. ### Step 3: Connect to the TiDB cluster on TiDB Cloud -1. Navigate to the **Active Clusters** page and click the name of your cluster. +1. Navigate to the **Active Clusters** page. +2. Find the area of your target cluster, and click **Connect** in the upper-right corner of the area. The connection dialog is displayed. You can see the **Status** of the VPC peering is **active**. If **Status** is still **system checking**, wait for about 5 minutes and open the dialog again. -2. Click **Connect**. The **Connect to TiDB** dialog displays. You could see the **Status** of the VPC peering is **active**. + > **Tip:** + > + > Alternatively, you can also click the name of your target cluster on the **Active Clusters** page and click **Connect** in the upper-right corner. 3. Access the TiDB Cluster from the instance within the VPC. See [Connect to Your TiDB Cluster](/tidb-cloud/connect-to-tidb-cluster.md). diff --git a/tidb-cloud/size-your-cluster.md b/tidb-cloud/size-your-cluster.md index 6779e3b374c68..2ae808ff42fca 100644 --- a/tidb-cloud/size-your-cluster.md +++ b/tidb-cloud/size-your-cluster.md @@ -21,13 +21,13 @@ You can configure both node size and node quantity for TiDB. The supported node sizes include the following: -- 4 vCPU, 16 GiB (Beta) +- 4 vCPU, 16 GiB - 8 vCPU, 16 GiB - 16 vCPU, 32 GiB > **Note:** > -> If the node size of TiDB is set as **4 vCPU, 16 GiB (Beta)**, note the following restrictions: +> If the node size of TiDB is set as **4 vCPU, 16 GiB**, note the following restrictions: > > - The node quantity of TiDB can only be set to 1 or 2, and the node quantity of TiKV is fixed to 3. > - TiDB can only be used with 4 vCPU TiKV. @@ -49,14 +49,14 @@ You can configure node size, node quantity, and storage size for TiKV. The supported node sizes include the following: -- 4 vCPU, 16 GiB (Beta) +- 4 vCPU, 16 GiB - 8 vCPU, 32 GiB - 8 vCPU, 64 GiB - 16 vCPU, 64 GiB > **Note:** > -> If the node size of TiKV is set as **4 vCPU, 16 GiB (Beta)**, note the following restrictions: +> If the node size of TiKV is set as **4 vCPU, 16 GiB**, note the following restrictions: > > - The node quantity of TiDB can only be set to 1 or 2, and the node quantity of TiKV is fixed to 3. > - TiKV can only be used with 4 vCPU TiDB. @@ -104,7 +104,7 @@ The supported node sizes include the following: - 8 vCPU, 64 GiB - 16 vCPU, 128 GiB -Note that TiFlash is unavailable when the vCPU size of TiDB or TiKV is set as **4 vCPU, 16 GiB (Beta)**. +Note that TiFlash is unavailable when the vCPU size of TiDB or TiKV is set as **4 vCPU, 16 GiB**. ### TiFlash node quantity diff --git a/tidb-cloud/tidb-cloud-billing-tcu.md b/tidb-cloud/tidb-cloud-billing-tcu.md index e449f07707572..869bc3df319ce 100644 --- a/tidb-cloud/tidb-cloud-billing-tcu.md +++ b/tidb-cloud/tidb-cloud-billing-tcu.md @@ -22,16 +22,17 @@ For each TiDB cluster, the number of TiCDC Capacity Units is set up by TiDB Clou The following table lists the price of TiDB Cloud for each TiCDC Capacity Unit (TCU): -| Region | TCU Price ($/hr) | -|---------------------|------------------| -| aws/us-west-2 | $0.1307 | -| aws/us-east-1 | $0.1307 | -| aws/ap-northeast-1 | $0.1669 | -| aws/ap-southeast-1 | $0.1623 | -| aws/eu-central-1 | $0.1564 | -| aws/ap-south-1 | $0.1393 | -| gcp/us-west1 | $0.1452 | -| gcp/us-central1 | $0.1452 | -| gcp/asia-northeast1 | $0.1868 | -| gcp/asia-southeast1 | $0.1746 | -| gcp/asia-east1 | $0.1628 | +| Cloud provider | Region | TCU Price ($/hr) | +|----------------|-----------------------------|------------------| +| AWS | Oregon (us-west-2) | $0.1307 | +| AWS | N. Virginia (us-east-1) | $0.1307 | +| AWS | Mumbai (ap-south-1) | $0.1393 | +| AWS | Singapore (ap-southeast-1) | $0.1623 | +| AWS | Tokyo (ap-northeast-1) | $0.1669 | +| AWS | Frankfurt (eu-central-1) | $0.1564 | +| GCP | Oregon (us-west1) | $0.1452 | +| GCP | Iowa (us-central1) | $0.1452 | +| GCP | Singapore (asia-southeast1) | $0.1746 | +| GCP | Taiwan (asia-east1) | $0.1628 | +| GCP | Tokyo (asia-northeast1) | $0.1868 | +| GCP | Osaka (asia-northeast2) | $0.1868 | diff --git a/tidb-cloud/tidb-cloud-billing.md b/tidb-cloud/tidb-cloud-billing.md index d3c8108e5884a..a86a8c586d668 100644 --- a/tidb-cloud/tidb-cloud-billing.md +++ b/tidb-cloud/tidb-cloud-billing.md @@ -68,6 +68,10 @@ To make these charges easier to view, your TiDB Cloud bills and invoices aggrega - Data transfer – Cross Region - Data transfer – Internet +## Support service cost + +If you have subscribed to one of the **Standard**, **Enterprise**, or **Premium** [support plan](/tidb-cloud/tidb-cloud-support.md), you are charged for the support service on a monthly basis. The billing information is included in your monthly bill. + ## Invoices If you are the owner or billing administrator of your organization, you can manage the invoice information of TiDB Cloud. Otherwise, skip this section. @@ -76,7 +80,7 @@ If you are the owner or billing administrator of your organization, you can mana > > If you sign up for TiDB Cloud through [AWS Marketplace](https://aws.amazon.com/marketplace), you can pay through your AWS account directly but cannot add payment methods or download invoices in the TiDB Cloud portal. -After you set up the payment method, TiDB Cloud will generate the invoice for the previous month at the beginning of each month. Invoice costs include TiDB cluster usage consumption, discounts, backup storage costs, and data transmission costs in your organization. +After you set up the payment method, TiDB Cloud will generate the invoice for the previous month at the beginning of each month. Invoice costs include TiDB cluster usage consumption, discounts, backup storage costs, support service cost, and data transmission costs in your organization. - TiDB Cloud provides the invoice to you on the ninth of each month. From the first to the ninth day, you cannot view the last month's cost details, but can obtain the cluster usage information of this month via the billing console. - The default method for paying invoices is credit card deduction. If you want to use other payment methods, please send a ticket request to let us know. @@ -96,7 +100,7 @@ To view the list of invoices, perform the following steps: If you are the owner or billing administrator of the organization, you can view and export the billing details of TiDB Cloud. Otherwise, skip this section. -After setting the payment method, TiDB Cloud will generate the invoice and billing details of the historical months, and generate the bill details of the current month at the beginning of each month. The billing details include your organization's TiDB cluster usage consumption, discounts, backup storage costs, data transmission costs, and project splitting information. +After setting the payment method, TiDB Cloud will generate the invoice and billing details of the historical months, and generate the bill details of the current month at the beginning of each month. The billing details include your organization's TiDB cluster usage consumption, discounts, backup storage costs, data transmission costs, support service cost, and project splitting information. > **Note:** > diff --git a/tidb-cloud/tidb-cloud-poc.md b/tidb-cloud/tidb-cloud-poc.md index c6d391acf76ec..6bf67087e4d7f 100644 --- a/tidb-cloud/tidb-cloud-poc.md +++ b/tidb-cloud/tidb-cloud-poc.md @@ -175,7 +175,7 @@ Now the workload testing is finished, you can explore more features, for example - Backup - To avoid vendor lock-in, you can use daily full backup to migrate data to a new cluster and use [Dumpling](/dumpling-overview.md) to export data. For more information, see [Export Data from TiDB](/tidb-cloud/export-data-from-tidb-cloud.md). + To avoid vendor lock-in, you can use daily full backup to migrate data to a new cluster and use [Dumpling](https://docs.pingcap.com/tidb/stable/dumpling-overview) to export data. For more information, see [Export Data from TiDB](/tidb-cloud/export-data-from-tidb-cloud.md). ## Step 8. Clean up the environment and finish the PoC diff --git a/tidb-cloud/tidb-cloud-quickstart.md b/tidb-cloud/tidb-cloud-quickstart.md index 1fbe6d2ac1168..c29d77220b3c7 100644 --- a/tidb-cloud/tidb-cloud-quickstart.md +++ b/tidb-cloud/tidb-cloud-quickstart.md @@ -31,13 +31,13 @@ To create a free Developer Tier cluster, take the following steps: 3. On the plan selection page, click **Get Started for Free** in the **Developer Tier** plan. -4. On the **Create a Cluster (Developer Tier)** page, update the default cluster name if necessary, and then select the region where you want to create your cluster. +4. On the **Create Cluster** page, **Developer Tier** is selected by default. Update the default cluster name if necessary, and then select the region where you want to create your cluster. 5. Click **Create**. - The cluster creation process starts and the **Security Quick Start** dialog box is displayed. + The cluster creation process starts and the **Security Settings** dialog box is displayed. -6. In the **Security Quick Start** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. +6. In the **Security Settings** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. Your TiDB Cloud cluster will be created in approximately 5 to 15 minutes. @@ -59,12 +59,12 @@ To create a Dedicated Tier cluster, take the following steps: 3. On the plan selection page, click **Get Full Access Today** in the **Dedicated Tier** plan. +4. On the **Create a Cluster** page, **Dedicated Tier** is selected by default. Update the default cluster name and port number if necessary, choose a cloud provider and a region, and then click **Next**. + > **Note:** > > If you want to get a 14-day free trial of TiDB Cloud Dedicated Tier first, see [Perform a Proof of Concept (PoC) with TiDB Cloud](/tidb-cloud/tidb-cloud-poc.md). -4. On the **Create a Cluster** page, update the default cluster name and port number if necessary, choose a cloud provider and a region, and then click **Next**. - 5. If this is the first cluster of your current project and CIDR has not been configured for this project, you need to set the project CIDR, and then click **Next**. If you do not see the **project CIDR** field, it means that CIDR has already been configured for this project. > **Note:** @@ -73,15 +73,15 @@ To create a Dedicated Tier cluster, take the following steps: 6. Configure the [cluster size](/tidb-cloud/size-your-cluster.md) for TiDB, TiKV, and TiFlash (optional) respectively, and then click **Next**. -7. Confirm the cluster information in the middle area and also the billing information in the right pane. +7. Confirm the cluster information on the page and also the billing information in the lower-left corner. -8. Click **Add Credit Card** in the right pane to add a credit card for your account. +8. Click **Add Credit Card** in the lower-right corner to add a credit card for your account. 9. Click **Create**. - The cluster creation process starts and the **Security Quick Start** dialog box is displayed. + The cluster creation process starts and the **Security Settings** dialog box is displayed. -10. In the **Security Quick Start** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. +10. In the **Security Settings** dialog box, set the root password and allowed IP addresses to connect to your cluster, and then click **Apply**. Your TiDB Cloud cluster will be created in approximately 5 to 15 minutes. @@ -90,11 +90,13 @@ To create a Dedicated Tier cluster, take the following steps: ## Step 2. Connect to your TiDB cluster -1. On the **Active Clusters** page, click the name of your newly created cluster. +1. Navigate to the **Active Clusters** page. - The overview page of your newly created cluster is displayed. +2. In the area of your newly created cluster, click **Connect** in the upper-right corner. A connection dialog box is displayed. -2. Click **Connect**. The **Connect to TiDB** dialog box is displayed. + > **Tip:** + > + > Alternatively, you can also click the name of your newly created cluster on the **Active Clusters** page and click **Connect** in the upper-right corner. 3. Under **Step 2: Connect with a SQL client** in the dialog box, click the tab of your preferred connection method, and then connect to your cluster with the connection string. @@ -117,9 +119,12 @@ To create a Dedicated Tier cluster, take the following steps: We provide Capital Bikeshare sample data for you to easily import data and run sample queries. -1. Navigate to the **Active Clusters** page and click the name of your newly created cluster. The overview page of your cluster is displayed. +1. Navigate to the **Active Clusters** page. +2. In the area of your newly created cluster, click **Import Data** in the upper-right corner. The **Data Import Task** page is displayed. -2. In the cluster information pane on the left, click **Import**. The **Data Import Task** page is displayed. + > **Tip:** + > + > Alternatively, you can also click the name of your newly created cluster on the **Active Clusters** page and click **Import Data** in the upper-right corner. 3. Fill in the import parameters: @@ -132,9 +137,7 @@ We provide Capital Bikeshare sample data for you to easily import data and run s - **Bucket URL**: enter the sample data URL `s3://tidbcloud-samples/data-ingestion/`. - **Data Format**: select **TiDB Dumpling**. - **Setup Credentials**: enter `arn:aws:iam::385595570414:role/import-sample-access` for Role-ARN. - - **Target Database**: - - **Username**: `root`. - - **Password**: enter your root password. + - **Target Cluster**: fill in the **Username** and **Password** fields. - **DB/Tables Filter**: leave this field blank. @@ -146,9 +149,7 @@ We provide Capital Bikeshare sample data for you to easily import data and run s - **Data Source Type**: `Google Cloud Stroage`. - **Bucket URL**: enter the sample data URL `gcs://tidbcloud-samples-us-west1`. - **Data Format**: select **TiDB Dumpling**. - - **Target Database**: - - **Username**: `root`. - - **Password**: enter your root password. + - **Target Cluster**: fill in the **Username** and **Password** fields. - **DB/Tables Filter**: leave this field blank. diff --git a/tidb-cloud/tidb-cloud-sql-tuning-overview.md b/tidb-cloud/tidb-cloud-sql-tuning-overview.md index e2a1f29063661..e06ba3fb225c3 100644 --- a/tidb-cloud/tidb-cloud-sql-tuning-overview.md +++ b/tidb-cloud/tidb-cloud-sql-tuning-overview.md @@ -114,4 +114,4 @@ In Key Visualizer, there are [four common heat map results](https://docs.pingcap In both cases of X-axis and Y-axis alternating bright and dark, you need to address read and write pressure. -For more information about SQL performance optimization, see [SQL Optimization](https://docs.pingcap.com/tidb/stable/sql-faq#sql-optimization) in SQL FAQs. +For more information about SQL performance optimization, see [SQL Optimization](https://docs.pingcap.com/tidb/stable/sql-faq#sql-optimization) in SQL FAQs. \ No newline at end of file diff --git a/tidb-cloud/tidb-cloud-support.md b/tidb-cloud/tidb-cloud-support.md index 27f2295493518..157086dfc175f 100644 --- a/tidb-cloud/tidb-cloud-support.md +++ b/tidb-cloud/tidb-cloud-support.md @@ -5,17 +5,76 @@ summary: Learn how to contact the support team of TiDB Cloud. # TiDB Cloud Support -TiDB Cloud offers support services to your organization. To contact our support team, perform the following steps: +TiDB Cloud offers a free **Basic** support plan for each user and you can upgrade to a paid plan for extended services. -1. Click **Support** on the upper right of the top navigation bar, and click **Request Support**. Or directly click the **Help** icon on the lower right. +The information of each support plan is available on the support page of the TiDB Cloud console. To access the page, see [Check or upgrade your support plan](#check-or-upgrade-your-support-plan). -2. In the **Leave us a message** dialog, fill in the ticket. +## Request support -3. Click **Send** to submit your ticket. After the support team receives your ticket, they will reply to you via email. +To request support for your organization, project, or cluster, perform the following steps: -4. Check your email to view the reply, your ticket history, and communication records. +1. On the TiDB Cloud console, click **Support** in the upper-right corner of the top navigation bar, and click **Create New Case**. A **Submit a request** page is displayed. - Before the ticket is closed, you can continue to communicate with the support team via either of the following two methods: + > **Tip:** + > + > You can also directly click the **Help** icon on the lower right of the TiDB Cloud console. - - Replying to the email sent by the support team directly - - Clicking the ticket link attached in the email to go to the support portal to check and add comments to your ticket +2. On the **Submit a request** page, select **TiDB Cloud Support** from the drop-down list. + +3. Fill in the request information. + +4. Click **Submit** to submit your ticket. After the support team receives your ticket, they will reply to you via email. The response time depends on the [support plan](#check-or-upgrade-your-support-plan) of your organization. + +## Check your submitted request + +After you have submitted a request, to check the status of your request, perform the following steps: + +1. On your TiDB Cloud console, click **Support** in the upper-right corner of the top navigation bar, and click **View Support Plan**. The support page is displayed. +2. In the **Request Support** area, click **View Support Portal**. The **My request** page is displayed. + + On the **My request** page, you can view the status of your request. + +Alternatively, you can also access the [Customer Support](https://support.pingcap.com/hc/en-us) page, log in to the support portal, and then click **View my requests**. + +## Check or upgrade your support plan + +To check or upgrade your support plan, perform the following steps: + +1. On your TiDB Cloud console, click **Support** in the upper-right corner of the top navigation bar, and click **View Support Plan**. + + The support page is displayed. On this page, you can see your **Current Plan**. By default, the **Basic** free plan is selected. + +2. Choose your desired support plan. + + +
+ + To upgrade to **Standard**: + + 1. Click **Select Plan** in the **Standard** pane. A **Finish and Start Using Support** page is displayed. + 2. Check the billing information in the lower-left corner of the page. + 3. Add your payment information in the **Billing Profile** and **Credit Card** areas. + + For more information about billing, see [TiDB Cloud Payment Method](/tidb-cloud/tidb-cloud-billing.md#payment-method). + + 4. Click **Confirm and Start Using Support** in the lower-right corner of the page. + + After the payment is finished, you have upgraded you plan to **Standard**. + +
+
+ + To upgrade your plan to **Enterprise** or **Premium**: + + 1. Click **Contact Sales** in the **Enterprise** or **Premium** pane. A **Contact Us** page is displayed. + 2. Fill in and submit your contact information on the page. Then, the support team will contact you and help you with your subscription. + +
+ + +## Downgrade your support plan + +To downgrade your support plan to **Basic**, perform the following steps: + +1. On your TiDB Cloud console, click **Support** in the upper-right corner of the top navigation bar, and click **View Support Plan**. +2. At the bottom of the **Support** page, click **Downgrade to basic plan**. diff --git a/tidb-cloud/tune-performance.md b/tidb-cloud/tune-performance.md index d18807aed1120..6466a401f48fe 100644 --- a/tidb-cloud/tune-performance.md +++ b/tidb-cloud/tune-performance.md @@ -15,11 +15,13 @@ TiDB Cloud provides [Statement Analysis](#statement-analysis) and [Key Visualize To use the statement analysis, perform the following steps: -1. Navigate to the **Diagnostics** tab of a cluster. The **Statement** sub-tab displays by default. +1. Navigate to the **Diagnosis** tab of a cluster. -2. Select the time period to be analyzed in the time interval box. Then you can get the execution statistics of SQL statements of all databases in this period. +2. Click the **Statement** tab. -3. (Optional) If you only care about certain databases, you can select the corresponding schema(s) in the next box to filter the results. +3. Select the time period to be analyzed in the time interval box. Then you can get the execution statistics of SQL statements of all databases in this period. + +4. (Optional) If you only care about certain databases, you can select the corresponding schema(s) in the next box to filter the results. The results are displayed in the form of a table, and you can sort the results by different columns. @@ -31,9 +33,9 @@ For details, see [Statement Execution Details in TiDB Dashboard](https://docs.pi To view the key analytics, perform the following steps: -1. Navigate to the **Diagnostics** tab of a cluster. +1. Navigate to the **Diagnosis** tab of a cluster. -2. Select the **Key Visualizer** tab. +2. Click the **Key Visualizer** tab. ![Key Visualizer](/media/tidb-cloud/key-visualizer.png) diff --git a/tidb-limitations.md b/tidb-limitations.md index 083671216f456..af333a97fdc44 100644 --- a/tidb-limitations.md +++ b/tidb-limitations.md @@ -44,14 +44,24 @@ This document describes the common usage limitations of TiDB, including the maxi | Size | unlimited | | Partitions | 8192 | + + * The upper limit of `Columns` can be modified via [`table-column-count-limit`](/tidb-configuration-file.md#table-column-count-limit-new-in-v50). * The upper limit of `Indexes` can be modified via [`index-limit`](/tidb-configuration-file.md#index-limit-new-in-v50). + + ## Limitation on a single row | Type | Upper limit | |:----------|:----------| -| Size | 6 MB by default. You can adjust the size limit via the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item. | +| Size | 6 MB | + + + +You can adjust the size limit via the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item. + + ## Limitation on a single column @@ -74,7 +84,13 @@ This document describes the common usage limitations of TiDB, including the maxi | Type | Upper limit | |:----------|:----------| -| The maximum number of SQL statements in a single transaction | When the optimistic transaction is used and the transaction retry is enabled, the default upper limit is 5000, which can be modified using [`stmt-count-limit`](/tidb-configuration-file.md#stmt-count-limit). | +| The maximum number of SQL statements in a single transaction | When the optimistic transaction is used and the transaction retry is enabled, the upper limit is 5000. | + + + +You can modify the limit via the [`stmt-count-limit`](/tidb-configuration-file.md#stmt-count-limit) configuration item. + + ## Limitations on TiKV version diff --git a/tidb-scheduling.md b/tidb-scheduling.md index 4beb76c372467..27a0eb12d3510 100644 --- a/tidb-scheduling.md +++ b/tidb-scheduling.md @@ -75,7 +75,7 @@ Scheduling is based on information collection. In short, the PD scheduling compo * Data read/write speed * The number of snapshots that are sent/received (The data might be replicated between replicas through snapshots) * Whether the store is overloaded - * Labels (See [Perception of Topology](/schedule-replicas-by-topology-labels.md)) + * Labels (See [Perception of Topology](https://docs.pingcap.com/tidb/stable/schedule-replicas-by-topology-labels)) You can use PD control to check the status of a TiKV store, which can be Up, Disconnect, Offline, Down, or Tombstone. The following is a description of all statuses and their relationship. diff --git a/tiflash/create-tiflash-replicas.md b/tiflash/create-tiflash-replicas.md index fc0978bcf2b71..91568f96ec8b7 100644 --- a/tiflash/create-tiflash-replicas.md +++ b/tiflash/create-tiflash-replicas.md @@ -133,6 +133,14 @@ SELECT TABLE_NAME FROM information_schema.tables where TABLE_SCHEMA = " ## Set available zones + + +> **Note:** +> +> This section is not applicable to TiDB Cloud. + + + When configuring replicas, if you need to distribute TiFlash replicas to multiple data centers for disaster recovery, you can configure available zones by following the steps below: 1. Specify labels for TiFlash nodes in the cluster configuration file. @@ -197,4 +205,8 @@ When configuring replicas, if you need to distribute TiFlash replicas to multipl ... ``` + + For more information about scheduling replicas by using labels, see [Schedule Replicas by Topology Labels](/schedule-replicas-by-topology-labels.md), [Multiple Data Centers in One City Deployment](/multi-data-centers-in-one-city-deployment.md), and [Three Data Centers in Two Cities Deployment](/three-data-centers-in-two-cities-deployment.md). + + diff --git a/tiflash/tiflash-overview.md b/tiflash/tiflash-overview.md index 1b900def910a8..15637b4dbbe97 100644 --- a/tiflash/tiflash-overview.md +++ b/tiflash/tiflash-overview.md @@ -10,6 +10,12 @@ aliases: ['/docs/dev/tiflash/tiflash-overview/','/docs/dev/reference/tiflash/ove In TiFlash, the columnar replicas are asynchronously replicated according to the Raft Learner consensus algorithm. When these replicas are read, the Snapshot Isolation level of consistency is achieved by validating Raft index and multi-version concurrency control (MVCC). + + +With TiDB Cloud, you can create an HTAP cluster easily by specifying one or more TiFlash nodes according to your HTAP workload. If the TiFlash node count is not specified when you create the cluster or you want to add more TiFlash nodes, you can change the node count by [scaling the cluster](/tidb-cloud/scale-tidb-cluster.md). + + + ## Architecture ![TiFlash Architecture](/media/tidb-storage-architecture.png) @@ -77,13 +83,25 @@ You can either use TiDB to read TiFlash replicas for medium-scale analytical pro - [Create TiFlash Replicas](/tiflash/create-tiflash-replicas.md) - [Use TiDB to Read TiFlash Replicas](/tiflash/use-tidb-to-read-tiflash.md) + + + - [Use TiSpark to Read TiFlash Replicas](/tiflash/use-tispark-to-read-tiflash.md) + + + - [Use MPP Mode](/tiflash/use-tiflash-mpp-mode.md) + + To experience the whole process from importing data to querying in a TPC-H dataset, refer to [Quick Start Guide for TiDB HTAP](/quick-start-with-htap.md). + + ## See also + + - To deploy a new cluster with TiFlash nodes, see [Deploy a TiDB cluster using TiUP](/production-deployment-using-tiup.md). - To add a TiFlash node in a deployed cluster, see [Scale out a TiFlash cluster](/scale-tidb-using-tiup.md#scale-out-a-tiflash-cluster). - [Maintain a TiFlash cluster](/tiflash/maintain-tiflash.md). @@ -95,3 +113,13 @@ To experience the whole process from importing data to querying in a TPC-H datas - [Supported push-down calculations in TiFlash](/tiflash/tiflash-supported-pushdown-calculations.md) - [Data validation in TiFlash](/tiflash/tiflash-data-validation.md) - [TiFlash compatibility](/tiflash/tiflash-compatibility.md) + + + + + +- [Tune TiFlash performance](/tiflash/tune-tiflash-performance.md). +- [Supported push-down calculations in TiFlash](/tiflash/tiflash-supported-pushdown-calculations.md) +- [TiFlash compatibility](/tiflash/tiflash-compatibility.md) + + diff --git a/tiflash/use-tidb-to-read-tiflash.md b/tiflash/use-tidb-to-read-tiflash.md index f4ce5860d76f2..b114f966da35a 100644 --- a/tiflash/use-tidb-to-read-tiflash.md +++ b/tiflash/use-tidb-to-read-tiflash.md @@ -52,7 +52,11 @@ Note that if a table has only a single TiFlash replica and the related node cann ## Engine isolation -Engine isolation is to specify that all queries use a replica of the specified engine by configuring the corresponding variable. The optional engines are "tikv", "tidb" (indicates the internal memory table area of TiDB, which stores some TiDB system tables and cannot be actively used by users), and "tiflash", with the following two configuration levels: +Engine isolation is to specify that all queries use a replica of the specified engine by configuring the corresponding variable. The optional engines are "tikv", "tidb" (indicates the internal memory table area of TiDB, which stores some TiDB system tables and cannot be actively used by users), and "tiflash". + + + +You can specify the engines at the following two configuration levels: * TiDB instance-level, namely, INSTANCE level. Add the following configuration item in the TiDB configuration file: @@ -87,6 +91,24 @@ The final engine configuration is the session-level configuration, that is, the > > Because [TiDB Dashboard](/dashboard/dashboard-intro.md) and other components need to read some system tables stored in the TiDB memory table area, it is recommended to always add the "tidb" engine to the instance-level engine configuration. + + + + +You can specify the engines using the following statement: + +```sql +set @@session.tidb_isolation_read_engines = "engine list separated by commas"; +``` + +or + +```sql +set SESSION tidb_isolation_read_engines = "engine list separated by commas"; +``` + + + If the queried table does not have a replica of the specified engine (for example, the engine is configured as "tiflash" but the table does not have a TiFlash replica), the query returns an error. ## Manual hint diff --git a/tiflash/use-tiflash-mpp-mode.md b/tiflash/use-tiflash-mpp-mode.md index e5f2ef4c5df7e..023fd83746350 100644 --- a/tiflash/use-tiflash-mpp-mode.md +++ b/tiflash/use-tiflash-mpp-mode.md @@ -47,8 +47,12 @@ set @@session.tidb_allow_mpp=1; set @@session.tidb_enforce_mpp=1; ``` + + The initial value of the `tidb_enforce_mpp` session variable is equal to the [`enforce-mpp`](/tidb-configuration-file.md#enforce-mpp) configuration value of this tidb-server instance (which is `false` by default). If multiple tidb-server instances in a TiDB cluster only perform analytical queries and you want to make sure that the MPP mode is used on these instances, you can change their [`enforce-mpp`](/tidb-configuration-file.md#enforce-mpp) configuration values to `true`. + + > **Note:** > > When `tidb_enforce_mpp=1` takes effect, the TiDB optimizer will ignore the cost estimation to choose the MPP mode. However, if other factors block the MPP mode, TiDB will not select the MPP mode. These factors include the absence of TiFlash replica, unfinished replication of TiFlash replicas, and statements containing operators or functions that are not supported by the MPP mode. diff --git a/transaction-isolation-levels.md b/transaction-isolation-levels.md index 39eb6b15e7608..3ab2e0ca9e9d1 100644 --- a/transaction-isolation-levels.md +++ b/transaction-isolation-levels.md @@ -6,8 +6,18 @@ aliases: ['/docs/dev/transaction-isolation-levels/','/docs/dev/reference/transac # TiDB Transaction Isolation Levels + + Transaction isolation is one of the foundations of database transaction processing. Isolation is one of the four key properties of a transaction (commonly referred as [ACID](/glossary.md#acid)). + + + + +Transaction isolation is one of the foundations of database transaction processing. Isolation is one of the four key properties of a transaction (commonly referred as [ACID](/tidb-cloud/tidb-cloud-glossary.md#acid)). + + + The SQL-92 standard defines four levels of transaction isolation: Read Uncommitted, Read Committed, Repeatable Read, and Serializable. See the following table for details: | Isolation Level | Dirty Write | Dirty Read | Fuzzy Read | Phantom | @@ -21,9 +31,9 @@ TiDB implements Snapshot Isolation (SI) consistency, which it advertises as `REP > **Note:** > -> In TiDB v3.0, the automatic retry of transactions is disabled by default. It is not recommended to enable the automatic retry because it might **break the transaction isolation level**. Refer to [Transaction Retry](/optimistic-transaction.md#automatic-retry) for details. +> Starting from TiDB v3.0, the automatic retry of transactions is disabled by default. It is not recommended to enable the automatic retry because it might **break the transaction isolation level**. Refer to [Transaction Retry](/optimistic-transaction.md#automatic-retry) for details. > -> Starting from TiDB [v3.0.8](/releases/release-3.0.8.md#tidb), newly created TiDB clusters use the [pessimistic transaction mode](/pessimistic-transaction.md) by default. The current read (`for update` read) is **non-repeatable read**. Refer to [pessimistic transaction mode](/pessimistic-transaction.md) for details. +> Starting from TiDB v3.0.8, newly created TiDB clusters use the [pessimistic transaction mode](/pessimistic-transaction.md) by default. The current read (`for update` read) is **non-repeatable read**. Refer to [pessimistic transaction mode](/pessimistic-transaction.md) for details. ## Repeatable Read isolation level @@ -54,7 +64,7 @@ The Repeatable Read isolation level in TiDB differs from that in MySQL. The MySQ ## Read Committed isolation level -Starting from TiDB [v4.0.0-beta](/releases/release-4.0.0-beta.md#tidb), TiDB supports the Read Committed isolation level. +Starting from TiDB v4.0.0-beta, TiDB supports the Read Committed isolation level. For historical reasons, the Read Committed isolation level of current mainstream databases is essentially the [Consistent Read isolation level defined by Oracle](https://docs.oracle.com/cd/B19306_01/server.102/b14220/consist.htm). In order to adapt to this situation, the Read Committed isolation level in TiDB pessimistic transactions is also a consistent read behavior in essence.