From 8501025bc09ec63cb1c896e8a0ec92460c72b648 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 5 Nov 2024 17:52:59 +0800 Subject: [PATCH 01/89] Create release-8.5.0.md --- releases/release-8.5.0.md | 236 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 236 insertions(+) create mode 100644 releases/release-8.5.0.md diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md new file mode 100644 index 0000000000000..5937c37248d70 --- /dev/null +++ b/releases/release-8.5.0.md @@ -0,0 +1,236 @@ +--- +title: TiDB 8.5.0 Release Notes +summary: Learn about the new features, compatibility changes, improvements, and bug fixes in TiDB 8.5.0. +--- + +# TiDB 8.5.0 Release Notes + + + +Release date: xx xx, 2024 + +TiDB version: 8.5.0 + +Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.5/quick-start-with-tidb) | [Production deployment](https://docs.pingcap.com/tidb/v8.5/production-deployment-using-tiup) + +TiDB 8.5.0 is a Long-Term Support Release (LTS). + +Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, and bug fixes released in [8.2.0-DMR](/releases/release-8.2.0.md), [8.3.0-DMR](/releases/release-8.3.0.md), and [8.4.0-DMR](/releases/release-8.4.0.md). When you upgrade from 8.1.x to 8.5.0, you can download the [TiDB Release Notes PDF](https://download.pingcap.org/tidb-v8.1-to-v8.5-en-release-notes.pdf) to view all release notes between the two LTS versions. The following table lists some highlights from 8.1.0 to 8.5.0: + + + +## Feature details + +### Scalability + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### Performance + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### Reliability + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### Availability + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### SQL + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### DB operations + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### Observability + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### Security + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +### Data migration + +* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** + + Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + + For more information, see [Documentation](link). + +## Compatibility changes + +> **Note:** +> +> This section provides compatibility changes you need to know when you upgrade from v8.1.0 to the current version (v8.2.0). If you are upgrading from v8.0.0 or earlier versions to the current version, you might also need to check the compatibility changes introduced in intermediate versions. + +### Behavior changes + +### MySQL compatibility + +### System variables + +| Variable name | Change type | Description | +|--------|------------------------------|------| +| | | | +| | | | +| | | | + +### Configuration parameters + +| Configuration file or component | Configuration parameter | Change type | Description | +| -------- | -------- | -------- | -------- | +| | | | | +| | | | | +| | | | | + +### System tables + +### Other changes + +## Offline package changes + +## Removed features + +## Deprecated features + +* The following features are planned for deprecation in future versions: + +## Improvements + ++ TiDB + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ TiKV + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ PD + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ TiFlash + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ Tools + + + Backup & Restore (BR) + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiCDC + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiDB Data Migration (DM) + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiDB Lightning + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiUP + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + +## Bug fixes + ++ TiDB + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ TiKV + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ PD + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ TiFlash + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + ++ Tools + + + Backup & Restore (BR) + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiCDC + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiDB Data Migration (DM) + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiDB Lightning + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + + + TiUP + + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + +## Contributors + +We would like to thank the following contributors from the TiDB community: + +- [Contributor-GitHub-ID](id-link) \ No newline at end of file From 8d805ecac7ffc2b72f0cb8b96b972f0d78d20204 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 6 Nov 2024 09:57:13 +0800 Subject: [PATCH 02/89] Update release-8.5.0.md --- releases/release-8.5.0.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 5937c37248d70..c28801ec9f7ef 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -131,6 +131,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, * The following features are planned for deprecation in future versions: + * TiDB introduces the system variable [`tidb_enable_auto_analyze_priority_queue`](/system-variables.md#tidb_enable_auto_analyze_priority_queue-new-in-v800), which controls whether priority queues are enabled to optimize the ordering of tasks that automatically collect statistics. In future releases, the priority queue will be the only way to order tasks for automatically collecting statistics, so this system variable will be deprecated. + * TiDB introduces the system variable [`tidb_enable_async_merge_global_stats`](/system-variables.md#tidb_enable_async_merge_global_stats-new-in-v750) in v7.5.0. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated. + * It is planned to redesign [the automatic evolution of execution plan bindings](/sql-plan-management.md#baseline-evolution) in subsequent releases, and the related variables and behavior will change. + * In v8.0.0, TiDB introduces the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable will be deprecated. + * The TiDB Lightning parameter [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) is planned for deprecation in a future release and will be subsequently removed. This parameter will be replaced by [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task), which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task. + * Starting from v6.3.0, partitioned tables use [dynamic pruning mode](/partitioned-table.md#dynamic-pruning-mode) by default. Compared with static pruning mode, dynamic pruning mode supports features such as IndexJoin and plan cache with better performance. Therefore, static pruning mode will be deprecated. + ## Improvements + TiDB From af2c27ba2a7b44135c2cf63ea7257e2498ccf23f Mon Sep 17 00:00:00 2001 From: Ben Meadowcroft Date: Wed, 6 Nov 2024 16:17:50 -0800 Subject: [PATCH 03/89] Added BR Log Encryption GA Release Note --- releases/release-8.5.0.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index c28801ec9f7ef..a4300117e7a1f 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -79,11 +79,15 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Security -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** +* BR supports client-side encryption of log backup data (GA) [#56433](https://github.com/pingcap/tidb/issues/56433) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + TiDB v8.4.0 introduced an experimental feature to encrypt, on the client side, log backup data. Starting from v8.5.0, this feature is now Generally Avaialble. Before uploading log backup data to your backup storage, you can encrypt the backup data to ensure its security via one of the following methods: - For more information, see [Documentation](link). + - Encrypt using a custom fixed key + - Encrypt using a master key stored on a local disk + - Encrypt using a master key managed by a Key Management Service (KMS) + + For more information, see [documentation](/br/br-pitr-manual.md#encrypt-the-log-backup-data). ### Data migration @@ -240,4 +244,4 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, We would like to thank the following contributors from the TiDB community: -- [Contributor-GitHub-ID](id-link) \ No newline at end of file +- [Contributor-GitHub-ID](id-link) From a90a7d3b622f7a3eb6a1fca0231e5cdc7aa99030 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 7 Nov 2024 18:03:23 +0800 Subject: [PATCH 04/89] Apply suggestions from code review Co-authored-by: Frank945946 <108602632+Frank945946@users.noreply.github.com> --- releases/release-8.5.0.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index a4300117e7a1f..2d410e817fed5 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -28,7 +28,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. For more information, see [Documentation](link). +* - The Schema cache memory limit feature is now Generally Available (GA), reducing memory usage in large-scale data scenarios. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd + In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows the system to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. Starting from v8.4, this feature is enabled by default with a default value of 512MiB, and users can adjust it as needed through the variable [tidb_schema_cache_size](/system-variables#tidb_schema_cache_size-new-in-v800). + + For more information, see [Documentation](link). ### Performance * Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** @@ -36,7 +40,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. For more information, see [Documentation](link). +* The database and table creation acceleration feature is now Generally Available (GA), significantly reducing the time required for data migration and cluster initialization. [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt + In v8.0.0, the system variable [tidb_enable_fast_create_table](/system-variables#tidb_enable_fast_create_table-new-in-v800) was introduced to improve the performance of batch database and table creation. During data migration and cluster initialization, it enables the rapid creation of tables at the million-scale level, significantly reducing the time required. + + For more information, see [Documentation](link). ### Reliability * Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** @@ -60,7 +68,21 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. For more information, see [Documentation](link). +*The Foreign Key feature is now Generally Available (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 + + The Foreign Key feature is now GA, enabling the use of foreign key constraints to enhance data consistency and integrity. Users can easily create foreign key constraints between tables, with support for cascading updates and deletions, making data management more convenient. This feature provides better support for applications with complex data constraints. + + For more information, see [Documentation](link). + +*Introduction of ADMIN ALTER DDL JOBS syntax to support online modification of DDL job variables [# ](https://github.com/pingcap/tidb/issues/ ) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd + + Starting from v8.3, it is now possible to set the variables [tidb_ddl_reorg_batch_size](/system-variables#tidb_ddl_reorg_batch_size) and [tidb_ddl_reorg_worker_cnt](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, the DDL job must first be canceled, the variables adjusted, and then resubmit the job. + Since v8.5, the ADMIN ALTER DDL JOBS syntax has been introduced, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example: +- ADMIN ALTER DDL JOBS job_id THREAD = 8; — Adjusts the tidb_ddl_reorg_worker_cnt for the specified DDL task. +- ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256; — Adjusts the tidb_ddl_reorg_batch_size for the specified DDL task. +- ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB'; — Adjusts the write traffic size for index data on each TiKV node. + For more information, see [Documentation](link). ### DB operations * Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** From 577f602b08932d25c31b102908910dc0ceeb24ef Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 11 Nov 2024 11:19:59 +0800 Subject: [PATCH 05/89] minor format updates --- releases/release-8.5.0.md | 44 +++++++++++++++++++++++---------------- 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 2d410e817fed5..c7bf51c233c3d 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -28,11 +28,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. For more information, see [Documentation](link). -* - The Schema cache memory limit feature is now Generally Available (GA), reducing memory usage in large-scale data scenarios. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd + +* The Schema cache memory limit feature is now Generally Available (GA), reducing memory usage in large-scale data scenarios. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows the system to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. Starting from v8.4, this feature is enabled by default with a default value of 512MiB, and users can adjust it as needed through the variable [tidb_schema_cache_size](/system-variables#tidb_schema_cache_size-new-in-v800). For more information, see [Documentation](link). + ### Performance * Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** @@ -40,11 +42,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. For more information, see [Documentation](link). -* The database and table creation acceleration feature is now Generally Available (GA), significantly reducing the time required for data migration and cluster initialization. [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt + +* The database and table creation acceleration feature is now Generally Available (GA), significantly reducing the time required for data migration and cluster initialization. [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt In v8.0.0, the system variable [tidb_enable_fast_create_table](/system-variables#tidb_enable_fast_create_table-new-in-v800) was introduced to improve the performance of batch database and table creation. During data migration and cluster initialization, it enables the rapid creation of tables at the million-scale level, significantly reducing the time required. For more information, see [Documentation](link). + ### Reliability * Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** @@ -68,21 +72,25 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. For more information, see [Documentation](link). -*The Foreign Key feature is now Generally Available (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 - The Foreign Key feature is now GA, enabling the use of foreign key constraints to enhance data consistency and integrity. Users can easily create foreign key constraints between tables, with support for cascading updates and deletions, making data management more convenient. This feature provides better support for applications with complex data constraints. +* The Foreign Key feature is now Generally Available (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 - For more information, see [Documentation](link). + The Foreign Key feature is now GA, enabling the use of foreign key constraints to enhance data consistency and integrity. Users can easily create foreign key constraints between tables, with support for cascading updates and deletions, making data management more convenient. This feature provides better support for applications with complex data constraints. -*Introduction of ADMIN ALTER DDL JOBS syntax to support online modification of DDL job variables [# ](https://github.com/pingcap/tidb/issues/ ) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd + For more information, see [Documentation](link). + +* Introduction of ADMIN ALTER DDL JOBS syntax to support online modification of DDL job variables [#57229](hhttps://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd + + Starting from v8.3, it is now possible to set the variables [tidb_ddl_reorg_batch_size](/system-variables#tidb_ddl_reorg_batch_size) and [tidb_ddl_reorg_worker_cnt](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, the DDL job must first be canceled, the variables adjusted, and then resubmit the job. - Starting from v8.3, it is now possible to set the variables [tidb_ddl_reorg_batch_size](/system-variables#tidb_ddl_reorg_batch_size) and [tidb_ddl_reorg_worker_cnt](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, the DDL job must first be canceled, the variables adjusted, and then resubmit the job. - Since v8.5, the ADMIN ALTER DDL JOBS syntax has been introduced, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example: -- ADMIN ALTER DDL JOBS job_id THREAD = 8; — Adjusts the tidb_ddl_reorg_worker_cnt for the specified DDL task. -- ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256; — Adjusts the tidb_ddl_reorg_batch_size for the specified DDL task. -- ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB'; — Adjusts the write traffic size for index data on each TiKV node. + Since v8.5, the ADMIN ALTER DDL JOBS syntax has been introduced, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example: + + - `ADMIN ALTER DDL JOBS job_id THREAD = 8;` — Adjusts the `tidb_ddl_reorg_worker_cnt` for the specified DDL task. + - `ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;` — Adjusts the `tidb_ddl_reorg_batch_size` for the specified DDL task. + - `ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';` — Adjusts the write traffic size for index data on each TiKV node. For more information, see [Documentation](link). + ### DB operations * Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** @@ -155,14 +163,14 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ## Deprecated features -* The following features are planned for deprecation in future versions: +The following features are planned for deprecation in future versions: - * TiDB introduces the system variable [`tidb_enable_auto_analyze_priority_queue`](/system-variables.md#tidb_enable_auto_analyze_priority_queue-new-in-v800), which controls whether priority queues are enabled to optimize the ordering of tasks that automatically collect statistics. In future releases, the priority queue will be the only way to order tasks for automatically collecting statistics, so this system variable will be deprecated. - * TiDB introduces the system variable [`tidb_enable_async_merge_global_stats`](/system-variables.md#tidb_enable_async_merge_global_stats-new-in-v750) in v7.5.0. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated. - * It is planned to redesign [the automatic evolution of execution plan bindings](/sql-plan-management.md#baseline-evolution) in subsequent releases, and the related variables and behavior will change. - * In v8.0.0, TiDB introduces the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable will be deprecated. - * The TiDB Lightning parameter [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) is planned for deprecation in a future release and will be subsequently removed. This parameter will be replaced by [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task), which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task. - * Starting from v6.3.0, partitioned tables use [dynamic pruning mode](/partitioned-table.md#dynamic-pruning-mode) by default. Compared with static pruning mode, dynamic pruning mode supports features such as IndexJoin and plan cache with better performance. Therefore, static pruning mode will be deprecated. +* TiDB introduces the system variable [`tidb_enable_auto_analyze_priority_queue`](/system-variables.md#tidb_enable_auto_analyze_priority_queue-new-in-v800), which controls whether priority queues are enabled to optimize the ordering of tasks that automatically collect statistics. In future releases, the priority queue will be the only way to order tasks for automatically collecting statistics, so this system variable will be deprecated. +* TiDB introduces the system variable [`tidb_enable_async_merge_global_stats`](/system-variables.md#tidb_enable_async_merge_global_stats-new-in-v750) in v7.5.0. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated. +* It is planned to redesign [the automatic evolution of execution plan bindings](/sql-plan-management.md#baseline-evolution) in subsequent releases, and the related variables and behavior will change. +* In v8.0.0, TiDB introduces the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable will be deprecated. +* The TiDB Lightning parameter [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) is planned for deprecation in a future release and will be subsequently removed. This parameter will be replaced by [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task), which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task. +* Starting from v6.3.0, partitioned tables use [dynamic pruning mode](/partitioned-table.md#dynamic-pruning-mode) by default. Compared with static pruning mode, dynamic pruning mode supports features such as IndexJoin and plan cache with better performance. Therefore, static pruning mode will be deprecated. ## Improvements From 5e23349469111f7378e07091a307ce9d95a8a1fe Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 11 Nov 2024 13:32:09 +0800 Subject: [PATCH 06/89] Update releases/release-8.5.0.md --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index c7bf51c233c3d..6a23e9589b8b6 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -79,7 +79,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Documentation](link). -* Introduction of ADMIN ALTER DDL JOBS syntax to support online modification of DDL job variables [#57229](hhttps://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Introduction of ADMIN ALTER DDL JOBS syntax to support online modification of DDL job variables [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd Starting from v8.3, it is now possible to set the variables [tidb_ddl_reorg_batch_size](/system-variables#tidb_ddl_reorg_batch_size) and [tidb_ddl_reorg_worker_cnt](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, the DDL job must first be canceled, the variables adjusted, and then resubmit the job. From 17d1e677da39670eeeb1e53929b8a9a370eebd64 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 11 Nov 2024 16:54:52 +0800 Subject: [PATCH 07/89] add two PD features --- releases/release-8.5.0.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 6a23e9589b8b6..133381e4d0a82 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -23,17 +23,19 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Scalability -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** +* The Schema cache memory limit feature is now Generally Available (GA), reducing memory usage in large-scale data scenarios. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. + In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows the system to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. Starting from v8.4, this feature is enabled by default with a default value of 512MiB, and users can adjust it as needed through the variable [tidb_schema_cache_size](/system-variables#tidb_schema_cache_size-new-in-v800). For more information, see [Documentation](link). -* The Schema cache memory limit feature is now Generally Available (GA), reducing memory usage in large-scale data scenarios. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Use the Active PD Follower feature to enhance the scalability of PD's Region information query service (General Availability) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) - In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows the system to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. Starting from v8.4, this feature is enabled by default with a default value of 512MiB, and users can adjust it as needed through the variable [tidb_schema_cache_size](/system-variables#tidb_schema_cache_size-new-in-v800). + In a TiDB cluster with a large number of Regions, the PD leader might experience high CPU load due to the increased overhead of handling heartbeats and scheduling tasks. If the cluster has many TiDB instances, and there is a high concurrency of requests for Region information, the CPU pressure on the PD leader increases further and might cause PD services to become unavailable. - For more information, see [Documentation](link). + To ensure high availability and also enhance the scalability of PD's Region information query service. You can enable the Active PD Follower feature by setting the system variable [`pd_enable_follower_handle_region`](/system-variables.md#pd_enable_follower_handle_region-new-in-v760) to `ON`. After this feature is enabled, TiDB evenly distributes Region information requests to all PD servers, and PD followers can also handle Region requests, thereby reducing the CPU pressure on the PD leader. + + For more information, see [documentation](/tune-region-performance.md#use-the-active-pd-follower-feature-to-enhance-the-scalability-of-pds-region-information-query-service) ### Performance @@ -51,11 +53,9 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Reliability -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** - - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. +* Enabling rate limiter can protect PD from being crash under a large number of sudden requests and improve the stability of PD (General Availability) [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) - For more information, see [Documentation](link). + You can adjust the rate limiter configuration through pd-ctl, for more information, see [PD Control User Guide | TiDB Docs](/stable/pd-control.md). ### Availability From 75dd8ad5cc18be08f15ec4f020da169afbc2cf59 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 12 Nov 2024 13:51:05 +0800 Subject: [PATCH 08/89] Update releases/release-8.5.0.md Co-authored-by: Frank945946 <108602632+Frank945946@users.noreply.github.com> --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 133381e4d0a82..638cd89f6c3f5 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -141,7 +141,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | Variable name | Change type | Description | |--------|------------------------------|------| -| | | | +|tidb_ddl_reorg_max_write_speed | Newly added |Used to control the speed at which TiDB writes index data to a single TiKV node. For example, setting the value to 200 MiB limits the maximum write speed to 200 MiB/s. | | | | | | | | | From 0a0a459daefe15ce90b86c16440852583345da6f Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 12 Nov 2024 13:52:16 +0800 Subject: [PATCH 09/89] Apply suggestions from code review --- releases/release-8.5.0.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 638cd89f6c3f5..d9cdf671cc1a7 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -53,9 +53,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Reliability -* Enabling rate limiter can protect PD from being crash under a large number of sudden requests and improve the stability of PD (General Availability) [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) +* Enabling rate limiter can protect PD from being crash under a large number of sudden requests and improve the stability of PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) - You can adjust the rate limiter configuration through pd-ctl, for more information, see [PD Control User Guide | TiDB Docs](/stable/pd-control.md). + You can adjust the rate limiter configuration through pd-ctl. + + For more information, see [Documentation](/stable/pd-control.md). ### Availability From cd199392843a293254c8a5e9cae33fa4ad287fb6 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 15 Nov 2024 22:53:10 +0800 Subject: [PATCH 10/89] Apply suggestions from code review --- releases/release-8.5.0.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index d9cdf671cc1a7..a934cbb5921d6 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -23,11 +23,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Scalability -* The Schema cache memory limit feature is now Generally Available (GA), reducing memory usage in large-scale data scenarios. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd +* The schema cache memory limit feature is now generally available (GA), reducing memory usage in scenarios with hundreds of thousands or even millions of databases or tables. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd - In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows the system to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. Starting from v8.4, this feature is enabled by default with a default value of 512MiB, and users can adjust it as needed through the variable [tidb_schema_cache_size](/system-variables#tidb_schema_cache_size-new-in-v800). + In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows TiDB to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. + + Starting from v8.4.0, this feature is enabled by default with a default value of `536870912` (that is, 512MiB). You can adjust it as needed through the variable [`tidb_schema_cache_size`](/system-variables.md#tidb_schema_cache_size-new-in-v800). - For more information, see [Documentation](link). + For more information, see [Documentation](/schema-cache.md). * Use the Active PD Follower feature to enhance the scalability of PD's Region information query service (General Availability) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) @@ -81,17 +83,17 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Documentation](link). -* Introduction of ADMIN ALTER DDL JOBS syntax to support online modification of DDL job variables [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd - Starting from v8.3, it is now possible to set the variables [tidb_ddl_reorg_batch_size](/system-variables#tidb_ddl_reorg_batch_size) and [tidb_ddl_reorg_worker_cnt](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, the DDL job must first be canceled, the variables adjusted, and then resubmit the job. + Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. - Since v8.5, the ADMIN ALTER DDL JOBS syntax has been introduced, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example: + TiDB v8.5.0 introduces the `ADMIN ALTER DDL JOBS` statement, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example: - - `ADMIN ALTER DDL JOBS job_id THREAD = 8;` — Adjusts the `tidb_ddl_reorg_worker_cnt` for the specified DDL task. - - `ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;` — Adjusts the `tidb_ddl_reorg_batch_size` for the specified DDL task. - - `ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';` — Adjusts the write traffic size for index data on each TiKV node. + - `ADMIN ALTER DDL JOBS job_id THREAD = 8;`: adjusts the `tidb_ddl_reorg_worker_cnt` for the specified DDL task. + - `ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;`: adjusts the `tidb_ddl_reorg_batch_size` for the specified DDL task. + - `ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';`: adjusts the write traffic size for index data on each TiKV node. - For more information, see [Documentation](link). + For more information, see [Documentation](/sql-statements/sql-statement-admin-alter-ddl.md). ### DB operations From ba2b4c0989dfe69d6588770e3272e8d614dc282e Mon Sep 17 00:00:00 2001 From: long0712 <87366891+ljun0712@users.noreply.github.com> Date: Mon, 18 Nov 2024 19:00:57 +0800 Subject: [PATCH 11/89] Update release-8.5.0.md Add GCP KMS content --- releases/release-8.5.0.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index a934cbb5921d6..7ec93870f6166 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -123,6 +123,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [documentation](/br/br-pitr-manual.md#encrypt-the-log-backup-data). +* TiKV encryption at rest supports Google [Key Management Service (Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive?hl) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) + + In v8.0.0, TiKV has supported this feature as an experimental feature. TiKV supports configuring Google Cloud KMS-based master keys for encryption at rest. Starting from v8.5.0, this feature is now Generally Avaialble. + To enable encryption at rest based on Google Cloud KMS, you need to create a key on Google Cloud and then configure the `[security.encryption.master-key]` section in the TiKV configuration file. + + For more information, see [documentation](/encryption-at-rest.md#tikv-encryption-at-rest). + ### Data migration * Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** From 3457fc33f2e762b3ca7fddfac613cc86bfd4fda3 Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 19 Nov 2024 12:01:12 +0800 Subject: [PATCH 12/89] update tidb_enable_fast_create_table --- releases/release-8.5.0.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 7ec93870f6166..ccbf057bdaf90 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -47,11 +47,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Documentation](link). -* The database and table creation acceleration feature is now Generally Available (GA), significantly reducing the time required for data migration and cluster initialization. [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt +* TiDB accelerated table creation becomes generally available (GA), significantly reducing data migration and cluster initialization time [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt - In v8.0.0, the system variable [tidb_enable_fast_create_table](/system-variables#tidb_enable_fast_create_table-new-in-v800) was introduced to improve the performance of batch database and table creation. During data migration and cluster initialization, it enables the rapid creation of tables at the million-scale level, significantly reducing the time required. + TiDB v7.6.0 introduces accelerated table creation as an experimental feature, controlled by the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_ddl_version-new-in-v760). Staring from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). - For more information, see [Documentation](link). + In v8.5.0, TiDB accelerated table creation becomes generally available (GA) and is enabled by default. During data migration and cluster initialization, this feature supports rapid creation of millions of tables, significantly reducing operation time. + + For more information, see [Documentation](/accelerated-table-creation.md). ### Reliability From 31849eab1cfda88d05b735d4dfa982c77fb2612a Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 19 Nov 2024 14:51:21 +0800 Subject: [PATCH 13/89] add compatibility changes > system variables: tidb_enable_fast_create_table --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index ccbf057bdaf90..2537c02b218ef 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -155,7 +155,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | Variable name | Change type | Description | |--------|------------------------------|------| |tidb_ddl_reorg_max_write_speed | Newly added |Used to control the speed at which TiDB writes index data to a single TiKV node. For example, setting the value to 200 MiB limits the maximum write speed to 200 MiB/s. | -| | | | +| [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800) | Modified | Changes the default value from `OFF` to `ON` after further tests, meaning that the [accelerated table creation](/accelerated-table-creation.md) feature is enabled by default. | | | | | ### Configuration parameters From 30d5d9dd8dfe938374b5a641a351c56ae60f59d7 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 20 Nov 2024 17:27:56 +0800 Subject: [PATCH 14/89] add the highlight table --- releases/release-8.5.0.md | 96 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 95 insertions(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 2537c02b218ef..a1a2ae138399d 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -17,7 +17,101 @@ TiDB 8.5.0 is a Long-Term Support Release (LTS). Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, and bug fixes released in [8.2.0-DMR](/releases/release-8.2.0.md), [8.3.0-DMR](/releases/release-8.3.0.md), and [8.4.0-DMR](/releases/release-8.4.0.md). When you upgrade from 8.1.x to 8.5.0, you can download the [TiDB Release Notes PDF](https://download.pingcap.org/tidb-v8.1-to-v8.5-en-release-notes.pdf) to view all release notes between the two LTS versions. The following table lists some highlights from 8.1.0 to 8.5.0: - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
分类功能描述
可扩展性与性能 多维度降低数据处理延迟 **tw@qiancai**通过不断挖掘数据处理的细节，TiDB 持续提升自身性能，力求满足金融场景对 SQL 处理时延的要求。 包括以下关键更新： +
  • 并行排序 (v8.2.0 引入)
    • +
  • 优化 KV 请求批处理策略 (v8.3.0 引入)
    • +
  • 并行获取 TSO (v8.4.0 引入)
    • +
  • 删除语句只获取必要的列 (v8.4.0 引入)
    • +
  • 优化缓存表场景性能 (v8.4.0 引入)
    • +
  • Hash Join 算法演进 (v8.4.0 引入)
    • +
    Active PD Follower 成为正式功能 **tw@Oreoxmt 2015**TiDB v7.6.0 引入了 Active PD Follower 特性，允许 PD follower 提供 Region 信息查询服务。在 TiDB 节点数量较多和 Region 数量较多的集群中，该特性可以提升 PD 集群处理 GetRegion、ScanRegions 请求的能力，减轻 PD leader 的 CPU 压力。在 v8.5.0，Active PD Follower 成为正式功能。
    实例级执行计划缓存（实验特性）（v8.4.0 引入）实例级执行计划缓存允许同一个 TiDB 实例的所有会话共享执行计划缓存。与现有的会话级执行计划缓存相比，实例级执行计划缓存能够在内存中缓存更多执行计划，减少 SQL 编译时间，从而降低 SQL 整体运行时间，提升 OLTP 的性能和吞吐，同时更好地控制内存使用，提升数据库稳定性。
    分区表全局索引（v8.4.0 起成为正式功能）全局索引可以有效提高检索非分区列的效率，并且消除了唯一键必须包含分区键的限制。该功能扩展了 TiDB 分区表的使用场景，避免了数据迁移过程中的一些应用修改工作。
    默认允许将 Projection 算子下推到存储引擎（v8.3.0 引入） Projection 算子下推可以将负载分散到存储节点，同时减少节点间的数据传输。这有助于降低部分 SQL 的执行时间，提升数据库的整体性能。
    统计信息收集忽略不必要的列（v8.3.0 引入） 在保证优化器能够获取到必要信息的前提下，加快了统计信息收集的速度，提升统计信息的时效性，进而保证选择最优的执行计划，提升集群性能。同时也降低了系统开销，改善了资源利用率。
    稳定性与高可用提升超大规模集群的稳定性 **tw@hfxsd 1976**对于使用 TiDB 运行多租户应用或者 SaaS 应用的公司，经常需要存储大量的表，TiDB 在 v8.5.0 着力增强了大规模集群的稳定性。 Schema 缓存控制以及Stats 缓存控制已经成为正式功能，减少了内存过度消耗带来的稳定性问题。 PD 通过 Active Follower 应对大量 Region 带来的压力，并将 PD 所承担的服务逐步解耦，独立部署。通过增加并发度，以及减少收集对象的数量，统计信息收集和加载效率得到提升，保证了大集群执行计划的稳定性。
    Runaway Queries 支持更多触发条件，并能够切换资源组 （v8.4.0 引入）Runaway Queries 提供了有效的手段来降低突发的 SQL 性能问题对系统产生的影响。v8.4.0 中新增 Coprocessor 处理的 Key 的数量 (PROCESSED_KEYS) 和 Request Unit (RU) 作为识别条件，并可以将识别到的查询置入指定资源组，对 Runaway Queries 进行更精确的识别与控制。
    支持为资源管控的后台任务设置资源使用上限 (实验特性)（v8.4.0 引入）为资源管控的后台任务设置百分比上限，针对不同业务系统的需求，控制后台任务的消耗，从而将后台任务的消耗限制在一个很低的水平，保证在线业务的服务质量。
    增强并扩展 TiProxy 的使用场景 **tw@Oreoxmt**作为 TiDB 高可用的重要组成，TiProxy 在做好 SQL 流量接入和转发的同时，开始尝试对集群变更进行评估。主要包括： +
  • TiProxy 流量捕获和回放（实验特性）（v8.4.0 引入）
    • +
  • TiProxy 内置虚拟 IP 管理（v8.3.0 引入）
    • +
  • TiProxy 支持多种负载均衡策略 （v8.2.0 引入）
    • +
    并行 HashAgg 算法支持数据落盘 （v8.2.0 起成为正式功能）HashAgg 是 TiDB 中常用的聚合算子，用于快速聚合具有相同字段值的行。TiDB v8.0.0 引入并行 HashAgg 作为实验特性，以进一步提升处理速度。当内存资源不足时，并行 HashAgg 可以将临时排序数据落盘，避免因内存使用过度而导致的 OOM 风险，从而提升查询性能和节点稳定性。该功能在 v8.2.0 成为正式功能，并默认开启，用户可以通过 tidb_executor_concurrency 安全地设置并行 HashAgg 的并发度。
    SQL 外键约束成为正式功能 **tw@lilin90 1894**外键（Foreign Key）是数据库中的一种约束，用于建立表与表之间的关联关系，确保数据一致性和完整性。它可以限制子表中引用的数据必须存在于主表中，防止无效数据插入。同时，外键支持级联操作（如删除或更新时自动同步），简化了业务逻辑的实现，减少了手动维护数据关联的复杂性。
    支持向量搜索功能（实验特性）(v8.4.0 引入）向量搜索是一种基于数据语义的搜索方法，可以提供更相关的搜索结果。作为 AI 和大语言模型 (LLM) 的核心功能之一，向量搜索可用于检索增强生成 (Retrieval-Augmented Generation, RAG)、语义搜索、推荐系统等多种场景。
    数据库管理与可观测性在内存表中显示 TiKV 和 TiDB 的 CPU 时间 （v8.4.0 引入）将 CPU 时间合入系统表中展示，与会话或 SQL 的其他指标并列，方便你从多角度对高 CPU 消耗的操作进行观测，提升诊断效率。尤其适用于诊断实例 CPU 飙升或集群读写热点等场景。
    按表或数据库维度聚合 TiKV 消耗的 CPU 时间（v8.4.0 引入）当热点问题不是由个别 SQL 语句引起时，利用 Top SQL 中按表或者数据库聚合的 CPU 时间，能够协助用户快速发现造成热点的表或者应用程序，从而大大提升热点问题和 CPU 消耗问题的诊断效率。
    支持对开启了 IMDSv2 服务的 TiKV 实例做备份 （v8.4.0 引入）目前 AWS EC2 的默认元数据服务是 IMDSv2。TiDB 支持从开启了 IMDSv2 的 TiKV 实例中备份数据，协助你更好地在公有云服务中运行 TiDB 集群。
    安全日志备份数据支持客户端加密在上传日志备份到备份存储之前，你可以对日志备份数据进行加密，确保数据在存储和传输过程中的安全性。
    ## Feature details From 5b25d7942315cc2606f78aef479b05732807281f Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 20 Nov 2024 17:36:44 +0800 Subject: [PATCH 15/89] Apply suggestions from code review Co-authored-by: Wenqi Mou --- releases/release-8.5.0.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index a1a2ae138399d..ae35c7cb69eda 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -209,9 +209,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Security -* BR supports client-side encryption of log backup data (GA) [#56433](https://github.com/pingcap/tidb/issues/56433) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai +* BR supports client-side encryption of full backup (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) @[joccau](https://github.com/joccau) and log backup data (GA) [#56433] (https://github.com/pingcap/tidb/issues/56433) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai +TiDB V5.3.0 introduced an experimental feature to encrypt full backup data on the client side, you can encrypt the backup data using a custom fixed key. This feature is Generally Available in v8.5.0 - TiDB v8.4.0 introduced an experimental feature to encrypt, on the client side, log backup data. Starting from v8.5.0, this feature is now Generally Avaialble. Before uploading log backup data to your backup storage, you can encrypt the backup data to ensure its security via one of the following methods: +For more information, see [documentation](/br/br-snapshot-manual.md#encrypt-the-backup-data). + TiDB v8.4.0 introduced an experimental feature to encrypt log backup data on the client side. Starting from v8.5.0, this feature is now Generally Available. Before uploading log backup data to your backup storage, you can encrypt the log backup data to ensure its security via one of the following methods: - Encrypt using a custom fixed key - Encrypt using a master key stored on a local disk From 4a4da4ebfa4456c4cd6954c396b8e9caf7928317 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 22 Nov 2024 10:17:44 +0800 Subject: [PATCH 16/89] Update releases/release-8.5.0.md --- releases/release-8.5.0.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index ae35c7cb69eda..dfc53735ca59b 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -209,17 +209,19 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Security -* BR supports client-side encryption of full backup (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) @[joccau](https://github.com/joccau) and log backup data (GA) [#56433] (https://github.com/pingcap/tidb/issues/56433) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai -TiDB V5.3.0 introduced an experimental feature to encrypt full backup data on the client side, you can encrypt the backup data using a custom fixed key. This feature is Generally Available in v8.5.0 +* BR supports client-side encryption of both full backup data and log backup data (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) [#56433] (https://github.com/pingcap/tidb/issues/56433) @[joccau](https://github.com/joccau) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai -For more information, see [documentation](/br/br-snapshot-manual.md#encrypt-the-backup-data). - TiDB v8.4.0 introduced an experimental feature to encrypt log backup data on the client side. Starting from v8.5.0, this feature is now Generally Available. Before uploading log backup data to your backup storage, you can encrypt the log backup data to ensure its security via one of the following methods: + * Client-side encryption of full backup data (introduced as experimental in TiDB v5.3.0) enables you to encrypt backup data on the client side using a custom fixed key. - - Encrypt using a custom fixed key - - Encrypt using a master key stored on a local disk - - Encrypt using a master key managed by a Key Management Service (KMS) + * Client-side encryption of log backup data (introduced as experimental in TiDB v8.4.0) enables you to encrypt log backup data on the client side using one of the following methods: - For more information, see [documentation](/br/br-pitr-manual.md#encrypt-the-log-backup-data). + * Encrypt using a custom fixed key + * Encrypt using a master key stored on a local disk + * Encrypt using a master key managed by a Key Management Service (KMS) + + Starting from v8.5.0, both features become generally available (GA), offering enhanced client-side data security. + + For more information, see [Encrypt the backup data](/br/br-snapshot-manual.md#encrypt-the-backup-data) and [Encrypt the log backup data](/br/br-pitr-manual.md#encrypt-the-log-backup-data). * TiKV encryption at rest supports Google [Key Management Service (Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive?hl) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) From 0ed8ce1bf797c6fae47785b4d666a2af37482346 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 22 Nov 2024 10:21:23 +0800 Subject: [PATCH 17/89] update the description for #5739 --- releases/release-8.5.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index dfc53735ca59b..25f29036cfaa3 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -151,11 +151,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Reliability -* Enabling rate limiter can protect PD from being crash under a large number of sudden requests and improve the stability of PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) +* Support configuring the rate limit and concurrency for gRPC API requests sent to PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) **tw@qiancai** - You can adjust the rate limiter configuration through pd-ctl. - - For more information, see [Documentation](/stable/pd-control.md). + When a sudden influx of requests is sent to PD, it can lead to high workloads and potentially affect PD performance. Starting from v8.5.0, you can use [`pd-ctl`](/pd-control.md) to configure rate limits and concurrency for gRPC API requests sent to PD, improving its stability. + + For more information, see [Documentation](/pd-control.md). ### Availability From 4f6e5d5e62fa4c842670bad6545b178c79200393 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 22 Nov 2024 14:31:23 +0800 Subject: [PATCH 18/89] update the description for #8906 --- releases/release-8.5.0.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 25f29036cfaa3..4b959623212f8 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -223,10 +223,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Encrypt the backup data](/br/br-snapshot-manual.md#encrypt-the-backup-data) and [Encrypt the log backup data](/br/br-pitr-manual.md#encrypt-the-log-backup-data). -* TiKV encryption at rest supports Google [Key Management Service (Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive?hl) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) +* TiKV encryption at rest supports Google [Key Management Service (Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive?hl) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) **tw@qiancai** - In v8.0.0, TiKV has supported this feature as an experimental feature. TiKV supports configuring Google Cloud KMS-based master keys for encryption at rest. Starting from v8.5.0, this feature is now Generally Avaialble. - To enable encryption at rest based on Google Cloud KMS, you need to create a key on Google Cloud and then configure the `[security.encryption.master-key]` section in the TiKV configuration file. + TiKV ensures data security by using the encryption at rest technique to encrypt stored data. The core aspect of this technique is proper key management. In v8.0.0, TiKV encryption at rest experimentally supports using Google Cloud KMS for master key management. + + Starting from v8.5.0, encryption at rest using Google Cloud KMS becomes generally available (GA). To use this feature, first create a key on Google Cloud, and then configure the `[security.encryption.master-key]` section in the TiKV configuration file. For more information, see [documentation](/encryption-at-rest.md#tikv-encryption-at-rest). From 39bac8019d592c66891d95a11132f7b49931df9c Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 22 Nov 2024 18:12:54 +0800 Subject: [PATCH 19/89] update Active PD Follower Signed-off-by: Aolin --- releases/release-8.5.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 4b959623212f8..1c588b589a0d5 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -125,13 +125,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Documentation](/schema-cache.md). -* Use the Active PD Follower feature to enhance the scalability of PD's Region information query service (General Availability) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) +* Use the Active PD Follower feature to enhance the scalability of PD's Region information query service (GA) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) tw@Oreoxmt In a TiDB cluster with a large number of Regions, the PD leader might experience high CPU load due to the increased overhead of handling heartbeats and scheduling tasks. If the cluster has many TiDB instances, and there is a high concurrency of requests for Region information, the CPU pressure on the PD leader increases further and might cause PD services to become unavailable. - To ensure high availability and also enhance the scalability of PD's Region information query service. You can enable the Active PD Follower feature by setting the system variable [`pd_enable_follower_handle_region`](/system-variables.md#pd_enable_follower_handle_region-new-in-v760) to `ON`. After this feature is enabled, TiDB evenly distributes Region information requests to all PD servers, and PD followers can also handle Region requests, thereby reducing the CPU pressure on the PD leader. + To ensure high availability, TiDB v7.6.0 introduces Active PD Follower as an experimental feature to enhance the scalability of PD's Region information query service. In v8.5.0, this feature becomes generally available (GA). You can enable the Active PD Follower feature by setting the system variable [`pd_enable_follower_handle_region`](/system-variables.md#pd_enable_follower_handle_region-new-in-v760) to `ON`. After this feature is enabled, TiDB evenly distributes Region information requests to all PD servers, and PD followers can also handle Region requests, thereby reducing the CPU pressure on the PD leader. - For more information, see [documentation](/tune-region-performance.md#use-the-active-pd-follower-feature-to-enhance-the-scalability-of-pds-region-information-query-service) + For more information, see [documentation](/tune-region-performance.md#use-the-active-pd-follower-feature-to-enhance-the-scalability-of-pds-region-information-query-service). ### Performance From b587f806ab7ef22559b480d989bf572338c30604 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 22 Nov 2024 18:31:32 +0800 Subject: [PATCH 20/89] highlights: update Active PD Followers and TiProxy Signed-off-by: Aolin --- releases/release-8.5.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 1c588b589a0d5..0ffd99400a0ef 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -39,8 +39,8 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - Active PD Follower 成为正式功能 **tw@Oreoxmt 2015** - TiDB v7.6.0 引入了 Active PD Follower 特性，允许 PD follower 提供 Region 信息查询服务。在 TiDB 节点数量较多和 Region 数量较多的集群中，该特性可以提升 PD 集群处理 GetRegion、ScanRegions 请求的能力，减轻 PD leader 的 CPU 压力。在 v8.5.0，Active PD Follower 成为正式功能。 + Use Active PD Followers to enhance PD's Region information query service (GA in v8.5.0) **tw@Oreoxmt 2015** + TiDB v7.6.0 introduces an experimental feature "Active PD Follower", which allows PD followers to provide Region information query services. This feature improves the capability of the PD cluster to handle GetRegion and ScanRegions requests in clusters with a large number of TiDB nodes and Regions, thereby reducing the CPU pressure on PD leaders. In v8.5.0, this feature becomes generally available (GA). 实例级执行计划缓存（实验特性）（v8.4.0 引入） @@ -72,11 +72,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, 为资源管控的后台任务设置百分比上限，针对不同业务系统的需求，控制后台任务的消耗，从而将后台任务的消耗限制在一个很低的水平，保证在线业务的服务质量。 - 增强并扩展 TiProxy 的使用场景 **tw@Oreoxmt** - 作为 TiDB 高可用的重要组成，TiProxy 在做好 SQL 流量接入和转发的同时，开始尝试对集群变更进行评估。主要包括： -
  • TiProxy 流量捕获和回放（实验特性）（v8.4.0 引入）
    • -
  • TiProxy 内置虚拟 IP 管理（v8.3.0 引入）
    • -
  • TiProxy 支持多种负载均衡策略 （v8.2.0 引入）
    • + Enhance and expand TiProxy use cases **tw@Oreoxmt** + As a crucial component of the high availability of TiDB, TiProxy extends its capabilities beyond SQL traffic access and forwarding to support cluster change evaluation. Key features include: +
  • TiProxy supports traffic capture and replay (experimental, introduced in v8.4.0)
    • +
  • Built-in virtual IP management in TiProxy (introduced in v8.3.0)
    • +
  • TiProxy supports multiple load balancing policies (introduced in v8.2.0)
    • From 3e11ebb63e12e96f38406a814cced240fc6edc94 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 25 Nov 2024 09:22:10 +0800 Subject: [PATCH 21/89] translate highlights --- releases/release-8.5.0.md | 44 +++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 0ffd99400a0ef..b9565dc4ae2c4 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -43,20 +43,20 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, TiDB v7.6.0 introduces an experimental feature "Active PD Follower", which allows PD followers to provide Region information query services. This feature improves the capability of the PD cluster to handle GetRegion and ScanRegions requests in clusters with a large number of TiDB nodes and Regions, thereby reducing the CPU pressure on PD leaders. In v8.5.0, this feature becomes generally available (GA). - 实例级执行计划缓存（实验特性）（v8.4.0 引入） - 实例级执行计划缓存允许同一个 TiDB 实例的所有会话共享执行计划缓存。与现有的会话级执行计划缓存相比，实例级执行计划缓存能够在内存中缓存更多执行计划，减少 SQL 编译时间，从而降低 SQL 整体运行时间，提升 OLTP 的性能和吞吐，同时更好地控制内存使用，提升数据库稳定性。 + Instance-level execution plan cache (experimental) (introduced in v8.4.0) + Instance-level plan cache allows all sessions within the same TiDB instance to share the plan cache. Compared with session-level plan cache, this feature reduces SQL compilation time by caching more execution plans in memory, decreasing overall SQL execution time. It improves OLTP performance and throughput while providing better control over memory usage and enhancing database stability. - 分区表全局索引（v8.4.0 起成为正式功能） - 全局索引可以有效提高检索非分区列的效率，并且消除了唯一键必须包含分区键的限制。该功能扩展了 TiDB 分区表的使用场景，避免了数据迁移过程中的一些应用修改工作。 + Global indexes for partitioned tables (GA in v8.4.0) + Global indexes can effectively improve the efficiency of retrieving non-partitioned columns, and remove the restriction that a unique key must contain the partition key. This feature extends the usage scenarios of TiDB partitioned tables, and avoids some of the application modification work required for data migration. - 默认允许将 Projection 算子下推到存储引擎（v8.3.0 引入） - Projection 算子下推可以将负载分散到存储节点，同时减少节点间的数据传输。这有助于降低部分 SQL 的执行时间，提升数据库的整体性能。 + Default pushdown of the Projection operator to the storage engine (introduced in v8.3.0) + Pushing the Projection operator down to the storage engine can distribute the load across storage nodes while reducing data transfer between nodes. This optimization helps to reduce the execution time for certain SQL queries and improves the overall database performance. - 统计信息收集忽略不必要的列（v8.3.0 引入） - 在保证优化器能够获取到必要信息的前提下，加快了统计信息收集的速度，提升统计信息的时效性，进而保证选择最优的执行计划，提升集群性能。同时也降低了系统开销，改善了资源利用率。 + Ignoring unnecessary columns when collecting statistics (introduced in v8.3.0) + Under the premise of ensuring that the optimizer can obtain the necessary information, TiDB speeds up statistics collection, improves the timeliness of statistics, and thus ensures that the optimal execution plan is selected, improving the performance of the cluster. Meanwhile, TiDB also reduces the system overhead and improves the resource utilization. 稳定性与高可用 @@ -80,8 +80,8 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - 并行 HashAgg 算法支持数据落盘 （v8.2.0 起成为正式功能） - HashAgg 是 TiDB 中常用的聚合算子，用于快速聚合具有相同字段值的行。TiDB v8.0.0 引入并行 HashAgg 作为实验特性，以进一步提升处理速度。当内存资源不足时，并行 HashAgg 可以将临时排序数据落盘，避免因内存使用过度而导致的 OOM 风险，从而提升查询性能和节点稳定性。该功能在 v8.2.0 成为正式功能，并默认开启，用户可以通过 tidb_executor_concurrency 安全地设置并行 HashAgg 的并发度。 + The parallel HashAgg algorithm of TiDB supports disk spill (GA in v8.2.0) + HashAgg is a widely used aggregation operator in TiDB for efficiently aggregating rows with the same field values. TiDB v8.0.0 introduces parallel HashAgg as an experimental feature to further enhance processing speed. When memory resources are insufficient, parallel HashAgg spills temporary sorted data to disk, avoiding potential OOM risks caused by excessive memory usage. This improves query performance while maintaining node stability. In v8.2.0, this feature becomes generally available (GA) and is enabled by default, enabling you to safely configure the concurrency of parallel HashAgg using tidb_executor_concurrency. SQL @@ -89,26 +89,26 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, 外键（Foreign Key）是数据库中的一种约束，用于建立表与表之间的关联关系，确保数据一致性和完整性。它可以限制子表中引用的数据必须存在于主表中，防止无效数据插入。同时，外键支持级联操作（如删除或更新时自动同步），简化了业务逻辑的实现，减少了手动维护数据关联的复杂性。 - 支持向量搜索功能（实验特性）(v8.4.0 引入） - 向量搜索是一种基于数据语义的搜索方法，可以提供更相关的搜索结果。作为 AI 和大语言模型 (LLM) 的核心功能之一，向量搜索可用于检索增强生成 (Retrieval-Augmented Generation, RAG)、语义搜索、推荐系统等多种场景。 + Vector search (experimental, introduced in v8.4.0) + Vector search is a search method based on data semantics, which provides more relevant search results. As one of the core functions of AI and large language models (LLMs), vector search can be used in various scenarios such as Retrieval-Augmented Generation (RAG), semantic search, and recommendation systems. - 数据库管理与可观测性 - 在内存表中显示 TiKV 和 TiDB 的 CPU 时间 （v8.4.0 引入） - 将 CPU 时间合入系统表中展示，与会话或 SQL 的其他指标并列，方便你从多角度对高 CPU 消耗的操作进行观测，提升诊断效率。尤其适用于诊断实例 CPU 飙升或集群读写热点等场景。 + DB Operations and Observability + Display TiKV and TiDB CPU times in memory tables (introduced in v8.4.0) + The CPU time is now integrated into a system table, displayed alongside other metrics for sessions or SQL, letting you observe high CPU consumption operations from multiple perspectives, and improving diagnostic efficiency. This is especially useful for diagnosing scenarios such as CPU spikes in instances or read/write hotspots in clusters. - 按表或数据库维度聚合 TiKV 消耗的 CPU 时间（v8.4.0 引入） - 当热点问题不是由个别 SQL 语句引起时，利用 Top SQL 中按表或者数据库聚合的 CPU 时间，能够协助用户快速发现造成热点的表或者应用程序，从而大大提升热点问题和 CPU 消耗问题的诊断效率。 + Support viewing aggregated TiKV CPU time by table or database (introduced in v8.4.0) + When hotspot issues are not caused by individual SQL statements, using the aggregated CPU time by table or database level in Top SQL can help you quickly identify the tables or applications responsible for the hotspots, significantly improving the efficiency of diagnosing hotspot and CPU consumption issues. - 支持对开启了 IMDSv2 服务的 TiKV 实例做备份 （v8.4.0 引入） - 目前 AWS EC2 的默认元数据服务是 IMDSv2。TiDB 支持从开启了 IMDSv2 的 TiKV 实例中备份数据，协助你更好地在公有云服务中运行 TiDB 集群。 + Support backing up TiKV instances with IMDSv2 service enabled (introduced in v8.4.0) + <AWS EC2 now uses IMDSv2 as the default metadata service. TiDB supports backing up data from TiKV instances that have IMDSv2 enabled, helping you run TiDB clusters more effectively in public cloud services. - 安全 - 日志备份数据支持客户端加密 - 在上传日志备份到备份存储之前，你可以对日志备份数据进行加密，确保数据在存储和传输过程中的安全性。 + Security + Client-side encryption of snapshot backup data和log backup data (GA in v8.5.0)**tw@qiancai 1998** + Before uploading backup data to your backup storage, you can encrypt the backup data to ensure its security during storage and transmission. From 0e2f9016488f153798a3b74284c2d6d6e79b9a79 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 25 Nov 2024 09:24:27 +0800 Subject: [PATCH 22/89] update descriptions of two features --- releases/release-8.5.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index b9565dc4ae2c4..ca70374aea464 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -151,9 +151,9 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Reliability -* Support configuring the rate limit and concurrency for gRPC API requests sent to PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) **tw@qiancai** +* Support limiting the maximum rate and concurrency of requests processed by PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) **tw@qiancai** - When a sudden influx of requests is sent to PD, it can lead to high workloads and potentially affect PD performance. Starting from v8.5.0, you can use [`pd-ctl`](/pd-control.md) to configure rate limits and concurrency for gRPC API requests sent to PD, improving its stability. + When a sudden influx of requests is sent to PD, it can lead to high workloads and potentially affect PD performance. Starting from v8.5.0, you can use [`pd-ctl`](/pd-control.md) to limit the maximum rate and concurrency of requests processed by PD, improving its stability. For more information, see [Documentation](/pd-control.md). @@ -219,7 +219,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, * Encrypt using a master key stored on a local disk * Encrypt using a master key managed by a Key Management Service (KMS) - Starting from v8.5.0, both features become generally available (GA), offering enhanced client-side data security. + Starting from v8.5.0, both encryption features become generally available (GA), offering enhanced client-side data security. For more information, see [Encrypt the backup data](/br/br-snapshot-manual.md#encrypt-the-backup-data) and [Encrypt the log backup data](/br/br-pitr-manual.md#encrypt-the-log-backup-data). From 8087f51f376dc64f3b42a5d59639208fd9b9e531 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 25 Nov 2024 11:17:32 +0800 Subject: [PATCH 23/89] translate highlights --- releases/release-8.5.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index ca70374aea464..608f29fd479b1 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -59,17 +59,17 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Under the premise of ensuring that the optimizer can obtain the necessary information, TiDB speeds up statistics collection, improves the timeliness of statistics, and thus ensures that the optimal execution plan is selected, improving the performance of the cluster. Meanwhile, TiDB also reduces the system overhead and improves the resource utilization. - 稳定性与高可用 + Reliability and availability 提升超大规模集群的稳定性 **tw@hfxsd 1976** 对于使用 TiDB 运行多租户应用或者 SaaS 应用的公司，经常需要存储大量的表，TiDB 在 v8.5.0 着力增强了大规模集群的稳定性。 Schema 缓存控制以及Stats 缓存控制已经成为正式功能，减少了内存过度消耗带来的稳定性问题。 PD 通过 Active Follower 应对大量 Region 带来的压力，并将 PD 所承担的服务逐步解耦，独立部署。通过增加并发度，以及减少收集对象的数量，统计信息收集和加载效率得到提升，保证了大集群执行计划的稳定性。 - Runaway Queries 支持更多触发条件，并能够切换资源组 （v8.4.0 引入） - Runaway Queries 提供了有效的手段来降低突发的 SQL 性能问题对系统产生的影响。v8.4.0 中新增 Coprocessor 处理的 Key 的数量 (PROCESSED_KEYS) 和 Request Unit (RU) 作为识别条件，并可以将识别到的查询置入指定资源组，对 Runaway Queries 进行更精确的识别与控制。 + Support more triggers for runaway queries, and support switching resource groups (introduced in v8.4.0) + Runaway Queries offer an effective way to mitigate the impact of unexpected SQL performance issues on systems. TiDB v8.4.0 introduces the number of keys processed by the Coprocessor (PROCESSED_KEYS) and request units (RU) as identifying conditions, and puts identified queries into the specified resource group for more precise identification and control of runaway queries. - 支持为资源管控的后台任务设置资源使用上限 (实验特性)（v8.4.0 引入） - 为资源管控的后台任务设置百分比上限，针对不同业务系统的需求，控制后台任务的消耗，从而将后台任务的消耗限制在一个很低的水平，保证在线业务的服务质量。 + Support setting the maximum limit on resource usage for background tasks of resource control (experimental, introduced in v8.4.0) + By setting a maximum percentage limit on background tasks of resource control, you can control their resource consumption based on the needs of different application systems. This keeps background task consumption at a low level and ensures the quality of online services. Enhance and expand TiProxy use cases **tw@Oreoxmt** From fecddac99d7c770d0a5d3209725c250de1651ff5 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 25 Nov 2024 14:23:03 +0800 Subject: [PATCH 24/89] update highlights --- releases/release-8.5.0.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 608f29fd479b1..6e9570297e5c2 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -28,14 +28,14 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, 可扩展性与性能 - 多维度降低数据处理延迟 **tw@qiancai** - 通过不断挖掘数据处理的细节，TiDB 持续提升自身性能，力求满足金融场景对 SQL 处理时延的要求。 包括以下关键更新： -
  • 并行排序 (v8.2.0 引入)
    • -
  • 优化 KV 请求批处理策略 (v8.3.0 引入)
    • -
  • 并行获取 TSO (v8.4.0 引入)
    • -
  • 删除语句只获取必要的列 (v8.4.0 引入)
    • -
  • 优化缓存表场景性能 (v8.4.0 引入)
    • -
  • Hash Join 算法演进 (v8.4.0 引入)
    • + Reduce data processing latency in multiple dimensions **tw@qiancai** + TiDB continuously refines data processing to enhance performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include: +
  • Support parallel sorting (introduced in v8.2.0)
    • +
  • Optimize batch processing strategy for KV (key-value) requests (introduced in v8.3.0)
    • +
  • Support parallel mode for TSO requests (introduced in v8.4.0)
    • +
  • Reduce the resource overhead of DELETE operations (introduced in v8.4.0)
    • +
  • Improve query performance for cached tables (introduced in v8.4.0)
    • +
  • Introduce an optimized version of Hash Join (introduced in v8.4.0)
    • @@ -43,7 +43,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, TiDB v7.6.0 introduces an experimental feature "Active PD Follower", which allows PD followers to provide Region information query services. This feature improves the capability of the PD cluster to handle GetRegion and ScanRegions requests in clusters with a large number of TiDB nodes and Regions, thereby reducing the CPU pressure on PD leaders. In v8.5.0, this feature becomes generally available (GA). - Instance-level execution plan cache (experimental) (introduced in v8.4.0) + Instance-level execution plan cache (experimental, introduced in v8.4.0) Instance-level plan cache allows all sessions within the same TiDB instance to share the plan cache. Compared with session-level plan cache, this feature reduces SQL compilation time by caching more execution plans in memory, decreasing overall SQL execution time. It improves OLTP performance and throughput while providing better control over memory usage and enhancing database stability. From 92ee70bad99b7edd0d39eef82efd121eacb27d7f Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 25 Nov 2024 14:26:42 +0800 Subject: [PATCH 25/89] update highlights --- releases/release-8.5.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 6e9570297e5c2..31f27809450ce 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -20,9 +20,9 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - - - + + + @@ -103,11 +103,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + - + From 56a25416f200cc2749e8fc9b8d9cf49e248cd60d Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 25 Nov 2024 14:29:44 +0800 Subject: [PATCH 26/89] update highlights for data processing --- releases/release-8.5.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 31f27809450ce..500de9b124529 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -30,12 +30,12 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, From 81e9b73b31f4c9c68fed109a33111e81bc836a40 Mon Sep 17 00:00:00 2001 From: lilin90 Date: Mon, 25 Nov 2024 15:07:09 +0800 Subject: [PATCH 27/89] Update foreign key descriptions --- releases/release-8.5.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 500de9b124529..d0f3747742bd6 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -85,8 +85,8 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - - + + @@ -173,11 +173,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Documentation](link). -* The Foreign Key feature is now Generally Available (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 +* Support foreign keys (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 - The Foreign Key feature is now GA, enabling the use of foreign key constraints to enhance data consistency and integrity. Users can easily create foreign key constraints between tables, with support for cascading updates and deletions, making data management more convenient. This feature provides better support for applications with complex data constraints. + The foreign key feature becomes generally available (GA) in v8.5.0. Foreign key constraints help ensure data consistency and integrity. You can easily establish foreign key relationships between tables, with support for cascading updates and deletions, simplifying data management. This feature enhances support for applications with complex data relationships. - For more information, see [Documentation](link). + For more information, see [Documentation](/foreign-key.md). * Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd From 2d32e781d09a490870621c3980d16aac0af166c7 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Mon, 25 Nov 2024 16:01:49 +0800 Subject: [PATCH 28/89] Add a link to foreign key --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index d0f3747742bd6..7e54099fb9e41 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -85,7 +85,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + From 8ee66e2f29e834bbe38db41ac36a2fb4ca128509 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Mon, 25 Nov 2024 17:06:32 +0800 Subject: [PATCH 29/89] Update translation --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 7e54099fb9e41..87817d9758c2a 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -27,7 +27,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + - - + + @@ -123,7 +123,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Starting from v8.4.0, this feature is enabled by default with a default value of `536870912` (that is, 512MiB). You can adjust it as needed through the variable [`tidb_schema_cache_size`](/system-variables.md#tidb_schema_cache_size-new-in-v800). - For more information, see [Documentation](/schema-cache.md). + For more information, see [documentation](/schema-cache.md). * Use the Active PD Follower feature to enhance the scalability of PD's Region information query service (GA) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) tw@Oreoxmt @@ -149,6 +149,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Documentation](/accelerated-table-creation.md). +* TiKV supports MVCC In-memory Engine (IME). It can accelerate queries that require scanning of a large number of MVCC historical versions. [#16141](https://github.com/tikv/tikv/issues/16141) [@SpadeA-Tang](https://github.com/SpadeA-Tang) [@glorv](https://github.com/glorv) [@overvenus](https://github.com/overvenus) + + In scenarios where records are updated frequently or TiDB is required to retain historical versions for a long period of time (for example, 24 hours), a buildup of MVCC versions can lead to degradation of scanning performance. TiKV MVCC In-memory Engine improves scanning performance by caching the most recent MVCC versions in memory and deleting the historical versions from memory through a fast GC mechanism. + + Starting from v8.5.0, TiKV introduces MVCC In-memory Engine. When scanning performance is degraded due to the buildup of MVCC versions in a TiKV cluster, you can enable the TiKV MVCC memory engine to improve scanning performance by setting the TiKV configuration parameter [`in-memory-engine.enable`](/tikv-in-memory-engine.md#usage). + + For more information, see [documentation](/tikv-in-memory-engine.md). ### Reliability * Support limiting the maximum rate and concurrency of requests processed by PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) **tw@qiancai** From 651b5e087b79ca7c2d8131d232cf73b64ed99189 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 25 Nov 2024 22:56:26 +0800 Subject: [PATCH 33/89] Apply suggestions from code review --- releases/release-8.5.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 120d3cf0fa46b..34caaab36d7d6 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -121,7 +121,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows TiDB to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. - Starting from v8.4.0, this feature is enabled by default with a default value of `536870912` (that is, 512MiB). You can adjust it as needed through the variable [`tidb_schema_cache_size`](/system-variables.md#tidb_schema_cache_size-new-in-v800). + Starting from v8.4.0, this feature is enabled by default with a default value of `536870912` (that is, 512 MiB). You can adjust it as needed through the variable [`tidb_schema_cache_size`](/system-variables.md#tidb_schema_cache_size-new-in-v800). For more information, see [documentation](/schema-cache.md). @@ -188,7 +188,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, * Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd - Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. + Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. TiDB v8.5.0 introduces the `ADMIN ALTER DDL JOBS` statement, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example: @@ -196,7 +196,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - `ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;`: adjusts the `tidb_ddl_reorg_batch_size` for the specified DDL task. - `ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';`: adjusts the write traffic size for index data on each TiKV node. - For more information, see [Documentation](/sql-statements/sql-statement-admin-alter-ddl.md). + For more information, see [documentation](/sql-statements/sql-statement-admin-alter-ddl.md). ### DB operations From c3c90bd2a95cd6d63d108e671fb2ac693b309a2c Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 26 Nov 2024 10:08:42 +0800 Subject: [PATCH 34/89] Update releases/release-8.5.0.md --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 34caaab36d7d6..d1b05fdbfea7b 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -61,7 +61,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + From a72cf08ccb862a6babc868aa0ef16722fa884477 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 26 Nov 2024 17:45:18 +0800 Subject: [PATCH 35/89] Apply suggestions from code review Co-authored-by: Aolin --- releases/release-8.5.0.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index d1b05fdbfea7b..688eecbf1c669 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -28,12 +28,12 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - - + @@ -162,7 +162,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, When a sudden influx of requests is sent to PD, it can lead to high workloads and potentially affect PD performance. Starting from v8.5.0, you can use [`pd-ctl`](/pd-control.md) to limit the maximum rate and concurrency of requests processed by PD, improving its stability. - For more information, see [Documentation](/pd-control.md). + For more information, see [documentation](/pd-control.md). ### Availability @@ -216,11 +216,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Security -* BR supports client-side encryption of both full backup data and log backup data (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) [#56433] (https://github.com/pingcap/tidb/issues/56433) @[joccau](https://github.com/joccau) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai +* BR supports client-side encryption of both full backup data and log backup data (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) [#56433](https://github.com/pingcap/tidb/issues/56433) @[joccau](https://github.com/joccau) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai * Client-side encryption of full backup data (introduced as experimental in TiDB v5.3.0) enables you to encrypt backup data on the client side using a custom fixed key. - * Client-side encryption of log backup data (introduced as experimental in TiDB v8.4.0) enables you to encrypt log backup data on the client side using one of the following methods: + * Client-side encryption of log backup data (introduced as experimental in TiDB v8.4.0) enables you to encrypt log backup data on the client side using one of the following methods: * Encrypt using a custom fixed key * Encrypt using a master key stored on a local disk @@ -230,7 +230,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Encrypt the backup data](/br/br-snapshot-manual.md#encrypt-the-backup-data) and [Encrypt the log backup data](/br/br-pitr-manual.md#encrypt-the-log-backup-data). -* TiKV encryption at rest supports Google [Key Management Service (Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive?hl) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) **tw@qiancai** +* TiKV encryption at rest supports [Google Cloud Key Management Service (Google Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) **tw@qiancai** TiKV ensures data security by using the encryption at rest technique to encrypt stored data. The core aspect of this technique is proper key management. In v8.0.0, TiKV encryption at rest experimentally supports using Google Cloud KMS for master key management. From 5f5a9bcbdf984369f169a40922d03932c1b2011f Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Tue, 26 Nov 2024 17:57:39 +0800 Subject: [PATCH 36/89] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-8.5.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 688eecbf1c669..4db816150fefe 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -86,7 +86,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + @@ -184,7 +184,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, The foreign key feature becomes generally available (GA) in v8.5.0. Foreign key constraints help ensure data consistency and integrity. You can easily establish foreign key relationships between tables, with support for cascading updates and deletions, simplifying data management. This feature enhances support for applications with complex data relationships. - For more information, see [Documentation](/foreign-key.md). + For more information, see [documentation](/foreign-key.md). * Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd From 431686d77004a7a5eb08904803c09c3391447a2c Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 26 Nov 2024 18:59:08 +0800 Subject: [PATCH 37/89] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-8.5.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 4db816150fefe..4a18cb9c8e0a9 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -75,7 +75,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, @@ -125,7 +125,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [documentation](/schema-cache.md). -* Use the Active PD Follower feature to enhance the scalability of PD's Region information query service (GA) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) tw@Oreoxmt +* Provide the Active PD Follower feature to enhance the scalability of PD's Region information query service (GA) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) tw@Oreoxmt In a TiDB cluster with a large number of Regions, the PD leader might experience high CPU load due to the increased overhead of handling heartbeats and scheduling tasks. If the cluster has many TiDB instances, and there is a high concurrency of requests for Region information, the CPU pressure on the PD leader increases further and might cause PD services to become unavailable. From 94c0acc9085122e3ba7f1c9fe86fda654d0e5c27 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 27 Nov 2024 08:28:05 +0800 Subject: [PATCH 38/89] add improvements and bug fixes --- releases/release-8.5.0.md | 193 ++++++++++++++++++++++++-------------- 1 file changed, 122 insertions(+), 71 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 4a18cb9c8e0a9..cd08cd68c7c79 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -156,6 +156,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Starting from v8.5.0, TiKV introduces MVCC In-memory Engine. When scanning performance is degraded due to the buildup of MVCC versions in a TiKV cluster, you can enable the TiKV MVCC memory engine to improve scanning performance by setting the TiKV configuration parameter [`in-memory-engine.enable`](/tikv-in-memory-engine.md#usage). For more information, see [documentation](/tikv-in-memory-engine.md). + ### Reliability * Support limiting the maximum rate and concurrency of requests processed by PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) **tw@qiancai** @@ -294,101 +295,151 @@ The following features are planned for deprecation in future versions: ## Improvements -+ TiDB ++ TiDB - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - 提升关闭分布式框架的 fast reorg 加索引任务对任务取消的响应速度 [#56017](https://github.com/pingcap/tidb/issues/56017) @[lance6716](https://github.com/lance6716) + - 提升小表加索引的速度 [#54230](https://github.com/pingcap/tidb/issues/54230) @[tangenta](https://github.com/tangenta) + - 增加系统变量 tidb_ddl_reorg_max_write_speed 来限制加索引时 ingest 阶段速度上限 [#57156](https://github.com/pingcap/tidb/issues/57156) @[CbcWestwolf](https://github.com/CbcWestwolf) + - 提升某些情况下查询 information_schema.tables 的性能 [#57295](https://github.com/pingcap/tidb/issues/57295) @[tangenta](https://github.com/tangenta) + - 支持动态调整更多 DDL 任务参数 [#57526](https://github.com/pingcap/tidb/issues/57526) @[fzzf678](https://github.com/fzzf678) + - 允许全局索引包含分区表达式中的所有列 [#56230](https://github.com/pingcap/tidb/issues/56230) @[Defined2014](https://github.com/Defined2014) + - 对于 list 分区表，支持对非"点查"的场景(range 查询)进行分区裁剪 [#56673](https://github.com/pingcap/tidb/issues/56673) @[Defined2014](https://github.com/Defined2014) + - 默认开启 FixControl#46177，避免某些情况选择全表扫没有选择索引范围扫描的问题 [#46177](https://github.com/pingcap/tidb/issues/46177) @[terry1purcell](https://github.com/terry1purcell) + - 改进内部估算逻辑，使其能够更充分地利用多列多值索引的统计信息，提升某些涉及多值索引的查询的估算精度 [#56915](https://github.com/pingcap/tidb/issues/56915) @[time-and-fate](https://github.com/time-and-fate) + - 提高特定情况下全表扫描的代价估算，减少错误的选择全表扫描的概率 [#57085](https://github.com/pingcap/tidb/issues/57085) @[terry1purcell](https://github.com/terry1purcell) + - 优化统计信息同步加载所需的数据量，提升加载性能 [#56812](https://github.com/pingcap/tidb/issues/56812) @[winoros](https://github.com/winoros) + - 优化特定情况下，OUTER JOIN 含有唯一索引且有 ORDER BY LIMIT 语句时的执行计划，提高执行效率 [#56321](https://github.com/pingcap/tidb/issues/56321) @[winoros](https://github.com/winoros) -+ TiKV ++ TiKV - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - 利用单独的线程来进行副本清理工作，保证 Raft 读写关键路径的延迟稳定 [#16001](https://github.com/tikv/tikv/issues/16001) @[hbisheng](https://github.com/hbisheng) + - 提升向量距离函数性能，为向量距离函数添加 SIMD 支持 [#17290](https://github.com/tikv/tikv/issues/17290) @[EricZequan](https://github.com/EricZequan) -+ PD ++ PD - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - 支持 tso 微服务模式和非微服务模式间动态切换 [#8477](https://github.com/tikv/pd/issues/8477) @[rleungx](https://github.com/rleungx) + - 改进 pd-ctl config 的输出 [#8694](https://github.com/tikv/pd/issues/8694) @[lhy1024](https://github.com/lhy1024) -+ TiFlash ++ TiFlash - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - (dup): release-7.1.6.md > 改进提升> TiFlash - 提升聚簇索引表在后台回收过期数据的速度 [#9529](https://github.com/pingcap/tiflash/issues/9529) @[JaySon-Huang](https://github.com/JaySon-Huang) + - 提升带有更新数据场景下 Vector Search 的查询性能 [#9599](https://github.com/pingcap/tiflash/issues/9599) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - 添加关于构建 Vector Index CPU 使用率的监控指标 [#9032](https://github.com/pingcap/tiflash/issues/9032) @[JaySon-Huang](https://github.com/JaySon-Huang) + - 优化逻辑运算符的执行速度 [#9146](https://github.com/pingcap/tiflash/issues/9146) @[windtalker](https://github.com/windtalker) + Tools - + Backup & Restore (BR) - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - + TiCDC - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - + TiDB Data Migration (DM) + + Backup & Restore (BR) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - (dup): release-7.1.6.md > 改进提升> Tools> Backup & Restore (BR) - 减少备份过程中无效日志的打印 [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) + - 改善 crypter key 错误时的错误信息 [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) + - 增大了 BR 创建 DB 时的并发性，改善了恢复时的性能 [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) + - 关闭了全量备份时默认开启的 checksum 以提高生产坏境的性能，只在测试环境中继续开启。此 checksum 只是用来内部验证 BR 的内部逻辑，而对验证备份数据完整性的 checksum 依然开启。[#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) + - 将重制超时从 gRPC 层面细化到 store 层面 [#55526](https://github.com/pingcap/tidb/pull/55526) @[3pointer](https://github.com/3pointer) - + TiDB Lightning + + TiDB Data Migration (DM) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - + TiUP - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - 在 DM 集群启动时，增加 dm-worker 连接到 dm-master 的重试 [#4287](https://github.com/pingcap/tiflow/issues/4287) @[GMHDBJD](https://github.com/GMHDBJD) ## Bug fixes -+ TiDB - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - -+ TiKV - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - -+ PD - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - -+ TiFlash - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) ++ TiDB + + - 修复 PD 返回 region 信息不包含 leader 的情况，没有重试可能导致执行报错的问题 [#56757](https://github.com/pingcap/tidb/issues/56757) @[cfzjywxk](https://github.com/cfzjywxk) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复写冲突时 TTL 任务可能无法取消的问题 [#56422](https://github.com/pingcap/tidb/issues/56422) @[YangKeao](https://github.com/YangKeao) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复取消 TTL 任务时，没有强制 Kill 对应 SQL 的问题 [#56511](https://github.com/pingcap/tidb/issues/56511) @[lcwangchao](https://github.com/lcwangchao) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复集群从 v6.5 升级到 v7.5 或更高版本后，已有 TTL 任务执行意外频繁的问题 [#56539](https://github.com/pingcap/tidb/issues/56539) @[lcwangchao](https://github.com/lcwangchao) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复 `INSERT ... ON DUPLICATE KEY` 语句不兼容 `mysql_insert_id` 的问题 [#55965](https://github.com/pingcap/tidb/issues/55965) @[tiancaiamao](https://github.com/tiancaiamao) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复 TTL 在未选用 TiKV 作为存储引擎时可能失败的问题 [#56402](https://github.com/pingcap/tidb/issues/56402) @[YangKeao](https://github.com/YangKeao) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复通过 `IMPORT INTO` 导入数据后，`AUTO_INCREMENT` 字段没有正确设置的问题 [#56476](https://github.com/pingcap/tidb/issues/56476) @[D3Hunter](https://github.com/D3Hunter) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `ADD INDEX` 时，未检查索引长度限制的问题 [#56930](https://github.com/pingcap/tidb/issues/56930) @[fzzf678](https://github.com/fzzf678) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `RECOVER TABLE BY JOB JOB_ID;` 可能导致 panic 的问题 [#55113](https://github.com/pingcap/tidb/issues/55113) @[crazycs520](https://github.com/crazycs520) + - (dup): release-7.1.6.md > 错误修复> TiDB - 修复由于 stale read 未对读操作的时间戳进行严格校验，导致 TSO 和真实物理时间存在偏移，有小概率影响事务一致性的问题 [#56809](https://github.com/pingcap/tidb/issues/56809) @[MyonKeminta](https://github.com/MyonKeminta) + - 修复 DDL owner 节点切换时，reorg 类型的 DDL 任务无法从切换前的进度继续的问题 [#56506](https://github.com/pingcap/tidb/issues/56506) @[tangenta](https://github.com/tangenta) + - 修复分布式执行框架监控面板不准确的问题 [#57172](https://github.com/pingcap/tidb/issues/57172) @[fzzf678](https://github.com/fzzf678) [#56942](https://github.com/pingcap/tidb/issues/56942) @[fzzf678](https://github.com/fzzf678) + - 修复某些情况下 REORGANIZE PARTITION 无法展示报错原因的问题 [#56634](https://github.com/pingcap/tidb/issues/56634) @[mjonss](https://github.com/mjonss) + - 修复查询 information_schema.tables 大小写敏感的问题 [#56987](https://github.com/pingcap/tidb/issues/56987) @[joechenrh](https://github.com/joechenrh) + - 修复 `CTE` 有多个数据消费者，且某个消费者在没有读取任何数据情况下退出可能导致的非法内存访问问题 [#57294](https://github.com/pingcap/tidb/issues/57294) @[windtalker](https://github.com/windtalker) + - 修复 `IndexHashJoin` 在非正常退出情况下可能卡住问题 [#54055](https://github.com/pingcap/tidb/issues/54055) @[wshwsh12](https://github.com/wshwsh12) + - 修复 `TRUNCATE` 处理 `NULL` 值不正确的问题 [#53546](https://github.com/pingcap/tidb/issues/53546) @[tuziemon](https://github.com/tuziemon) + - 修复类型推导错误导致的 `CAST AS CHAR` 函数结果问题 [#56640](https://github.com/pingcap/tidb/issues/56640) @[zimulala](https://github.com/zimulala) + - 修复类型推导错误导致的字符串被截断问题 [#56587](https://github.com/pingcap/tidb/issues/56587) @[joechenrh](https://github.com/joechenrh) + - 修复 `ADDTIME` 和 `SUBTIME` 函数第一个参数是日期类型时结果错误问题 [#57569](https://github.com/pingcap/tidb/issues/57569) @[xzhangxian1008](https://github.com/xzhangxian1008) + - 修复在非严格模式下 (sql_mode = '')，非法空值被插入的问题 [#56381](https://github.com/pingcap/tidb/issues/56381) @[joechenrh](https://github.com/joechenrh) + - 修复 `UPDATE` 语句中 `ENUM` 类型的值更新错误的问题 [#56832](https://github.com/pingcap/tidb/issues/56832) @[xhebox](https://github.com/xhebox) + - 修复开启 `tidb_low_resolution_tso` 后 `SELECT FOR UPDATE` 语句有资源泄漏的问题 [#55468](https://github.com/pingcap/tidb/issues/55468) @[tiancaiamao](https://github.com/tiancaiamao) + - 修复 JSON 函数没有检查参数类型的问题 [#54029](https://github.com/pingcap/tidb/issues/54029) @[YangKeao](https://github.com/YangKeao) + - 修复当 `PREPARE` 命令中包含 JSON 函数时可能失败的问题 [#54044](https://github.com/pingcap/tidb/issues/54044) @[YangKeao](https://github.com/YangKeao) + - 修复当执行 `bool like cast(bit as char)` 时 TiKV 崩溃的问题 [#56494](https://github.com/pingcap/tidb/issues/56494) @[lcwangchao](https://github.com/lcwangchao) + - 修复 `CREATE VIEW` 语句中带有变量或参数时不报错的问题 [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) + - 修复 `JSON_VALID()` 函数执行结果错误的问题 [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) + - 修复关闭 `tidb_ttl_job_enable` 后 TTL 任务没有被取消的问题 [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) + - 修复同时使用 `RANGE COLUMNS` 分区函数和 `utf8mb4_0900_ai_ci` 排序规则时，查询结果错误的问题 [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) + - 修复了执行以换行符开头的 prepared statement 会导致数组越界的运行时错误 [#54283](https://github.com/pingcap/tidb/issues/54283) @[Defined2014](https://github.com/Defined2014) + - 修复了时间/日期函数中设置无效精度的一些问题，比如精度设置得太大 [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) + - 在 `UPDATE/INSERT/DELETE IGNORE` statement 中忽略掉一些外键错误 [#56678](https://github.com/pingcap/tidb/issues/56678) @[YangKeao](https://github.com/YangKeao) + - 修正了查询 slow_query 表时，如果不加时间过滤条件则只会查询最新的慢日志文件的行为 [#56100](https://github.com/pingcap/tidb/issues/56100) @[crazycs520](https://github.com/crazycs520) + - 修复了 TTL 表的内存泄漏问题 [#56934](https://github.com/pingcap/tidb/issues/56934) @[lcwangchao](https://github.com/lcwangchao) + - 避免使用 non-public 状态的表，该改动可以修复 write_only 状态的表外键约束未生效的一处问题 [#55813](https://github.com/pingcap/tidb/issues/55813) @[YangKeao](https://github.com/YangKeao) + - 修复使用 NATURAL JOIN 或者 USING CLAUSE 之后再使用子查询可能会报错的问题 [#53766](https://github.com/pingcap/tidb/issues/53766) @[dash12653](https://github.com/dash12653) + - 修复如果 CTE 包含 orderby/limit/distinct 子句并且被另外一个 CTE 的 recursive part 所引用时，可能被错误的 INLINE 导致执行报错的问题 [#56603](https://github.com/pingcap/tidb/issues/56603) @[elsa0520](https://github.com/elsa0520) + - 修复 VIEW 中定义的 CTE 被错误 INLINE 的问题 [#56582](https://github.com/pingcap/tidb/issues/56582) @[elsa0520](https://github.com/elsa0520) + - 修复 Plan Replayer 导入含有外键的表结构时可能报错的问题 [#56456](https://github.com/pingcap/tidb/issues/56456) @[hawkingrei](https://github.com/hawkingrei) + - 修复 Plan Replayer 导入含有 Placement Rule 的表结构时可能报错的问题 [#54961](https://github.com/pingcap/tidb/issues/54961) @[hawkingrei](https://github.com/hawkingrei) + - 修复使用 ANALYZE 收集含有引用了虚拟生成列的表达式索引的表的统计信息时执行会报错的问题 [#57079](https://github.com/pingcap/tidb/issues/57079) @[hawkingrei](https://github.com/hawkingrei) + - 修复 DROP DATABASE 语句没有正确触发统计信息对应变更的问题 [#57227](https://github.com/pingcap/tidb/issues/57227) @[Rustin170506](https://github.com/Rustin170506) + - 修复在 CTE 中解析数据库名时，得到错误的数据库名的问题 [#54582](https://github.com/pingcap/tidb/issues/54582) @[hawkingrei](https://github.com/hawkingrei) + - 修复在 DUMP STATS 生成 json 的过程中导致直方图上下界数据受损的问题 [#56083](https://github.com/pingcap/tidb/issues/56083) @[hawkingrei](https://github.com/hawkingrei) + - 修复 EXISTS 子查询的结果继续参与代数运算时，结果和 MySQL 不一致的问题 [#56641](https://github.com/pingcap/tidb/issues/56641) @[windtalker](https://github.com/windtalker) + - 修复无法为带别名的多表删除创建计划绑定的问题 [#56726](https://github.com/pingcap/tidb/issues/56726) @[hawkingrei](https://github.com/hawkingrei) + - 修复优化器在简化复杂谓词时没有考虑字符集及排序规则导致执行可能报错的问题 [#56479](https://github.com/pingcap/tidb/issues/56479) @[dash12653](https://github.com/dash12653) + - 修复 Grafana 中 STATS_HEALTHY 图标可能统计出错的问题 [#57176](https://github.com/pingcap/tidb/issues/57176) @[hawkingrei](https://github.com/hawkingrei) + ++ TiKV + + - (dup): release-7.1.6.md > 错误修复> TiKV - 修复读线程在从 Raft Engine 中的 MemTable 读取过时索引时出现的 panic 问题 [#17383](https://github.com/tikv/tikv/issues/17383) @[LykxSassinator](https://github.com/LykxSassinator) + - (dup): release-7.5.4.md > 错误修复> TiKV - 修复当大量事务在排队等待同一个 key 上的锁被释放且该 key 被频繁更新时，TiKV 可能因死锁检测压力过大而出现 OOM 的问题 [#17394](https://github.com/tikv/tikv/issues/17394) @[MyonKeminta](https://github.com/MyonKeminta) + - 修复资源管控后台任务 CPU 可能被重复统计的错误 [#17603](https://github.com/tikv/tikv/issues/17603) @[glorv](https://github.com/glorv) + - 修复 CDC 过多内部未完成任务堆积导致 TiKV OOM 的问题 [#17696](https://github.com/tikv/tikv/issues/17696) @[3AceShowHand](https://github.com/3AceShowHand) + - 修复设置 raft entry max size 过大时写入 batch 可能过大引起抖动的问题 [#17701](https://github.com/tikv/tikv/issues/17701) @[SpadeA-Tang](https://github.com/SpadeA-Tang) + - 修复 Split 后可能无法快速选出 Leader 的问题 [#17602](https://github.com/tikv/tikv/issues/17602) @[LykxSassinator](https://github.com/LykxSassinator) + - 修复 radians 和 degree 函数的 panic 问题 [#17852](https://github.com/tikv/tikv/issues/17852) @[gengliqi](https://github.com/gengliqi) + - 修复 hibernated Region 集中唤醒可能导致写入抖动的问题 [#17101](https://github.com/tikv/tikv/issues/17101) @[hhwyt](https://github.com/hhwyt) + ++ PD + + - (dup): release-7.1.6.md > 错误修复> PD - 修复热点缓存中可能存在的内存泄露问题 [#8698](https://github.com/tikv/pd/issues/8698) @[lhy1024](https://github.com/lhy1024) + - 使得 resource group 选择器对所有面板生效 [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) + - 修复 syncer 加载过程中日志不清晰的问题 [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) + - 修复已删除的 resource group 仍然出现在监控面板中的问题 [#8716](https://github.com/tikv/pd/issues/8716) @[AndreMouche](https://github.com/AndreMouche) + ++ TiFlash + + - (dup): release-7.1.6.md > 错误修复> TiFlash - 修复 `SUBSTRING()` 函数不支持部分整数类型的 `pos` 和 `len` 参数导致查询报错的问题 [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) + - 修复在存算分离架构下，Vector Search 查询性能在 TiFlash write node 扩容之后，可能出现性能下降的问题 [#9637](https://github.com/pingcap/tiflash/issues/9637) @[kolafish](https://github.com/kolafish) + - 修复当 `SUBSTRING` 函数的第二个参数为负数时，可能产生错误结果的问题 [#9604](https://github.com/pingcap/tiflash/issues/9604) @[guo-shaoge](https://github.com/guo-shaoge) + - 修复当 `REPLACE` 函数的第一个参数为常数时，可能报错的问题 [#9522](https://github.com/pingcap/tiflash/issues/9522) @[guo-shaoge](https://github.com/guo-shaoge) + - 修复当 `LPAD` 和 `RPAD` 函数在某些情况下返回错误结果的问题 [#9465](https://github.com/pingcap/tiflash/issues/9465) @[guo-shaoge](https://github.com/guo-shaoge) + Tools - + Backup & Restore (BR) - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - + TiCDC - - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - + TiDB Data Migration (DM) + + Backup & Restore (BR) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - 减少了预先分配的内存从而避免当 incomplete range 过多时的 OOM [#53529](https://github.com/pingcap/tidb/issues/53529) @[Leavrth](https://github.com/Leavrth) + - 修复了备份时无法备份 global index 的错误 [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014)] + - 修复了日志可能打印出加密信息的错误 [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) + - 修复了 advancer 无法处理锁冲突的情形 [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) - + TiDB Lightning + + TiCDC - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - 修复 Debezium 协议下 Kafka 消息中未填写 Key 的问题。 [#1799](https://github.com/pingcap/tiflow/issues/1799) @[wk989898](https://github.com/wk989898) + - 修复 Redo 模块无法正确上报错误的问题。 [#11744](https://github.com/pingcap/tiflow/issues/11744) @[CharlesCheung96](https://github.com/CharlesCheung96) + - 修复 TiDB Owner 变更导致 DDL Jobs 的 schema version 出现非递增场景下 TiCDC 错误丢弃 DDL 任务的问题。 [#11714](https://github.com/pingcap/tiflow/issues/11714) @[wlwilliamx](https://github.com/wlwilliamx) - + TiUP + + TiDB Lightning - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) - - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) + - (dup): release-7.1.6.md > 错误修复> Tools> TiDB Lightning - 修复 TiDB Lightning 因 TiKV 发送的消息过大而接收失败的问题 [#56114](https://github.com/pingcap/tidb/issues/56114) @[fishiu](https://github.com/fishiu) + - 修复物理导入后 AUTO INCREMENT 值设置过大的问题 [#56814](https://github.com/pingcap/tidb/issues/56814) @[D3Hunter](https://github.com/D3Hunter) ## Contributors From c70d893c7d26f6d7ce2484463e295dd2a5936a1e Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 27 Nov 2024 11:30:32 +0800 Subject: [PATCH 39/89] Apply suggestions from code review --- releases/release-8.5.0.md | 40 +++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index cd08cd68c7c79..fa0d0712a3d6c 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -297,23 +297,23 @@ The following features are planned for deprecation in future versions: + TiDB - - 提升关闭分布式框架的 fast reorg 加索引任务对任务取消的响应速度 [#56017](https://github.com/pingcap/tidb/issues/56017) @[lance6716](https://github.com/lance6716) - - 提升小表加索引的速度 [#54230](https://github.com/pingcap/tidb/issues/54230) @[tangenta](https://github.com/tangenta) - - 增加系统变量 tidb_ddl_reorg_max_write_speed 来限制加索引时 ingest 阶段速度上限 [#57156](https://github.com/pingcap/tidb/issues/57156) @[CbcWestwolf](https://github.com/CbcWestwolf) - - 提升某些情况下查询 information_schema.tables 的性能 [#57295](https://github.com/pingcap/tidb/issues/57295) @[tangenta](https://github.com/tangenta) - - 支持动态调整更多 DDL 任务参数 [#57526](https://github.com/pingcap/tidb/issues/57526) @[fzzf678](https://github.com/fzzf678) - - 允许全局索引包含分区表达式中的所有列 [#56230](https://github.com/pingcap/tidb/issues/56230) @[Defined2014](https://github.com/Defined2014) - - 对于 list 分区表，支持对非"点查"的场景(range 查询)进行分区裁剪 [#56673](https://github.com/pingcap/tidb/issues/56673) @[Defined2014](https://github.com/Defined2014) + - Improve the response speed of `ADD INDEX` acceleration to task cancelation when the Distributed eXecution Framework (DXF) is disabled [#56017](https://github.com/pingcap/tidb/issues/56017) @[lance6716](https://github.com/lance6716) + - Improve the speed of adding indexes to small tables [#54230](https://github.com/pingcap/tidb/issues/54230) @[tangenta](https://github.com/tangenta) + - Add the system variable `tidb_ddl_reorg_max_write_speed` to limit the maximum speed of the ingest phase when adding indexes [#57156](https://github.com/pingcap/tidb/issues/57156) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Improve the performance of querying `information_schema.tables` in some cases [#57295](https://github.com/pingcap/tidb/issues/57295) @[tangenta](https://github.com/tangenta) + - Support dynamically adjusting more DDL task parameters [#57526](https://github.com/pingcap/tidb/issues/57526) @[fzzf678](https://github.com/fzzf678) + - Support global indexes containing all columns in a partitioned expression [#56230](https://github.com/pingcap/tidb/issues/56230) @[Defined2014](https://github.com/Defined2014) + - Support partition pruning for list partitioned tables in range query scenarios [#56673](https://github.com/pingcap/tidb/issues/56673) @[Defined2014](https://github.com/Defined2014) - 默认开启 FixControl#46177，避免某些情况选择全表扫没有选择索引范围扫描的问题 [#46177](https://github.com/pingcap/tidb/issues/46177) @[terry1purcell](https://github.com/terry1purcell) - - 改进内部估算逻辑，使其能够更充分地利用多列多值索引的统计信息，提升某些涉及多值索引的查询的估算精度 [#56915](https://github.com/pingcap/tidb/issues/56915) @[time-and-fate](https://github.com/time-and-fate) - - 提高特定情况下全表扫描的代价估算，减少错误的选择全表扫描的概率 [#57085](https://github.com/pingcap/tidb/issues/57085) @[terry1purcell](https://github.com/terry1purcell) - - 优化统计信息同步加载所需的数据量，提升加载性能 [#56812](https://github.com/pingcap/tidb/issues/56812) @[winoros](https://github.com/winoros) - - 优化特定情况下，OUTER JOIN 含有唯一索引且有 ORDER BY LIMIT 语句时的执行计划，提高执行效率 [#56321](https://github.com/pingcap/tidb/issues/56321) @[winoros](https://github.com/winoros) + - Improve the internal estimation logic so that it can better use the statistics of multi-column and multi-value indexes to improve the estimation accuracy of certain queries involving multi-value indexes [#56915](https://github.com/pingcap/tidb/issues/56915) @[time-and-fate](https://github.com/time-and-fate) + - Improve the cost estimation of full table scan in specific cases and reduce the probability of incorrectly choosing the full table scan [#57085](https://github.com/pingcap/tidb/issues/57085) @[terry1purcell](https://github.com/terry1purcell) + - Optimize the amount of data required for synchronous loading of statistics to improve loading performance [#56812](https://github.com/pingcap/tidb/issues/56812) @[winoros](https://github.com/winoros) + - Optimize the execution plan when `OUTER JOIN` contains a unique index and there is an `ORDER BY LIMIT` statement in certain cases to improve execution efficiency [#56321](https://github.com/pingcap/tidb/issues/56321) @[winoros](https://github.com/winoros) + TiKV - - 利用单独的线程来进行副本清理工作，保证 Raft 读写关键路径的延迟稳定 [#16001](https://github.com/tikv/tikv/issues/16001) @[hbisheng](https://github.com/hbisheng) - - 提升向量距离函数性能，为向量距离函数添加 SIMD 支持 [#17290](https://github.com/tikv/tikv/issues/17290) @[EricZequan](https://github.com/EricZequan) + - Use a separate thread for replica cleanup to ensure stable latency for critical paths of Raft reads and writes [#16001](https://github.com/tikv/tikv/issues/16001) @[hbisheng](https://github.com/hbisheng) + - Improve the performance of vector distance function by supporting SIMD [#17290](https://github.com/tikv/tikv/issues/17290) @[EricZequan](https://github.com/EricZequan) + PD @@ -409,9 +409,9 @@ The following features are planned for deprecation in future versions: + PD - (dup): release-7.1.6.md > 错误修复> PD - 修复热点缓存中可能存在的内存泄露问题 [#8698](https://github.com/tikv/pd/issues/8698) @[lhy1024](https://github.com/lhy1024) - - 使得 resource group 选择器对所有面板生效 [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) - - 修复 syncer 加载过程中日志不清晰的问题 [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) - - 修复已删除的 resource group 仍然出现在监控面板中的问题 [#8716](https://github.com/tikv/pd/issues/8716) @[AndreMouche](https://github.com/AndreMouche) + - Fix the issue that the resource group selector does not take effect for all panels [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) + - Fix the issue that deleted resource groups still appear in the monitoring panel [#8716](https://github.com/tikv/pd/issues/8716) @[AndreMouche](https://github.com/AndreMouche) + - Fix the issue that log description is not clear when loading syncer [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) + TiFlash @@ -425,10 +425,10 @@ The following features are planned for deprecation in future versions: + Backup & Restore (BR) - - 减少了预先分配的内存从而避免当 incomplete range 过多时的 OOM [#53529](https://github.com/pingcap/tidb/issues/53529) @[Leavrth](https://github.com/Leavrth) - - 修复了备份时无法备份 global index 的错误 [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014)] - - 修复了日志可能打印出加密信息的错误 [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) - - 修复了 advancer 无法处理锁冲突的情形 [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) + - Fix the OOM issue when there are too many incomplete ranges that have not been backed up yet during backup [#53529](https://github.com/pingcap/tidb/issues/53529) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014)] + - Fix the issue that logs might print out encrypted information [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) + - Fix the issue that Advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) + TiCDC From 966f7a15396c5ef0cd5faf3edc90ebea3a9a1c6d Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 27 Nov 2024 12:30:12 +0800 Subject: [PATCH 40/89] improvement and bug fixes: update TiFlash, TiKV, TiCDC, and TiDB Lightning Signed-off-by: Aolin --- releases/release-8.5.0.md | 42 +++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index fa0d0712a3d6c..d151c502d07d7 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -323,19 +323,19 @@ The following features are planned for deprecation in future versions: + TiFlash - (dup): release-7.1.6.md > 改进提升> TiFlash - 提升聚簇索引表在后台回收过期数据的速度 [#9529](https://github.com/pingcap/tiflash/issues/9529) @[JaySon-Huang](https://github.com/JaySon-Huang) - - 提升带有更新数据场景下 Vector Search 的查询性能 [#9599](https://github.com/pingcap/tiflash/issues/9599) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - - 添加关于构建 Vector Index CPU 使用率的监控指标 [#9032](https://github.com/pingcap/tiflash/issues/9032) @[JaySon-Huang](https://github.com/JaySon-Huang) - - 优化逻辑运算符的执行速度 [#9146](https://github.com/pingcap/tiflash/issues/9146) @[windtalker](https://github.com/windtalker) + - Improve query performance of vector search in data update scenarios [#9599](https://github.com/pingcap/tiflash/issues/9599) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - Add monitoring metrics for CPU usage during vector index building [#9032](https://github.com/pingcap/tiflash/issues/9032) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Improve the execution efficiency of logical operators [#9146](https://github.com/pingcap/tiflash/issues/9146) @[windtalker](https://github.com/windtalker) + Tools + Backup & Restore (BR) - (dup): release-7.1.6.md > 改进提升> Tools> Backup & Restore (BR) - 减少备份过程中无效日志的打印 [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) - - 改善 crypter key 错误时的错误信息 [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) - - 增大了 BR 创建 DB 时的并发性，改善了恢复时的性能 [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) - - 关闭了全量备份时默认开启的 checksum 以提高生产坏境的性能，只在测试环境中继续开启。此 checksum 只是用来内部验证 BR 的内部逻辑，而对验证备份数据完整性的 checksum 依然开启。[#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) - - 将重制超时从 gRPC 层面细化到 store 层面 [#55526](https://github.com/pingcap/tidb/pull/55526) @[3pointer](https://github.com/3pointer) + - Optimize error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) + - Increase concurrency during BR database creation to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) + - Disable checksum by default during full backups to improve production environment performance while keeping it enabled in testing environments. This checksum is used only for the internal verification logic of BR, while the checksum that verifies backup data integrity remains enabled [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) + - Fix the issue that backup might hang when processing a large number of tables [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) + TiDB Data Migration (DM) @@ -399,12 +399,12 @@ The following features are planned for deprecation in future versions: - (dup): release-7.1.6.md > 错误修复> TiKV - 修复读线程在从 Raft Engine 中的 MemTable 读取过时索引时出现的 panic 问题 [#17383](https://github.com/tikv/tikv/issues/17383) @[LykxSassinator](https://github.com/LykxSassinator) - (dup): release-7.5.4.md > 错误修复> TiKV - 修复当大量事务在排队等待同一个 key 上的锁被释放且该 key 被频繁更新时，TiKV 可能因死锁检测压力过大而出现 OOM 的问题 [#17394](https://github.com/tikv/tikv/issues/17394) @[MyonKeminta](https://github.com/MyonKeminta) - - 修复资源管控后台任务 CPU 可能被重复统计的错误 [#17603](https://github.com/tikv/tikv/issues/17603) @[glorv](https://github.com/glorv) - - 修复 CDC 过多内部未完成任务堆积导致 TiKV OOM 的问题 [#17696](https://github.com/tikv/tikv/issues/17696) @[3AceShowHand](https://github.com/3AceShowHand) - - 修复设置 raft entry max size 过大时写入 batch 可能过大引起抖动的问题 [#17701](https://github.com/tikv/tikv/issues/17701) @[SpadeA-Tang](https://github.com/SpadeA-Tang) - - 修复 Split 后可能无法快速选出 Leader 的问题 [#17602](https://github.com/tikv/tikv/issues/17602) @[LykxSassinator](https://github.com/LykxSassinator) - - 修复 radians 和 degree 函数的 panic 问题 [#17852](https://github.com/tikv/tikv/issues/17852) @[gengliqi](https://github.com/gengliqi) - - 修复 hibernated Region 集中唤醒可能导致写入抖动的问题 [#17101](https://github.com/tikv/tikv/issues/17101) @[hhwyt](https://github.com/hhwyt) + - Fix the issue that CPU usage for background tasks of resource control is counted multiple times [#17603](https://github.com/tikv/tikv/issues/17603) @[glorv](https://github.com/glorv) + - Fix the issue that TiKV OOM might occur due to the accumulation of CDC internal tasks [#17696](https://github.com/tikv/tikv/issues/17696) @[3AceShowHand](https://github.com/3AceShowHand) + - Fix the issue that large batch writes cause jitter when `raft-entry-max-size` is set too high [#17701](https://github.com/tikv/tikv/issues/17701) @[SpadeA-Tang](https://github.com/SpadeA-Tang) + - Fix the issue that the leader could not be quickly elected after Region split [#17602](https://github.com/tikv/tikv/issues/17602) @[LykxSassinator](https://github.com/LykxSassinator) + - Fix the issue that TiKV might panic when using `RADIANS()` or `DEGREES()` [#17852](https://github.com/tikv/tikv/issues/17852) @[gengliqi](https://github.com/gengliqi) + - Fix the issue that write jitter might occur when all hibernated Regions are awakened [#17101](https://github.com/tikv/tikv/issues/17101) @[hhwyt](https://github.com/hhwyt) + PD @@ -416,10 +416,10 @@ The following features are planned for deprecation in future versions: + TiFlash - (dup): release-7.1.6.md > 错误修复> TiFlash - 修复 `SUBSTRING()` 函数不支持部分整数类型的 `pos` 和 `len` 参数导致查询报错的问题 [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) - - 修复在存算分离架构下，Vector Search 查询性能在 TiFlash write node 扩容之后，可能出现性能下降的问题 [#9637](https://github.com/pingcap/tiflash/issues/9637) @[kolafish](https://github.com/kolafish) - - 修复当 `SUBSTRING` 函数的第二个参数为负数时，可能产生错误结果的问题 [#9604](https://github.com/pingcap/tiflash/issues/9604) @[guo-shaoge](https://github.com/guo-shaoge) - - 修复当 `REPLACE` 函数的第一个参数为常数时，可能报错的问题 [#9522](https://github.com/pingcap/tiflash/issues/9522) @[guo-shaoge](https://github.com/guo-shaoge) - - 修复当 `LPAD` 和 `RPAD` 函数在某些情况下返回错误结果的问题 [#9465](https://github.com/pingcap/tiflash/issues/9465) @[guo-shaoge](https://github.com/guo-shaoge) + - Fix the issue that vector search performance might degrade after scaling out TiFlash write nodes in the disaggregated storage and compute architecture [#9637](https://github.com/pingcap/tiflash/issues/9637) @[kolafish](https://github.com/kolafish) + - Fix the issue that the `SUBSTRING()` function returns incorrect results when the second parameter is negative [#9604](https://github.com/pingcap/tiflash/issues/9604) @[guo-shaoge](https://github.com/guo-shaoge) + - Fix the issue that the `REPLACE()` function returns an error when the first parameter is a constant [#9522](https://github.com/pingcap/tiflash/issues/9522) @[guo-shaoge](https://github.com/guo-shaoge) + - Fix the issue that `LPAD()` and `RPAD()` functions return incorrect results in some cases [#9465](https://github.com/pingcap/tiflash/issues/9465) @[guo-shaoge](https://github.com/guo-shaoge) + Tools @@ -432,14 +432,14 @@ The following features are planned for deprecation in future versions: + TiCDC - - 修复 Debezium 协议下 Kafka 消息中未填写 Key 的问题。 [#1799](https://github.com/pingcap/tiflow/issues/1799) @[wk989898](https://github.com/wk989898) - - 修复 Redo 模块无法正确上报错误的问题。 [#11744](https://github.com/pingcap/tiflow/issues/11744) @[CharlesCheung96](https://github.com/CharlesCheung96) - - 修复 TiDB Owner 变更导致 DDL Jobs 的 schema version 出现非递增场景下 TiCDC 错误丢弃 DDL 任务的问题。 [#11714](https://github.com/pingcap/tiflow/issues/11714) @[wlwilliamx](https://github.com/wlwilliamx) + - Fix the issue that the Kafka messages lack Key fields when using the Debezium protocol [#1799](https://github.com/pingcap/tiflow/issues/1799) @[wk989898](https://github.com/wk989898) + - Fix the issue that the redo module fails to properly report errors [#11744](https://github.com/pingcap/tiflow/issues/11744) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Fix the issue that TiCDC mistakenly discards DDL tasks when the schema version of DDL tasks become non-incremental during TiDB Owner changes [#11714](https://github.com/pingcap/tiflow/issues/11714) @[wlwilliamx](https://github.com/wlwilliamx) + TiDB Lightning - (dup): release-7.1.6.md > 错误修复> Tools> TiDB Lightning - 修复 TiDB Lightning 因 TiKV 发送的消息过大而接收失败的问题 [#56114](https://github.com/pingcap/tidb/issues/56114) @[fishiu](https://github.com/fishiu) - - 修复物理导入后 AUTO INCREMENT 值设置过大的问题 [#56814](https://github.com/pingcap/tidb/issues/56814) @[D3Hunter](https://github.com/D3Hunter) + - Fix the issue that the `AUTO_INCREMENT` value is set too high after importing data using the physical import mode [#56814](https://github.com/pingcap/tidb/issues/56814) @[D3Hunter](https://github.com/D3Hunter) ## Contributors From 71741324ed9cae678c27112c406135956ead0f7c Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 27 Nov 2024 22:12:31 +0800 Subject: [PATCH 41/89] Update releases/release-8.5.0.md --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index d151c502d07d7..f1a1f13922664 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -304,7 +304,7 @@ The following features are planned for deprecation in future versions: - Support dynamically adjusting more DDL task parameters [#57526](https://github.com/pingcap/tidb/issues/57526) @[fzzf678](https://github.com/fzzf678) - Support global indexes containing all columns in a partitioned expression [#56230](https://github.com/pingcap/tidb/issues/56230) @[Defined2014](https://github.com/Defined2014) - Support partition pruning for list partitioned tables in range query scenarios [#56673](https://github.com/pingcap/tidb/issues/56673) @[Defined2014](https://github.com/Defined2014) - - 默认开启 FixControl#46177，避免某些情况选择全表扫没有选择索引范围扫描的问题 [#46177](https://github.com/pingcap/tidb/issues/46177) @[terry1purcell](https://github.com/terry1purcell) + - Enable FixControl#46177 by default to fix the issue that table full scan is incorrectly selected (instead of the index range scan) in some cases [#46177](https://github.com/pingcap/tidb/issues/46177) @[terry1purcell](https://github.com/terry1purcell) - Improve the internal estimation logic so that it can better use the statistics of multi-column and multi-value indexes to improve the estimation accuracy of certain queries involving multi-value indexes [#56915](https://github.com/pingcap/tidb/issues/56915) @[time-and-fate](https://github.com/time-and-fate) - Improve the cost estimation of full table scan in specific cases and reduce the probability of incorrectly choosing the full table scan [#57085](https://github.com/pingcap/tidb/issues/57085) @[terry1purcell](https://github.com/terry1purcell) - Optimize the amount of data required for synchronous loading of statistics to improve loading performance [#56812](https://github.com/pingcap/tidb/issues/56812) @[winoros](https://github.com/winoros) From 9609506222f6981415fd39b94ce6c232604b8c46 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 28 Nov 2024 08:20:53 +0800 Subject: [PATCH 42/89] Apply suggestions from code review --- releases/release-8.5.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index f1a1f13922664..9e47d9b77dd73 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -299,7 +299,7 @@ The following features are planned for deprecation in future versions: - Improve the response speed of `ADD INDEX` acceleration to task cancelation when the Distributed eXecution Framework (DXF) is disabled [#56017](https://github.com/pingcap/tidb/issues/56017) @[lance6716](https://github.com/lance6716) - Improve the speed of adding indexes to small tables [#54230](https://github.com/pingcap/tidb/issues/54230) @[tangenta](https://github.com/tangenta) - - Add the system variable `tidb_ddl_reorg_max_write_speed` to limit the maximum speed of the ingest phase when adding indexes [#57156](https://github.com/pingcap/tidb/issues/57156) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Add a new system variable `tidb_ddl_reorg_max_write_speed` to limit the maximum speed of the ingest phase when adding indexes [#57156](https://github.com/pingcap/tidb/issues/57156) @[CbcWestwolf](https://github.com/CbcWestwolf) - Improve the performance of querying `information_schema.tables` in some cases [#57295](https://github.com/pingcap/tidb/issues/57295) @[tangenta](https://github.com/tangenta) - Support dynamically adjusting more DDL task parameters [#57526](https://github.com/pingcap/tidb/issues/57526) @[fzzf678](https://github.com/fzzf678) - Support global indexes containing all columns in a partitioned expression [#56230](https://github.com/pingcap/tidb/issues/56230) @[Defined2014](https://github.com/Defined2014) @@ -313,7 +313,7 @@ The following features are planned for deprecation in future versions: + TiKV - Use a separate thread for replica cleanup to ensure stable latency for critical paths of Raft reads and writes [#16001](https://github.com/tikv/tikv/issues/16001) @[hbisheng](https://github.com/hbisheng) - - Improve the performance of vector distance function by supporting SIMD [#17290](https://github.com/tikv/tikv/issues/17290) @[EricZequan](https://github.com/EricZequan) + - Improve the performance of the vector distance function by supporting SIMD [#17290](https://github.com/tikv/tikv/issues/17290) @[EricZequan](https://github.com/EricZequan) + PD From 21cf6bec78ba0be54cef61826370560efbb044ef Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 28 Nov 2024 10:46:50 +0800 Subject: [PATCH 43/89] Apply suggestions from code review --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 9e47d9b77dd73..11a68fb873c61 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -335,7 +335,7 @@ The following features are planned for deprecation in future versions: - Optimize error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) - Increase concurrency during BR database creation to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) - Disable checksum by default during full backups to improve production environment performance while keeping it enabled in testing environments. This checksum is used only for the internal verification logic of BR, while the checksum that verifies backup data integrity remains enabled [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) - - Fix the issue that backup might hang when processing a large number of tables [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) + - Add independent connection timeout tracking and reset mechanisms for each storage node, enhancing the handling of slow nodes and preventing backup operation from hanging [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) + TiDB Data Migration (DM) From f4c2d815a9d1d65d0ad0f7cd1fbab974eb06f3b2 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 28 Nov 2024 11:42:36 +0800 Subject: [PATCH 44/89] Apply suggestions from code review --- releases/release-8.5.0.md | 42 +++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 11a68fb873c61..484047cefdb23 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -339,7 +339,7 @@ The following features are planned for deprecation in future versions: + TiDB Data Migration (DM) - - 在 DM 集群启动时，增加 dm-worker 连接到 dm-master 的重试 [#4287](https://github.com/pingcap/tiflow/issues/4287) @[GMHDBJD](https://github.com/GMHDBJD) + - Add retries for DM-worker to connect to DM-master when the DM cluster starts up [#4287](https://github.com/pingcap/tiflow/issues/4287) @[GMHDBJD](https://github.com/GMHDBJD) ## Bug fixes @@ -374,26 +374,26 @@ The following features are planned for deprecation in future versions: - 修复 `CREATE VIEW` 语句中带有变量或参数时不报错的问题 [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) - 修复 `JSON_VALID()` 函数执行结果错误的问题 [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) - 修复关闭 `tidb_ttl_job_enable` 后 TTL 任务没有被取消的问题 [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) - - 修复同时使用 `RANGE COLUMNS` 分区函数和 `utf8mb4_0900_ai_ci` 排序规则时，查询结果错误的问题 [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) - - 修复了执行以换行符开头的 prepared statement 会导致数组越界的运行时错误 [#54283](https://github.com/pingcap/tidb/issues/54283) @[Defined2014](https://github.com/Defined2014) - - 修复了时间/日期函数中设置无效精度的一些问题，比如精度设置得太大 [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) - - 在 `UPDATE/INSERT/DELETE IGNORE` statement 中忽略掉一些外键错误 [#56678](https://github.com/pingcap/tidb/issues/56678) @[YangKeao](https://github.com/YangKeao) - - 修正了查询 slow_query 表时，如果不加时间过滤条件则只会查询最新的慢日志文件的行为 [#56100](https://github.com/pingcap/tidb/issues/56100) @[crazycs520](https://github.com/crazycs520) - - 修复了 TTL 表的内存泄漏问题 [#56934](https://github.com/pingcap/tidb/issues/56934) @[lcwangchao](https://github.com/lcwangchao) - - 避免使用 non-public 状态的表，该改动可以修复 write_only 状态的表外键约束未生效的一处问题 [#55813](https://github.com/pingcap/tidb/issues/55813) @[YangKeao](https://github.com/YangKeao) - - 修复使用 NATURAL JOIN 或者 USING CLAUSE 之后再使用子查询可能会报错的问题 [#53766](https://github.com/pingcap/tidb/issues/53766) @[dash12653](https://github.com/dash12653) - - 修复如果 CTE 包含 orderby/limit/distinct 子句并且被另外一个 CTE 的 recursive part 所引用时，可能被错误的 INLINE 导致执行报错的问题 [#56603](https://github.com/pingcap/tidb/issues/56603) @[elsa0520](https://github.com/elsa0520) - - 修复 VIEW 中定义的 CTE 被错误 INLINE 的问题 [#56582](https://github.com/pingcap/tidb/issues/56582) @[elsa0520](https://github.com/elsa0520) - - 修复 Plan Replayer 导入含有外键的表结构时可能报错的问题 [#56456](https://github.com/pingcap/tidb/issues/56456) @[hawkingrei](https://github.com/hawkingrei) - - 修复 Plan Replayer 导入含有 Placement Rule 的表结构时可能报错的问题 [#54961](https://github.com/pingcap/tidb/issues/54961) @[hawkingrei](https://github.com/hawkingrei) - - 修复使用 ANALYZE 收集含有引用了虚拟生成列的表达式索引的表的统计信息时执行会报错的问题 [#57079](https://github.com/pingcap/tidb/issues/57079) @[hawkingrei](https://github.com/hawkingrei) - - 修复 DROP DATABASE 语句没有正确触发统计信息对应变更的问题 [#57227](https://github.com/pingcap/tidb/issues/57227) @[Rustin170506](https://github.com/Rustin170506) - - 修复在 CTE 中解析数据库名时，得到错误的数据库名的问题 [#54582](https://github.com/pingcap/tidb/issues/54582) @[hawkingrei](https://github.com/hawkingrei) - - 修复在 DUMP STATS 生成 json 的过程中导致直方图上下界数据受损的问题 [#56083](https://github.com/pingcap/tidb/issues/56083) @[hawkingrei](https://github.com/hawkingrei) - - 修复 EXISTS 子查询的结果继续参与代数运算时，结果和 MySQL 不一致的问题 [#56641](https://github.com/pingcap/tidb/issues/56641) @[windtalker](https://github.com/windtalker) - - 修复无法为带别名的多表删除创建计划绑定的问题 [#56726](https://github.com/pingcap/tidb/issues/56726) @[hawkingrei](https://github.com/hawkingrei) - - 修复优化器在简化复杂谓词时没有考虑字符集及排序规则导致执行可能报错的问题 [#56479](https://github.com/pingcap/tidb/issues/56479) @[dash12653](https://github.com/dash12653) - - 修复 Grafana 中 STATS_HEALTHY 图标可能统计出错的问题 [#57176](https://github.com/pingcap/tidb/issues/57176) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the query result is wrong when using the `RANGE COLUMNS` partition function and the `utf8mb4_0900_ai_ci` collation at the same time [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) + - Fix the runtime error caused by executing a prepared statement that begins with a newline character, resulting in an array out of bounds [#54283](https://github.com/pingcap/tidb/issues/54283) @[Defined2014](https://github.com/Defined2014) + - Fix the precision issue in the `UTC_TIMESTAMP()` function, for example, the precision is set too high [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) + - Fix the issue that foreign key errors are not omitted in `UPDATE`, `INSERT`, and `DELETE IGNORE` statements [#56678](https://github.com/pingcap/tidb/issues/56678) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that when querying the `information_schema.cluster_slow_query` table, if the time filter is not added, only the latest slow log file will be queried [#56100](https://github.com/pingcap/tidb/issues/56100) @[crazycs520](https://github.com/crazycs520) + - Fix memory leaks in TTL tables [#56934](https://github.com/pingcap/tidb/issues/56934) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that foreign key constraints do not take effect for tables with `write_only` status, and avoid using tables with `non-public` status [#55813](https://github.com/pingcap/tidb/issues/55813) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that using subqueries after using `NATURAL JOIN` or `USING CLAUSE` might result in errors [#53766](https://github.com/pingcap/tidb/issues/53766) @[dash12653](https://github.com/dash12653) + - Fix the issue that if a CTE contains `ORDER BY`, `LIMIT`, or `SELECT DISTINCT` clauses and is referenced by the recursive part of another CTE, it might be incorrectly inlined and result in an execution error [#56603](https://github.com/pingcap/tidb/issues/56603) @[elsa0520](https://github.com/elsa0520) + - Fix the issue that the CTE defined in `VIEW` is incorrectly inlined [#56582](https://github.com/pingcap/tidb/issues/56582) @[elsa0520](https://github.com/elsa0520) + - Fix the issue that Plan Replayer might report an error when importing a table structure containing foreign keys [#56456](https://github.com/pingcap/tidb/issues/56456) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that Plan Replayer might report an error when importing a table structure containing Placement Rules [#54961](https://github.com/pingcap/tidb/issues/54961) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that when using `ANALYZE` to collect statistics for a table, if the table contains expression indexes of virtually generated columns, the execution reports an error [#57079](https://github.com/pingcap/tidb/issues/57079) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the `DROP DATABASE` statement does not correctly trigger the corresponding change in statistics [#57227](https://github.com/pingcap/tidb/issues/57227) @[Rustin170506](https://github.com/Rustin170506) + - Fix the issue that when parsing a database name in CTE, it returns a wrong database name [#54582](https://github.com/pingcap/tidb/issues/54582) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the upper and lower bounds of the histogram are damaged when `DUMP STATS` is generating JSON [#56083](https://github.com/pingcap/tidb/issues/56083) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the result of `EXISTS` subquery is inconsistent with MySQL when it continues to participate in algebraic operations [#56641](https://github.com/pingcap/tidb/issues/56641) @[windtalker](https://github.com/windtalker) + - Fix the issue that plan bindings cannot be created for the multi-table deletion statement `DELETE` with aliases [#56726](https://github.com/pingcap/tidb/issues/56726) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the optimizer does not take into account the character set and collations when simplifying complex predicates, resulting in possible execution errors [#56479](https://github.com/pingcap/tidb/issues/56479) @[dash12653](https://github.com/dash12653) + - Fix the issue that the data for Stats Healthy Distribution in Grafana might be incorrect [#57176](https://github.com/pingcap/tidb/issues/57176) @[hawkingrei](https://github.com/hawkingrei) + TiKV From 3575d3121280571ceca6d6a2f476f71ed202ccd2 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 28 Nov 2024 14:49:03 +0800 Subject: [PATCH 45/89] update TiDB bug fixes --- releases/release-8.5.0.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 484047cefdb23..71a30b3f46dfd 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -355,25 +355,25 @@ The following features are planned for deprecation in future versions: - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `ADD INDEX` 时，未检查索引长度限制的问题 [#56930](https://github.com/pingcap/tidb/issues/56930) @[fzzf678](https://github.com/fzzf678) - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `RECOVER TABLE BY JOB JOB_ID;` 可能导致 panic 的问题 [#55113](https://github.com/pingcap/tidb/issues/55113) @[crazycs520](https://github.com/crazycs520) - (dup): release-7.1.6.md > 错误修复> TiDB - 修复由于 stale read 未对读操作的时间戳进行严格校验，导致 TSO 和真实物理时间存在偏移，有小概率影响事务一致性的问题 [#56809](https://github.com/pingcap/tidb/issues/56809) @[MyonKeminta](https://github.com/MyonKeminta) - - 修复 DDL owner 节点切换时，reorg 类型的 DDL 任务无法从切换前的进度继续的问题 [#56506](https://github.com/pingcap/tidb/issues/56506) @[tangenta](https://github.com/tangenta) - - 修复分布式执行框架监控面板不准确的问题 [#57172](https://github.com/pingcap/tidb/issues/57172) @[fzzf678](https://github.com/fzzf678) [#56942](https://github.com/pingcap/tidb/issues/56942) @[fzzf678](https://github.com/fzzf678) - - 修复某些情况下 REORGANIZE PARTITION 无法展示报错原因的问题 [#56634](https://github.com/pingcap/tidb/issues/56634) @[mjonss](https://github.com/mjonss) - - 修复查询 information_schema.tables 大小写敏感的问题 [#56987](https://github.com/pingcap/tidb/issues/56987) @[joechenrh](https://github.com/joechenrh) - - 修复 `CTE` 有多个数据消费者，且某个消费者在没有读取任何数据情况下退出可能导致的非法内存访问问题 [#57294](https://github.com/pingcap/tidb/issues/57294) @[windtalker](https://github.com/windtalker) - - 修复 `IndexHashJoin` 在非正常退出情况下可能卡住问题 [#54055](https://github.com/pingcap/tidb/issues/54055) @[wshwsh12](https://github.com/wshwsh12) - - 修复 `TRUNCATE` 处理 `NULL` 值不正确的问题 [#53546](https://github.com/pingcap/tidb/issues/53546) @[tuziemon](https://github.com/tuziemon) - - 修复类型推导错误导致的 `CAST AS CHAR` 函数结果问题 [#56640](https://github.com/pingcap/tidb/issues/56640) @[zimulala](https://github.com/zimulala) - - 修复类型推导错误导致的字符串被截断问题 [#56587](https://github.com/pingcap/tidb/issues/56587) @[joechenrh](https://github.com/joechenrh) - - 修复 `ADDTIME` 和 `SUBTIME` 函数第一个参数是日期类型时结果错误问题 [#57569](https://github.com/pingcap/tidb/issues/57569) @[xzhangxian1008](https://github.com/xzhangxian1008) - - 修复在非严格模式下 (sql_mode = '')，非法空值被插入的问题 [#56381](https://github.com/pingcap/tidb/issues/56381) @[joechenrh](https://github.com/joechenrh) - - 修复 `UPDATE` 语句中 `ENUM` 类型的值更新错误的问题 [#56832](https://github.com/pingcap/tidb/issues/56832) @[xhebox](https://github.com/xhebox) - - 修复开启 `tidb_low_resolution_tso` 后 `SELECT FOR UPDATE` 语句有资源泄漏的问题 [#55468](https://github.com/pingcap/tidb/issues/55468) @[tiancaiamao](https://github.com/tiancaiamao) - - 修复 JSON 函数没有检查参数类型的问题 [#54029](https://github.com/pingcap/tidb/issues/54029) @[YangKeao](https://github.com/YangKeao) - - 修复当 `PREPARE` 命令中包含 JSON 函数时可能失败的问题 [#54044](https://github.com/pingcap/tidb/issues/54044) @[YangKeao](https://github.com/YangKeao) - - 修复当执行 `bool like cast(bit as char)` 时 TiKV 崩溃的问题 [#56494](https://github.com/pingcap/tidb/issues/56494) @[lcwangchao](https://github.com/lcwangchao) - - 修复 `CREATE VIEW` 语句中带有变量或参数时不报错的问题 [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) - - 修复 `JSON_VALID()` 函数执行结果错误的问题 [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) - - 修复关闭 `tidb_ttl_job_enable` 后 TTL 任务没有被取消的问题 [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) + - 修复 DDL owner 节点切换后，无法按照切换前的进度继续执行 Reorg DDL 任务的问题 [#56506](https://github.com/pingcap/tidb/issues/56506) @[tangenta](https://github.com/tangenta) + - 修复分布式执行框架的监控面板数据不准确的问题 [#57172](https://github.com/pingcap/tidb/issues/57172) @[fzzf678](https://github.com/fzzf678) [#56942](https://github.com/pingcap/tidb/issues/56942) @[fzzf678](https://github.com/fzzf678) + - 修复某些情况下 `REORGANIZE PARTITION` 无法显示报错原因的问题 [#56634](https://github.com/pingcap/tidb/issues/56634) @[mjonss](https://github.com/mjonss) + - 修复查询 `INFORMATION_SCHEMA.TABLES` 时由于大小写敏感导致返回结果错误的问题 [#56987](https://github.com/pingcap/tidb/issues/56987) @[joechenrh](https://github.com/joechenrh) + - 修复当公共表表达式（CTE）有多个数据消费者时，如果某个消费者在未读取数据时就退出，可能导致非法内存访问的问题 [#55881](https://github.com/pingcap/tidb/issues/55881) @[windtalker](https://github.com/windtalker) + - 修复 `INDEX_HASH_JOIN` 在异常退出时可能卡住的问题 [#54055](https://github.com/pingcap/tidb/issues/54055) @[wshwsh12](https://github.com/wshwsh12) + - 修复 `TRUNCATE` 语句在处理 `NULL` 值时返回结果错误的问题 [#53546](https://github.com/pingcap/tidb/issues/53546) @[tuziemon](https://github.com/tuziemon) + - 修复由于类型推导错误导致 `CAST AS CHAR` 函数结果不正确的问题 [#56640](https://github.com/pingcap/tidb/issues/56640) @[zimulala](https://github.com/zimulala) + - 修复由于类型推导错误导致输出结果中字符串被截断的问题 [#56587](https://github.com/pingcap/tidb/issues/56587) @[joechenrh](https://github.com/joechenrh) + - 修复 `ADDTIME` 和 `SUBTIME` 函数的第一个参数为日期类型时返回错误结果的问题 [#57569](https://github.com/pingcap/tidb/issues/57569) @[xzhangxian1008](https://github.com/xzhangxian1008) + - 修复在非严格模式下 (`sql_mode = ''`) ，非法空值能被插入的问题 [#56381](https://github.com/pingcap/tidb/issues/56381) @[joechenrh](https://github.com/joechenrh) + - 修复 `UPDATE` 语句更新 `ENUM` 类型的值时更新错误的问题 [#56832](https://github.com/pingcap/tidb/issues/56832) @[xhebox](https://github.com/xhebox) + - 修复开启 `tidb_low_resolution_tso` 变量后，`SELECT FOR UPDATE` 语句出现资源泄漏的问题 [#55468](https://github.com/pingcap/tidb/issues/55468) @[tiancaiamao](https://github.com/tiancaiamao) + - 修复 `JSON_TYPE` 函数未检查其参数类型导致传入非 JSON 数据类型时未报错的问题 [#54029](https://github.com/pingcap/tidb/issues/54029) @[YangKeao](https://github.com/YangKeao) + - 修复在 `PREPARE` 语句中使用 JSON 函数可能导致执行失败的问题 [#54044](https://github.com/pingcap/tidb/issues/54044) @[YangKeao](https://github.com/YangKeao) + - 修复将数据从 `BIT` 类型换为 `CHAR` 类型时可能导致 TiKV 崩溃的问题 [#56494](https://github.com/pingcap/tidb/issues/56494) @[lcwangchao](https://github.com/lcwangchao) + - 修复在 `CREATE VIEW` 语句中使用变量或参数时未报错的问题 [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) + - 修复 `JSON_VALID()` 函数返回结果错误的问题 [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) + - 修复关闭 `tidb_ttl_job_enable` 变量后 TTL 任务未被取消的问题 [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the query result is wrong when using the `RANGE COLUMNS` partition function and the `utf8mb4_0900_ai_ci` collation at the same time [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) - Fix the runtime error caused by executing a prepared statement that begins with a newline character, resulting in an array out of bounds [#54283](https://github.com/pingcap/tidb/issues/54283) @[Defined2014](https://github.com/Defined2014) - Fix the precision issue in the `UTC_TIMESTAMP()` function, for example, the precision is set too high [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) From a22ff56d0f1a26ffc370702971edc9fd63c91317 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 28 Nov 2024 16:19:22 +0800 Subject: [PATCH 46/89] translate PD improvements and TiDB bug fixes --- releases/release-8.5.0.md | 44 +++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 71a30b3f46dfd..2b985ef342753 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -317,8 +317,8 @@ The following features are planned for deprecation in future versions: + PD - - 支持 tso 微服务模式和非微服务模式间动态切换 [#8477](https://github.com/tikv/pd/issues/8477) @[rleungx](https://github.com/rleungx) - - 改进 pd-ctl config 的输出 [#8694](https://github.com/tikv/pd/issues/8694) @[lhy1024](https://github.com/lhy1024) + - Support dynamic switching of the `tso` service between microservice and non-microservice modes [#8477](https://github.com/tikv/pd/issues/8477) @[rleungx](https://github.com/rleungx) + - Optimize the case format of certain fields in the `pd-ctl config` output [#8694](https://github.com/tikv/pd/issues/8694) @[lhy1024](https://github.com/lhy1024) + TiFlash @@ -345,7 +345,7 @@ The following features are planned for deprecation in future versions: + TiDB - - 修复 PD 返回 region 信息不包含 leader 的情况，没有重试可能导致执行报错的问题 [#56757](https://github.com/pingcap/tidb/issues/56757) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue that TiDB does not automatically retry requests when the Region metadata returned from PD lacks Leader information, potentially causing execution errors [#56757](https://github.com/pingcap/tidb/issues/56757) @[cfzjywxk](https://github.com/cfzjywxk) - (dup): release-7.1.6.md > 错误修复> TiDB - 修复写冲突时 TTL 任务可能无法取消的问题 [#56422](https://github.com/pingcap/tidb/issues/56422) @[YangKeao](https://github.com/YangKeao) - (dup): release-7.1.6.md > 错误修复> TiDB - 修复取消 TTL 任务时，没有强制 Kill 对应 SQL 的问题 [#56511](https://github.com/pingcap/tidb/issues/56511) @[lcwangchao](https://github.com/lcwangchao) - (dup): release-7.1.6.md > 错误修复> TiDB - 修复集群从 v6.5 升级到 v7.5 或更高版本后，已有 TTL 任务执行意外频繁的问题 [#56539](https://github.com/pingcap/tidb/issues/56539) @[lcwangchao](https://github.com/lcwangchao) @@ -355,25 +355,25 @@ The following features are planned for deprecation in future versions: - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `ADD INDEX` 时，未检查索引长度限制的问题 [#56930](https://github.com/pingcap/tidb/issues/56930) @[fzzf678](https://github.com/fzzf678) - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `RECOVER TABLE BY JOB JOB_ID;` 可能导致 panic 的问题 [#55113](https://github.com/pingcap/tidb/issues/55113) @[crazycs520](https://github.com/crazycs520) - (dup): release-7.1.6.md > 错误修复> TiDB - 修复由于 stale read 未对读操作的时间戳进行严格校验，导致 TSO 和真实物理时间存在偏移，有小概率影响事务一致性的问题 [#56809](https://github.com/pingcap/tidb/issues/56809) @[MyonKeminta](https://github.com/MyonKeminta) - - 修复 DDL owner 节点切换后，无法按照切换前的进度继续执行 Reorg DDL 任务的问题 [#56506](https://github.com/pingcap/tidb/issues/56506) @[tangenta](https://github.com/tangenta) - - 修复分布式执行框架的监控面板数据不准确的问题 [#57172](https://github.com/pingcap/tidb/issues/57172) @[fzzf678](https://github.com/fzzf678) [#56942](https://github.com/pingcap/tidb/issues/56942) @[fzzf678](https://github.com/fzzf678) - - 修复某些情况下 `REORGANIZE PARTITION` 无法显示报错原因的问题 [#56634](https://github.com/pingcap/tidb/issues/56634) @[mjonss](https://github.com/mjonss) - - 修复查询 `INFORMATION_SCHEMA.TABLES` 时由于大小写敏感导致返回结果错误的问题 [#56987](https://github.com/pingcap/tidb/issues/56987) @[joechenrh](https://github.com/joechenrh) - - 修复当公共表表达式（CTE）有多个数据消费者时，如果某个消费者在未读取数据时就退出，可能导致非法内存访问的问题 [#55881](https://github.com/pingcap/tidb/issues/55881) @[windtalker](https://github.com/windtalker) - - 修复 `INDEX_HASH_JOIN` 在异常退出时可能卡住的问题 [#54055](https://github.com/pingcap/tidb/issues/54055) @[wshwsh12](https://github.com/wshwsh12) - - 修复 `TRUNCATE` 语句在处理 `NULL` 值时返回结果错误的问题 [#53546](https://github.com/pingcap/tidb/issues/53546) @[tuziemon](https://github.com/tuziemon) - - 修复由于类型推导错误导致 `CAST AS CHAR` 函数结果不正确的问题 [#56640](https://github.com/pingcap/tidb/issues/56640) @[zimulala](https://github.com/zimulala) - - 修复由于类型推导错误导致输出结果中字符串被截断的问题 [#56587](https://github.com/pingcap/tidb/issues/56587) @[joechenrh](https://github.com/joechenrh) - - 修复 `ADDTIME` 和 `SUBTIME` 函数的第一个参数为日期类型时返回错误结果的问题 [#57569](https://github.com/pingcap/tidb/issues/57569) @[xzhangxian1008](https://github.com/xzhangxian1008) - - 修复在非严格模式下 (`sql_mode = ''`) ，非法空值能被插入的问题 [#56381](https://github.com/pingcap/tidb/issues/56381) @[joechenrh](https://github.com/joechenrh) - - 修复 `UPDATE` 语句更新 `ENUM` 类型的值时更新错误的问题 [#56832](https://github.com/pingcap/tidb/issues/56832) @[xhebox](https://github.com/xhebox) - - 修复开启 `tidb_low_resolution_tso` 变量后，`SELECT FOR UPDATE` 语句出现资源泄漏的问题 [#55468](https://github.com/pingcap/tidb/issues/55468) @[tiancaiamao](https://github.com/tiancaiamao) - - 修复 `JSON_TYPE` 函数未检查其参数类型导致传入非 JSON 数据类型时未报错的问题 [#54029](https://github.com/pingcap/tidb/issues/54029) @[YangKeao](https://github.com/YangKeao) - - 修复在 `PREPARE` 语句中使用 JSON 函数可能导致执行失败的问题 [#54044](https://github.com/pingcap/tidb/issues/54044) @[YangKeao](https://github.com/YangKeao) - - 修复将数据从 `BIT` 类型换为 `CHAR` 类型时可能导致 TiKV 崩溃的问题 [#56494](https://github.com/pingcap/tidb/issues/56494) @[lcwangchao](https://github.com/lcwangchao) - - 修复在 `CREATE VIEW` 语句中使用变量或参数时未报错的问题 [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) - - 修复 `JSON_VALID()` 函数返回结果错误的问题 [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) - - 修复关闭 `tidb_ttl_job_enable` 变量后 TTL 任务未被取消的问题 [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that TiDB could not resume Reorg DDL tasks from the previous progress after the DDL owner node is switched [#56506](https://github.com/pingcap/tidb/issues/56506) @[tangenta](https://github.com/tangenta) + - Fix the issue that some metrics in the monitoring panel of Distributed eXecution Framework (DXF) are inaccurate [#57172](https://github.com/pingcap/tidb/issues/57172) @[fzzf678](https://github.com/fzzf678) [#56942](https://github.com/pingcap/tidb/issues/56942) @[fzzf678](https://github.com/fzzf678) + - Fix the issue that `REORGANIZE PARTITION` fails to return error reasons in certain cases [#56634](https://github.com/pingcap/tidb/issues/56634) @[mjonss](https://github.com/mjonss) + - Fix the issue that querying `INFORMATION_SCHEMA.TABLES` returns incorrect results due to case sensitivity [#56987](https://github.com/pingcap/tidb/issues/56987) @[joechenrh](https://github.com/joechenrh) + - Fix the issue of illegal memory access that might occur when a Common Table Expression (CTE) has multiple data consumers and one consumer exits without reading any data [#55881](https://github.com/pingcap/tidb/issues/55881) @[windtalker](https://github.com/windtalker) + - Fix the issue that `INDEX_HASH_JOIN` might hang during an abnormal exit [#54055](https://github.com/pingcap/tidb/issues/54055) @[wshwsh12](https://github.com/wshwsh12) + - Fix the issue that the `TRUNCATE` statement returns incorrect results when handling `NULL` values [#53546](https://github.com/pingcap/tidb/issues/53546) @[tuziemon](https://github.com/tuziemon) + - Fix the issue that the `CAST AS CHAR` function returns incorrect results due to type inference errors [#56640](https://github.com/pingcap/tidb/issues/56640) @[zimulala](https://github.com/zimulala) + - Fix the issue of truncated strings in the output of some functions due to type inference errors [#56587](https://github.com/pingcap/tidb/issues/56587) @[joechenrh](https://github.com/joechenrh) + - Fix the issue that the `ADDTIME` and `SUBTIME` functions returns incorrect results when their first argument is a date type [#57569](https://github.com/pingcap/tidb/issues/57569) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix the issue that invalid `NULL` values can be inserted in non-strict mode (`sql_mode = ''`) [#56381](https://github.com/pingcap/tidb/issues/56381) @[joechenrh](https://github.com/joechenrh) + - Fix the issue that the `UPDATE` statement incorrectly updates values of the `ENUM` type [#56832](https://github.com/pingcap/tidb/issues/56832) @[xhebox](https://github.com/xhebox) + - Fix the issue that enabling the `tidb_low_resolution_tso` variable causes resource leaks during the execution of `SELECT FOR UPDATE` statements [#55468](https://github.com/pingcap/tidb/issues/55468) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the issue that the `JSON_TYPE` function does not validate parameter types, causing no errors returned when non-JSON data types are passed [#54029](https://github.com/pingcap/tidb/issues/54029) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that using JSON functions in `PREPARE` statements might cause execution failures [#54044](https://github.com/pingcap/tidb/issues/54044) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that converting data from the `BIT` type to the `CHAR` type might cause TiKV panics [#56494](https://github.com/pingcap/tidb/issues/56494) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that using variables or parameters in the `CREATE VIEW` statement does not report errors [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) + - Fix the issue that the `JSON_VALID()` function returns incorrect results [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that TTL tasks are not canceled after the `tidb_ttl_job_enable` variable is disabled [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the query result is wrong when using the `RANGE COLUMNS` partition function and the `utf8mb4_0900_ai_ci` collation at the same time [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) - Fix the runtime error caused by executing a prepared statement that begins with a newline character, resulting in an array out of bounds [#54283](https://github.com/pingcap/tidb/issues/54283) @[Defined2014](https://github.com/Defined2014) - Fix the precision issue in the `UTC_TIMESTAMP()` function, for example, the precision is set too high [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) From d1f9bf59b227202812a09f26bada61fa871b6bc3 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 29 Nov 2024 09:34:20 +0800 Subject: [PATCH 47/89] add #57627 --- releases/release-8.5.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 2b985ef342753..6233ae8f34253 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -394,6 +394,7 @@ The following features are planned for deprecation in future versions: - Fix the issue that plan bindings cannot be created for the multi-table deletion statement `DELETE` with aliases [#56726](https://github.com/pingcap/tidb/issues/56726) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that the optimizer does not take into account the character set and collations when simplifying complex predicates, resulting in possible execution errors [#56479](https://github.com/pingcap/tidb/issues/56479) @[dash12653](https://github.com/dash12653) - Fix the issue that the data for Stats Healthy Distribution in Grafana might be incorrect [#57176](https://github.com/pingcap/tidb/issues/57176) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that vector search might return incorrect results when querying tables with clustered indexes [#57627](https://github.com/pingcap/tidb/issues/57627) @[winoros](https://github.com/winoros) + TiKV From 26d3e95378adc9c99e2a5d5fca00b05988cdcd2d Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 29 Nov 2024 09:38:01 +0800 Subject: [PATCH 48/89] Apply suggestions from code review Co-authored-by: Aolin --- releases/release-8.5.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 6233ae8f34253..b05d51df4abd2 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -364,11 +364,11 @@ The following features are planned for deprecation in future versions: - Fix the issue that the `TRUNCATE` statement returns incorrect results when handling `NULL` values [#53546](https://github.com/pingcap/tidb/issues/53546) @[tuziemon](https://github.com/tuziemon) - Fix the issue that the `CAST AS CHAR` function returns incorrect results due to type inference errors [#56640](https://github.com/pingcap/tidb/issues/56640) @[zimulala](https://github.com/zimulala) - Fix the issue of truncated strings in the output of some functions due to type inference errors [#56587](https://github.com/pingcap/tidb/issues/56587) @[joechenrh](https://github.com/joechenrh) - - Fix the issue that the `ADDTIME` and `SUBTIME` functions returns incorrect results when their first argument is a date type [#57569](https://github.com/pingcap/tidb/issues/57569) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix the issue that the `ADDTIME()` and `SUBTIME()` functions returns incorrect results when their first argument is a date type [#57569](https://github.com/pingcap/tidb/issues/57569) @[xzhangxian1008](https://github.com/xzhangxian1008) - Fix the issue that invalid `NULL` values can be inserted in non-strict mode (`sql_mode = ''`) [#56381](https://github.com/pingcap/tidb/issues/56381) @[joechenrh](https://github.com/joechenrh) - Fix the issue that the `UPDATE` statement incorrectly updates values of the `ENUM` type [#56832](https://github.com/pingcap/tidb/issues/56832) @[xhebox](https://github.com/xhebox) - Fix the issue that enabling the `tidb_low_resolution_tso` variable causes resource leaks during the execution of `SELECT FOR UPDATE` statements [#55468](https://github.com/pingcap/tidb/issues/55468) @[tiancaiamao](https://github.com/tiancaiamao) - - Fix the issue that the `JSON_TYPE` function does not validate parameter types, causing no errors returned when non-JSON data types are passed [#54029](https://github.com/pingcap/tidb/issues/54029) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the `JSON_TYPE()` function does not validate the parameter type, causing no errors returned when a non-JSON data type is passed [#54029](https://github.com/pingcap/tidb/issues/54029) @[YangKeao](https://github.com/YangKeao) - Fix the issue that using JSON functions in `PREPARE` statements might cause execution failures [#54044](https://github.com/pingcap/tidb/issues/54044) @[YangKeao](https://github.com/YangKeao) - Fix the issue that converting data from the `BIT` type to the `CHAR` type might cause TiKV panics [#56494](https://github.com/pingcap/tidb/issues/56494) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue that using variables or parameters in the `CREATE VIEW` statement does not report errors [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) From aaff9d17793c35efce290850c523500ec88b8cf8 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 29 Nov 2024 10:45:41 +0800 Subject: [PATCH 49/89] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-8.5.0.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index b05d51df4abd2..93fc6e8c5e23a 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -332,10 +332,10 @@ The following features are planned for deprecation in future versions: + Backup & Restore (BR) - (dup): release-7.1.6.md > 改进提升> Tools> Backup & Restore (BR) - 减少备份过程中无效日志的打印 [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) - - Optimize error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) - - Increase concurrency during BR database creation to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) + - Optimize the error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) + - Increase concurrency in BR when creating databases to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) - Disable checksum by default during full backups to improve production environment performance while keeping it enabled in testing environments. This checksum is used only for the internal verification logic of BR, while the checksum that verifies backup data integrity remains enabled [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) - - Add independent connection timeout tracking and reset mechanisms for each storage node, enhancing the handling of slow nodes and preventing backup operation from hanging [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) + - Add a mechanism to independently track and reset the connection timeout for each storage node, enhancing the handling of slow nodes and preventing backup operations from hanging [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) + TiDB Data Migration (DM) @@ -400,11 +400,11 @@ The following features are planned for deprecation in future versions: - (dup): release-7.1.6.md > 错误修复> TiKV - 修复读线程在从 Raft Engine 中的 MemTable 读取过时索引时出现的 panic 问题 [#17383](https://github.com/tikv/tikv/issues/17383) @[LykxSassinator](https://github.com/LykxSassinator) - (dup): release-7.5.4.md > 错误修复> TiKV - 修复当大量事务在排队等待同一个 key 上的锁被释放且该 key 被频繁更新时，TiKV 可能因死锁检测压力过大而出现 OOM 的问题 [#17394](https://github.com/tikv/tikv/issues/17394) @[MyonKeminta](https://github.com/MyonKeminta) - - Fix the issue that CPU usage for background tasks of resource control is counted multiple times [#17603](https://github.com/tikv/tikv/issues/17603) @[glorv](https://github.com/glorv) + - Fix the issue that CPU usage for background tasks of resource control is counted twice [#17603](https://github.com/tikv/tikv/issues/17603) @[glorv](https://github.com/glorv) - Fix the issue that TiKV OOM might occur due to the accumulation of CDC internal tasks [#17696](https://github.com/tikv/tikv/issues/17696) @[3AceShowHand](https://github.com/3AceShowHand) - - Fix the issue that large batch writes cause jitter when `raft-entry-max-size` is set too high [#17701](https://github.com/tikv/tikv/issues/17701) @[SpadeA-Tang](https://github.com/SpadeA-Tang) + - Fix the issue that large batch writes cause performance jitter when `raft-entry-max-size` is set too high [#17701](https://github.com/tikv/tikv/issues/17701) @[SpadeA-Tang](https://github.com/SpadeA-Tang) - Fix the issue that the leader could not be quickly elected after Region split [#17602](https://github.com/tikv/tikv/issues/17602) @[LykxSassinator](https://github.com/LykxSassinator) - - Fix the issue that TiKV might panic when using `RADIANS()` or `DEGREES()` [#17852](https://github.com/tikv/tikv/issues/17852) @[gengliqi](https://github.com/gengliqi) + - Fix the issue that TiKV might panic when executing queries containing `RADIANS()` or `DEGREES()` functions [#17852](https://github.com/tikv/tikv/issues/17852) @[gengliqi](https://github.com/gengliqi) - Fix the issue that write jitter might occur when all hibernated Regions are awakened [#17101](https://github.com/tikv/tikv/issues/17101) @[hhwyt](https://github.com/hhwyt) + PD @@ -434,8 +434,8 @@ The following features are planned for deprecation in future versions: + TiCDC - Fix the issue that the Kafka messages lack Key fields when using the Debezium protocol [#1799](https://github.com/pingcap/tiflow/issues/1799) @[wk989898](https://github.com/wk989898) - - Fix the issue that the redo module fails to properly report errors [#11744](https://github.com/pingcap/tiflow/issues/11744) @[CharlesCheung96](https://github.com/CharlesCheung96) - - Fix the issue that TiCDC mistakenly discards DDL tasks when the schema version of DDL tasks become non-incremental during TiDB Owner changes [#11714](https://github.com/pingcap/tiflow/issues/11714) @[wlwilliamx](https://github.com/wlwilliamx) + - Fix the issue that the redo module fails to properly report errors [#11744](https://github.com/pingcap/tiflow/issues/11744) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Fix the issue that TiCDC mistakenly discards DDL tasks when the schema versions of DDL tasks become non-incremental during TiDB DDL owner changes [#11714](https://github.com/pingcap/tiflow/issues/11714) @[wlwilliamx](https://github.com/wlwilliamx) + TiDB Lightning From 455579eb98c3de171b67653dcdd2a99eb9da3669 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 29 Nov 2024 10:48:19 +0800 Subject: [PATCH 50/89] Apply suggestions from code review --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 93fc6e8c5e23a..ca1b9030422ee 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -334,7 +334,7 @@ The following features are planned for deprecation in future versions: - (dup): release-7.1.6.md > 改进提升> Tools> Backup & Restore (BR) - 减少备份过程中无效日志的打印 [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) - Optimize the error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) - Increase concurrency in BR when creating databases to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) - - Disable checksum by default during full backups to improve production environment performance while keeping it enabled in testing environments. This checksum is used only for the internal verification logic of BR, while the checksum that verifies backup data integrity remains enabled [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) + - During a full backup, the checksum for BR internal validation is disabled by default (`--checksum=false`) to improve backup performance. The checksum used to verify backup data integrity remains enabled [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) - Add a mechanism to independently track and reset the connection timeout for each storage node, enhancing the handling of slow nodes and preventing backup operations from hanging [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) + TiDB Data Migration (DM) From 19fa11414785e8ef0c2c423134790264d73e6445 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 29 Nov 2024 11:35:41 +0800 Subject: [PATCH 51/89] Apply suggestions from code review Co-authored-by: Lilian Lee --- releases/release-8.5.0.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index ca1b9030422ee..684034d29c7cf 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -61,7 +61,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + @@ -117,11 +117,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Scalability -* The schema cache memory limit feature is now generally available (GA), reducing memory usage in scenarios with hundreds of thousands or even millions of databases or tables. [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Setting the memory limit for schema cache is now generally available (GA), reducing memory usage in large-scale data scenarios [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd - In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows TiDB to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. + In some SaaS scenarios, where the number of tables reaches hundreds of thousands or even millions, schema metadata can consume a significant amount of memory. With this feature enabled, TiDB uses the Least Recently Used (LRU) algorithm to cache and evict the corresponding schema metadata, effectively reducing memory usage. - Starting from v8.4.0, this feature is enabled by default with a default value of `536870912` (that is, 512 MiB). You can adjust it as needed through the variable [`tidb_schema_cache_size`](/system-variables.md#tidb_schema_cache_size-new-in-v800). + Starting from v8.4.0, this feature is enabled by default with a default value of `536870912` (that is, 512 MiB). You can adjust it as needed using the variable [`tidb_schema_cache_size`](/system-variables.md#tidb_schema_cache_size-new-in-v800). For more information, see [documentation](/schema-cache.md). @@ -149,11 +149,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Documentation](/accelerated-table-creation.md). -* TiKV supports MVCC In-memory Engine (IME). It can accelerate queries that require scanning of a large number of MVCC historical versions. [#16141](https://github.com/tikv/tikv/issues/16141) [@SpadeA-Tang](https://github.com/SpadeA-Tang) [@glorv](https://github.com/glorv) [@overvenus](https://github.com/overvenus) +* TiKV supports the MVCC in-memory engine (IME), which accelerates queries involving scans of extensive MVCC historical versions [#16141](https://github.com/tikv/tikv/issues/16141) [@SpadeA-Tang](https://github.com/SpadeA-Tang) [@glorv](https://github.com/glorv) [@overvenus](https://github.com/overvenus) - In scenarios where records are updated frequently or TiDB is required to retain historical versions for a long period of time (for example, 24 hours), a buildup of MVCC versions can lead to degradation of scanning performance. TiKV MVCC In-memory Engine improves scanning performance by caching the most recent MVCC versions in memory and deleting the historical versions from memory through a fast GC mechanism. + When records are frequently updated, or TiDB is required to retain historical versions for extended periods (for example, 24 hours), the accumulation of MVCC versions can degrade scan performance. The TiKV MVCC in-memory engine improves scan performance by caching the latest MVCC versions in memory, and using a rapid GC mechanism to remove historical versions from memory. - Starting from v8.5.0, TiKV introduces MVCC In-memory Engine. When scanning performance is degraded due to the buildup of MVCC versions in a TiKV cluster, you can enable the TiKV MVCC memory engine to improve scanning performance by setting the TiKV configuration parameter [`in-memory-engine.enable`](/tikv-in-memory-engine.md#usage). + Starting from v8.5.0, TiKV introduces MVCC in-memory engine. If the accumulation of MVCC versions in the TiKV cluster leads to degraded scan performance, you can enable the TiKV MVCC in-memory engine to improve scan performance by setting the TiKV configuration parameter [`in-memory-engine.enable`](/tikv-in-memory-engine.md#usage). For more information, see [documentation](/tikv-in-memory-engine.md). @@ -191,11 +191,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. - TiDB v8.5.0 introduces the `ADMIN ALTER DDL JOBS` statement, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example: + TiDB v8.5.0 introduces the `ADMIN ALTER DDL JOBS` statement, letting you adjust the variable values of specific DDL jobs online. This enables flexible balancing of resource consumption and performance. The changes are limited to individual jobs, making the impact more controllable. For example: - - `ADMIN ALTER DDL JOBS job_id THREAD = 8;`: adjusts the `tidb_ddl_reorg_worker_cnt` for the specified DDL task. - - `ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;`: adjusts the `tidb_ddl_reorg_batch_size` for the specified DDL task. - - `ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';`: adjusts the write traffic size for index data on each TiKV node. + - `ADMIN ALTER DDL JOBS job_id THREAD = 8;`: adjusts the `tidb_ddl_reorg_worker_cnt` of the specified DDL job online. + - `ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;`: adjusts the `tidb_ddl_reorg_batch_size` of the specified job online. + - `ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';`: adjusts the write traffic of index data to each TiKV node online. For more information, see [documentation](/sql-statements/sql-statement-admin-alter-ddl.md). From ec490d5d702958e24d360962e8116fc45b0933db Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Fri, 29 Nov 2024 14:53:54 +0800 Subject: [PATCH 52/89] Fix format Co-authored-by: Aolin --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 684034d29c7cf..57e3fd25d101c 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -427,7 +427,7 @@ The following features are planned for deprecation in future versions: + Backup & Restore (BR) - Fix the OOM issue when there are too many incomplete ranges that have not been backed up yet during backup [#53529](https://github.com/pingcap/tidb/issues/53529) @[Leavrth](https://github.com/Leavrth) - - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014)] + - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014) - Fix the issue that logs might print out encrypted information [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) - Fix the issue that Advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) From f17f97ba568e350c694a14e0a8cfce1982e8d1e9 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 29 Nov 2024 15:20:23 +0800 Subject: [PATCH 53/89] Apply suggestions from code review Co-authored-by: Lilian Lee --- releases/release-8.5.0.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 57e3fd25d101c..2689396845338 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -297,22 +297,22 @@ The following features are planned for deprecation in future versions: + TiDB - - Improve the response speed of `ADD INDEX` acceleration to task cancelation when the Distributed eXecution Framework (DXF) is disabled [#56017](https://github.com/pingcap/tidb/issues/56017) @[lance6716](https://github.com/lance6716) + - Improve the response speed of job cancellation for the `ADD INDEX` acceleration feature when disabling the Distributed eXecution Framework (DXF) [#56017](https://github.com/pingcap/tidb/issues/56017) @[lance6716](https://github.com/lance6716) - Improve the speed of adding indexes to small tables [#54230](https://github.com/pingcap/tidb/issues/54230) @[tangenta](https://github.com/tangenta) - Add a new system variable `tidb_ddl_reorg_max_write_speed` to limit the maximum speed of the ingest phase when adding indexes [#57156](https://github.com/pingcap/tidb/issues/57156) @[CbcWestwolf](https://github.com/CbcWestwolf) - Improve the performance of querying `information_schema.tables` in some cases [#57295](https://github.com/pingcap/tidb/issues/57295) @[tangenta](https://github.com/tangenta) - - Support dynamically adjusting more DDL task parameters [#57526](https://github.com/pingcap/tidb/issues/57526) @[fzzf678](https://github.com/fzzf678) - - Support global indexes containing all columns in a partitioned expression [#56230](https://github.com/pingcap/tidb/issues/56230) @[Defined2014](https://github.com/Defined2014) + - Support dynamically adjusting more DDL job parameters [#57526](https://github.com/pingcap/tidb/issues/57526) @[fzzf678](https://github.com/fzzf678) + - Support global indexes that contain all columns from a partition expression [#56230](https://github.com/pingcap/tidb/issues/56230) @[Defined2014](https://github.com/Defined2014) - Support partition pruning for list partitioned tables in range query scenarios [#56673](https://github.com/pingcap/tidb/issues/56673) @[Defined2014](https://github.com/Defined2014) - - Enable FixControl#46177 by default to fix the issue that table full scan is incorrectly selected (instead of the index range scan) in some cases [#46177](https://github.com/pingcap/tidb/issues/46177) @[terry1purcell](https://github.com/terry1purcell) - - Improve the internal estimation logic so that it can better use the statistics of multi-column and multi-value indexes to improve the estimation accuracy of certain queries involving multi-value indexes [#56915](https://github.com/pingcap/tidb/issues/56915) @[time-and-fate](https://github.com/time-and-fate) - - Improve the cost estimation of full table scan in specific cases and reduce the probability of incorrectly choosing the full table scan [#57085](https://github.com/pingcap/tidb/issues/57085) @[terry1purcell](https://github.com/terry1purcell) + - Enable FixControl#46177 by default to fix the issue that a full table scan is incorrectly selected instead of an index range scan in some cases [#46177](https://github.com/pingcap/tidb/issues/46177) @[terry1purcell](https://github.com/terry1purcell) + - Improve the internal estimation logic to better utilize statistics of multi-column and multi-value indexes, enhancing estimation accuracy for certain queries involving multi-value indexes [#56915](https://github.com/pingcap/tidb/issues/56915) @[time-and-fate](https://github.com/time-and-fate) + - Improve the cost estimation for full table scans in specific scenarios, reducing the probability of incorrectly choosing a full table scan [#57085](https://github.com/pingcap/tidb/issues/57085) @[terry1purcell](https://github.com/terry1purcell) - Optimize the amount of data required for synchronous loading of statistics to improve loading performance [#56812](https://github.com/pingcap/tidb/issues/56812) @[winoros](https://github.com/winoros) - - Optimize the execution plan when `OUTER JOIN` contains a unique index and there is an `ORDER BY LIMIT` statement in certain cases to improve execution efficiency [#56321](https://github.com/pingcap/tidb/issues/56321) @[winoros](https://github.com/winoros) + - Optimize the execution plan in specific cases where an `OUTER JOIN` involves a unique index and an `ORDER BY ... LIMIT` clause, improving execution efficiency [#56321](https://github.com/pingcap/tidb/issues/56321) @[winoros](https://github.com/winoros) + TiKV - - Use a separate thread for replica cleanup to ensure stable latency for critical paths of Raft reads and writes [#16001](https://github.com/tikv/tikv/issues/16001) @[hbisheng](https://github.com/hbisheng) + - Use a separate thread to clean up replicas, ensuring stable latency for critical paths of Raft reads and writes [#16001](https://github.com/tikv/tikv/issues/16001) @[hbisheng](https://github.com/hbisheng) - Improve the performance of the vector distance function by supporting SIMD [#17290](https://github.com/tikv/tikv/issues/17290) @[EricZequan](https://github.com/EricZequan) + PD @@ -410,9 +410,9 @@ The following features are planned for deprecation in future versions: + PD - (dup): release-7.1.6.md > 错误修复> PD - 修复热点缓存中可能存在的内存泄露问题 [#8698](https://github.com/tikv/pd/issues/8698) @[lhy1024](https://github.com/lhy1024) - - Fix the issue that the resource group selector does not take effect for all panels [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) + - Fix the issue that the resource group selector does not take effect on any panel [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) - Fix the issue that deleted resource groups still appear in the monitoring panel [#8716](https://github.com/tikv/pd/issues/8716) @[AndreMouche](https://github.com/AndreMouche) - - Fix the issue that log description is not clear when loading syncer [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) + - Fix unclear log descriptions during the Region syncer loading process [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) + TiFlash @@ -426,10 +426,10 @@ The following features are planned for deprecation in future versions: + Backup & Restore (BR) - - Fix the OOM issue when there are too many incomplete ranges that have not been backed up yet during backup [#53529](https://github.com/pingcap/tidb/issues/53529) @[Leavrth](https://github.com/Leavrth) + - Fix the OOM issue during backups when there are too many uncompleted range gaps, reducing the amount of pre-allocated memory [#53529](https://github.com/pingcap/tidb/issues/53529) @[Leavrth](https://github.com/Leavrth) - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014) - Fix the issue that logs might print out encrypted information [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) - - Fix the issue that Advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) + - Fix the issue that the advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) + TiCDC From 5fbe41e6ada96e0afacd180e3950177e4a78c0f0 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Sat, 30 Nov 2024 19:53:26 +0800 Subject: [PATCH 54/89] update compatibility info and add contributor info --- releases/release-8.5.0.md | 73 +++++++-------------------------------- 1 file changed, 12 insertions(+), 61 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 2689396845338..e2232bf32d989 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -120,7 +120,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, * Setting the memory limit for schema cache is now generally available (GA), reducing memory usage in large-scale data scenarios [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd In some SaaS scenarios, where the number of tables reaches hundreds of thousands or even millions, schema metadata can consume a significant amount of memory. With this feature enabled, TiDB uses the Least Recently Used (LRU) algorithm to cache and evict the corresponding schema metadata, effectively reducing memory usage. - + Starting from v8.4.0, this feature is enabled by default with a default value of `536870912` (that is, 512 MiB). You can adjust it as needed using the variable [`tidb_schema_cache_size`](/system-variables.md#tidb_schema_cache_size-new-in-v800). For more information, see [documentation](/schema-cache.md). @@ -135,12 +135,6 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Performance -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** - - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. - - For more information, see [Documentation](link). - * TiDB accelerated table creation becomes generally available (GA), significantly reducing data migration and cluster initialization time [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt TiDB v7.6.0 introduces accelerated table creation as an experimental feature, controlled by the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_ddl_version-new-in-v760). Staring from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). @@ -165,29 +159,15 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [documentation](/pd-control.md). -### Availability - -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** - - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. - - For more information, see [Documentation](link). - ### SQL -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** - - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. - - For more information, see [Documentation](link). - * Support foreign keys (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 The foreign key feature becomes generally available (GA) in v8.5.0. Foreign key constraints help ensure data consistency and integrity. You can easily establish foreign key relationships between tables, with support for cascading updates and deletions, simplifying data management. This feature enhances support for applications with complex data relationships. For more information, see [documentation](/foreign-key.md). -* Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. @@ -199,22 +179,6 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [documentation](/sql-statements/sql-statement-admin-alter-ddl.md). -### DB operations - -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** - - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. - - For more information, see [Documentation](link). - -### Observability - -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** - - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. - - For more information, see [Documentation](link). - ### Security * BR supports client-side encryption of both full backup data and log backup data (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) [#56433](https://github.com/pingcap/tidb/issues/56433) @[joccau](https://github.com/joccau) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai @@ -236,16 +200,8 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, TiKV ensures data security by using the encryption at rest technique to encrypt stored data. The core aspect of this technique is proper key management. In v8.0.0, TiKV encryption at rest experimentally supports using Google Cloud KMS for master key management. Starting from v8.5.0, encryption at rest using Google Cloud KMS becomes generally available (GA). To use this feature, first create a key on Google Cloud, and then configure the `[security.encryption.master-key]` section in the TiKV configuration file. - - For more information, see [documentation](/encryption-at-rest.md#tikv-encryption-at-rest). - -### Data migration -* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** - - Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well. - - For more information, see [Documentation](link). + For more information, see [documentation](/encryption-at-rest.md#tikv-encryption-at-rest). ## Compatibility changes @@ -255,31 +211,22 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Behavior changes -### MySQL compatibility +- In non-strict mode (`sql_mode = ''`), inserting `NULL` values into non-`NULL` columns now returns an error for MySQL compatibility. [#55457](https://github.com/pingcap/tidb/issues/55457) @[joechenrh](https://github.com/joechenrh) +- The `ALTER TABLE ... DROP FOREIGN KEY IF EXISTS ...` statement is no longer supported. [#56703](https://github.com/pingcap/tidb/pull/56703) @[YangKeao](https://github.com/YangKeao) ### System variables | Variable name | Change type | Description | |--------|------------------------------|------| -|tidb_ddl_reorg_max_write_speed | Newly added |Used to control the speed at which TiDB writes index data to a single TiKV node. For example, setting the value to 200 MiB limits the maximum write speed to 200 MiB/s. | | [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800) | Modified | Changes the default value from `OFF` to `ON` after further tests, meaning that the [accelerated table creation](/accelerated-table-creation.md) feature is enabled by default. | -| | | | +| [`tidb_ddl_reorg_max_write_speed`](/system-variables.md#tidb_ddl_reorg_max_write_speed-new-in-v850) | Newly added | Limits the write bandwidth for each TiKV node and only takes effect when index creation acceleration is enabled (controlled by the [`tidb_ddl_enable_fast_reorg`](#tidb_ddl_enable_fast_reorg-new-in-v630) variable). For example, setting the variable to `200MiB` limits the maximum write speed to 200 MiB/s. | ### Configuration parameters | Configuration file or component | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | +| TiDB | [`deprecate-integer-display-length`](/tidb-configuration-file.md#deprecate-integer-display-length) | Modified | Starting with v8.5.0, the integer display width feature is deprecated. The default value of this configuration item is changed from `false` to `true`. | | TiKV | [`raft-client-queue-size`](/tikv-configuration-file.md#raft-client-queue-size) | Modified | Changes the default value from `8192` to `16384`. | -| | | | | -| | | | | - -### System tables - -### Other changes - -## Offline package changes - -## Removed features ## Deprecated features @@ -446,4 +393,8 @@ The following features are planned for deprecation in future versions: We would like to thank the following contributors from the TiDB community: -- [Contributor-GitHub-ID](id-link) +- [dash12653](https://github.com/dash12653) (First-time contributor) +- [chagelo](https://github.com/chagelo) (First-time contributor) +- [LindaSummer](https://github.com/LindaSummer) +- [songzhibin97](https://github.com/songzhibin97) +- [Hexilee](https://github.com/Hexilee) \ No newline at end of file From 4eb8678ba5204ae7e960fa6e589466604381dcd5 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Sat, 30 Nov 2024 20:02:36 +0800 Subject: [PATCH 55/89] replace zh dup notes with en dup notes --- releases/release-8.5.0.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index e2232bf32d989..faddc2280abaf 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -269,7 +269,7 @@ The following features are planned for deprecation in future versions: + TiFlash - - (dup): release-7.1.6.md > 改进提升> TiFlash - 提升聚簇索引表在后台回收过期数据的速度 [#9529](https://github.com/pingcap/tiflash/issues/9529) @[JaySon-Huang](https://github.com/JaySon-Huang) + - (dup): release-7.1.6.md > Improvements> TiFlash - Improve the garbage collection speed of outdated data in the background for tables with clustered indexes [#9529](https://github.com/pingcap/tiflash/issues/9529) @[JaySon-Huang](https://github.com/JaySon-Huang) - Improve query performance of vector search in data update scenarios [#9599](https://github.com/pingcap/tiflash/issues/9599) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - Add monitoring metrics for CPU usage during vector index building [#9032](https://github.com/pingcap/tiflash/issues/9032) @[JaySon-Huang](https://github.com/JaySon-Huang) - Improve the execution efficiency of logical operators [#9146](https://github.com/pingcap/tiflash/issues/9146) @[windtalker](https://github.com/windtalker) @@ -278,7 +278,7 @@ The following features are planned for deprecation in future versions: + Backup & Restore (BR) - - (dup): release-7.1.6.md > 改进提升> Tools> Backup & Restore (BR) - 减少备份过程中无效日志的打印 [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) + - (dup): release-7.1.6.md > Improvements> Tools> Backup & Restore (BR) - Reduce unnecessary log printing during backup [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) - Optimize the error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) - Increase concurrency in BR when creating databases to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) - During a full backup, the checksum for BR internal validation is disabled by default (`--checksum=false`) to improve backup performance. The checksum used to verify backup data integrity remains enabled [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) @@ -293,15 +293,15 @@ The following features are planned for deprecation in future versions: + TiDB - Fix the issue that TiDB does not automatically retry requests when the Region metadata returned from PD lacks Leader information, potentially causing execution errors [#56757](https://github.com/pingcap/tidb/issues/56757) @[cfzjywxk](https://github.com/cfzjywxk) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复写冲突时 TTL 任务可能无法取消的问题 [#56422](https://github.com/pingcap/tidb/issues/56422) @[YangKeao](https://github.com/YangKeao) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复取消 TTL 任务时，没有强制 Kill 对应 SQL 的问题 [#56511](https://github.com/pingcap/tidb/issues/56511) @[lcwangchao](https://github.com/lcwangchao) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复集群从 v6.5 升级到 v7.5 或更高版本后，已有 TTL 任务执行意外频繁的问题 [#56539](https://github.com/pingcap/tidb/issues/56539) @[lcwangchao](https://github.com/lcwangchao) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复 `INSERT ... ON DUPLICATE KEY` 语句不兼容 `mysql_insert_id` 的问题 [#55965](https://github.com/pingcap/tidb/issues/55965) @[tiancaiamao](https://github.com/tiancaiamao) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复 TTL 在未选用 TiKV 作为存储引擎时可能失败的问题 [#56402](https://github.com/pingcap/tidb/issues/56402) @[YangKeao](https://github.com/YangKeao) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复通过 `IMPORT INTO` 导入数据后，`AUTO_INCREMENT` 字段没有正确设置的问题 [#56476](https://github.com/pingcap/tidb/issues/56476) @[D3Hunter](https://github.com/D3Hunter) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `ADD INDEX` 时，未检查索引长度限制的问题 [#56930](https://github.com/pingcap/tidb/issues/56930) @[fzzf678](https://github.com/fzzf678) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复执行 `RECOVER TABLE BY JOB JOB_ID;` 可能导致 panic 的问题 [#55113](https://github.com/pingcap/tidb/issues/55113) @[crazycs520](https://github.com/crazycs520) - - (dup): release-7.1.6.md > 错误修复> TiDB - 修复由于 stale read 未对读操作的时间戳进行严格校验，导致 TSO 和真实物理时间存在偏移，有小概率影响事务一致性的问题 [#56809](https://github.com/pingcap/tidb/issues/56809) @[MyonKeminta](https://github.com/MyonKeminta) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that TTL tasks cannot be canceled when there is a write conflict [#56422](https://github.com/pingcap/tidb/issues/56422) @[YangKeao](https://github.com/YangKeao) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that when canceling a TTL task, the corresponding SQL is not killed forcibly [#56511](https://github.com/pingcap/tidb/issues/56511) @[lcwangchao](https://github.com/lcwangchao) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that existing TTL tasks are executed unexpectedly frequently in a cluster that is upgraded from v6.5 to v7.5 or later [#56539](https://github.com/pingcap/tidb/issues/56539) @[lcwangchao](https://github.com/lcwangchao) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that the `INSERT ... ON DUPLICATE KEY` statement is not compatible with `mysql_insert_id` [#55965](https://github.com/pingcap/tidb/issues/55965) @[tiancaiamao](https://github.com/tiancaiamao) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that TTL might fail if TiKV is not selected as the storage engine [#56402](https://github.com/pingcap/tidb/issues/56402) @[YangKeao](https://github.com/YangKeao) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that the `AUTO_INCREMENT` field is not correctly set after importing data using the `IMPORT INTO` statement [#56476](https://github.com/pingcap/tidb/issues/56476) @[D3Hunter](https://github.com/D3Hunter) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that TiDB does not check the index length limitation when executing `ADD INDEX` [#56930](https://github.com/pingcap/tidb/issues/56930) @[fzzf678](https://github.com/fzzf678) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that executing `RECOVER TABLE BY JOB JOB_ID;` might cause TiDB to panic [#55113](https://github.com/pingcap/tidb/issues/55113) @[crazycs520](https://github.com/crazycs520) + - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that stale read does not strictly verify the timestamp of the read operation, resulting in a small probability of affecting the consistency of the transaction when an offset exists between the TSO and the real physical time [#56809](https://github.com/pingcap/tidb/issues/56809) @[MyonKeminta](https://github.com/MyonKeminta) - Fix the issue that TiDB could not resume Reorg DDL tasks from the previous progress after the DDL owner node is switched [#56506](https://github.com/pingcap/tidb/issues/56506) @[tangenta](https://github.com/tangenta) - Fix the issue that some metrics in the monitoring panel of Distributed eXecution Framework (DXF) are inaccurate [#57172](https://github.com/pingcap/tidb/issues/57172) @[fzzf678](https://github.com/fzzf678) [#56942](https://github.com/pingcap/tidb/issues/56942) @[fzzf678](https://github.com/fzzf678) - Fix the issue that `REORGANIZE PARTITION` fails to return error reasons in certain cases [#56634](https://github.com/pingcap/tidb/issues/56634) @[mjonss](https://github.com/mjonss) @@ -345,8 +345,8 @@ The following features are planned for deprecation in future versions: + TiKV - - (dup): release-7.1.6.md > 错误修复> TiKV - 修复读线程在从 Raft Engine 中的 MemTable 读取过时索引时出现的 panic 问题 [#17383](https://github.com/tikv/tikv/issues/17383) @[LykxSassinator](https://github.com/LykxSassinator) - - (dup): release-7.5.4.md > 错误修复> TiKV - 修复当大量事务在排队等待同一个 key 上的锁被释放且该 key 被频繁更新时，TiKV 可能因死锁检测压力过大而出现 OOM 的问题 [#17394](https://github.com/tikv/tikv/issues/17394) @[MyonKeminta](https://github.com/MyonKeminta) + - (dup): release-7.1.6.md > Bug fixes> TiKV - Fix the panic issue that occurs when read threads access outdated indexes in the MemTable of the Raft Engine [#17383](https://github.com/tikv/tikv/issues/17383) @[LykxSassinator](https://github.com/LykxSassinator) + - (dup): release-7.5.4.md > Bug fixes> TiKV - Fix the issue that when a large number of transactions are queuing for lock release on the same key and the key is frequently updated, excessive pressure on deadlock detection might cause TiKV OOM issues [#17394](https://github.com/tikv/tikv/issues/17394) @[MyonKeminta](https://github.com/MyonKeminta) - Fix the issue that CPU usage for background tasks of resource control is counted twice [#17603](https://github.com/tikv/tikv/issues/17603) @[glorv](https://github.com/glorv) - Fix the issue that TiKV OOM might occur due to the accumulation of CDC internal tasks [#17696](https://github.com/tikv/tikv/issues/17696) @[3AceShowHand](https://github.com/3AceShowHand) - Fix the issue that large batch writes cause performance jitter when `raft-entry-max-size` is set too high [#17701](https://github.com/tikv/tikv/issues/17701) @[SpadeA-Tang](https://github.com/SpadeA-Tang) @@ -356,14 +356,14 @@ The following features are planned for deprecation in future versions: + PD - - (dup): release-7.1.6.md > 错误修复> PD - 修复热点缓存中可能存在的内存泄露问题 [#8698](https://github.com/tikv/pd/issues/8698) @[lhy1024](https://github.com/lhy1024) + - (dup): release-7.1.6.md > Bug fixes> PD - Fix the memory leak issue in hotspot cache [#8698](https://github.com/tikv/pd/issues/8698) @[lhy1024](https://github.com/lhy1024) - Fix the issue that the resource group selector does not take effect on any panel [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) - Fix the issue that deleted resource groups still appear in the monitoring panel [#8716](https://github.com/tikv/pd/issues/8716) @[AndreMouche](https://github.com/AndreMouche) - Fix unclear log descriptions during the Region syncer loading process [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) + TiFlash - - (dup): release-7.1.6.md > 错误修复> TiFlash - 修复 `SUBSTRING()` 函数不支持部分整数类型的 `pos` 和 `len` 参数导致查询报错的问题 [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) + - (dup): release-7.1.6.md > Bug fixes> TiFlash - Fix the issue that the `SUBSTRING()` function does not support the `pos` and `len` arguments for certain integer types, causing query errors [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) - Fix the issue that vector search performance might degrade after scaling out TiFlash write nodes in the disaggregated storage and compute architecture [#9637](https://github.com/pingcap/tiflash/issues/9637) @[kolafish](https://github.com/kolafish) - Fix the issue that the `SUBSTRING()` function returns incorrect results when the second parameter is negative [#9604](https://github.com/pingcap/tiflash/issues/9604) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that the `REPLACE()` function returns an error when the first parameter is a constant [#9522](https://github.com/pingcap/tiflash/issues/9522) @[guo-shaoge](https://github.com/guo-shaoge) @@ -386,7 +386,7 @@ The following features are planned for deprecation in future versions: + TiDB Lightning - - (dup): release-7.1.6.md > 错误修复> Tools> TiDB Lightning - 修复 TiDB Lightning 因 TiKV 发送的消息过大而接收失败的问题 [#56114](https://github.com/pingcap/tidb/issues/56114) @[fishiu](https://github.com/fishiu) + - (dup): release-7.1.6.md > Bug fixes> Tools> TiDB Lightning - Fix the issue that TiDB Lightning fails to receive oversized messages sent from TiKV [#56114](https://github.com/pingcap/tidb/issues/56114) @[fishiu](https://github.com/fishiu) - Fix the issue that the `AUTO_INCREMENT` value is set too high after importing data using the physical import mode [#56814](https://github.com/pingcap/tidb/issues/56814) @[D3Hunter](https://github.com/D3Hunter) ## Contributors From 0d454f6b0c64e725876333507ea127a4a276b078 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 3 Dec 2024 10:53:49 +0800 Subject: [PATCH 56/89] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-8.5.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index faddc2280abaf..2fbc7421d0b76 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -232,11 +232,11 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, The following features are planned for deprecation in future versions: -* TiDB introduces the system variable [`tidb_enable_auto_analyze_priority_queue`](/system-variables.md#tidb_enable_auto_analyze_priority_queue-new-in-v800), which controls whether priority queues are enabled to optimize the ordering of tasks that automatically collect statistics. In future releases, the priority queue will be the only way to order tasks for automatically collecting statistics, so this system variable will be deprecated. -* TiDB introduces the system variable [`tidb_enable_async_merge_global_stats`](/system-variables.md#tidb_enable_async_merge_global_stats-new-in-v750) in v7.5.0. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated. +* In v8.0.0, TiDB introduces the [`tidb_enable_auto_analyze_priority_queue`](/system-variables.md#tidb_enable_auto_analyze_priority_queue-new-in-v800) system variable to control whether priority queues are enabled to optimize the ordering of tasks that automatically collect statistics. In future releases, the priority queue will be the only way to order tasks for automatically collecting statistics, so this system variable will be deprecated. +* In v7.5.0, TiDB introduces the [`tidb_enable_async_merge_global_stats`](/system-variables.md#tidb_enable_async_merge_global_stats-new-in-v750) system variable. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated. * It is planned to redesign [the automatic evolution of execution plan bindings](/sql-plan-management.md#baseline-evolution) in subsequent releases, and the related variables and behavior will change. -* In v8.0.0, TiDB introduces the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable will be deprecated. -* In v5.1, TiDB introduces the system variable [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-new-in-v51) to set whether to enable the dynamic pruning mode for partitioned tables. Starting from v8.5.0, a warning is returned when you set this variable to `static` or `static-only`. In future versions, this system variable will be deprecated. +* In v8.0.0, TiDB introduces the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, this system variable will be deprecated. +* In v5.1, TiDB introduces the [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-new-in-v51) system variable to control whether to enable the dynamic pruning mode for partitioned tables. Starting from v8.5.0, a warning is returned when you set this variable to `static` or `static-only`. In future versions, this system variable will be deprecated. * The TiDB Lightning parameter [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) is planned for deprecation in a future release and will be subsequently removed. This parameter will be replaced by [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task), which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task. * Starting from v6.3.0, partitioned tables use [dynamic pruning mode](/partitioned-table.md#dynamic-pruning-mode) by default. Compared with static pruning mode, dynamic pruning mode supports features such as IndexJoin and plan cache with better performance. Therefore, static pruning mode will be deprecated. From 55642faa28ca0133178e0e3b85e4cf302d1ee282 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Tue, 3 Dec 2024 11:51:42 +0800 Subject: [PATCH 57/89] Update wording for consistency --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 2fbc7421d0b76..ac4baa159f627 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -225,7 +225,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | Configuration file or component | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | -| TiDB | [`deprecate-integer-display-length`](/tidb-configuration-file.md#deprecate-integer-display-length) | Modified | Starting with v8.5.0, the integer display width feature is deprecated. The default value of this configuration item is changed from `false` to `true`. | +| TiDB | [`deprecate-integer-display-length`](/tidb-configuration-file.md#deprecate-integer-display-length) | Modified | Starting from v8.5.0, the integer display width feature is deprecated. The default value of this configuration item is changed from `false` to `true`. | | TiKV | [`raft-client-queue-size`](/tikv-configuration-file.md#raft-client-queue-size) | Modified | Changes the default value from `8192` to `16384`. | ## Deprecated features From 50124576285bb1c8c6e76fd3b00855700280152c Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 3 Dec 2024 16:43:49 +0800 Subject: [PATCH 58/89] add 3 bug fixes --- releases/release-8.5.0.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index ac4baa159f627..d9b6a0be1c018 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -360,7 +360,8 @@ The following features are planned for deprecation in future versions: - Fix the issue that the resource group selector does not take effect on any panel [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) - Fix the issue that deleted resource groups still appear in the monitoring panel [#8716](https://github.com/tikv/pd/issues/8716) @[AndreMouche](https://github.com/AndreMouche) - Fix unclear log descriptions during the Region syncer loading process [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) - + - (dup): release-7.1.6.md > Bug fixes> PD - Fix the memory leak issue in label statistics [#8700](https://github.com/tikv/pd/issues/8700) @[lhy1024](https://github.com/lhy1024) + - Fix the issue that configuring `tidb_enable_tso_follower_proxy` to `0` or `OFF` fails to disable the TSO Follower Proxy feature [#8709](https://github.com/tikv/pd/issues/8709) @[JmPotato](https://github.com/JmPotato) + TiFlash - (dup): release-7.1.6.md > Bug fixes> TiFlash - Fix the issue that the `SUBSTRING()` function does not support the `pos` and `len` arguments for certain integer types, causing query errors [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) @@ -377,6 +378,7 @@ The following features are planned for deprecation in future versions: - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014) - Fix the issue that logs might print out encrypted information [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) - Fix the issue that the advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) + - Fix the issue that PITR tasks might fail for tables with millions of rows (https://github.com/pingcap/tidb/issues/57743) @[Tristan1900](https://github.com/Tristan1900) + TiCDC From db59b7b4dad046f5e7da258c36976c0cf03bc51f Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 3 Dec 2024 18:00:01 +0800 Subject: [PATCH 59/89] add PD: improvements --- releases/release-8.5.0.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index d9b6a0be1c018..84c8de189dde9 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -266,6 +266,8 @@ The following features are planned for deprecation in future versions: - Support dynamic switching of the `tso` service between microservice and non-microservice modes [#8477](https://github.com/tikv/pd/issues/8477) @[rleungx](https://github.com/rleungx) - Optimize the case format of certain fields in the `pd-ctl config` output [#8694](https://github.com/tikv/pd/issues/8694) @[lhy1024](https://github.com/lhy1024) + - [Store limit v2](/configure-store-limit.md#principles-of-store-limit-v2) becomes generally available (GA) [#8865](https://github.com/tikv/pd/issues/8865) @[lhy1024](https://github.com/lhy1024) + - Support configuring Region inspection concurrency [#8866](https://github.com/tikv/pd/issues/8866) @[lhy1024](https://github.com/lhy1024) + TiFlash From 57f19f42e14f231e3d9e4358978c3ea11724c020 Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 3 Dec 2024 18:24:17 +0800 Subject: [PATCH 60/89] fix format --- releases/release-8.5.0.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 84c8de189dde9..862cb6a3b06de 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -364,6 +364,7 @@ The following features are planned for deprecation in future versions: - Fix unclear log descriptions during the Region syncer loading process [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) - (dup): release-7.1.6.md > Bug fixes> PD - Fix the memory leak issue in label statistics [#8700](https://github.com/tikv/pd/issues/8700) @[lhy1024](https://github.com/lhy1024) - Fix the issue that configuring `tidb_enable_tso_follower_proxy` to `0` or `OFF` fails to disable the TSO Follower Proxy feature [#8709](https://github.com/tikv/pd/issues/8709) @[JmPotato](https://github.com/JmPotato) + + TiFlash - (dup): release-7.1.6.md > Bug fixes> TiFlash - Fix the issue that the `SUBSTRING()` function does not support the `pos` and `len` arguments for certain integer types, causing query errors [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) @@ -380,7 +381,7 @@ The following features are planned for deprecation in future versions: - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014) - Fix the issue that logs might print out encrypted information [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) - Fix the issue that the advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) - - Fix the issue that PITR tasks might fail for tables with millions of rows (https://github.com/pingcap/tidb/issues/57743) @[Tristan1900](https://github.com/Tristan1900) + - Fix the issue that PITR tasks might fail for tables with millions of rows [#57743](https://github.com/pingcap/tidb/issues/57743) @[Tristan1900](https://github.com/Tristan1900) + TiCDC From 5faef439c0114537ab084c98d210c814aa692e3c Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 5 Dec 2024 11:13:48 +0800 Subject: [PATCH 61/89] mark Region inspection concurrency as experimental --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 862cb6a3b06de..ec1ddaa171de2 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -267,7 +267,7 @@ The following features are planned for deprecation in future versions: - Support dynamic switching of the `tso` service between microservice and non-microservice modes [#8477](https://github.com/tikv/pd/issues/8477) @[rleungx](https://github.com/rleungx) - Optimize the case format of certain fields in the `pd-ctl config` output [#8694](https://github.com/tikv/pd/issues/8694) @[lhy1024](https://github.com/lhy1024) - [Store limit v2](/configure-store-limit.md#principles-of-store-limit-v2) becomes generally available (GA) [#8865](https://github.com/tikv/pd/issues/8865) @[lhy1024](https://github.com/lhy1024) - - Support configuring Region inspection concurrency [#8866](https://github.com/tikv/pd/issues/8866) @[lhy1024](https://github.com/lhy1024) + - Support configuring Region inspection concurrency (experimental) [#8866](https://github.com/tikv/pd/issues/8866) @[lhy1024](https://github.com/lhy1024) + TiFlash From 385221aeceb544ca0199648623d7e00de37ead8a Mon Sep 17 00:00:00 2001 From: lilin90 Date: Thu, 5 Dec 2024 11:17:04 +0800 Subject: [PATCH 62/89] Update wording --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 862cb6a3b06de..4d29c383c0efc 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -288,7 +288,7 @@ The following features are planned for deprecation in future versions: + TiDB Data Migration (DM) - - Add retries for DM-worker to connect to DM-master when the DM cluster starts up [#4287](https://github.com/pingcap/tiflow/issues/4287) @[GMHDBJD](https://github.com/GMHDBJD) + - Add retries for DM-worker to connect to DM-master during DM cluster startup [#4287](https://github.com/pingcap/tiflow/issues/4287) @[GMHDBJD](https://github.com/GMHDBJD) ## Bug fixes From 602c17897a653bc61409f29ee1814535c0d38ee5 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 5 Dec 2024 15:55:28 +0800 Subject: [PATCH 63/89] Apply suggestions from code review Co-authored-by: Frank945946 <108602632+Frank945946@users.noreply.github.com> --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 94cea5e1c121b..87d6f7cf308f6 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -48,7 +48,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + From d1eef60decf59cf7d5cf6884001079360a177a75 Mon Sep 17 00:00:00 2001 From: lilin90 Date: Thu, 5 Dec 2024 18:14:58 +0800 Subject: [PATCH 64/89] releases: update wording --- releases/release-8.5.0.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 87d6f7cf308f6..f861b3e09e619 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -323,26 +323,26 @@ The following features are planned for deprecation in future versions: - Fix the issue that using variables or parameters in the `CREATE VIEW` statement does not report errors [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) - Fix the issue that the `JSON_VALID()` function returns incorrect results [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) - Fix the issue that TTL tasks are not canceled after the `tidb_ttl_job_enable` variable is disabled [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the query result is wrong when using the `RANGE COLUMNS` partition function and the `utf8mb4_0900_ai_ci` collation at the same time [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that using the `RANGE COLUMNS` partition function and the `utf8mb4_0900_ai_ci` collation at the same time could result in incorrect query results [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) - Fix the runtime error caused by executing a prepared statement that begins with a newline character, resulting in an array out of bounds [#54283](https://github.com/pingcap/tidb/issues/54283) @[Defined2014](https://github.com/Defined2014) - - Fix the precision issue in the `UTC_TIMESTAMP()` function, for example, the precision is set too high [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) + - Fix the precision issue in the `UTC_TIMESTAMP()` function, such as setting the precision too high [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) - Fix the issue that foreign key errors are not omitted in `UPDATE`, `INSERT`, and `DELETE IGNORE` statements [#56678](https://github.com/pingcap/tidb/issues/56678) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that when querying the `information_schema.cluster_slow_query` table, if the time filter is not added, only the latest slow log file will be queried [#56100](https://github.com/pingcap/tidb/issues/56100) @[crazycs520](https://github.com/crazycs520) - - Fix memory leaks in TTL tables [#56934](https://github.com/pingcap/tidb/issues/56934) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that foreign key constraints do not take effect for tables with `write_only` status, and avoid using tables with `non-public` status [#55813](https://github.com/pingcap/tidb/issues/55813) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that using subqueries after using `NATURAL JOIN` or `USING CLAUSE` might result in errors [#53766](https://github.com/pingcap/tidb/issues/53766) @[dash12653](https://github.com/dash12653) - - Fix the issue that if a CTE contains `ORDER BY`, `LIMIT`, or `SELECT DISTINCT` clauses and is referenced by the recursive part of another CTE, it might be incorrectly inlined and result in an execution error [#56603](https://github.com/pingcap/tidb/issues/56603) @[elsa0520](https://github.com/elsa0520) + - Fix the issue that when querying the `information_schema.cluster_slow_query` table, if the time filter is not added, only the latest slow log file is queried [#56100](https://github.com/pingcap/tidb/issues/56100) @[crazycs520](https://github.com/crazycs520) + - Fix the issue of memory leaks in TTL tables [#56934](https://github.com/pingcap/tidb/issues/56934) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that foreign key constraints do not take effect for tables in `write_only` status, preventing using tables in `non-public` status [#55813](https://github.com/pingcap/tidb/issues/55813) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that using subqueries after the `NATURAL JOIN` or `USING` clause might result in errors [#53766](https://github.com/pingcap/tidb/issues/53766) @[dash12653](https://github.com/dash12653) + - Fix the issue that if a CTE contains the `ORDER BY`, `LIMIT`, or `SELECT DISTINCT` clause and is referenced by the recursive part of another CTE, it might be incorrectly inlined and result in an execution error [#56603](https://github.com/pingcap/tidb/issues/56603) @[elsa0520](https://github.com/elsa0520) - Fix the issue that the CTE defined in `VIEW` is incorrectly inlined [#56582](https://github.com/pingcap/tidb/issues/56582) @[elsa0520](https://github.com/elsa0520) - Fix the issue that Plan Replayer might report an error when importing a table structure containing foreign keys [#56456](https://github.com/pingcap/tidb/issues/56456) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that Plan Replayer might report an error when importing a table structure containing Placement Rules [#54961](https://github.com/pingcap/tidb/issues/54961) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that when using `ANALYZE` to collect statistics for a table, if the table contains expression indexes of virtually generated columns, the execution reports an error [#57079](https://github.com/pingcap/tidb/issues/57079) @[hawkingrei](https://github.com/hawkingrei) - - Fix the issue that the `DROP DATABASE` statement does not correctly trigger the corresponding change in statistics [#57227](https://github.com/pingcap/tidb/issues/57227) @[Rustin170506](https://github.com/Rustin170506) + - Fix the issue that the `DROP DATABASE` statement does not correctly trigger the corresponding update in statistics [#57227](https://github.com/pingcap/tidb/issues/57227) @[Rustin170506](https://github.com/Rustin170506) - Fix the issue that when parsing a database name in CTE, it returns a wrong database name [#54582](https://github.com/pingcap/tidb/issues/54582) @[hawkingrei](https://github.com/hawkingrei) - - Fix the issue that the upper and lower bounds of the histogram are damaged when `DUMP STATS` is generating JSON [#56083](https://github.com/pingcap/tidb/issues/56083) @[hawkingrei](https://github.com/hawkingrei) - - Fix the issue that the result of `EXISTS` subquery is inconsistent with MySQL when it continues to participate in algebraic operations [#56641](https://github.com/pingcap/tidb/issues/56641) @[windtalker](https://github.com/windtalker) - - Fix the issue that plan bindings cannot be created for the multi-table deletion statement `DELETE` with aliases [#56726](https://github.com/pingcap/tidb/issues/56726) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the upper bound and lower bound of the histogram are corrupted when `DUMP STATS` is transforming statistics into JSON [#56083](https://github.com/pingcap/tidb/issues/56083) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that `EXISTS` subquery results, when further involved in algebraic operations, could differ from the results in MySQL [#56641](https://github.com/pingcap/tidb/issues/56641) @[windtalker](https://github.com/windtalker) + - Fix the issue that execution plan bindings cannot be created for the multi-table `DELETE` statement with aliases [#56726](https://github.com/pingcap/tidb/issues/56726) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that the optimizer does not take into account the character set and collations when simplifying complex predicates, resulting in possible execution errors [#56479](https://github.com/pingcap/tidb/issues/56479) @[dash12653](https://github.com/dash12653) - - Fix the issue that the data for Stats Healthy Distribution in Grafana might be incorrect [#57176](https://github.com/pingcap/tidb/issues/57176) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the data in the **Stats Healthy Distribution** panel of Grafana might be incorrect [#57176](https://github.com/pingcap/tidb/issues/57176) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that vector search might return incorrect results when querying tables with clustered indexes [#57627](https://github.com/pingcap/tidb/issues/57627) @[winoros](https://github.com/winoros) + TiKV From 55eae2c4fd4c42fc1239e3f2a507416e0958bf2e Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 9 Dec 2024 10:40:42 +0800 Subject: [PATCH 65/89] Update releases/release-8.5.0.md Co-authored-by: Frank945946 <108602632+Frank945946@users.noreply.github.com> --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index f861b3e09e619..961d6e3fffb8e 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -117,7 +117,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Scalability -* Setting the memory limit for schema cache is now generally available (GA), reducing memory usage in large-scale data scenarios [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Setting the memory limit for schema cache is now generally available (GA). When the number of tables reaches hundreds of thousands or even millions, this feature significantly reduces the memory usage of schema metadata [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd In some SaaS scenarios, where the number of tables reaches hundreds of thousands or even millions, schema metadata can consume a significant amount of memory. With this feature enabled, TiDB uses the Least Recently Used (LRU) algorithm to cache and evict the corresponding schema metadata, effectively reducing memory usage. From 4e9bdb1064c9787e32a1ec25a6641717350f8537 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 9 Dec 2024 11:59:06 +0800 Subject: [PATCH 66/89] highlight-AWS SDK for Rust --- releases/release-8.5.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 961d6e3fffb8e..6d859b11f2e14 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -102,8 +102,8 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - - + + From 338ce92e2b67b8ae9a8967144ae3342d44448bd3 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 9 Dec 2024 16:10:36 +0800 Subject: [PATCH 67/89] update br bug fixes descriptions (keep only one #57743) --- releases/release-8.5.0.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 6d859b11f2e14..ed55f90ece105 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -381,7 +381,8 @@ The following features are planned for deprecation in future versions: - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014) - Fix the issue that logs might print out encrypted information [#57585](https://github.com/pingcap/tidb/issues/57585) @[kennytm](https://github.com/kennytm) - Fix the issue that the advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) - - Fix the issue that PITR tasks might fail for tables with millions of rows [#57743](https://github.com/pingcap/tidb/issues/57743) @[Tristan1900](https://github.com/Tristan1900) + - Fix potential security vulnerabilities by upgrading the `k8s.io/api` library version [#57790](https://github.com/pingcap/tidb/issues/57790) @[BornChanger](https://github.com/BornChanger) + - Fix the issue that PITR tasks might return the `Information schema is out of date` error when there are a large number of tables in the cluster but the actual data size is small [#57743](https://github.com/pingcap/tidb/issues/57743) @[Tristan1900](https://github.com/Tristan1900) + TiCDC From 19c46fdf34df445fcadab0c5f04d3fe71656a7b1 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 9 Dec 2024 16:14:10 +0800 Subject: [PATCH 68/89] Update releases/release-8.5.0.md Co-authored-by: xixirangrang --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index ed55f90ece105..5497c741056b9 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -103,7 +103,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + From 4780e93d18cf1f74a7e2d821bf86477c3a901a24 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 9 Dec 2024 16:15:22 +0800 Subject: [PATCH 69/89] add a missing space --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 5497c741056b9..f1a01b68c51bf 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -103,7 +103,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + From 791da1980eea379cbed53c9c7190975c22d9661a Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Mon, 9 Dec 2024 17:23:31 +0800 Subject: [PATCH 70/89] Add BR "--checksum" Co-authored-by: Grace Cai --- releases/release-8.5.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index f1a01b68c51bf..1b87e0256c97d 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -227,6 +227,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | -------- | -------- | -------- | -------- | | TiDB | [`deprecate-integer-display-length`](/tidb-configuration-file.md#deprecate-integer-display-length) | Modified | Starting from v8.5.0, the integer display width feature is deprecated. The default value of this configuration item is changed from `false` to `true`. | | TiKV | [`raft-client-queue-size`](/tikv-configuration-file.md#raft-client-queue-size) | Modified | Changes the default value from `8192` to `16384`. | +| BR | [`--checksum`](/br/use-br-command-line-tool.md) | Modified | Changes the default value from `true` to `false`, meaning that BR does not calculate the table-level checksum during full backups by default, to improve backup performance. | ## Deprecated features From e0c7ff330b3a8a3a92c5e9988acb4279b80a0ef8 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 9 Dec 2024 17:54:21 +0800 Subject: [PATCH 71/89] fix link format issues --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 1b87e0256c97d..984b51b05af4c 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -169,7 +169,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, * Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd - Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. + Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables.md#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables.md#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. TiDB v8.5.0 introduces the `ADMIN ALTER DDL JOBS` statement, letting you adjust the variable values of specific DDL jobs online. This enables flexible balancing of resource consumption and performance. The changes are limited to individual jobs, making the impact more controllable. For example: From 09430ed4448aafb74ed6980f7b3849ecb537dfb3 Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 9 Dec 2024 18:58:01 +0800 Subject: [PATCH 72/89] update BR checksum=false Co-authored-by: Grace Cai --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 984b51b05af4c..6d0b288eee9a8 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -284,7 +284,7 @@ The following features are planned for deprecation in future versions: - (dup): release-7.1.6.md > Improvements> Tools> Backup & Restore (BR) - Reduce unnecessary log printing during backup [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) - Optimize the error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) - Increase concurrency in BR when creating databases to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) - - During a full backup, the checksum for BR internal validation is disabled by default (`--checksum=false`) to improve backup performance. The checksum used to verify backup data integrity remains enabled [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) + - Disable the table-level checksum calculation during full backups by default (`--checksum=false`) to improve backup performance [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) - Add a mechanism to independently track and reset the connection timeout for each storage node, enhancing the handling of slow nodes and preventing backup operations from hanging [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) + TiDB Data Migration (DM) From 41a17dbcc0a97b1ea4fd1054045e05306cc65e8d Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 10 Dec 2024 09:54:01 +0800 Subject: [PATCH 73/89] fix a broken link --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 6d0b288eee9a8..1174a4749957d 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -219,7 +219,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | Variable name | Change type | Description | |--------|------------------------------|------| | [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800) | Modified | Changes the default value from `OFF` to `ON` after further tests, meaning that the [accelerated table creation](/accelerated-table-creation.md) feature is enabled by default. | -| [`tidb_ddl_reorg_max_write_speed`](/system-variables.md#tidb_ddl_reorg_max_write_speed-new-in-v850) | Newly added | Limits the write bandwidth for each TiKV node and only takes effect when index creation acceleration is enabled (controlled by the [`tidb_ddl_enable_fast_reorg`](#tidb_ddl_enable_fast_reorg-new-in-v630) variable). For example, setting the variable to `200MiB` limits the maximum write speed to 200 MiB/s. | +| [`tidb_ddl_reorg_max_write_speed`](/system-variables.md#tidb_ddl_reorg_max_write_speed-new-in-v850) | Newly added | Limits the write bandwidth for each TiKV node and only takes effect when index creation acceleration is enabled (controlled by the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) variable). For example, setting the variable to `200MiB` limits the maximum write speed to 200 MiB/s. | ### Configuration parameters From 595c0c8bf4f4bf6fa7993f4b051e82150339652e Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 10 Dec 2024 11:56:57 +0800 Subject: [PATCH 74/89] clean up comments --- releases/release-8.5.0.md | 92 +++++++++++++++++++-------------------- 1 file changed, 46 insertions(+), 46 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 1174a4749957d..0462d55164c70 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -28,8 +28,8 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - - + - + @@ -60,7 +60,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + @@ -72,7 +72,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + - + @@ -107,7 +107,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + @@ -117,7 +117,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Scalability -* Setting the memory limit for schema cache is now generally available (GA). When the number of tables reaches hundreds of thousands or even millions, this feature significantly reduces the memory usage of schema metadata [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Setting the memory limit for schema cache is now generally available (GA). When the number of tables reaches hundreds of thousands or even millions, this feature significantly reduces the memory usage of schema metadata [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) In some SaaS scenarios, where the number of tables reaches hundreds of thousands or even millions, schema metadata can consume a significant amount of memory. With this feature enabled, TiDB uses the Least Recently Used (LRU) algorithm to cache and evict the corresponding schema metadata, effectively reducing memory usage. @@ -125,7 +125,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [documentation](/schema-cache.md). -* Provide the Active PD Follower feature to enhance the scalability of PD's Region information query service (GA) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) tw@Oreoxmt +* Provide the Active PD Follower feature to enhance the scalability of PD's Region information query service (GA) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang) In a TiDB cluster with a large number of Regions, the PD leader might experience high CPU load due to the increased overhead of handling heartbeats and scheduling tasks. If the cluster has many TiDB instances, and there is a high concurrency of requests for Region information, the CPU pressure on the PD leader increases further and might cause PD services to become unavailable. @@ -135,7 +135,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Performance -* TiDB accelerated table creation becomes generally available (GA), significantly reducing data migration and cluster initialization time [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt +* TiDB accelerated table creation becomes generally available (GA), significantly reducing data migration and cluster initialization time [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) TiDB v7.6.0 introduces accelerated table creation as an experimental feature, controlled by the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_ddl_version-new-in-v760). Staring from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). @@ -153,7 +153,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Reliability -* Support limiting the maximum rate and concurrency of requests processed by PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) **tw@qiancai** +* Support limiting the maximum rate and concurrency of requests processed by PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx) When a sudden influx of requests is sent to PD, it can lead to high workloads and potentially affect PD performance. Starting from v8.5.0, you can use [`pd-ctl`](/pd-control.md) to limit the maximum rate and concurrency of requests processed by PD, improving its stability. @@ -161,13 +161,13 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### SQL -* Support foreign keys (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 +* Support foreign keys (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) The foreign key feature becomes generally available (GA) in v8.5.0. Foreign key constraints help ensure data consistency and integrity. You can easily establish foreign key relationships between tables, with support for cascading updates and deletions, simplifying data management. This feature enhances support for applications with complex data relationships. For more information, see [documentation](/foreign-key.md). -* Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd +* Introduce the `ADMIN ALTER DDL JOBS` statement to support modifying the DDL jobs online [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) Starting from v8.3.0, you can set the variables [`tidb_ddl_reorg_batch_size`](/system-variables.md#tidb_ddl_reorg_batch_size) and [`tidb_ddl_reorg_worker_cnt`](/system-variables.md#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, you need to cancel the DDL job first, adjust the variables, and then resubmit the job. @@ -181,7 +181,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ### Security -* BR supports client-side encryption of both full backup data and log backup data (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) [#56433](https://github.com/pingcap/tidb/issues/56433) @[joccau](https://github.com/joccau) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai +* BR supports client-side encryption of both full backup data and log backup data (GA) [#28640](https://github.com/pingcap/tidb/issues/28640) [#56433](https://github.com/pingcap/tidb/issues/56433) @[joccau](https://github.com/joccau) @[Tristan1900](https://github.com/Tristan1900) * Client-side encryption of full backup data (introduced as experimental in TiDB v5.3.0) enables you to encrypt backup data on the client side using a custom fixed key. @@ -195,7 +195,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, For more information, see [Encrypt the backup data](/br/br-snapshot-manual.md#encrypt-the-backup-data) and [Encrypt the log backup data](/br/br-pitr-manual.md#encrypt-the-log-backup-data). -* TiKV encryption at rest supports [Google Cloud Key Management Service (Google Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) **tw@qiancai** +* TiKV encryption at rest supports [Google Cloud Key Management Service (Google Cloud KMS)](https://cloud.google.com/docs/security/key-management-deep-dive) (GA) [#8906](https://github.com/tikv/tikv/issues/8906) @[glorv](https://github.com/glorv) TiKV ensures data security by using the encryption at rest technique to encrypt stored data. The core aspect of this technique is proper key management. In v8.0.0, TiKV encryption at rest experimentally supports using Google Cloud KMS for master key management. @@ -243,7 +243,7 @@ The following features are planned for deprecation in future versions: ## Improvements -+ TiDB ++ TiDB - Improve the response speed of job cancellation for the `ADD INDEX` acceleration feature when disabling the Distributed eXecution Framework (DXF) [#56017](https://github.com/pingcap/tidb/issues/56017) @[lance6716](https://github.com/lance6716) - Improve the speed of adding indexes to small tables [#54230](https://github.com/pingcap/tidb/issues/54230) @[tangenta](https://github.com/tangenta) @@ -258,53 +258,53 @@ The following features are planned for deprecation in future versions: - Optimize the amount of data required for synchronous loading of statistics to improve loading performance [#56812](https://github.com/pingcap/tidb/issues/56812) @[winoros](https://github.com/winoros) - Optimize the execution plan in specific cases where an `OUTER JOIN` involves a unique index and an `ORDER BY ... LIMIT` clause, improving execution efficiency [#56321](https://github.com/pingcap/tidb/issues/56321) @[winoros](https://github.com/winoros) -+ TiKV ++ TiKV - Use a separate thread to clean up replicas, ensuring stable latency for critical paths of Raft reads and writes [#16001](https://github.com/tikv/tikv/issues/16001) @[hbisheng](https://github.com/hbisheng) - Improve the performance of the vector distance function by supporting SIMD [#17290](https://github.com/tikv/tikv/issues/17290) @[EricZequan](https://github.com/EricZequan) -+ PD ++ PD - Support dynamic switching of the `tso` service between microservice and non-microservice modes [#8477](https://github.com/tikv/pd/issues/8477) @[rleungx](https://github.com/rleungx) - Optimize the case format of certain fields in the `pd-ctl config` output [#8694](https://github.com/tikv/pd/issues/8694) @[lhy1024](https://github.com/lhy1024) - [Store limit v2](/configure-store-limit.md#principles-of-store-limit-v2) becomes generally available (GA) [#8865](https://github.com/tikv/pd/issues/8865) @[lhy1024](https://github.com/lhy1024) - Support configuring Region inspection concurrency (experimental) [#8866](https://github.com/tikv/pd/issues/8866) @[lhy1024](https://github.com/lhy1024) -+ TiFlash ++ TiFlash - - (dup): release-7.1.6.md > Improvements> TiFlash - Improve the garbage collection speed of outdated data in the background for tables with clustered indexes [#9529](https://github.com/pingcap/tiflash/issues/9529) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Improve the garbage collection speed of outdated data in the background for tables with clustered indexes [#9529](https://github.com/pingcap/tiflash/issues/9529) @[JaySon-Huang](https://github.com/JaySon-Huang) - Improve query performance of vector search in data update scenarios [#9599](https://github.com/pingcap/tiflash/issues/9599) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - Add monitoring metrics for CPU usage during vector index building [#9032](https://github.com/pingcap/tiflash/issues/9032) @[JaySon-Huang](https://github.com/JaySon-Huang) - Improve the execution efficiency of logical operators [#9146](https://github.com/pingcap/tiflash/issues/9146) @[windtalker](https://github.com/windtalker) + Tools - + Backup & Restore (BR) + + Backup & Restore (BR) - - (dup): release-7.1.6.md > Improvements> Tools> Backup & Restore (BR) - Reduce unnecessary log printing during backup [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) + - Reduce unnecessary log printing during backup [#55902](https://github.com/pingcap/tidb/issues/55902) @[Leavrth](https://github.com/Leavrth) - Optimize the error message for the encryption key `--crypter.key` [#56388](https://github.com/pingcap/tidb/issues/56388) @[Tristan1900](https://github.com/Tristan1900) - Increase concurrency in BR when creating databases to improve data restore performance [#56866](https://github.com/pingcap/tidb/issues/56866) @[Leavrth](https://github.com/Leavrth) - Disable the table-level checksum calculation during full backups by default (`--checksum=false`) to improve backup performance [#56373](https://github.com/pingcap/tidb/issues/56373) @[Tristan1900](https://github.com/Tristan1900) - Add a mechanism to independently track and reset the connection timeout for each storage node, enhancing the handling of slow nodes and preventing backup operations from hanging [#57666](https://github.com/pingcap/tidb/issues/57666) @[3pointer](https://github.com/3pointer) - + TiDB Data Migration (DM) + + TiDB Data Migration (DM) - Add retries for DM-worker to connect to DM-master during DM cluster startup [#4287](https://github.com/pingcap/tiflow/issues/4287) @[GMHDBJD](https://github.com/GMHDBJD) ## Bug fixes -+ TiDB ++ TiDB - Fix the issue that TiDB does not automatically retry requests when the Region metadata returned from PD lacks Leader information, potentially causing execution errors [#56757](https://github.com/pingcap/tidb/issues/56757) @[cfzjywxk](https://github.com/cfzjywxk) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that TTL tasks cannot be canceled when there is a write conflict [#56422](https://github.com/pingcap/tidb/issues/56422) @[YangKeao](https://github.com/YangKeao) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that when canceling a TTL task, the corresponding SQL is not killed forcibly [#56511](https://github.com/pingcap/tidb/issues/56511) @[lcwangchao](https://github.com/lcwangchao) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that existing TTL tasks are executed unexpectedly frequently in a cluster that is upgraded from v6.5 to v7.5 or later [#56539](https://github.com/pingcap/tidb/issues/56539) @[lcwangchao](https://github.com/lcwangchao) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that the `INSERT ... ON DUPLICATE KEY` statement is not compatible with `mysql_insert_id` [#55965](https://github.com/pingcap/tidb/issues/55965) @[tiancaiamao](https://github.com/tiancaiamao) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that TTL might fail if TiKV is not selected as the storage engine [#56402](https://github.com/pingcap/tidb/issues/56402) @[YangKeao](https://github.com/YangKeao) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that the `AUTO_INCREMENT` field is not correctly set after importing data using the `IMPORT INTO` statement [#56476](https://github.com/pingcap/tidb/issues/56476) @[D3Hunter](https://github.com/D3Hunter) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that TiDB does not check the index length limitation when executing `ADD INDEX` [#56930](https://github.com/pingcap/tidb/issues/56930) @[fzzf678](https://github.com/fzzf678) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that executing `RECOVER TABLE BY JOB JOB_ID;` might cause TiDB to panic [#55113](https://github.com/pingcap/tidb/issues/55113) @[crazycs520](https://github.com/crazycs520) - - (dup): release-7.1.6.md > Bug fixes> TiDB - Fix the issue that stale read does not strictly verify the timestamp of the read operation, resulting in a small probability of affecting the consistency of the transaction when an offset exists between the TSO and the real physical time [#56809](https://github.com/pingcap/tidb/issues/56809) @[MyonKeminta](https://github.com/MyonKeminta) + - Fix the issue that TTL tasks cannot be canceled when there is a write conflict [#56422](https://github.com/pingcap/tidb/issues/56422) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that when canceling a TTL task, the corresponding SQL is not killed forcibly [#56511](https://github.com/pingcap/tidb/issues/56511) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that existing TTL tasks are executed unexpectedly frequently in a cluster that is upgraded from v6.5 to v7.5 or later [#56539](https://github.com/pingcap/tidb/issues/56539) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that the `INSERT ... ON DUPLICATE KEY` statement is not compatible with `mysql_insert_id` [#55965](https://github.com/pingcap/tidb/issues/55965) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the issue that TTL might fail if TiKV is not selected as the storage engine [#56402](https://github.com/pingcap/tidb/issues/56402) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the `AUTO_INCREMENT` field is not correctly set after importing data using the `IMPORT INTO` statement [#56476](https://github.com/pingcap/tidb/issues/56476) @[D3Hunter](https://github.com/D3Hunter) + - Fix the issue that TiDB does not check the index length limitation when executing `ADD INDEX` [#56930](https://github.com/pingcap/tidb/issues/56930) @[fzzf678](https://github.com/fzzf678) + - Fix the issue that executing `RECOVER TABLE BY JOB JOB_ID;` might cause TiDB to panic [#55113](https://github.com/pingcap/tidb/issues/55113) @[crazycs520](https://github.com/crazycs520) + - Fix the issue that stale read does not strictly verify the timestamp of the read operation, resulting in a small probability of affecting the consistency of the transaction when an offset exists between the TSO and the real physical time [#56809](https://github.com/pingcap/tidb/issues/56809) @[MyonKeminta](https://github.com/MyonKeminta) - Fix the issue that TiDB could not resume Reorg DDL tasks from the previous progress after the DDL owner node is switched [#56506](https://github.com/pingcap/tidb/issues/56506) @[tangenta](https://github.com/tangenta) - Fix the issue that some metrics in the monitoring panel of Distributed eXecution Framework (DXF) are inaccurate [#57172](https://github.com/pingcap/tidb/issues/57172) @[fzzf678](https://github.com/fzzf678) [#56942](https://github.com/pingcap/tidb/issues/56942) @[fzzf678](https://github.com/fzzf678) - Fix the issue that `REORGANIZE PARTITION` fails to return error reasons in certain cases [#56634](https://github.com/pingcap/tidb/issues/56634) @[mjonss](https://github.com/mjonss) @@ -323,7 +323,7 @@ The following features are planned for deprecation in future versions: - Fix the issue that converting data from the `BIT` type to the `CHAR` type might cause TiKV panics [#56494](https://github.com/pingcap/tidb/issues/56494) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue that using variables or parameters in the `CREATE VIEW` statement does not report errors [#53176](https://github.com/pingcap/tidb/issues/53176) @[mjonss](https://github.com/mjonss) - Fix the issue that the `JSON_VALID()` function returns incorrect results [#56293](https://github.com/pingcap/tidb/issues/56293) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that TTL tasks are not canceled after the `tidb_ttl_job_enable` variable is disabled [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that TTL tasks are not canceled after the `tidb_ttl_job_enable` variable is disabled [#57404](https://github.com/pingcap/tidb/issues/57404) @[YangKeao](https://github.com/YangKeao) - Fix the issue that using the `RANGE COLUMNS` partition function and the `utf8mb4_0900_ai_ci` collation at the same time could result in incorrect query results [#57261](https://github.com/pingcap/tidb/issues/57261) @[Defined2014](https://github.com/Defined2014) - Fix the runtime error caused by executing a prepared statement that begins with a newline character, resulting in an array out of bounds [#54283](https://github.com/pingcap/tidb/issues/54283) @[Defined2014](https://github.com/Defined2014) - Fix the precision issue in the `UTC_TIMESTAMP()` function, such as setting the precision too high [#56451](https://github.com/pingcap/tidb/issues/56451) @[chagelo](https://github.com/chagelo) @@ -346,10 +346,10 @@ The following features are planned for deprecation in future versions: - Fix the issue that the data in the **Stats Healthy Distribution** panel of Grafana might be incorrect [#57176](https://github.com/pingcap/tidb/issues/57176) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that vector search might return incorrect results when querying tables with clustered indexes [#57627](https://github.com/pingcap/tidb/issues/57627) @[winoros](https://github.com/winoros) -+ TiKV ++ TiKV - - (dup): release-7.1.6.md > Bug fixes> TiKV - Fix the panic issue that occurs when read threads access outdated indexes in the MemTable of the Raft Engine [#17383](https://github.com/tikv/tikv/issues/17383) @[LykxSassinator](https://github.com/LykxSassinator) - - (dup): release-7.5.4.md > Bug fixes> TiKV - Fix the issue that when a large number of transactions are queuing for lock release on the same key and the key is frequently updated, excessive pressure on deadlock detection might cause TiKV OOM issues [#17394](https://github.com/tikv/tikv/issues/17394) @[MyonKeminta](https://github.com/MyonKeminta) + - Fix the panic issue that occurs when read threads access outdated indexes in the MemTable of the Raft Engine [#17383](https://github.com/tikv/tikv/issues/17383) @[LykxSassinator](https://github.com/LykxSassinator) + - Fix the issue that when a large number of transactions are queuing for lock release on the same key and the key is frequently updated, excessive pressure on deadlock detection might cause TiKV OOM issues [#17394](https://github.com/tikv/tikv/issues/17394) @[MyonKeminta](https://github.com/MyonKeminta) - Fix the issue that CPU usage for background tasks of resource control is counted twice [#17603](https://github.com/tikv/tikv/issues/17603) @[glorv](https://github.com/glorv) - Fix the issue that TiKV OOM might occur due to the accumulation of CDC internal tasks [#17696](https://github.com/tikv/tikv/issues/17696) @[3AceShowHand](https://github.com/3AceShowHand) - Fix the issue that large batch writes cause performance jitter when `raft-entry-max-size` is set too high [#17701](https://github.com/tikv/tikv/issues/17701) @[SpadeA-Tang](https://github.com/SpadeA-Tang) @@ -357,18 +357,18 @@ The following features are planned for deprecation in future versions: - Fix the issue that TiKV might panic when executing queries containing `RADIANS()` or `DEGREES()` functions [#17852](https://github.com/tikv/tikv/issues/17852) @[gengliqi](https://github.com/gengliqi) - Fix the issue that write jitter might occur when all hibernated Regions are awakened [#17101](https://github.com/tikv/tikv/issues/17101) @[hhwyt](https://github.com/hhwyt) -+ PD ++ PD - - (dup): release-7.1.6.md > Bug fixes> PD - Fix the memory leak issue in hotspot cache [#8698](https://github.com/tikv/pd/issues/8698) @[lhy1024](https://github.com/lhy1024) + - Fix the memory leak issue in hotspot cache [#8698](https://github.com/tikv/pd/issues/8698) @[lhy1024](https://github.com/lhy1024) - Fix the issue that the resource group selector does not take effect on any panel [#56572](https://github.com/pingcap/tidb/issues/56572) @[glorv](https://github.com/glorv) - Fix the issue that deleted resource groups still appear in the monitoring panel [#8716](https://github.com/tikv/pd/issues/8716) @[AndreMouche](https://github.com/AndreMouche) - Fix unclear log descriptions during the Region syncer loading process [#8717](https://github.com/tikv/pd/issues/8717) @[lhy1024](https://github.com/lhy1024) - - (dup): release-7.1.6.md > Bug fixes> PD - Fix the memory leak issue in label statistics [#8700](https://github.com/tikv/pd/issues/8700) @[lhy1024](https://github.com/lhy1024) + - Fix the memory leak issue in label statistics [#8700](https://github.com/tikv/pd/issues/8700) @[lhy1024](https://github.com/lhy1024) - Fix the issue that configuring `tidb_enable_tso_follower_proxy` to `0` or `OFF` fails to disable the TSO Follower Proxy feature [#8709](https://github.com/tikv/pd/issues/8709) @[JmPotato](https://github.com/JmPotato) -+ TiFlash ++ TiFlash - - (dup): release-7.1.6.md > Bug fixes> TiFlash - Fix the issue that the `SUBSTRING()` function does not support the `pos` and `len` arguments for certain integer types, causing query errors [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) + - Fix the issue that the `SUBSTRING()` function does not support the `pos` and `len` arguments for certain integer types, causing query errors [#9473](https://github.com/pingcap/tiflash/issues/9473) @[gengliqi](https://github.com/gengliqi) - Fix the issue that vector search performance might degrade after scaling out TiFlash write nodes in the disaggregated storage and compute architecture [#9637](https://github.com/pingcap/tiflash/issues/9637) @[kolafish](https://github.com/kolafish) - Fix the issue that the `SUBSTRING()` function returns incorrect results when the second parameter is negative [#9604](https://github.com/pingcap/tiflash/issues/9604) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that the `REPLACE()` function returns an error when the first parameter is a constant [#9522](https://github.com/pingcap/tiflash/issues/9522) @[guo-shaoge](https://github.com/guo-shaoge) @@ -376,7 +376,7 @@ The following features are planned for deprecation in future versions: + Tools - + Backup & Restore (BR) + + Backup & Restore (BR) - Fix the OOM issue during backups when there are too many uncompleted range gaps, reducing the amount of pre-allocated memory [#53529](https://github.com/pingcap/tidb/issues/53529) @[Leavrth](https://github.com/Leavrth) - Fix the issue that global indexes cannot be backed up [#57469](https://github.com/pingcap/tidb/issues/57469) @[Defined2014](https://github.com/Defined2014) @@ -385,15 +385,15 @@ The following features are planned for deprecation in future versions: - Fix potential security vulnerabilities by upgrading the `k8s.io/api` library version [#57790](https://github.com/pingcap/tidb/issues/57790) @[BornChanger](https://github.com/BornChanger) - Fix the issue that PITR tasks might return the `Information schema is out of date` error when there are a large number of tables in the cluster but the actual data size is small [#57743](https://github.com/pingcap/tidb/issues/57743) @[Tristan1900](https://github.com/Tristan1900) - + TiCDC + + TiCDC - Fix the issue that the Kafka messages lack Key fields when using the Debezium protocol [#1799](https://github.com/pingcap/tiflow/issues/1799) @[wk989898](https://github.com/wk989898) - Fix the issue that the redo module fails to properly report errors [#11744](https://github.com/pingcap/tiflow/issues/11744) @[CharlesCheung96](https://github.com/CharlesCheung96) - Fix the issue that TiCDC mistakenly discards DDL tasks when the schema versions of DDL tasks become non-incremental during TiDB DDL owner changes [#11714](https://github.com/pingcap/tiflow/issues/11714) @[wlwilliamx](https://github.com/wlwilliamx) - + TiDB Lightning + + TiDB Lightning - - (dup): release-7.1.6.md > Bug fixes> Tools> TiDB Lightning - Fix the issue that TiDB Lightning fails to receive oversized messages sent from TiKV [#56114](https://github.com/pingcap/tidb/issues/56114) @[fishiu](https://github.com/fishiu) + - Fix the issue that TiDB Lightning fails to receive oversized messages sent from TiKV [#56114](https://github.com/pingcap/tidb/issues/56114) @[fishiu](https://github.com/fishiu) - Fix the issue that the `AUTO_INCREMENT` value is set too high after importing data using the physical import mode [#56814](https://github.com/pingcap/tidb/issues/56814) @[D3Hunter](https://github.com/D3Hunter) ## Contributors From bb7c2526a5d683e61f95f2ddf58f20b0aa474ccb Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 10 Dec 2024 15:10:32 +0800 Subject: [PATCH 75/89] add patrol-region-worker-count --- releases/release-8.5.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 0462d55164c70..eb3e3a4f923ca 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -227,6 +227,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | -------- | -------- | -------- | -------- | | TiDB | [`deprecate-integer-display-length`](/tidb-configuration-file.md#deprecate-integer-display-length) | Modified | Starting from v8.5.0, the integer display width feature is deprecated. The default value of this configuration item is changed from `false` to `true`. | | TiKV | [`raft-client-queue-size`](/tikv-configuration-file.md#raft-client-queue-size) | Modified | Changes the default value from `8192` to `16384`. | +| PD | [`patrol-region-worker-count`](/pd-configuration-file.md#patrol-region-worker-count-new-in-v850) | Newly added | Controls the number of concurrent [operators](/glossary.md#operator) created by the checker when inspecting the health state of a Region. | | BR | [`--checksum`](/br/use-br-command-line-tool.md) | Modified | Changes the default value from `true` to `false`, meaning that BR does not calculate the table-level checksum during full backups by default, to improve backup performance. | ## Deprecated features From 418483f04b09f1364b282b1dea211c7b504687cb Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 10 Dec 2024 15:30:14 +0800 Subject: [PATCH 76/89] Update releases/release-8.5.0.md Co-authored-by: Aolin --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index eb3e3a4f923ca..4d40ccae0c683 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -141,7 +141,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, In v8.5.0, TiDB accelerated table creation becomes generally available (GA) and is enabled by default. During data migration and cluster initialization, this feature supports rapid creation of millions of tables, significantly reducing operation time. - For more information, see [Documentation](/accelerated-table-creation.md). + For more information, see [documentation](/accelerated-table-creation.md). * TiKV supports the MVCC in-memory engine (IME), which accelerates queries involving scans of extensive MVCC historical versions [#16141](https://github.com/tikv/tikv/issues/16141) [@SpadeA-Tang](https://github.com/SpadeA-Tang) [@glorv](https://github.com/glorv) [@overvenus](https://github.com/overvenus) From 917998357d328d7ff7e63c237ef3e35174f6d04d Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 10 Dec 2024 17:23:23 +0800 Subject: [PATCH 77/89] update PD highlights --- releases/release-8.5.0.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 4d40ccae0c683..022ba1ce551fb 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -61,7 +61,12 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, - + From d42c870de256f322d038f744f448b184df728942 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 10 Dec 2024 17:24:21 +0800 Subject: [PATCH 78/89] update the link for --checksum --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 022ba1ce551fb..b5622185d2df1 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -233,7 +233,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | TiDB | [`deprecate-integer-display-length`](/tidb-configuration-file.md#deprecate-integer-display-length) | Modified | Starting from v8.5.0, the integer display width feature is deprecated. The default value of this configuration item is changed from `false` to `true`. | | TiKV | [`raft-client-queue-size`](/tikv-configuration-file.md#raft-client-queue-size) | Modified | Changes the default value from `8192` to `16384`. | | PD | [`patrol-region-worker-count`](/pd-configuration-file.md#patrol-region-worker-count-new-in-v850) | Newly added | Controls the number of concurrent [operators](/glossary.md#operator) created by the checker when inspecting the health state of a Region. | -| BR | [`--checksum`](/br/use-br-command-line-tool.md) | Modified | Changes the default value from `true` to `false`, meaning that BR does not calculate the table-level checksum during full backups by default, to improve backup performance. | +| BR | [`--checksum`](/br/br-snapshot-manual.md) | Modified | Changes the default value from `true` to `false`, meaning that BR does not calculate the table-level checksum during full backups by default, to improve backup performance. | ## Deprecated features From def98be437d2c92b77aea39a7db27763d126e54b Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 11 Dec 2024 10:31:37 +0800 Subject: [PATCH 79/89] add the performance test section --- releases/release-8.5.0.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index b5622185d2df1..59088854810c8 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -402,6 +402,10 @@ The following features are planned for deprecation in future versions: - Fix the issue that TiDB Lightning fails to receive oversized messages sent from TiKV [#56114](https://github.com/pingcap/tidb/issues/56114) @[fishiu](https://github.com/fishiu) - Fix the issue that the `AUTO_INCREMENT` value is set too high after importing data using the physical import mode [#56814](https://github.com/pingcap/tidb/issues/56814) @[D3Hunter](https://github.com/D3Hunter) +## Performance test + +To learn about the performance of TiDB v8.5.0, you can refer to the [TPC-C performance test report](https://docs.pingcap.com/tidbcloud/v8.5-performance-benchmarking-with-tpcc) and [Sysbench performance test report](https://docs.pingcap.com/tidbcloud/v8.5-performance-benchmarking-with-sysbench) of the TiDB Cloud Dedicated cluster. + ## Contributors We would like to thank the following contributors from the TiDB community: From 372511cbbcbb3b1e4434d82ce5aa036c0e12c5e9 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 11 Dec 2024 11:47:14 +0800 Subject: [PATCH 80/89] Apply suggestions from code review Co-authored-by: Aolin --- releases/release-8.5.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 59088854810c8..29334acc3cb16 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -30,12 +30,12 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, @@ -65,7 +65,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements,
  • Schema cache control and setting the memory quota for the TiDB statistics cache are generally available (GA), reducing stability issues caused by excessive memory consumption.
  • PD introduces Active Follower to handle the pressure brought by numerous Regions, gradually decouples the services handled by PD for independent deployment.
  • PD improves the performance of Region heartbeat processing and supports tens of millions of Regions for a single cluster.
    • -
  • You can increase concurrency and reduce the number of collected objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans in large clusters.
    • +
  • You can increase concurrency and reduce the number of collected objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans in large clusters.
    • @@ -212,7 +212,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, > **Note:** > -> This section provides compatibility changes you need to know when you upgrade from v8.1.0 to the current version (v8.2.0). If you are upgrading from v8.0.0 or earlier versions to the current version, you might also need to check the compatibility changes introduced in intermediate versions. +> This section provides compatibility changes you need to know when you upgrade from v8.4.0 to the current version (v8.5.0). If you are upgrading from v8.3.0 or earlier versions to the current version, you might also need to check the compatibility changes introduced in intermediate versions. ### Behavior changes From 2770381e08d762795e2e1d0d6dd6250eae56bd98 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 12 Dec 2024 09:42:18 +0800 Subject: [PATCH 81/89] add TOC changes --- TOC.md | 4 +++- releases/release-notes.md | 6 +++++- releases/release-timeline.md | 1 + 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/TOC.md b/TOC.md index 232b373f3d6f8..e3604742d6c3e 100644 --- a/TOC.md +++ b/TOC.md @@ -4,7 +4,7 @@ - [Docs Home](https://docs.pingcap.com/) - About TiDB Self-Managed - [What is TiDB Self-Managed](/overview.md) - - [TiDB 8.4 Release Notes](/releases/release-8.4.0.md) + - [TiDB 8.5 Release Notes](/releases/release-8.5.0.md) - [Features](/basic-features.md) - [MySQL Compatibility](/mysql-compatibility.md) - [TiDB Limitations](/tidb-limitations.md) @@ -1075,6 +1075,8 @@ - [Release Timeline](/releases/release-timeline.md) - [TiDB Versioning](/releases/versioning.md) - [TiDB Installation Packages](/binary-package.md) + - v8.5 + - [8.5.0](/releases/release-8.5.0.md) - v8.4 - [8.4.0-DMR](/releases/release-8.4.0.md) - v8.3 diff --git a/releases/release-notes.md b/releases/release-notes.md index eeb53d72bd12c..3136e11b5968b 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -1,13 +1,17 @@ --- title: Release Notes aliases: ['/docs/dev/releases/release-notes/','/docs/dev/releases/rn/'] -summary: TiDB has released multiple versions, including 8.3.0-DMR, 8.2.0-DMR, 8.1.0, 8.0.0-DMR, 7.6.0-DMR, 7.5.1, 7.5.0, 7.4.0-DMR, 7.3.0-DMR, 7.2.0-DMR, 7.1.4, 7.1.3, 7.1.2, 7.1.1, 7.1.0, 7.0.0-DMR, 6.6.0-DMR, 6.5.9, 6.5.8, 6.5.7, 6.5.6, 6.5.5, 6.5.4, 6.5.3, 6.5.2, 6.5.1, 6.5.0, 6.4.0-DMR, 6.3.0-DMR, 6.2.0-DMR, 6.1.7, 6.1.6, 6.1.5, 6.1.4, 6.1.3, 6.1.2, 6.1.1, 6.1.0, 6.0.0-DMR, 5.4.3, 5.4.2, 5.4.1, 5.4.0, 5.3.4, 5.3.3, 5.3.2, 5.3.1, 5.3.0, 5.2.4, 5.2.3, 5.2.2, 5.2.1, 5.2.0, 5.1.5, 5.1.4, 5.1.3, 5.1.2, 5.1.1, 5.1.0, 5.0.6, 5.0.5, 5.0.4, 5.0.3, 5.0.2, 5.0.1, 5.0.0, 5.0.0-rc, 4.0.16, 4.0.15, 4.0.14, 4.0.13, 4.0.12, 4.0.11, 4.0.10, 4.0.9, 4.0.8, 4.0.7, 4.0.6, 4.0.5, 4.0.4, 4.0.3, 4.0.2, 4.0.1, 4.0.0, 4.0.0-rc.2, 4.0.0-rc.1, 4.0.0-rc, 4.0.0-beta.2, 4.0.0-beta.1, 4.0.0-beta, 3.1.2, 3.1.1, 3.1.0, 3.1.0-rc, 3.1.0-beta.2, 3.1.0-beta.1, 3.1.0-beta, 3.0.20, 3.0.19, 3.0.18, 3.0.17, 3.0.16, 3.0.15, 3.0.14, 3.0.13, 3.0.12, 3.0.11, 3.0.10, 3.0.9, 3.0.8, 3.0.7, 3.0.6, 3.0.5, 3.0.4, 3.0.3, 3.0.2, 3.0.1, 3.0.0, 3.0.0-rc.3, 3.0.0-rc.2, 3.0.0-rc.1, 3.0.0-beta.1, 3.0.0-beta, 2.1.19, 2.1.18, 2.1.17, 2.1.16, 2.1.15, 2.1.14, 2.1.13, 2.1.12, 2.1.11, 2.1.10, 2.1.9, 2.1.8, 2.1.7, 2.1.6, 2.1.5, 2.1.4, 2.1.3, 2.1.2, 2.1.1, 2.1.0, 2.1.0-rc.5, 2.1.0-rc.4, 2.1.0-rc.3, 2.1.0-rc.2, 2.1.0-rc.1, 2.1.0-beta, 2.0.11, 2.0.10, 2.0.9, 2.0.8, 2.0.7, 2.0.6, 2.0.5, 2.0.4, 2.0.3, 2.0.2, 2.0.1, 2.0.0, 2.0.0-rc.5, 2.0.0-rc.4, 2.0.0-rc.3, 2.0.0-rc.1, 1.1.0-beta, 1.1.0-alpha, 1.0.8, 1.0.7, 1.0.6, 1.0.5, 1.0.4, 1.0.3, 1.0.2, 1.0.1, 1.0.0, Pre-GA, rc4, rc3, rc2, rc1. +summary: TiDB has released multiple versions, including 8.5.0, 8.4.0-DMR, 8.3.0-DMR, 8.2.0-DMR, 8.1.0, 8.0.0-DMR, 7.6.0-DMR, 7.5.1, 7.5.0, 7.4.0-DMR, 7.3.0-DMR, 7.2.0-DMR, 7.1.4, 7.1.3, 7.1.2, 7.1.1, 7.1.0, 7.0.0-DMR, 6.6.0-DMR, 6.5.9, 6.5.8, 6.5.7, 6.5.6, 6.5.5, 6.5.4, 6.5.3, 6.5.2, 6.5.1, 6.5.0, 6.4.0-DMR, 6.3.0-DMR, 6.2.0-DMR, 6.1.7, 6.1.6, 6.1.5, 6.1.4, 6.1.3, 6.1.2, 6.1.1, 6.1.0, 6.0.0-DMR, 5.4.3, 5.4.2, 5.4.1, 5.4.0, 5.3.4, 5.3.3, 5.3.2, 5.3.1, 5.3.0, 5.2.4, 5.2.3, 5.2.2, 5.2.1, 5.2.0, 5.1.5, 5.1.4, 5.1.3, 5.1.2, 5.1.1, 5.1.0, 5.0.6, 5.0.5, 5.0.4, 5.0.3, 5.0.2, 5.0.1, 5.0.0, 5.0.0-rc, 4.0.16, 4.0.15, 4.0.14, 4.0.13, 4.0.12, 4.0.11, 4.0.10, 4.0.9, 4.0.8, 4.0.7, 4.0.6, 4.0.5, 4.0.4, 4.0.3, 4.0.2, 4.0.1, 4.0.0, 4.0.0-rc.2, 4.0.0-rc.1, 4.0.0-rc, 4.0.0-beta.2, 4.0.0-beta.1, 4.0.0-beta, 3.1.2, 3.1.1, 3.1.0, 3.1.0-rc, 3.1.0-beta.2, 3.1.0-beta.1, 3.1.0-beta, 3.0.20, 3.0.19, 3.0.18, 3.0.17, 3.0.16, 3.0.15, 3.0.14, 3.0.13, 3.0.12, 3.0.11, 3.0.10, 3.0.9, 3.0.8, 3.0.7, 3.0.6, 3.0.5, 3.0.4, 3.0.3, 3.0.2, 3.0.1, 3.0.0, 3.0.0-rc.3, 3.0.0-rc.2, 3.0.0-rc.1, 3.0.0-beta.1, 3.0.0-beta, 2.1.19, 2.1.18, 2.1.17, 2.1.16, 2.1.15, 2.1.14, 2.1.13, 2.1.12, 2.1.11, 2.1.10, 2.1.9, 2.1.8, 2.1.7, 2.1.6, 2.1.5, 2.1.4, 2.1.3, 2.1.2, 2.1.1, 2.1.0, 2.1.0-rc.5, 2.1.0-rc.4, 2.1.0-rc.3, 2.1.0-rc.2, 2.1.0-rc.1, 2.1.0-beta, 2.0.11, 2.0.10, 2.0.9, 2.0.8, 2.0.7, 2.0.6, 2.0.5, 2.0.4, 2.0.3, 2.0.2, 2.0.1, 2.0.0, 2.0.0-rc.5, 2.0.0-rc.4, 2.0.0-rc.3, 2.0.0-rc.1, 1.1.0-beta, 1.1.0-alpha, 1.0.8, 1.0.7, 1.0.6, 1.0.5, 1.0.4, 1.0.3, 1.0.2, 1.0.1, 1.0.0, Pre-GA, rc4, rc3, rc2, rc1. --- # TiDB Release Notes +## 8.5 + +- [8.5.0-DMR](/releases/release-8.4.0.md): 2024-12-xx + ## 8.4 - [8.4.0-DMR](/releases/release-8.4.0.md): 2024-11-11 diff --git a/releases/release-timeline.md b/releases/release-timeline.md index 5984664b488ac..5ebff4cf7aeb1 100644 --- a/releases/release-timeline.md +++ b/releases/release-timeline.md @@ -11,6 +11,7 @@ This document shows all the released TiDB versions in reverse chronological orde | Version | Release Date | | :--- | :--- | +| [8.5.0](/releases/release-8.5.0.md) | 2024-12-xx | | [7.1.6](/releases/release-7.1.6.md) | 2024-11-21 | | [8.4.0-DMR](/releases/release-8.4.0.md) | 2024-11-11 | | [7.5.4](/releases/release-7.5.4.md) | 2024-10-15 | From f69da3d60f8caa2685e81380a7f424860285aab9 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 12 Dec 2024 09:43:17 +0800 Subject: [PATCH 82/89] Update releases/release-notes.md --- releases/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-notes.md b/releases/release-notes.md index 3136e11b5968b..6391aa4d5c3dc 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -10,7 +10,7 @@ summary: TiDB has released multiple versions, including 8.5.0, 8.4.0-DMR, 8.3.0- ## 8.5 -- [8.5.0-DMR](/releases/release-8.4.0.md): 2024-12-xx +- [8.5.0](/releases/release-8.4.0.md): 2024-12-xx ## 8.4 From 753ec542e0b1f339c508a8275324c208fb312abe Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 12 Dec 2024 09:43:35 +0800 Subject: [PATCH 83/89] Update releases/release-notes.md --- releases/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-notes.md b/releases/release-notes.md index 6391aa4d5c3dc..ee4e56991f4c8 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -10,7 +10,7 @@ summary: TiDB has released multiple versions, including 8.5.0, 8.4.0-DMR, 8.3.0- ## 8.5 -- [8.5.0](/releases/release-8.4.0.md): 2024-12-xx +- [8.5.0](/releases/release-8.5.0.md): 2024-12-xx ## 8.4 From 3f3ca075dcc22b3d2bca203804a34ed69994054c Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 12 Dec 2024 09:45:09 +0800 Subject: [PATCH 84/89] Update releases/release-8.5.0.md --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 29334acc3cb16..a676309ba9585 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -7,7 +7,7 @@ summary: Learn about the new features, compatibility changes, improvements, and -Release date: xx xx, 2024 +Release date: December xx, 2024 TiDB version: 8.5.0 From 3ed9110149609713d26064a36fcdec82b45acf60 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 12 Dec 2024 10:54:27 +0800 Subject: [PATCH 85/89] add "Removed features" according to v8.4.0 release notes --- releases/release-8.5.0.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index a676309ba9585..fb8d85290d83e 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -235,6 +235,16 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | PD | [`patrol-region-worker-count`](/pd-configuration-file.md#patrol-region-worker-count-new-in-v850) | Newly added | Controls the number of concurrent [operators](/glossary.md#operator) created by the checker when inspecting the health state of a Region. | | BR | [`--checksum`](/br/br-snapshot-manual.md) | Modified | Changes the default value from `true` to `false`, meaning that BR does not calculate the table-level checksum during full backups by default, to improve backup performance. | +## Removed features + +* The following features are removed starting from v8.4.0: + + * In v8.4.0, [TiDB Binlog](https://docs.pingcap.com/tidb/v8.3/tidb-binlog-overview) is removed. Starting from v8.3.0, TiDB Binlog is fully deprecated. For incremental data replication, use [TiCDC](/ticdc/ticdc-overview.md) instead. For point-in-time recovery (PITR), use [PITR](/br/br-pitr-guide.md). Before you upgrade your TiDB cluster to v8.4.0 or later versions, be sure to switch to TiCDC and PITR. + +* The following features are planned for removal in future versions: + + * Starting from v8.0.0, TiDB Lightning deprecates the [old version of conflict detection](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#the-old-version-of-conflict-detection-deprecated-in-v800) strategy for the physical import mode, and enables you to control the conflict detection strategy for both logical and physical import modes via the [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) parameter. The [`duplicate-resolution`](/tidb-lightning/tidb-lightning-configuration.md) parameter for the old version of conflict detection will be removed in a future release. + ## Deprecated features The following features are planned for deprecation in future versions: From 33a534d332771b071a5e3a2f4281908919b85222 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 12 Dec 2024 11:06:44 +0800 Subject: [PATCH 86/89] Update wording --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index fb8d85290d83e..63c1d7e0d0d8b 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -237,7 +237,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, ## Removed features -* The following features are removed starting from v8.4.0: +* The following feature has been removed: * In v8.4.0, [TiDB Binlog](https://docs.pingcap.com/tidb/v8.3/tidb-binlog-overview) is removed. Starting from v8.3.0, TiDB Binlog is fully deprecated. For incremental data replication, use [TiCDC](/ticdc/ticdc-overview.md) instead. For point-in-time recovery (PITR), use [PITR](/br/br-pitr-guide.md). Before you upgrade your TiDB cluster to v8.4.0 or later versions, be sure to switch to TiCDC and PITR. From 2045fc842e2f64b9cabc9068012532cb55ef3d79 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 12 Dec 2024 11:39:19 +0800 Subject: [PATCH 87/89] update PDF name --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 63c1d7e0d0d8b..5e8156201da82 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -15,7 +15,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.5/quick-start-with- TiDB 8.5.0 is a Long-Term Support Release (LTS). -Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, and bug fixes released in [8.2.0-DMR](/releases/release-8.2.0.md), [8.3.0-DMR](/releases/release-8.3.0.md), and [8.4.0-DMR](/releases/release-8.4.0.md). When you upgrade from 8.1.x to 8.5.0, you can download the [TiDB Release Notes PDF](https://download.pingcap.org/tidb-v8.1-to-v8.5-en-release-notes.pdf) to view all release notes between the two LTS versions. The following table lists some highlights from 8.1.0 to 8.5.0: +Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, and bug fixes released in [8.2.0-DMR](/releases/release-8.2.0.md), [8.3.0-DMR](/releases/release-8.3.0.md), and [8.4.0-DMR](/releases/release-8.4.0.md). When you upgrade from 8.1.x to 8.5.0, you can download the [TiDB Release Notes PDF](https://download.pingcap.org/tidb-v8.2-to-v8.5-en-release-notes.pdf) to view all release notes between the two LTS versions. The following table lists some highlights from 8.1.0 to 8.5.0:
    分类功能描述CategoryFeature/EnhancementDescription
    Support backing up TiKV instances with IMDSv2 service enabled (introduced in v8.4.0) <AWS EC2 now uses IMDSv2 as the default metadata service. TiDB supports backing up data from TiKV instances that have IMDSv2 enabled, helping you run TiDB clusters more effectively in public cloud services.AWS EC2 now uses IMDSv2 as the default metadata service. TiDB supports backing up data from TiKV instances that have IMDSv2 enabled, helping you run TiDB clusters more effectively in public cloud services.
    SecurityClient-side encryption of snapshot backup data和log backup data (GA in v8.5.0)**tw@qiancai 1998**Client-side encryption of snapshot backup data and log backup data (GA in v8.5.0)**tw@qiancai 1998** Before uploading backup data to your backup storage, you can encrypt the backup data to ensure its security during storage and transmission.
    可扩展性与性能 Reduce data processing latency in multiple dimensions **tw@qiancai** TiDB continuously refines data processing to enhance performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include: -
  • Support parallel sorting (introduced in v8.2.0)
    • -
  • Optimize batch processing strategy for KV (key-value) requests (introduced in v8.3.0)
    • -
  • Support parallel mode for TSO requests (introduced in v8.4.0)
    • +
  • Support parallel sorting (introduced in v8.2.0)
    • +
  • Optimize batch processing strategy for KV (key-value) requests (introduced in v8.3.0)
    • +
  • Support parallel mode for TSO requests (introduced in v8.4.0)
  • Reduce the resource overhead of DELETE operations (introduced in v8.4.0)
  • Improve query performance for cached tables (introduced in v8.4.0)
    • -
  • Introduce an optimized version of Hash Join (introduced in v8.4.0)
    • +
  • Introduce an optimized version of Hash Join (introduced in v8.4.0)
    • SQL 外键约束成为正式功能 **tw@lilin90 1894**外键（Foreign Key）是数据库中的一种约束，用于建立表与表之间的关联关系，确保数据一致性和完整性。它可以限制子表中引用的数据必须存在于主表中，防止无效数据插入。同时，外键支持级联操作（如删除或更新时自动同步），简化了业务逻辑的实现，减少了手动维护数据关联的复杂性。Foreign key (GA in v8.5.0) **tw@lilin90 1894**Foreign keys are constraints in a database that establish relationships between tables, ensuring data consistency and integrity. They restrict the data referenced in a child table to exist in the parent table, preventing the insertion of invalid data. Foreign keys also support cascading operations (such as automatic synchronization during deletion or update), simplifying business logic implementation and reducing the complexity of manually maintaining data relationships.
    Vector search (experimental, introduced in v8.4.0)
    SQL Foreign key (GA in v8.5.0) **tw@lilin90 1894**Foreign key (GA in v8.5.0) **tw@lilin90 1894** Foreign keys are constraints in a database that establish relationships between tables, ensuring data consistency and integrity. They restrict the data referenced in a child table to exist in the parent table, preventing the insertion of invalid data. Foreign keys also support cascading operations (such as automatic synchronization during deletion or update), simplifying business logic implementation and reducing the complexity of manually maintaining data relationships.
    可扩展性与性能Scalability and Performance Reduce data processing latency in multiple dimensions **tw@qiancai** TiDB continuously refines data processing to enhance performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include:
  • Support parallel sorting (introduced in v8.2.0)
    • From 2b4100413baaeaba9131c9ee6b94d4f9bcee7523 Mon Sep 17 00:00:00 2001 From: lilin90 Date: Mon, 25 Nov 2024 17:14:54 +0800 Subject: [PATCH 30/89] Add change of TiKV config raft-client-queue-size --- releases/release-8.5.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 87817d9758c2a..1172753f8e961 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -261,7 +261,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, | Configuration file or component | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | -| | | | | +| TiKV | [`raft-client-queue-size`](/tikv-configuration-file.md#raft-client-queue-size) | Modified | Changes the default value from `8192` to `16384`. | | | | | | | | | | | From 94712e27445e7805508e55fdfe0fed465d9d91d9 Mon Sep 17 00:00:00 2001 From: lilin90 Date: Mon, 25 Nov 2024 17:31:34 +0800 Subject: [PATCH 31/89] Update deprecation --- releases/release-8.5.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 1172753f8e961..c313d5a2eb422 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -281,6 +281,7 @@ The following features are planned for deprecation in future versions: * TiDB introduces the system variable [`tidb_enable_async_merge_global_stats`](/system-variables.md#tidb_enable_async_merge_global_stats-new-in-v750) in v7.5.0. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated. * It is planned to redesign [the automatic evolution of execution plan bindings](/sql-plan-management.md#baseline-evolution) in subsequent releases, and the related variables and behavior will change. * In v8.0.0, TiDB introduces the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable will be deprecated. +* In v5.1, TiDB introduces the system variable [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-new-in-v51) to set whether to enable the dynamic pruning mode for partitioned tables. Starting from v8.5.0, a warning is returned when you set this variable to `static` or `static-only`. In future versions, this system variable will be deprecated. * The TiDB Lightning parameter [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) is planned for deprecation in a future release and will be subsequently removed. This parameter will be replaced by [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task), which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task. * Starting from v6.3.0, partitioned tables use [dynamic pruning mode](/partitioned-table.md#dynamic-pruning-mode) by default. Compared with static pruning mode, dynamic pruning mode supports features such as IndexJoin and plan cache with better performance. Therefore, static pruning mode will be deprecated. From df04b24f57502e6e5ee4fcacfb90f23e03e9ec9c Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 25 Nov 2024 17:32:51 +0800 Subject: [PATCH 32/89] Apply suggestions from code review --- releases/release-8.5.0.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index c313d5a2eb422..120d3cf0fa46b 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -60,8 +60,8 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements,
    Reliability and availability提升超大规模集群的稳定性 **tw@hfxsd 1976**对于使用 TiDB 运行多租户应用或者 SaaS 应用的公司，经常需要存储大量的表，TiDB 在 v8.5.0 着力增强了大规模集群的稳定性。 Schema 缓存控制以及Stats 缓存控制已经成为正式功能，减少了内存过度消耗带来的稳定性问题。 PD 通过 Active Follower 应对大量 Region 带来的压力，并将 PD 所承担的服务逐步解耦，独立部署。通过增加并发度，以及减少收集对象的数量，统计信息收集和加载效率得到提升，保证了大集群执行计划的稳定性。Improve the stability of large-scale clusters **tw@hfxsd 1976**Companies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In TiDB v8.5.0, significant efforts have been made to enhance the stability of large-scale clusters. Schema cache control and the memory quota for the TiDB statistics cache become generally available (GA). They reduce stability problems caused by excessive memory consumption. PD uses Active Follower to address the pressures of a large number of Regions. It also decouples the services undertaken by the PD and deploys them independently. By increasing concurrency, and reducing the number of collection objects, the efficiency of statistics collection and loading is improved, ensuring the stability of execution plans for large clusters.
    Support more triggers for runaway queries, and support switching resource groups (introduced in v8.4.0)
    Reliability and availability Improve the stability of large-scale clusters **tw@hfxsd 1976**Companies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In TiDB v8.5.0, significant efforts have been made to enhance the stability of large-scale clusters. Schema cache control and the memory quota for the TiDB statistics cache become generally available (GA). They reduce stability problems caused by excessive memory consumption. PD uses Active Follower to address the pressures of a large number of Regions. It also decouples the services undertaken by the PD and deploys them independently. By increasing concurrency, and reducing the number of collection objects, the efficiency of statistics collection and loading is improved, ensuring the stability of execution plans for large clusters.Companies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In TiDB v8.5.0, significant efforts have been made to enhance the stability of large-scale clusters. Schema cache control and the memory quota for the TiDB statistics cache become generally available (GA). They reduce stability problems caused by excessive memory consumption. PD uses Active Follower to address the pressures of a large number of Regions. It also decouples the services undertaken by the PD and deploys them independently. You can increase concurrency and reduce the number of collection objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans for large clusters.
    Support more triggers for runaway queries, and support switching resource groups (introduced in v8.4.0)
    Scalability and Performance Reduce data processing latency in multiple dimensions **tw@qiancai**TiDB continuously refines data processing to enhance performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include: + Reduce data processing latency in multiple dimensions **tw@qiancai**TiDB continuously refines data processing to improve performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include:
  • Support parallel sorting (introduced in v8.2.0)
  • Optimize batch processing strategy for KV (key-value) requests (introduced in v8.3.0)
    • -
  • Support parallel mode for TSO requests (introduced in v8.4.0)
    • -
  • Reduce the resource overhead of DELETE operations (introduced in v8.4.0)
    • +
  • Support parallel mode for TSO requests (introduced in v8.4.0)
    • +
  • Reduce the resource overhead of DELETE operations (introduced in v8.4.0)
  • Improve query performance for cached tables (introduced in v8.4.0)
  • Introduce an optimized version of Hash Join (introduced in v8.4.0)
    • SQL Foreign key (GA in v8.5.0) **tw@lilin90 1894**Foreign keys are constraints in a database that establish relationships between tables, ensuring data consistency and integrity. They restrict the data referenced in a child table to exist in the parent table, preventing the insertion of invalid data. Foreign keys also support cascading operations (such as automatic synchronization during deletion or update), simplifying business logic implementation and reducing the complexity of manually maintaining data relationships.Foreign keys are constraints in a database that establish relationships between tables, ensuring data consistency and integrity. They ensure that the data referenced in a child table exist in the parent table, preventing the insertion of invalid data. Foreign keys also support cascading operations (such as automatic synchronization during deletion or update), simplifying business logic implementation and reducing the complexity of manually maintaining data relationships.
    Vector search (experimental, introduced in v8.4.0) Enhance and expand TiProxy use cases **tw@Oreoxmt** As a crucial component of the high availability of TiDB, TiProxy extends its capabilities beyond SQL traffic access and forwarding to support cluster change evaluation. Key features include:
  • TiProxy supports traffic capture and replay (experimental, introduced in v8.4.0)
    • -
  • Built-in virtual IP management in TiProxy (introduced in v8.3.0)
    • +
  • TiProxy supports built-in virtual IP management (introduced in v8.3.0)
  • TiProxy supports multiple load balancing policies (introduced in v8.2.0)
    		•
    Reliability and availability Improve the stability of large-scale clusters **tw@hfxsd 1976**Companies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In TiDB v8.5.0, significant efforts have been made to enhance the stability of large-scale clusters. Schema cache control and the memory quota for the TiDB statistics cache become generally available (GA). They reduce stability problems caused by excessive memory consumption. PD uses Active Follower to address the pressures of a large number of Regions. It also decouples the services undertaken by the PD and deploys them independently. You can increase concurrency and reduce the number of collection objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans for large clusters.Companies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In v8.5.0, TiDB significantly enhances the stability of large-scale clusters. Schema cache control and setting the memory quota for the TiDB statistics cache are generally available (GA), reducing stability issues caused by excessive memory consumption. PD introduces Active Follower to handle the pressure brought by numerous Regions, and gradually decouples the services handled by PD for independent deployment. You can increase concurrency and reduce the number of collected objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans in large clusters.
    Support more triggers for runaway queries, and support switching resource groups (introduced in v8.4.0)
    Global indexes for partitioned tables (GA in v8.4.0)Global indexes can effectively improve the efficiency of retrieving non-partitioned columns, and remove the restriction that a unique key must contain the partition key. This feature extends the usage scenarios of TiDB partitioned tables, and avoids some of the application modification work required for data migration.Global indexes can effectively improve the efficiency of retrieving non-partitioned columns, and remove the restriction that a unique key must contain the partition key. This feature extends the usage scenarios of TiDB partitioned tables, improves the performance of partitioned tables, and reduces resource consumption in certain query scenarios.
    Default pushdown of the Projection operator to the storage engine (introduced in v8.3.0) When hotspot issues are not caused by individual SQL statements, using the aggregated CPU time by table or database level in Top SQL can help you quickly identify the tables or applications responsible for the hotspots, significantly improving the efficiency of diagnosing hotspot and CPU consumption issues.
    Support backing up TiKV instances with IMDSv2 service enabled (introduced in v8.4.0) AWS EC2 now uses IMDSv2 as the default metadata service. TiDB supports backing up data from TiKV instances that have IMDSv2 enabled, helping you run TiDB clusters more effectively in public cloud services.Backup & Restore (BR) uses AWS SDK for Rust to access external storage (introduced in v8.5.0)BR replaces the original Rusoto library with AWS SDK for Rust to access external storage such as Amazon S3 from TiKV. This change enhances compatibility with AWS features such as IMDSv2 and EKS Pod Identity.
    Security
    Backup & Restore (BR) uses AWS SDK for Rust to access external storage (introduced in v8.5.0)BR replaces the original Rusoto library with AWS SDK for Rust to access external storage such as Amazon S3 from TiKV. This change enhances compatibility with AWS features such as IMDSv2 and EKS Pod Identity.BR replaces the original Rusoto library withAWS SDK for Rust to access external storage such as Amazon S3 from TiKV. This change enhances compatibility with AWS features such as IMDSv2 and EKS Pod Identity.
    Security
    Backup & Restore (BR) uses AWS SDK for Rust to access external storage (introduced in v8.5.0)BR replaces the original Rusoto library withAWS SDK for Rust to access external storage such as Amazon S3 from TiKV. This change enhances compatibility with AWS features such as IMDSv2 and EKS Pod Identity.BR replaces the original Rusoto library with AWS SDK for Rust to access external storage such as Amazon S3 from TiKV. This change enhances compatibility with AWS features such as IMDSv2 and EKS Pod Identity.
    Security
    Scalability and PerformanceReduce data processing latency in multiple dimensions **tw@qiancai**TiDB continuously refines data processing to improve performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include: + Reduce data processing latency in multiple dimensionsTiDB continuously refines data processing to improve performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include:
  • Support parallel sorting (introduced in v8.2.0)
  • Optimize batch processing strategy for KV (key-value) requests (introduced in v8.3.0)
  • Support parallel mode for TSO requests (introduced in v8.4.0)
    • @@ -39,7 +39,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements,
    Use Active PD Followers to enhance PD's Region information query service (GA in v8.5.0) **tw@Oreoxmt 2015**Use Active PD Followers to enhance PD's Region information query service (GA in v8.5.0) TiDB v7.6.0 introduces an experimental feature "Active PD Follower", which allows PD followers to provide Region information query services. This feature improves the capability of the PD cluster to handle GetRegion and ScanRegions requests in clusters with a large number of TiDB nodes and Regions, thereby reducing the CPU pressure on PD leaders. In v8.5.0, this feature becomes generally available (GA).
    Reliability and availabilityImprove the stability of large-scale clusters **tw@hfxsd 1976**Improve the stability of large-scale clusters Companies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In v8.5.0, TiDB significantly enhances the stability of large-scale clusters. Schema cache control and setting the memory quota for the TiDB statistics cache are generally available (GA), reducing stability issues caused by excessive memory consumption. PD introduces Active Follower to handle the pressure brought by numerous Regions, and gradually decouples the services handled by PD for independent deployment. You can increase concurrency and reduce the number of collected objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans in large clusters.
    By setting a maximum percentage limit on background tasks of resource control, you can control their resource consumption based on the needs of different application systems. This keeps background task consumption at a low level and ensures the quality of online services.
    Enhance and expand TiProxy use cases **tw@Oreoxmt**Enhance and expand TiProxy use cases As a crucial component of the high availability of TiDB, TiProxy extends its capabilities beyond SQL traffic access and forwarding to support cluster change evaluation. Key features include:
  • TiProxy supports traffic capture and replay (experimental, introduced in v8.4.0)
  • TiProxy supports built-in virtual IP management (introduced in v8.3.0)
    • @@ -85,7 +85,7 @@ Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements,
    SQL Foreign key (GA in v8.5.0) **tw@lilin90 1894**Foreign key (GA in v8.5.0) Foreign keys are constraints in a database that establish relationships between tables, ensuring data consistency and integrity. They ensure that the data referenced in a child table exist in the parent table, preventing the insertion of invalid data. Foreign keys also support cascading operations (such as automatic synchronization during deletion or update), simplifying business logic implementation and reducing the complexity of manually maintaining data relationships.
    SecurityClient-side encryption of snapshot backup data and log backup data (GA in v8.5.0)**tw@qiancai 1998**Client-side encryption of snapshot backup data and log backup data (GA in v8.5.0) Before uploading backup data to your backup storage, you can encrypt the backup data to ensure its security during storage and transmission.
    Reliability and availability Improve the stability of large-scale clustersCompanies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In v8.5.0, TiDB significantly enhances the stability of large-scale clusters. Schema cache control and setting the memory quota for the TiDB statistics cache are generally available (GA), reducing stability issues caused by excessive memory consumption. PD introduces Active Follower to handle the pressure brought by numerous Regions, and gradually decouples the services handled by PD for independent deployment. You can increase concurrency and reduce the number of collected objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans in large clusters.Companies that use TiDB to run multi-tenant or SaaS applications often need to store a large number of tables. In v8.5.0, TiDB significantly enhances the stability of large-scale clusters. +
  • Schema cache control and setting the memory quota for the TiDB statistics cache are generally available (GA), reducing stability issues caused by excessive memory consumption.
    • +
  • PD introduces Active Follower to handle the pressure brought by numerous Regions, gradually decouples the services handled by PD for independent deployment.
    • +
  • PD improves the performance of Region heartbeat processing and supports tens of millions of Regions for a single cluster.
    • +
  • You can increase concurrency and reduce the number of collected objects to improve the efficiency of statistics collection and loading, ensuring the stability of execution plans in large clusters.
    • +
    Support more triggers for runaway queries, and support switching resource groups (introduced in v8.4.0) Scalability and Performance Reduce data processing latency in multiple dimensions TiDB continuously refines data processing to improve performance, effectively meeting the low-latency SQL processing requirements in financial scenarios. Key updates include: -
  • Support parallel sorting (introduced in v8.2.0)
    • -
  • Optimize batch processing strategy for KV (key-value) requests (introduced in v8.3.0)
    • +
  • Support parallel sorting (introduced in v8.2.0)
    • +
  • Optimize batch processing strategy for KV (key-value) requests (introduced in v8.3.0)
  • Support parallel mode for TSO requests (introduced in v8.4.0)
  • Reduce the resource overhead of DELETE operations (introduced in v8.4.0)
  • Improve query performance for cached tables (introduced in v8.4.0)
    • -
  • Introduce an optimized version of Hash Join (introduced in v8.4.0)
    • +
  • Introduce an optimized version of Hash Join (experimental, introduced in v8.4.0)
    • From 27700226a410beff02a2d0445d52dce37c38e3f9 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 16 Dec 2024 10:27:36 +0800 Subject: [PATCH 88/89] add #9714 for TiFlash bug fixes Co-authored-by: JaySon --- releases/release-8.5.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 5e8156201da82..2fdee32239123 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -389,6 +389,7 @@ The following features are planned for deprecation in future versions: - Fix the issue that the `SUBSTRING()` function returns incorrect results when the second parameter is negative [#9604](https://github.com/pingcap/tiflash/issues/9604) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that the `REPLACE()` function returns an error when the first parameter is a constant [#9522](https://github.com/pingcap/tiflash/issues/9522) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that `LPAD()` and `RPAD()` functions return incorrect results in some cases [#9465](https://github.com/pingcap/tiflash/issues/9465) @[guo-shaoge](https://github.com/guo-shaoge) + - Fix the issue that after creating a vector index, if the internal task for building the vector index is unexpectedly interrupted, it could result in TiFlash writing corrupted data and being unable to restart [#9714](https://github.com/pingcap/tiflash/issues/9714) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools From 73e88fd84240dfb324d1447bc6b5cfbf374f8265 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 16 Dec 2024 11:03:51 +0800 Subject: [PATCH 89/89] add two bug fixes for BR --- releases/release-8.5.0.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/releases/release-8.5.0.md b/releases/release-8.5.0.md index 2fdee32239123..2cea66425b5a7 100644 --- a/releases/release-8.5.0.md +++ b/releases/release-8.5.0.md @@ -401,6 +401,8 @@ The following features are planned for deprecation in future versions: - Fix the issue that the advancer cannot handle lock conflicts [#57134](https://github.com/pingcap/tidb/issues/57134) @[3pointer](https://github.com/3pointer) - Fix potential security vulnerabilities by upgrading the `k8s.io/api` library version [#57790](https://github.com/pingcap/tidb/issues/57790) @[BornChanger](https://github.com/BornChanger) - Fix the issue that PITR tasks might return the `Information schema is out of date` error when there are a large number of tables in the cluster but the actual data size is small [#57743](https://github.com/pingcap/tidb/issues/57743) @[Tristan1900](https://github.com/Tristan1900) + - Fix the issue that log backup might unexpectedly enter a paused state when the advancer owner switches [#58031](https://github.com/pingcap/tidb/issues/58031) @[3pointer](https://github.com/3pointer) + - Fix the issue that the `tiup br restore` command omits checking whether the target cluster table already exists during database or table restoration, which might overwrite existing tables [#58168](https://github.com/pingcap/tidb/issues/58168) @[RidRisR](https://github.com/RidRisR) + TiCDC