-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add info about slow query tables #262
Open
dveeden
wants to merge
3
commits into
pingcap:master
Choose a base branch
from
dveeden:slow_query
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# System tables |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# slow_query |
3 changes: 3 additions & 0 deletions
3
src/understand-tidb/system-tables/information_schema/introduction.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# information_schema | ||
|
||
* [slow_query](slow_query.md) |
74 changes: 74 additions & 0 deletions
74
src/understand-tidb/system-tables/information_schema/slow_query.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
# slow_query | ||
|
||
## Overview | ||
|
||
Slow queries logs are written to `tidb-slow.log` (set with `slow-query-file` in the config) if they are over the `slow-threshold` (300ms by default). If `record-plan-in-slow-log` is enabled this will include the execution plan. | ||
|
||
Each TiDB node writes its own slow logs. | ||
|
||
### SLOW_QUERY | ||
``` | ||
+------------+ | ||
| App or CLI | | ||
+------------+ | ||
| | ||
SQL Query | ||
SLOW_QUERY | ||
| | ||
+--------+ | ||
| TiDB | | ||
+--------+ | ||
| | ||
+---------------+ | ||
| tidb-slow.log | | ||
+---------------+ | ||
``` | ||
|
||
The slow log can be viewed directly or via the `information_schema.SLOW_QUERY` table. | ||
|
||
The column definition is in [`slowQueryCols`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/infoschema/tables.go#L874-L956). | ||
|
||
The [`SlowQueryExtractor`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/planner/core/memtable_predicate_extractor.go#L1282) helps to extract some predicates of `slow_query`. | ||
|
||
A lot of the other logic can be found in [`slow_query.go`](https://github.com/pingcap/tidb/blob/master/pkg/executor/slow_query.go). | ||
|
||
This table has [`RuntimeStats`](https://github.com/pingcap/tidb/blob/2e51209f483bb7909be1eb0b55e5f18f0c437a25/pkg/executor/slow_query.go#L1114-L1119), which adds stats to `EXPLAIN ANALYZE` output. | ||
|
||
### CLUSTER_SLOW_QUERY | ||
|
||
There is also the `information_schema.CLUSTER_SLOW_QUERY` table that combines the slow log from all nodes into a single table. | ||
|
||
The [TiDB Dashboard](https://docs.pingcap.com/tidb/stable/dashboard-slow-query) uses the `CLUSTER_SLOW_QUERY` table to display the slow queries in a webpage. | ||
|
||
``` | ||
+----------------+ | ||
| TiDB Dashboard | | ||
| or App/CLI | | ||
+----------------+ | ||
| | ||
SQL Query | ||
CLUSTER_SLOW_QUERY | ||
| | ||
+--------+ | ||
+--gRPC-------| TiDB 1 |------gRPC--+ | ||
| +--------+ | | ||
+--------+ | +--------+ | ||
| TiDB 0 | | | TiDB 2 | | ||
+--------+ | +--------+ | ||
| | | | ||
+---------------+ +---------------+ +---------------+ | ||
| tidb-slow.log | | tidb-slow.log | | tidb-slow.log | | ||
+---------------+ +---------------+ +---------------+ | ||
``` | ||
|
||
The protobuf messages are defined in [`diagnosticspb.proto`](https://github.com/pingcap/kvproto/blob/master/proto/diagnosticspb.proto) in the `pingcap/kvproto` repo. Key messages are `SearchLogRequest` and `SearchLogResponse`. | ||
|
||
This table uses the [`clusterLogRetriever`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/executor/memtable_reader.go#L361) with the [`ClusterLogTableExtractor`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/planner/core/memtable_predicate_extractor.go#L756). | ||
|
||
There are no `RuntimeStats` for this table, see [this issue](https://github.com/pingcap/tidb/issues/56707) for a request to add that. | ||
|
||
## Documentation | ||
|
||
- User docs | ||
- [information_schema.slow_query](https://docs.pingcap.com/tidb/stable/information-schema-slow-query) | ||
- [Identify Slow Queries](https://docs.pingcap.com/tidb/stable/identify-slow-queries#query-the-number-of-slow-queries-for-each-tidb-node-in-a-cluster) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# System tables | ||
|
||
There are multiple types of system tables: | ||
|
||
- `mysql` schema: There are mostly real tables stored on TiKV | ||
- [`information_schema`](information_schema/introduction.md): These tables are generated on request and not stored on disk. | ||
- `performance_schema`: These tables are generated on request and not stored on disk. | ||
- `metrics_schema`: These are metrics from Prometheus, not real tables. | ||
- `SYS`: these are views etc to make `performance_schema` easier to use. | ||
|
||
See also the docs: | ||
- [TiDB mysql schema](https://docs.pingcap.com/tidb/stable/mysql-schema) | ||
- [TiDB information_schema](https://docs.pingcap.com/tidb/stable/information-schema) | ||
- [TiDB performance_schema](https://docs.pingcap.com/tidb/stable/performance-schema) | ||
- [TiDB sys schema](https://docs.pingcap.com/tidb/stable/sys-schema) | ||
- [TiDB metrics_schema](https://docs.pingcap.com/tidb/stable/metrics-schema) | ||
- [MySQL performance_schema](https://dev.mysql.com/doc/refman/8.4/en/performance-schema.html) | ||
- [MySQL information_schema](https://dev.mysql.com/doc/refman/8.4/en/information-schema.html) | ||
- [MySQL sys schema](https://dev.mysql.com/doc/refman/8.4/en/sys-schema.html) | ||
- [MySQL mysql schema](https://dev.mysql.com/doc/refman/8.4/en/system-schema.html) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These descriptions are not quite right, CLUSTER_SLOW_QUERY is actually implemented through the
coprocessor
grpc interface, which is the same as a normal cop request, except that it sends the cop request to the TiDB node, which allows for the reuse of query conditions push down and column prune, to reduces transfer unnecessary data.https://github.com/pingcap/tidb/blob/f347b9d334495c72d27c53532dfef522213dbaac/pkg/store/copr/coprocessor.go#L594