Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add info about slow query tables #262

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@
* [Session](understand-tidb/session.md)
* [Privilege](understand-tidb/privilege.md)
* [Plugin](understand-tidb/plugin.md)
* [System tables](understand-tidb/system-tables/introduction.md)
* [information_schema](understand-tidb/system-tables/information_schema/introduction.md)
* [slow_query](understand-tidb/system-tables/information_schema/slow_query.md)

* [Project Management](project-management/introduction.md)
* [Releases Train Model](project-management/release-train-model.md)
Expand Down
1 change: 1 addition & 0 deletions src/system-tables/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# System tables
1 change: 1 addition & 0 deletions src/system-tables/slow_query.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# slow_query
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# information_schema

* [slow_query](slow_query.md)
74 changes: 74 additions & 0 deletions src/understand-tidb/system-tables/information_schema/slow_query.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# slow_query

## Overview

Slow queries logs are written to `tidb-slow.log` (set with `slow-query-file` in the config) if they are over the `slow-threshold` (300ms by default). If `record-plan-in-slow-log` is enabled this will include the execution plan.

Each TiDB node writes its own slow logs.

### SLOW_QUERY
```
+------------+
| App or CLI |
+------------+
|
SQL Query
SLOW_QUERY
|
+--------+
| TiDB |
+--------+
|
+---------------+
| tidb-slow.log |
+---------------+
```

The slow log can be viewed directly or via the `information_schema.SLOW_QUERY` table.

The column definition is in [`slowQueryCols`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/infoschema/tables.go#L874-L956).

The [`SlowQueryExtractor`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/planner/core/memtable_predicate_extractor.go#L1282) helps to extract some predicates of `slow_query`.

A lot of the other logic can be found in [`slow_query.go`](https://github.com/pingcap/tidb/blob/master/pkg/executor/slow_query.go).

This table has [`RuntimeStats`](https://github.com/pingcap/tidb/blob/2e51209f483bb7909be1eb0b55e5f18f0c437a25/pkg/executor/slow_query.go#L1114-L1119), which adds stats to `EXPLAIN ANALYZE` output.

### CLUSTER_SLOW_QUERY

There is also the `information_schema.CLUSTER_SLOW_QUERY` table that combines the slow log from all nodes into a single table.

The [TiDB Dashboard](https://docs.pingcap.com/tidb/stable/dashboard-slow-query) uses the `CLUSTER_SLOW_QUERY` table to display the slow queries in a webpage.

```
+----------------+
| TiDB Dashboard |
| or App/CLI |
+----------------+
|
SQL Query
CLUSTER_SLOW_QUERY
|
+--------+
+--gRPC-------| TiDB 1 |------gRPC--+
| +--------+ |
+--------+ | +--------+
| TiDB 0 | | | TiDB 2 |
+--------+ | +--------+
| | |
+---------------+ +---------------+ +---------------+
| tidb-slow.log | | tidb-slow.log | | tidb-slow.log |
+---------------+ +---------------+ +---------------+
```

The protobuf messages are defined in [`diagnosticspb.proto`](https://github.com/pingcap/kvproto/blob/master/proto/diagnosticspb.proto) in the `pingcap/kvproto` repo. Key messages are `SearchLogRequest` and `SearchLogResponse`.

This table uses the [`clusterLogRetriever`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/executor/memtable_reader.go#L361) with the [`ClusterLogTableExtractor`](https://github.com/pingcap/tidb/blob/1521bf723dd023da655add0f883acaab5ee69683/pkg/planner/core/memtable_predicate_extractor.go#L756).
Comment on lines +64 to +66

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These descriptions are not quite right, CLUSTER_SLOW_QUERY is actually implemented through the coprocessor grpc interface, which is the same as a normal cop request, except that it sends the cop request to the TiDB node, which allows for the reuse of query conditions push down and column prune, to reduces transfer unnecessary data.

https://github.com/pingcap/tidb/blob/f347b9d334495c72d27c53532dfef522213dbaac/pkg/store/copr/coprocessor.go#L594


There are no `RuntimeStats` for this table, see [this issue](https://github.com/pingcap/tidb/issues/56707) for a request to add that.

## Documentation

- User docs
- [information_schema.slow_query](https://docs.pingcap.com/tidb/stable/information-schema-slow-query)
- [Identify Slow Queries](https://docs.pingcap.com/tidb/stable/identify-slow-queries#query-the-number-of-slow-queries-for-each-tidb-node-in-a-cluster)
20 changes: 20 additions & 0 deletions src/understand-tidb/system-tables/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# System tables

There are multiple types of system tables:

- `mysql` schema: There are mostly real tables stored on TiKV
- [`information_schema`](information_schema/introduction.md): These tables are generated on request and not stored on disk.
- `performance_schema`: These tables are generated on request and not stored on disk.
- `metrics_schema`: These are metrics from Prometheus, not real tables.
- `SYS`: these are views etc to make `performance_schema` easier to use.

See also the docs:
- [TiDB mysql schema](https://docs.pingcap.com/tidb/stable/mysql-schema)
- [TiDB information_schema](https://docs.pingcap.com/tidb/stable/information-schema)
- [TiDB performance_schema](https://docs.pingcap.com/tidb/stable/performance-schema)
- [TiDB sys schema](https://docs.pingcap.com/tidb/stable/sys-schema)
- [TiDB metrics_schema](https://docs.pingcap.com/tidb/stable/metrics-schema)
- [MySQL performance_schema](https://dev.mysql.com/doc/refman/8.4/en/performance-schema.html)
- [MySQL information_schema](https://dev.mysql.com/doc/refman/8.4/en/information-schema.html)
- [MySQL sys schema](https://dev.mysql.com/doc/refman/8.4/en/sys-schema.html)
- [MySQL mysql schema](https://dev.mysql.com/doc/refman/8.4/en/system-schema.html)