Skip to content

Commit

Permalink
address feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
andygrove committed Dec 18, 2024
1 parent e21367a commit bdcec72
Showing 1 changed file with 8 additions and 4 deletions.
12 changes: 8 additions & 4 deletions docs/source/user-guide/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,12 @@ Set `spark.comet.metrics.detailed=true` to see all available Comet metrics.

Comet adds some additional metrics:

| Metric | Description |
| ------------------------------- | ----------------------------------------------------------------------------------------- |
| `encoding and compression time` | Time to encode batches in IPC format and compress using ZSTD. |
| `native shuffle time` | Total time spent in native shuffle writer, excluding the execution time of the input plan |
| Metric | Description |
| ------------------------------- | ------------------------------------------------------------- |
| `native shuffle time` | Total time in native code excluding any child operators. |
| `repartition time` | Time to repartition batches. |
| `memory pool time` | Time interacting with memory pool. |
| `encoding and compression time` | Time to encode batches in IPC format and compress using ZSTD. |

## Native Metrics

Expand All @@ -58,5 +60,7 @@ Here is a guide to some of the native metrics.
| Metric | Description |
| ----------------- | ------------------------------------------------------------- |
| `elapsed_compute` | Total time excluding any child operators. |
| `repart_time` | Time to repartition batches. |
| `ipc_time` | Time to encode batches in IPC format and compress using ZSTD. |
| `mempool_time` | Time interacting with memory pool. |
| `write_time` | Time spent writing bytes to disk. |

0 comments on commit bdcec72

Please sign in to comment.