Skip to content

Commit

Permalink
address feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
andygrove committed Dec 17, 2024
1 parent 73b4f9b commit bf039f7
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions docs/source/user-guide/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,16 @@ Set `spark.comet.metrics.detailed=true` to see all available Comet metrics.

### CometScanExec

| Metric | Description |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `scan time` | Total time to scan a Parquet file. This is not comparable to the same metric in Spark because Comet's scan metric is more accurate (records time in nanoseconds rather than milliseconds) |
| Metric | Description |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `scan time` | Total time to scan a Parquet file. This is not comparable to the same metric in Spark because Comet's scan metric is more accurate. Although both Comet and Spark measure the time in nanoseconds, Spark rounds this time to the nearest millisecond per batch and Comet does not. |

### Exchange

Comet adds some additional metrics:

| Metric | Description |
| ------------------------------- |-------------------------------------------------------------------------------------------|
| ------------------------------- | ----------------------------------------------------------------------------------------- |
| `ipc time` | Time to encode batches in IPC format. Includes compression time. |
| `native shuffle time` | Total time spent in native shuffle writer, excluding the execution time of the input plan |
| `native shuffle input time` | Time spend executing the shuffle input plan and fetching batches. |
Expand Down

0 comments on commit bf039f7

Please sign in to comment.