Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
andygrove committed May 7, 2024
1 parent eb9c21c commit 22409f4
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion docs/source/user-guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@ $SPARK_HOME/bin/spark-shell \
--conf spark.sql.extensions=org.apache.comet.CometSparkSessionExtensions \
--conf spark.comet.enabled=true \
--conf spark.comet.exec.enabled=true \
--conf spark.comet.exec.all.enabled=true
--conf spark.comet.exec.all.enabled=true \
--conf spark.comet.explainFallback.enabled=true
```

### Verify Comet enabled for Spark SQL query
Expand Down Expand Up @@ -95,6 +96,16 @@ INFO src/lib.rs: Comet native library initialized
PushedFilters: [IsNotNull(a), GreaterThan(a,5)], ReadSchema: struct<a:int>
```

With the configuration `spark.comet.explainFallback.enabled=true`, Comet will log any reasons that prevent a plan from
being executed natively.

```scala
scala> spark.sql("select * from t1 where cast(cast(a as double) as decimal(10,2)) > 5").show
WARN CometSparkSessionExtensions$CometExecRule: Comet cannot execute this plan natively because:
- Comet does not guarantee correct results for cast from DoubleType to DecimalType(10,2) with timezone Some(America/Denver) and evalMode LEGACY (No overflow check). To enable all incompatible casts, set spark.comet.cast.allowIncompatible=true
- CollectLimit is not supported
```

### Enable Comet shuffle

Comet shuffle feature is disabled by default. To enable it, please add related configs:
Expand Down

0 comments on commit 22409f4

Please sign in to comment.