Skip to content

[WIP][CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files #12579

[WIP][CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files

[WIP][CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files #12579

Triggered via pull request November 26, 2024 13:57
Status Failure
Total duration 13m 37s
Artifacts

maven.yml

on: pull_request
Matrix: mr
Matrix: service
Matrix: spark2
Matrix: spark3
Fit to window
Zoom out
Zoom in

Annotations

19 errors and 19 warnings
spark3 (11, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (17, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
Process completed with exit code 1.
spark2 (8, 2.4)
Process completed with exit code 1.
spark3 (17, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (17, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (17, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark2 (8, 2.4)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.