[SPARK-48135][INFRA] Run buf and ui only in PR builders and Java …

…21 Daily CI

### What changes were proposed in this pull request?

This PR aims to run `buf` and `ui` tests only in PR builders and Java 21 Daily CI.

### Why are the changes needed?

Currently, Apache Spark CI is running `buf` and `ui` tests always because they finish quickly.

https://github.com/apache/spark/blob/32ba5c1db62caaaa2674e8acced56f89ed840bf9/.github/workflows/build_and_test.yml#L102-L103

- `buf` job
https://github.com/apache/spark/blob/32ba5c1db62caaaa2674e8acced56f89ed840bf9/.github/workflows/build_and_test.yml#L571-L574

- `ui` job
https://github.com/apache/spark/blob/32ba5c1db62caaaa2674e8acced56f89ed840bf9/.github/workflows/build_and_test.yml#L1049-L1052

However, ASF Infra team's guideline recommends to maintain the job concurrency level under or equal to `15`. We had better offload `buf` and `ui` from per-commit CI.

- https://infra.apache.org/github-actions-policy.html

> All workflows SHOULD have a job concurrency level less than or equal to 15.

### Does this PR introduce _any_ user-facing change?

No because this is an infra update.

### How was this patch tested?

Pass the CIs and manual review because PR builders will not be affected by this.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#46392 from dongjoon-hyun/SPARK-48135.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

.github/workflows/build_and_test.yml

@@ -82,10 +82,14 @@ jobs:
             pandas=$pyspark
             kubernetes=`./dev/is-changed.py -m kubernetes`
             sparkr=`./dev/is-changed.py -m sparkr`
+            buf=true
+            ui=true
           else
             pandas=false
             kubernetes=false
             sparkr=false
+            buf=false
+            ui=false
           fi
           # 'build' is always true for now.
           # It does not save significant time and most of PRs trigger the build.
@@ -99,8 +103,8 @@ jobs:
               \"docker-integration-tests\": \"false\",
               \"lint\" : \"true\",
               \"k8s-integration-tests\" : \"$kubernetes\",
-              \"buf\" : \"true\",
-              \"ui\" : \"true\",
+              \"buf\" : \"$buf\",
+              \"ui\" : \"$ui\",
             }"
           echo $precondition # For debugging
           # Remove `\n` to avoid "Invalid format" error

.github/workflows/build_java21.yml

@@ -47,5 +47,7 @@ jobs:
           "sparkr": "true",
           "tpcds-1g": "true",
           "docker-integration-tests": "true",
-          "k8s-integration-tests": "true"
+          "k8s-integration-tests": "true",
+          "buf": "true",
+          "ui": "true"
         }