-
Notifications
You must be signed in to change notification settings - Fork 208
Open
Description
Summary
Add support for Spark 4.0 and Spark 4.1 to the dev/reformat script with automatic JDK and Scala version switching.
Changes
1. Fix Flink Maven profile (dev/reformat:55)
- Before:
-Pflink,flink-1.18 - After:
-Pflink-1.18 - Reason: Avoid activating non-existent
flinkprofile
2. Add Spark 4.x support (dev/reformat:59-68)
- Add
spark-4.0andspark-4.1to the version sweep list - Auto-switch to
scala-2.13profile for Spark 4.x (Spark 4.x requires Scala 2.13) - Auto-switch to JDK 17 for Spark 4.x (Spark 4.x requires JDK 17+)
- Auto-switch back to JDK 8 for Spark 3.x versions
3. Optimize Spark 3.5 coverage (dev/reformat:52)
- Remove
spark-3.5from the main sweep loop - Keep it only in third-party build section to avoid duplicate formatting runs
- Third-party dependencies (Celeborn, Uniffle, Paimon, Flink, Iceberg) are still formatted with Spark 3.5
4. Update CI workflow (.github/workflows/style.yml)
- Add JDK 17 setup alongside existing JDK 8
- Enable style check to work with both Spark 3.x and Spark 4.x versions
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels