Skip to content

[AURON #2020] [BUILD] Add --threads option to control Maven build parallelism. #2021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
slfan1989 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[AURON #2020] [BUILD] Add --threads option to control Maven build parallelism. #2021

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ Run `./auron-build.sh --help` to see all available options, including:
- `--celeborn`, `--uniffle`, `--paimon`, `--iceberg`: Optional integrations
- `--skiptests`: Skip unit tests (default: true)
- `--sparktests`: Run Spark integration tests
- `--threads`: Maven build threads (e.g. 1, 4, 1C). Defaults to single-threaded local builds; Docker defaults to 8 unless overridden.

### Running Tests

Expand Down
22 changes: 20 additions & 2 deletions auron-build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ print_help() {
echo " --skiptests <true|false> Skip unit tests (default: true)"
echo " --sparktests <true|false> Run spark tests (default: false)"
echo " --docker <true|false> Build in Docker environment (default: false)"
echo " --threads <N|NC> Maven build threads (e.g. 1, 4, 1C). Default: local unset, docker 8"
IFS=','; echo " --image <NAME> Docker image to use (e.g. ${SUPPORTED_OS_IMAGES[*]}, default: ${SUPPORTED_OS_IMAGES[*]:0:1})"; unset IFS
IFS=','; echo " --sparkver <VERSION> Specify Spark version (e.g. ${SUPPORTED_SPARK_VERSIONS[*]})"; unset IFS
IFS=','; echo " --flinkver <VERSION> Specify Flink version (e.g. ${SUPPORTED_FLINK_VERSIONS[*]})"; unset IFS
Expand Down Expand Up @@ -126,6 +127,7 @@ RELEASE_PROFILE=false
CLEAN=true
SKIP_TESTS=true
SPARK_TESTS=false
THREADS=""
SPARK_VER=""
FLINK_VER=""
SCALA_VER=""
Expand Down Expand Up @@ -158,6 +160,15 @@ while [[ $# -gt 0 ]]; do
exit 1
fi
;;
--threads)
if [[ -n "$2" && "$2" != -* ]]; then
THREADS="$2"
shift 2
else
echo "ERROR: --threads requires a value (e.g. 1, 4, 1C)" >&2
exit 1
fi
;;
--image)
if [[ -n "$2" && "$2" != -* ]]; then
IMAGE_NAME="$2"
Expand Down Expand Up @@ -396,6 +407,15 @@ if [[ -n "$ICEBERG_VER" ]]; then
BUILD_ARGS+=("-Piceberg-$ICEBERG_VER")
fi

# Configure Maven build threads:
# - local builds default to Maven's single-threaded behavior
# - docker builds default to -T8 unless overridden
if [[ -n "$THREADS" ]]; then
BUILD_ARGS+=("-T$THREADS")
elif [[ "$USE_DOCKER" == true ]]; then
BUILD_ARGS+=("-T8")
fi

MVN_ARGS=("${CLEAN_ARGS[@]}" "${BUILD_ARGS[@]}")

# -----------------------------------------------------------------------------
Expand Down Expand Up @@ -470,8 +490,6 @@ fi
# -----------------------------------------------------------------------------
if [[ "$USE_DOCKER" == true ]]; then
echo "[INFO] Compiling inside Docker container using image: $IMAGE_NAME"
# In Docker mode, use multi-threaded Maven build with -T8 for faster compilation
BUILD_ARGS+=("-T8")
if [[ "$CLEAN" == true ]]; then
# Clean the host-side directory that is mounted into the Docker container.
# This avoids "device or resource busy" errors when running `mvn clean` inside the container.
Expand Down
Loading