Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 25 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Here is a quick guide on how to run Reflex repo locally so you can start contrib
**Prerequisites:**

- uv version >= 0.6.0 and add it to your path (see [UV Docs](https://docs.astral.sh/uv/getting-started/installation/) for more info).
- `act` installed and a working Docker daemon if you want to run the local CI script (see [act installation docs](https://nektosact.com/installation/)).

**1. Fork this repository:**
Fork this repository by clicking on the `Fork` button on the top right.
Expand Down Expand Up @@ -64,8 +65,30 @@ Once you solve a current issue or improvement to Reflex, you can make a PR, and

Before submitting, a pull request, ensure the following steps are taken and test passing.

In your `reflex` directory run make sure all the unit tests are still passing using the following command.
This will fail if code coverage is below 70%.
In your `reflex` directory, you can run the local CI startup script with:

```bash
bash scripts/run_all_tests.sh
```

The script runs selected GitHub Actions jobs locally through `act`:

- `unit-tests` on `ubuntu-latest` with Python `3.13`
- `integration-app-harness` on `ubuntu-22.04` with Python `3.13`, `memory`, and both split groups
- `benchmarks`
- `check_latest_node` for both split groups

Any extra arguments are forwarded to `act`. `-l` / `--list` lists the selected jobs without running them:

```bash
bash scripts/run_all_tests.sh -l
```

If Docker is unavailable and you still want to run against your host environment, set `REFLEX_ACT_ALLOW_SELF_HOSTED=1`. In that mode the script scrubs `CODESPACE_NAME`, `GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN`, and `PYTHONSAFEPATH` to reduce host-specific drift. Set `REFLEX_ACT_SELF_HOSTED_PRESERVE_HOST_ENV=1` if you need the raw host environment instead.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation omits GITHUB_CODESPACE_TOKEN from the list of scrubbed variables

The CONTRIBUTING.md describes three scrubbed variables (CODESPACE_NAME, GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN, PYTHONSAFEPATH), but scripts/run_all_tests.sh lines 129–135 also unsets GITHUB_CODESPACE_TOKEN. A user who needs that token preserved (e.g., for Codespaces auth in self-hosted mode) would not know to set REFLEX_ACT_SELF_HOSTED_PRESERVE_HOST_ENV=1.

Suggested change
If Docker is unavailable and you still want to run against your host environment, set `REFLEX_ACT_ALLOW_SELF_HOSTED=1`. In that mode the script scrubs `CODESPACE_NAME`, `GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN`, and `PYTHONSAFEPATH` to reduce host-specific drift. Set `REFLEX_ACT_SELF_HOSTED_PRESERVE_HOST_ENV=1` if you need the raw host environment instead.
If Docker is unavailable and you still want to run against your host environment, set `REFLEX_ACT_ALLOW_SELF_HOSTED=1`. In that mode the script scrubs `CODESPACE_NAME`, `GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN`, `GITHUB_CODESPACE_TOKEN`, and `PYTHONSAFEPATH` to reduce host-specific drift.


Browser-backed workflows still have lower parity in self-hosted mode. If you are running as `root`, prefer a non-root environment; `APP_HARNESS_DRIVER_ARGS=--no-sandbox` is only a local fallback and may still not match GitHub Actions behavior.

If you only need the fast local check, make sure all the unit tests are still passing using the following command. This will fail if code coverage is below 70%.

```bash
uv run pytest tests/units --cov --no-cov-on-fail --cov-report=
Expand Down
211 changes: 211 additions & 0 deletions scripts/run_all_tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
#!/usr/bin/env bash
set -euo pipefail

script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_root="$(cd "$script_dir/.." && pwd)"

show_help() {
cat <<'EOF'
Usage: bash scripts/run_all_tests.sh [act args]

Run selected Reflex GitHub Actions jobs locally with `act`.

Examples:
bash scripts/run_all_tests.sh
bash scripts/run_all_tests.sh -l

Environment:
ACT_BIN=/path/to/act Override the `act` binary.
REFLEX_ACT_EVENT=pull_request Override the event passed to `act`.
REFLEX_ACT_UNIT_TESTS_PYTHON_VERSION=3.13
REFLEX_ACT_INTEGRATION_PYTHON_VERSION=3.13
REFLEX_ACT_INTEGRATION_STATE_MANAGER=memory
REFLEX_ACT_PLATFORM_LATEST=...
REFLEX_ACT_PLATFORM_22_04=...
REFLEX_ACT_ALLOW_SELF_HOSTED=1 Run on the host when Docker is unavailable.
REFLEX_ACT_SELF_HOSTED_PRESERVE_HOST_ENV=1
Keep host-only env vars in self-hosted mode.
REFLEX_ACT_SKIP_BENCHMARKS=1 Skip the benchmark workflow.
EOF
}

has_arg() {
local needle="$1"
shift
local arg

for arg in "$@"; do
if [[ "$arg" == "$needle" ]]; then
return 0
fi
done

return 1
}

is_list_mode() {
local arg

for arg in "$@"; do
case "$arg" in
-l|--list|--list-options)
return 0
;;
esac
done

return 1
}

ensure_act() {
if [[ -n "$ACT_BIN" && -x "$ACT_BIN" ]]; then
return 0
fi

cat >&2 <<'EOF'
Error: act is required to run the local CI workflow script.

Install it from https://nektosact.com/installation/ or set ACT_BIN to an existing binary.
EOF
exit 1
}
Comment on lines +60 to +71
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ensure_act reports "not installed" when ACT_BIN is set to a bare name

[[ -x "$ACT_BIN" ]] performs a filesystem test against the literal path. If the user sets ACT_BIN=act (a bare name, not an absolute path), the test checks for ./act in the current directory and fails — even when act is available on PATH. The misleading "act is required / install it" error message surfaces instead of "the specified binary is not executable or not found".

A safer check would use command -v to honour PATH resolution:

ensure_act() {
  if [[ -n "$ACT_BIN" ]] && command -v "$ACT_BIN" >/dev/null 2>&1; then
    # Resolve to full path so later -x checks work
    ACT_BIN="$(command -v "$ACT_BIN")"
    return 0
  fi

  cat >&2 <<'EOF'
Error: act is required to run the local CI workflow script.

Install it from https://nektosact.com/installation/ or set ACT_BIN to an existing binary.
EOF
  exit 1
}


docker_daemon_available() {
command -v docker >/dev/null 2>&1 && docker info >/dev/null 2>&1
}

self_hosted_mode_enabled() {
[[ "${REFLEX_ACT_ALLOW_SELF_HOSTED:-0}" == "1" ]]
}

require_runner_support() {
if is_list_mode "${extra_act_args[@]}"; then
return 0
fi

if docker_daemon_available || self_hosted_mode_enabled; then
return 0
fi

cat >&2 <<'EOF'
Error: Docker is required to run Reflex GitHub Actions locally with act.

Use `bash scripts/run_all_tests.sh -l` to list the selected jobs without running them,
or set REFLEX_ACT_ALLOW_SELF_HOSTED=1 to execute directly on the host with lower CI parity.
EOF
exit 1
}

configure_platform_args() {
local latest_platform
local ubuntu_2204_platform

if self_hosted_mode_enabled; then
latest_platform="${REFLEX_ACT_PLATFORM_LATEST:--self-hosted}"
ubuntu_2204_platform="${REFLEX_ACT_PLATFORM_22_04:--self-hosted}"
echo "Warning: Docker unavailable or bypassed; using self-hosted act platform mapping." >&2
else
latest_platform="${REFLEX_ACT_PLATFORM_LATEST:-catthehacker/ubuntu:full-latest}"
ubuntu_2204_platform="${REFLEX_ACT_PLATFORM_22_04:-catthehacker/ubuntu:full-22.04}"
fi

platform_args=(
-P "ubuntu-latest=${latest_platform}"
-P "ubuntu-22.04=${ubuntu_2204_platform}"
)
}

configure_self_hosted_env() {
if ! self_hosted_mode_enabled; then
return 0
fi

if [[ "${REFLEX_ACT_SELF_HOSTED_PRESERVE_HOST_ENV:-0}" == "1" ]]; then
echo "Warning: preserving host environment in self-hosted mode." >&2
return 0
fi

# Keep host-specific shell state from leaking into workflow behavior.
act_env_prefix=(
env
-u CODESPACE_NAME
-u GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN
-u GITHUB_CODESPACE_TOKEN
-u PYTHONSAFEPATH
)

echo "Warning: self-hosted mode scrubs Codespaces and PYTHONSAFEPATH env vars for better CI parity." >&2

if [[ "$(id -u)" == "0" ]]; then
cat >&2 <<'EOF'
Warning: browser-based workflows may still fail as root.
Use a non-root environment for best parity. If you stay on root, try APP_HARNESS_DRIVER_ARGS=--no-sandbox as a local-only fallback.
EOF
fi
}

run_job() {
local workflow="$1"
local job="$2"
local label="$3"
shift 3

echo "==> ${label}"
"${act_env_prefix[@]}" "${ACT_BIN}" \
"${ACT_EVENT}" \
"${platform_args[@]}" \
-W ".github/workflows/${workflow}" \
-j "${job}" \
"$@" \
"${extra_act_args[@]}"
}
Comment on lines +154 to +161
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty array expansion breaks with set -u on Bash < 4.4

The script uses set -euo pipefail at the top, and both "${act_env_prefix[@]}" (here in run_job) and "${extra_act_args[@]}" (line 82 in require_runner_support) will cause a fatal unbound variable error when those arrays are empty on Bash ≤ 4.3 and macOS's default Bash 3.2.

Reproducer on Bash 3.2 / 4.3:

set -u; arr=(); echo "${arr[@]}"
# bash: arr[@]: unbound variable

The standard bash-safe idiom is "${arr[@]+"${arr[@]}"}" which expands to nothing when empty and to the array elements when non-empty.

Affected locations:

  • scripts/run_all_tests.sh:154"${act_env_prefix[@]}"
  • scripts/run_all_tests.sh:82is_list_mode "${extra_act_args[@]}"
Suggested change
"${act_env_prefix[@]}" "${ACT_BIN}" \
"${ACT_EVENT}" \
"${platform_args[@]}" \
-W ".github/workflows/${workflow}" \
-j "${job}" \
"$@" \
"${extra_act_args[@]}"
}
"${act_env_prefix[@]+"${act_env_prefix[@]}"}" "${ACT_BIN}" \
"${ACT_EVENT}" \
"${platform_args[@]}" \
-W ".github/workflows/${workflow}" \
-j "${job}" \
"$@" \
"${extra_act_args[@]+"${extra_act_args[@]}"}"


if has_arg "-h" "$@" || has_arg "--help" "$@"; then
show_help
exit 0
fi

ACT_BIN="${ACT_BIN:-$(command -v act || true)}"
ACT_EVENT="${REFLEX_ACT_EVENT:-pull_request}"
extra_act_args=("$@")
act_env_prefix=()
platform_args=()

ensure_act
require_runner_support
configure_platform_args
configure_self_hosted_env

cd "$repo_root"

run_job \
"unit_tests.yml" \
"unit-tests" \
"unit-tests (ubuntu-latest, python ${REFLEX_ACT_UNIT_TESTS_PYTHON_VERSION:-3.13})" \
--matrix "os:ubuntu-latest" \
--matrix "python-version:${REFLEX_ACT_UNIT_TESTS_PYTHON_VERSION:-3.13}"

for split_index in 1 2; do
run_job \
"integration_app_harness.yml" \
"integration-app-harness" \
"integration-app-harness (python ${REFLEX_ACT_INTEGRATION_PYTHON_VERSION:-3.13}, ${REFLEX_ACT_INTEGRATION_STATE_MANAGER:-memory}, split ${split_index})" \
--matrix "python-version:${REFLEX_ACT_INTEGRATION_PYTHON_VERSION:-3.13}" \
--matrix "state_manager:${REFLEX_ACT_INTEGRATION_STATE_MANAGER:-memory}" \
--matrix "split_index:${split_index}"
done

if [[ "${REFLEX_ACT_SKIP_BENCHMARKS:-0}" != "1" ]]; then
run_job \
"performance.yml" \
"benchmarks" \
"benchmarks"
fi

for split_index in 1 2; do
run_job \
"check_node_latest.yml" \
"check_latest_node" \
"check-node-latest (split ${split_index})" \
--matrix "split_index:${split_index}"
done