From cb5805912776ce2971f10b94fc703cac1d4ecc44 Mon Sep 17 00:00:00 2001
From: Spencer Bryngelson <sbryngelson@gmail.com>
Date: Mon, 23 Feb 2026 18:58:23 -0500
Subject: [PATCH 1/7] Add CLAUDE.md and .claude/rules/ for Claude Code guidance

Adds project-level instructions for Claude Code interactive sessions
and automated PR reviews. Core CLAUDE.md covers commands, development
workflow contract, architecture, and critical rules. Domain-specific
knowledge (Fortran conventions, GPU/MPI patterns, parameter system,
common pitfalls) lives in modular .claude/rules/ files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 .claude/rules/common-pitfalls.md     |  42 +++++++
 .claude/rules/fortran-conventions.md |  46 ++++++++
 .claude/rules/gpu-and-mpi.md         | 109 +++++++++++++++++
 .claude/rules/parameter-system.md    |  45 +++++++
 CLAUDE.md                            | 168 +++++++++++++++++++++++++++
 5 files changed, 410 insertions(+)
 create mode 100644 .claude/rules/common-pitfalls.md
 create mode 100644 .claude/rules/fortran-conventions.md
 create mode 100644 .claude/rules/gpu-and-mpi.md
 create mode 100644 .claude/rules/parameter-system.md
 create mode 100644 CLAUDE.md

diff --git a/.claude/rules/common-pitfalls.md b/.claude/rules/common-pitfalls.md
new file mode 100644
index 0000000000..d25cb0041a
--- /dev/null
+++ b/.claude/rules/common-pitfalls.md
@@ -0,0 +1,42 @@
+# Common Pitfalls
+
+## Array Bounds
+- Arrays use non-unity lower bounds with ghost cells
+- Riemann solver indexing: left state at `j`, right state at `j+1`
+- Off-by-one errors in ghost cell regions are a common source of bugs
+
+## Blast Radius
+- `src/common/` is shared by ALL three executables (pre_process, simulation, post_process)
+- Any change to common/ requires testing all three targets
+- Public subroutine signature changes affect all callers across all targets
+- Parameter default changes affect all existing case files
+
+## Physics Consistency
+- Pressure formula MUST match `model_eqns` setting
+- Model-specific conservative ↔ primitive conversion paths exist
+- Volume fractions must sum to 1.0
+- Boundary condition symmetry requirements must be maintained
+
+## Compiler-Specific Issues
+- Code must compile on gfortran, nvfortran, Cray ftn, and Intel ifx
+- Each compiler has different strictness levels and warning behavior
+- Fypp macros must expand correctly for both GPU and CPU builds
+- GPU builds only work with nvfortran, Cray ftn, and AMD flang
+
+## Test Golden Files
+- Tests compare output against golden files in `tests/<hash>/golden.txt`
+- If your change intentionally modifies output, regenerate golden files:
+  `./mfc.sh test --generate --only <affected_tests> -j 8`
+- Do not regenerate ALL golden files unless you understand every output change
+- Golden file diffs are compared with tolerance, not exact match
+
+## PR Checklist
+Before submitting a PR:
+- [ ] `./mfc.sh format -j 8` (auto-format)
+- [ ] `./mfc.sh precheck -j 8` (5 CI lint checks)
+- [ ] `./mfc.sh build -j 8` (compiles)
+- [ ] `./mfc.sh test --only <relevant> -j 8` (tests pass)
+- [ ] If adding parameters: all 3 locations updated
+- [ ] If modifying `src/common/`: all three targets tested
+- [ ] If changing output: golden files regenerated for affected tests
+- [ ] One logical change per commit
diff --git a/.claude/rules/fortran-conventions.md b/.claude/rules/fortran-conventions.md
new file mode 100644
index 0000000000..1b089b5475
--- /dev/null
+++ b/.claude/rules/fortran-conventions.md
@@ -0,0 +1,46 @@
+# Fortran Conventions
+
+## File Format
+- Source files use `.fpp` extension (Fortran + Fypp preprocessor macros)
+- Fypp preprocesses `.fpp` → `.f90` at build time via CMake
+- Fypp supports conditional compilation, code generation, and regex macros
+
+## Module Structure
+Every Fortran module follows this pattern:
+- File: `m_<feature>.fpp`
+- Module: `module m_<feature>`
+- `implicit none` required
+- Explicit `intent(in)`, `intent(out)`, or `intent(inout)` on ALL subroutine/function arguments
+- Initialization subroutine: `s_initialize_<feature>_module`
+- Finalization subroutine: `s_finalize_<feature>_module`
+
+## Naming
+- Modules: `m_<feature>`
+- Public subroutines: `s_<verb>_<noun>`
+- Public functions: `f_<verb>_<noun>`
+- Private/local variables: no prefix required
+- Constants: descriptive names, not ALL_CAPS
+
+## Forbidden Patterns
+These are caught by `./mfc.sh precheck` (source lint step 4/5):
+- `dsqrt`, `dexp`, `dlog`, `dble`, `dabs` → use `sqrt`, `exp`, `log`, `real(..., wp)`
+- `real(8)`, `real(4)` → use `wp` or `stp` kind parameters
+- `goto`, `COMMON` blocks, global `save` variables
+- `stop`, `error stop` → use `call s_mpi_abort()`
+- Raw `!$acc` or `!$omp` directives → use Fypp GPU_* macros from `parallel_macros.fpp`
+
+## Precision Types
+- `wp` (working precision): used for computation. Double by default.
+- `stp` (storage precision): used for I/O. Double by default.
+- In single-precision mode (`--single`): both become single.
+- In mixed-precision mode (`--mixed`): wp=double, stp=single.
+- MPI type matching: `mpi_p` must match `wp`, `mpi_io_p` must match `stp`.
+- Always use generic intrinsics: `sqrt` not `dsqrt`, `abs` not `dabs`.
+- Cast with `real(..., wp)` or `real(..., stp)`, never `dble(...)`.
+
+## Size Guidelines (soft)
+- Subroutine: ≤500 lines
+- Helper routine: ≤150 lines
+- Function: ≤100 lines
+- File: ≤1000 lines
+- Arguments: ≤6 preferred
diff --git a/.claude/rules/gpu-and-mpi.md b/.claude/rules/gpu-and-mpi.md
new file mode 100644
index 0000000000..90c96777c9
--- /dev/null
+++ b/.claude/rules/gpu-and-mpi.md
@@ -0,0 +1,109 @@
+# GPU and MPI Patterns
+
+## GPU Offloading Architecture
+
+Only `src/simulation/` is GPU-accelerated. Pre/post_process run on CPU only.
+
+MFC uses a **backend-agnostic GPU abstraction** via Fypp macros. The same source code
+compiles to either OpenACC or OpenMP target offload depending on the build flag:
+
+- `./mfc.sh build --gpu acc` → OpenACC backend (NVIDIA nvfortran, Cray ftn)
+- `./mfc.sh build --gpu mp`  → OpenMP target offload backend (Cray ftn, AMD flang)
+- `./mfc.sh build` (no --gpu) → CPU-only, GPU macros expand to plain Fortran
+
+### Macro Layers (in src/common/include/)
+- `parallel_macros.fpp` — **Use these.** Generic `GPU_*` macros that dispatch to the
+  correct backend based on `MFC_OpenACC` / `MFC_OpenMP` compile definitions.
+- `acc_macros.fpp` — OpenACC-specific `ACC_*` implementations (do not call directly)
+- `omp_macros.fpp` — OpenMP target offload `OMP_*` implementations (do not call directly)
+  - OMP macros generate **compiler-specific** directives: NVIDIA uses `target teams loop`,
+    Cray uses `target teams distribute parallel do simd`, AMD uses
+    `target teams distribute parallel do`
+- `shared_parallel_macros.fpp` — Shared helpers (collapse, private, reduction generators)
+
+### Key GPU Macros (always use the `GPU_*` prefix)
+- `@:GPU_PARALLEL_LOOP(collapse=N, private=[...], reduction=[...], reductionOp='+')` —
+  Parallel loop over GPU threads. Most common GPU macro.
+- `@:END_GPU_PARALLEL_LOOP()` — Required closing for GPU_PARALLEL_LOOP.
+- `@:GPU_PARALLEL(code, ...)` — GPU parallel region (wraps code block).
+- `@:GPU_LOOP(collapse=N, ...)` — Inner loop within a GPU parallel region.
+- `@:GPU_DATA(code, copy=..., create=..., ...)` — Scoped data region.
+- `@:GPU_ENTER_DATA(create=[...])` — Allocate device memory (unscoped).
+- `@:GPU_EXIT_DATA(delete=[...])` — Free device memory.
+- `@:GPU_UPDATE(host=[...])` — Copy device → host (before MPI send).
+- `@:GPU_UPDATE(device=[...])` — Copy host → device (after MPI receive).
+- `@:GPU_ROUTINE(function_name=..., nohost=True/False)` — Mark routine for device.
+- `@:GPU_DECLARE(copyin=[...], link=[...])` — Declare device-resident data.
+- `@:GPU_ATOMIC(atomic='update')` — Atomic operation on device.
+- `@:GPU_WAIT()` — Synchronization barrier.
+- `@:GPU_HOST_DATA(code, use_device_addr=[...])` — Host code with device pointers.
+
+NEVER write raw `!$acc` or `!$omp` directives. Always use `GPU_*` Fypp macros.
+The precheck source lint will catch raw directives and fail.
+
+### Memory Management Macros (from macros.fpp)
+- `@:ALLOCATE(var1, var2, ...)` — Fortran allocate + `GPU_ENTER_DATA(create=...)`
+- `@:DEALLOCATE(var1, var2, ...)` — `GPU_EXIT_DATA(delete=...)` + Fortran deallocate
+- Every `@:ALLOCATE` MUST have a matching `@:DEALLOCATE` in finalization
+- Conditional allocation MUST have conditional deallocation
+
+### GPU Field Setup (Cray-specific, from macros.fpp)
+- `@:ACC_SETUP_VFs(...)` / `@:ACC_SETUP_SFs(...)` — GPU pointer setup for vector/scalar fields
+- These compile only for Cray (`_CRAYFTN`); other compilers skip them
+
+### Compiler-Backend Matrix
+| Compiler        | `--gpu acc` (OpenACC) | `--gpu mp` (OpenMP) | CPU-only |
+|-----------------|----------------------|---------------------|----------|
+| GNU gfortran    | No                   | No                  | Yes      |
+| NVIDIA nvfortran| Yes (primary)        | Yes                 | Yes      |
+| Cray ftn (CCE)  | Yes                  | Yes (primary)       | Yes      |
+| Intel ifx       | No                   | No                  | Yes      |
+| AMD flang       | No                   | Yes                 | Yes      |
+
+## Preprocessor Defines (`#ifdef` / `#ifndef`)
+
+Raw `#ifdef` / `#ifndef` preprocessor guards are **normal and expected** in MFC.
+They are NOT the same as raw `!$acc`/`!$omp` pragmas (which are forbidden).
+
+Use `#ifdef` for feature, target, compiler, and library gating:
+
+### Feature gating
+- `MFC_MPI` — MPI-enabled build (`--mpi` flag, default ON)
+- `MFC_OpenACC` — OpenACC GPU backend (`--gpu acc`)
+- `MFC_OpenMP` — OpenMP target offload backend (`--gpu mp`)
+- `MFC_GPU` — Any GPU build (either OpenACC or OpenMP)
+- `MFC_DEBUG` — Debug build (`--debug`)
+- `MFC_SINGLE_PRECISION` — Single-precision mode (`--single`)
+- `MFC_MIXED_PRECISION` — Mixed-precision mode (`--mixed`)
+
+### Target gating (for code in `src/common/` shared across executables)
+- `MFC_PRE_PROCESS` — Only in pre_process builds
+- `MFC_SIMULATION` — Only in simulation builds
+- `MFC_POST_PROCESS` — Only in post_process builds
+
+### Compiler gating (for compiler-specific workarounds)
+- `_CRAYFTN` — Cray Fortran compiler
+- `__NVCOMPILER_GPU_UNIFIED_MEM` — NVIDIA unified memory (GH-200 / `--unified`)
+- `__PGI` — Legacy PGI/NVIDIA compiler
+- `__INTEL_COMPILER` — Intel compiler
+- `FRONTIER_UNIFIED` — Frontier HPC unified memory
+
+### Library-specific code
+- FFTW (`m_fftw.fpp`) uses heavy `#ifdef` gating for `MFC_GPU` and `__PGI`
+- CUDA Fortran (`cudafor` module) is gated behind `__NVCOMPILER_GPU_UNIFIED_MEM`
+- SILO/HDF5 interfaces may have conditional paths
+
+When adding new `#ifdef` blocks, always provide an `#else` or `#endif` path so
+the code compiles in all configurations (CPU-only, GPU-ACC, GPU-OMP, with/without MPI).
+
+## MPI
+
+### Halo Exchange
+- Pack/unpack offset calculations are error-prone — verify carefully
+- Buffer sizing depends on dimensionality and QBMM state
+- GPU coherence: always `GPU_UPDATE(host=...)` before MPI send,
+  `GPU_UPDATE(device=...)` after MPI receive
+
+### Error Handling
+- Use `call s_mpi_abort()` for fatal errors, never `stop` or `error stop`
+- MPI must be finalized before program exit
diff --git a/.claude/rules/parameter-system.md b/.claude/rules/parameter-system.md
new file mode 100644
index 0000000000..4371a0b3a7
--- /dev/null
+++ b/.claude/rules/parameter-system.md
@@ -0,0 +1,45 @@
+# Parameter System
+
+## Overview
+MFC has ~3,300 simulation parameters defined in Python and read by Fortran via namelist files.
+
+## Parameter Flow: Python → Fortran
+
+1. **Definition**: `toolchain/mfc/params/definitions.py` — source of truth
+   - Parameters are indexed families: `patch_icpp(i)%attr`, `fluid_pp(i)%attr`, etc.
+   - Each has type, default, constraints, and tags
+
+2. **Validation**: `toolchain/mfc/case_validator.py`
+   - JSON schema validation via fastjsonschema
+   - Physics constraint checking (e.g., volume fractions sum to 1)
+   - Dependency validation (required/recommended params)
+
+3. **Input Generation**: `toolchain/mfc/run/input.py`
+   - Python case dict → Fortran namelist `.inp` file
+   - Format: `&user_inputs / ... / &end`
+
+4. **Fortran Reading**: `src/*/m_start_up.fpp`
+   - Reads `&user_inputs` namelist
+   - Each parameter must be declared in the namelist statement
+
+## Adding a New Parameter (3-location checklist)
+
+YOU MUST update all 3 locations. Missing any causes silent failures.
+
+1. **`toolchain/mfc/params/definitions.py`**: Add parameter with type, default, constraints
+2. **`src/*/m_start_up.fpp`**: Add to the Fortran `namelist` declaration in the relevant
+   target(s). If the param is used by simulation only, add it there. If shared, add to
+   all three targets' m_start_up.fpp.
+3. **`toolchain/mfc/case_validator.py`**: Add validation rules if the parameter has
+   physics constraints. Include `PHYSICS_DOCS` entry with title, category, explanation.
+
+## Case Files
+- Case files are Python scripts (`.py`) that define a dict of parameters
+- Validated with `./mfc.sh validate case.py`
+- Examples in `examples/` directory
+- Create new cases with `./mfc.sh new <name>`
+- Search parameters with `./mfc.sh params <query>`
+
+## Analytical Initial Conditions
+String expressions in parameters become Fortran code via `case.py.__get_analytic_ic_fpp()`.
+These are compiled into the binary, so syntax errors cause build failures, not runtime errors.
diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 0000000000..c4168cb050
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1,168 @@
+# MFC — Multi-component Flow Code
+
+MFC is an exascale multi-physics CFD solver written in modern Fortran 2008+ with Fypp
+preprocessing. It has three executables (pre_process, simulation, post_process), a Python
+toolchain for building/running/testing, and supports GPU acceleration via OpenACC and
+OpenMP target offload. It must compile with gfortran, nvfortran, Cray ftn, and Intel ifx.
+
+## Commands
+
+Prefer using `./mfc.sh` as the entry point for building, running, testing, formatting,
+and linting. It handles virtual environments, module loading, dependency bootstrapping,
+and build configuration. Avoid invoking CMake, Python toolchain scripts, or Fortran
+compilers directly unless you have a specific reason.
+
+All commands run from the repo root via `./mfc.sh`.
+
+```bash
+# Building
+./mfc.sh build -j 8                        # Build all 3 targets (pre_process, simulation, post_process)
+./mfc.sh build -t simulation -j 8          # Build only simulation
+./mfc.sh build --gpu acc -j 8              # Build with OpenACC GPU support
+./mfc.sh build --gpu mp -j 8              # Build with OpenMP target offload GPU support
+./mfc.sh build --debug -j 8                # Debug build
+./mfc.sh build -i case.py --case-optimization -j 8  # Case-optimized build (10x speedup)
+
+# Running
+./mfc.sh run case.py -n 4                  # Run case with 4 MPI ranks
+./mfc.sh run case.py --no-build            # Run without rebuilding
+./mfc.sh run case.py -e batch -N 2 -n 4 -c phoenix -a ACCOUNT  # Batch submit on Phoenix
+
+# Testing
+./mfc.sh test -j 8                         # Run full test suite (500+ tests)
+./mfc.sh test --only 1D -j 8              # Only 1D tests
+./mfc.sh test --only 2D Bubbles -j 8      # Only 2D bubble tests
+./mfc.sh test --only <UUID> -j 8          # Run one specific test by UUID
+./mfc.sh test -l                           # List all tests with UUIDs and traces
+./mfc.sh test -% 10 -j 8                  # Run 10% random sample
+./mfc.sh test --generate --only <feature>  # Regenerate golden files after intentional output change
+
+# Verification (pre-commit CI checks)
+./mfc.sh precheck -j 8                     # Run all 5 lint checks (same as CI gate)
+./mfc.sh format -j 8                       # Auto-format Fortran (.fpp/.f90) + Python
+./mfc.sh lint                              # Pylint + Python unit tests
+./mfc.sh spelling                          # Spell check
+
+# Module loading (HPC clusters only — must use `source`)
+source ./mfc.sh load -c p -m g             # Load Phoenix GPU modules
+source ./mfc.sh load -c f -m g             # Load Frontier GPU modules
+source ./mfc.sh load -c p -m c             # Load Phoenix CPU modules
+
+# Other
+./mfc.sh validate case.py                  # Validate case file without running
+./mfc.sh params <query>                    # Search ~3300 case parameters
+./mfc.sh clean                             # Remove build artifacts
+./mfc.sh new <name>                        # Create new case from template
+```
+
+## System Identification and Module Loading
+
+MFC targets HPC clusters. Before building on a cluster, load the correct modules
+via `source ./mfc.sh load -c <slug> -m <mode>`.
+
+To identify the current system, check multiple signals — hostname alone is not always
+sufficient (compute nodes may differ from login nodes):
+
+```bash
+hostname                    # e.g., login-phoenix-gnr-2.pace.gatech.edu
+echo $LMOD_SYSHOST          # e.g., "phoenix" (most reliable when set)
+echo $CRAY_LD_LIBRARY_PATH  # Non-empty → Cray system (Frontier, Carpenter Cray)
+echo $MODULESHOME           # Confirms module system is available
+```
+
+Supported systems and their slugs (full list in `toolchain/modules`):
+
+| Slug | System | GPU Backend | Example |
+|------|--------|-------------|---------|
+| `p` | GT Phoenix | OpenACC (nvfortran) | `source ./mfc.sh load -c p -m g` |
+| `f` | OLCF Frontier | OpenACC (Cray ftn) | `source ./mfc.sh load -c f -m g` |
+| `tuo` | LLNL Tuolumne | OpenMP (Cray ftn) | `source ./mfc.sh load -c tuo -m g` |
+| `d` | NCSA Delta | OpenACC (nvfortran) | `source ./mfc.sh load -c d -m g` |
+| `b` | PSC Bridges2 | OpenACC (nvfortran) | `source ./mfc.sh load -c b -m g` |
+| `cc` | DoD Carpenter (Cray) | CPU only | `source ./mfc.sh load -c cc -m c` |
+| `c` | DoD Carpenter (GNU) | CPU only | `source ./mfc.sh load -c c -m c` |
+| `o` | Brown Oscar | OpenACC (nvfortran) | `source ./mfc.sh load -c o -m g` |
+| `h` | UF HiPerGator | OpenACC (nvfortran) | `source ./mfc.sh load -c h -m g` |
+
+The `-m` flag selects mode: `g`/`gpu` for GPU builds, `c`/`cpu` for CPU-only.
+Batch job templates for `./mfc.sh run -e batch -c <system>` are in `toolchain/templates/`.
+
+IMPORTANT: `source` (or `.`) is required for `load` — it sets environment variables
+in the current shell. Using `./mfc.sh load` without `source` will error.
+
+## Development Workflow Contract
+
+IMPORTANT: Follow this loop for ALL code changes. Do not skip steps.
+
+1. **Read first** — Read and understand relevant code before modifying it.
+2. **Plan** — For multi-file changes, outline your approach before implementing.
+3. **Implement** — Make small, focused changes. One logical change per commit.
+4. **Format** — Run `./mfc.sh format -j 8` to auto-format.
+5. **Verify** — Run `./mfc.sh precheck -j 8` (same 5 checks as CI lint gate).
+6. **Build** — Run `./mfc.sh build -j 8` to verify compilation.
+7. **Test** — Run relevant tests: `./mfc.sh test --only <feature> -j 8`.
+   For changes to `src/common/`, test ALL three targets: `./mfc.sh test -j 8`.
+8. **Commit** — Only after steps 4-7 pass. Do not commit untested code.
+
+YOU MUST run `./mfc.sh precheck` before any commit. This is enforced by pre-commit hooks.
+YOU MUST run tests relevant to your changes before claiming work is done.
+NEVER commit code that does not compile or fails tests.
+
+## Architecture
+
+```
+src/
+  common/         # Shared code (used by ALL three executables — wide blast radius)
+  pre_process/    # Grid generation and initial conditions
+  simulation/     # CFD solver (GPU-accelerated via OpenACC / OpenMP target offload)
+  post_process/   # Data output and visualization
+toolchain/        # Python CLI, build system, testing, parameter management
+  mfc/params/definitions.py   # ~3300 parameter definitions (source of truth)
+  mfc/case_validator.py       # Physics constraint validation
+  mfc/test/                   # Test runner and case generation
+examples/         # Example simulation cases (case.py files)
+tests/            # 500+ regression test golden files
+```
+
+Source files are `.fpp` (Fortran + Fypp macros), preprocessed to `.f90` by CMake.
+
+## Critical Rules
+
+NEVER use raw OpenACC/OpenMP pragmas (`!$acc`, `!$omp`). Use `GPU_*` Fypp macros instead.
+  Raw `#ifdef`/`#ifndef` preprocessor guards for feature/compiler/library gating ARE normal.
+NEVER use double-precision intrinsics: `dsqrt`, `dexp`, `dlog`, `dble`, `dabs`, `real(8)`, `real(4)`.
+  Use generic intrinsics (`sqrt`, `exp`, `log`) and precision types (`wp`, `stp`).
+NEVER use `stop` or `error stop`. Use `call s_mpi_abort()`.
+NEVER use `goto`, `COMMON` blocks, or global `save` variables.
+
+Every `@:ALLOCATE(...)` MUST have a matching `@:DEALLOCATE(...)`.
+Every new parameter MUST be added in 3 places:
+  1. `toolchain/mfc/params/definitions.py` (parameter definition)
+  2. Fortran namelist in `src/*/m_start_up.fpp` (namelist binding)
+  3. `toolchain/mfc/case_validator.py` (if constraints exist)
+
+Changes to `src/common/` affect ALL three executables. Test comprehensively.
+
+## Naming Conventions
+
+- Modules: `m_<feature>` (e.g., `m_bubbles`)
+- Public subroutines: `s_<verb>_<noun>` (e.g., `s_compute_pressure`)
+- Public functions: `f_<verb>_<noun>`
+- 2-space indentation, lowercase keywords, explicit `intent` on all arguments
+
+## Precision System
+
+- `wp` = working precision (computation). `stp` = storage precision (I/O).
+- Default: both double. Single mode: both single. Mixed: wp=double, stp=single.
+- MPI types must match: `mpi_p` ↔ `wp`, `mpi_io_p` ↔ `stp`.
+
+## Code Review Priorities
+
+When reviewing PRs, prioritize in this order:
+1. Correctness (logic bugs, numerical issues, array bounds)
+2. Precision discipline (stp vs wp mixing)
+3. Memory management (@:ALLOCATE/@:DEALLOCATE pairing, GPU pointer setup)
+4. MPI correctness (halo exchange, buffer sizing, GPU_UPDATE calls)
+5. GPU code (GPU_* Fypp macros only, no raw pragmas)
+6. Physics consistency (pressure formula matches model_eqns)
+7. Compiler portability (all four compilers)

From 7895c6f789348a10eb6fadfbd50f0ca1c02a043c Mon Sep 17 00:00:00 2001
From: Spencer Bryngelson <sbryngelson@gmail.com>
Date: Mon, 23 Feb 2026 19:14:57 -0500
Subject: [PATCH 2/7] Update copilot-instructions.md to mention OpenMP target
 offload

The GPU section previously only mentioned OpenACC. Now reflects that MFC
supports both OpenACC and OpenMP target offload via backend-agnostic
GPU_* Fypp macros.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 .github/copilot-instructions.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
index 2dbfbbf331..bad6e9ad45 100644
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -16,7 +16,7 @@ Formatting and linting are enforced by pre-commit hooks. Focus review effort on
   * Sources in `src/`, tests in `tests/`, examples in `examples/`, Python toolchain in `toolchain/`.
   * Most source files are `.fpp` (Fypp templates); CMake transpiles them to `.f90`.
 * **Fypp macros** are in `src/<subprogram>/include/`, where `<subprogram>` is `simulation`, `common`, `pre_process`, or `post_process`.
-* Only `simulation` (plus its `common` dependencies) is GPU-accelerated via **OpenACC**.
+* Only `simulation` (plus its `common` dependencies) is GPU-accelerated via **OpenACC** or **OpenMP target offload** (`--gpu acc` or `--gpu mp`). GPU code uses backend-agnostic `GPU_*` Fypp macros (in `src/common/include/parallel_macros.fpp`) that dispatch to the correct backend at compile time.
 * Code must compile with **GNU gfortran**, **NVIDIA nvfortran**, **Cray ftn**, and **Intel ifx**.
 * Precision modes: double (default), single, and mixed (`wp` = working precision, `stp` = storage precision).
 * **Python toolchain** requires **Python 3.10+** — do not suggest `from __future__` imports or other backwards-compatibility shims.

From 041cfa6e30631d4b7eaa1254aa9ef76c01c52890 Mon Sep 17 00:00:00 2001
From: Spencer Bryngelson <sbryngelson@gmail.com>
Date: Mon, 23 Feb 2026 19:23:35 -0500
Subject: [PATCH 3/7] Fix parameter checklist and validation attribution

- Expand 3-location to 4-location checklist: add m_global_parameters.fpp
  as required step for declaring Fortran variables (without this, adding
  a parameter to the namelist in m_start_up.fpp causes a compile error)
- Fix fastjsonschema attribution: JSON schema validation is in case.py
  and params/registry.py, not in case_validator.py which does physics
  constraint checking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 .claude/rules/parameter-system.md | 19 +++++++++++--------
 CLAUDE.md                         |  7 ++++---
 2 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/.claude/rules/parameter-system.md b/.claude/rules/parameter-system.md
index 4371a0b3a7..2cadc42ecb 100644
--- a/.claude/rules/parameter-system.md
+++ b/.claude/rules/parameter-system.md
@@ -9,10 +9,11 @@ MFC has ~3,300 simulation parameters defined in Python and read by Fortran via n
    - Parameters are indexed families: `patch_icpp(i)%attr`, `fluid_pp(i)%attr`, etc.
    - Each has type, default, constraints, and tags
 
-2. **Validation**: `toolchain/mfc/case_validator.py`
-   - JSON schema validation via fastjsonschema
-   - Physics constraint checking (e.g., volume fractions sum to 1)
-   - Dependency validation (required/recommended params)
+2. **Validation** (two layers):
+   - `toolchain/mfc/case.py` / `toolchain/mfc/params/registry.py` — JSON schema validation
+     via fastjsonschema (type checking, defaults)
+   - `toolchain/mfc/case_validator.py` — Physics constraint checking
+     (e.g., volume fractions sum to 1, dependency validation)
 
 3. **Input Generation**: `toolchain/mfc/run/input.py`
    - Python case dict → Fortran namelist `.inp` file
@@ -24,13 +25,15 @@ MFC has ~3,300 simulation parameters defined in Python and read by Fortran via n
 
 ## Adding a New Parameter (3-location checklist)
 
-YOU MUST update all 3 locations. Missing any causes silent failures.
+YOU MUST update all 4 locations. Missing any causes silent failures or compile errors.
 
 1. **`toolchain/mfc/params/definitions.py`**: Add parameter with type, default, constraints
-2. **`src/*/m_start_up.fpp`**: Add to the Fortran `namelist` declaration in the relevant
+2. **`src/*/m_global_parameters.fpp`**: Declare the Fortran variable in the relevant
    target(s). If the param is used by simulation only, add it there. If shared, add to
-   all three targets' m_start_up.fpp.
-3. **`toolchain/mfc/case_validator.py`**: Add validation rules if the parameter has
+   all three targets' m_global_parameters.fpp.
+3. **`src/*/m_start_up.fpp`**: Add to the Fortran `namelist` declaration in the relevant
+   target(s).
+4. **`toolchain/mfc/case_validator.py`**: Add validation rules if the parameter has
    physics constraints. Include `PHYSICS_DOCS` entry with title, category, explanation.
 
 ## Case Files
diff --git a/CLAUDE.md b/CLAUDE.md
index c4168cb050..7e494ecccf 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -136,10 +136,11 @@ NEVER use `stop` or `error stop`. Use `call s_mpi_abort()`.
 NEVER use `goto`, `COMMON` blocks, or global `save` variables.
 
 Every `@:ALLOCATE(...)` MUST have a matching `@:DEALLOCATE(...)`.
-Every new parameter MUST be added in 3 places:
+Every new parameter MUST be added in 4 places:
   1. `toolchain/mfc/params/definitions.py` (parameter definition)
-  2. Fortran namelist in `src/*/m_start_up.fpp` (namelist binding)
-  3. `toolchain/mfc/case_validator.py` (if constraints exist)
+  2. Fortran variable declaration in `src/*/m_global_parameters.fpp`
+  3. Fortran namelist in `src/*/m_start_up.fpp` (namelist binding)
+  4. `toolchain/mfc/case_validator.py` (if constraints exist)
 
 Changes to `src/common/` affect ALL three executables. Test comprehensively.
 

From 20da91fb48875c5b37ff7063e0e219065b4ef5c9 Mon Sep 17 00:00:00 2001
From: Spencer Bryngelson <sbryngelson@gmail.com>
Date: Mon, 23 Feb 2026 19:29:38 -0500
Subject: [PATCH 4/7] Fix mixed-precision stp description: half, not single

In mixed-precision mode (--mixed), stp is set to half_precision (kind=2),
not single_precision, as confirmed in src/common/m_precision_select.f90.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 .claude/rules/fortran-conventions.md | 2 +-
 CLAUDE.md                            | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/.claude/rules/fortran-conventions.md b/.claude/rules/fortran-conventions.md
index 1b089b5475..a9b410c163 100644
--- a/.claude/rules/fortran-conventions.md
+++ b/.claude/rules/fortran-conventions.md
@@ -33,7 +33,7 @@ These are caught by `./mfc.sh precheck` (source lint step 4/5):
 - `wp` (working precision): used for computation. Double by default.
 - `stp` (storage precision): used for I/O. Double by default.
 - In single-precision mode (`--single`): both become single.
-- In mixed-precision mode (`--mixed`): wp=double, stp=single.
+- In mixed-precision mode (`--mixed`): wp=double, stp=half.
 - MPI type matching: `mpi_p` must match `wp`, `mpi_io_p` must match `stp`.
 - Always use generic intrinsics: `sqrt` not `dsqrt`, `abs` not `dabs`.
 - Cast with `real(..., wp)` or `real(..., stp)`, never `dble(...)`.
diff --git a/CLAUDE.md b/CLAUDE.md
index 7e494ecccf..90a2a27ea0 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -154,7 +154,7 @@ Changes to `src/common/` affect ALL three executables. Test comprehensively.
 ## Precision System
 
 - `wp` = working precision (computation). `stp` = storage precision (I/O).
-- Default: both double. Single mode: both single. Mixed: wp=double, stp=single.
+- Default: both double. Single mode: both single. Mixed: wp=double, stp=half.
 - MPI types must match: `mpi_p` ↔ `wp`, `mpi_io_p` ↔ `stp`.
 
 ## Code Review Priorities

From 5427de1084d0b809f8fa0b58085b6248db97f7be Mon Sep 17 00:00:00 2001
From: Spencer Bryngelson <sbryngelson@gmail.com>
Date: Mon, 23 Feb 2026 19:35:34 -0500
Subject: [PATCH 5/7] Fix Fypp macro syntax and precheck coverage claims
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

- GPU macros use $: prefix (inline), not @: — matches all 400+ usages in src/
- GPU_PARALLEL, GPU_DATA, GPU_HOST_DATA are block macros using #:call/#:endcall
- Add block macro usage example
- Fix GPU_ROUTINE args: parallelism='[seq]', not function_name=
- Split forbidden patterns into precheck-enforced vs convention-enforced
  (goto, COMMON, save, stop are not checked by precheck.sh)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 .claude/rules/fortran-conventions.md |  7 +++--
 .claude/rules/gpu-and-mpi.md         | 43 +++++++++++++++++++---------
 2 files changed, 34 insertions(+), 16 deletions(-)

diff --git a/.claude/rules/fortran-conventions.md b/.claude/rules/fortran-conventions.md
index a9b410c163..90bf777dcc 100644
--- a/.claude/rules/fortran-conventions.md
+++ b/.claude/rules/fortran-conventions.md
@@ -22,12 +22,15 @@ Every Fortran module follows this pattern:
 - Constants: descriptive names, not ALL_CAPS
 
 ## Forbidden Patterns
-These are caught by `./mfc.sh precheck` (source lint step 4/5):
+
+Caught by `./mfc.sh precheck` (source lint step 4/5):
 - `dsqrt`, `dexp`, `dlog`, `dble`, `dabs` → use `sqrt`, `exp`, `log`, `real(..., wp)`
 - `real(8)`, `real(4)` → use `wp` or `stp` kind parameters
+- Raw `!$acc` or `!$omp` directives → use Fypp GPU_* macros from `parallel_macros.fpp`
+
+Enforced by convention/code review (not automated):
 - `goto`, `COMMON` blocks, global `save` variables
 - `stop`, `error stop` → use `call s_mpi_abort()`
-- Raw `!$acc` or `!$omp` directives → use Fypp GPU_* macros from `parallel_macros.fpp`
 
 ## Precision Types
 - `wp` (working precision): used for computation. Double by default.
diff --git a/.claude/rules/gpu-and-mpi.md b/.claude/rules/gpu-and-mpi.md
index 90c96777c9..508de5b427 100644
--- a/.claude/rules/gpu-and-mpi.md
+++ b/.claude/rules/gpu-and-mpi.md
@@ -22,21 +22,36 @@ compiles to either OpenACC or OpenMP target offload depending on the build flag:
 - `shared_parallel_macros.fpp` — Shared helpers (collapse, private, reduction generators)
 
 ### Key GPU Macros (always use the `GPU_*` prefix)
-- `@:GPU_PARALLEL_LOOP(collapse=N, private=[...], reduction=[...], reductionOp='+')` —
+
+Inline macros (use `$:` prefix):
+- `$:GPU_PARALLEL_LOOP(collapse=N, private=[...], reduction=[...], reductionOp='+')` —
   Parallel loop over GPU threads. Most common GPU macro.
-- `@:END_GPU_PARALLEL_LOOP()` — Required closing for GPU_PARALLEL_LOOP.
-- `@:GPU_PARALLEL(code, ...)` — GPU parallel region (wraps code block).
-- `@:GPU_LOOP(collapse=N, ...)` — Inner loop within a GPU parallel region.
-- `@:GPU_DATA(code, copy=..., create=..., ...)` — Scoped data region.
-- `@:GPU_ENTER_DATA(create=[...])` — Allocate device memory (unscoped).
-- `@:GPU_EXIT_DATA(delete=[...])` — Free device memory.
-- `@:GPU_UPDATE(host=[...])` — Copy device → host (before MPI send).
-- `@:GPU_UPDATE(device=[...])` — Copy host → device (after MPI receive).
-- `@:GPU_ROUTINE(function_name=..., nohost=True/False)` — Mark routine for device.
-- `@:GPU_DECLARE(copyin=[...], link=[...])` — Declare device-resident data.
-- `@:GPU_ATOMIC(atomic='update')` — Atomic operation on device.
-- `@:GPU_WAIT()` — Synchronization barrier.
-- `@:GPU_HOST_DATA(code, use_device_addr=[...])` — Host code with device pointers.
+- `$:END_GPU_PARALLEL_LOOP()` — Required closing for GPU_PARALLEL_LOOP.
+- `$:GPU_LOOP(collapse=N, ...)` — Inner loop within a GPU parallel region.
+- `$:GPU_ENTER_DATA(create=[...])` — Allocate device memory (unscoped).
+- `$:GPU_EXIT_DATA(delete=[...])` — Free device memory.
+- `$:GPU_UPDATE(host=[...])` — Copy device → host (before MPI send).
+- `$:GPU_UPDATE(device=[...])` — Copy host → device (after MPI receive).
+- `$:GPU_ROUTINE(parallelism='[seq]')` — Mark routine for device compilation.
+- `$:GPU_DECLARE(create=[...])` — Declare device-resident data.
+- `$:GPU_ATOMIC(atomic='update')` — Atomic operation on device.
+- `$:GPU_WAIT()` — Synchronization barrier.
+
+Block macros (use `#:call`/`#:endcall`):
+- `GPU_PARALLEL(...)` — GPU parallel region wrapping a code block.
+- `GPU_DATA(copy=..., create=..., ...)` — Scoped data region.
+- `GPU_HOST_DATA(use_device_addr=[...])` — Host code with device pointers.
+
+Block macro usage:
+```
+#:call GPU_PARALLEL(copyin='[var1]', copyout='[var2]')
+  $:GPU_LOOP(collapse=N)
+  do k = 0, n; do j = 0, m
+    ! loop body
+  end do; end do
+  $:END_GPU_LOOP()
+#:endcall GPU_PARALLEL
+```
 
 NEVER write raw `!$acc` or `!$omp` directives. Always use `GPU_*` Fypp macros.
 The precheck source lint will catch raw directives and fail.

From 3286cfa6616f3b475692f698df8ac4cf4f61630f Mon Sep 17 00:00:00 2001
From: Spencer Bryngelson <sbryngelson@gmail.com>
Date: Mon, 23 Feb 2026 19:47:31 -0500
Subject: [PATCH 6/7] Fix documentation accuracy: checklist counts, phantom
 macro, missing patterns

- Fix "3-location checklist" heading to "4-location" (parameter-system.md)
- Fix "all 3 locations" to "all 4 locations" in PR checklist (common-pitfalls.md)
- Remove non-existent $:END_GPU_LOOP() from block macro example (gpu-and-mpi.md)
- Add d0 literals, double precision, dcos/dsin/dtan to forbidden patterns list
- Reference toolchain/bootstrap/precheck.sh for full forbidden pattern list
- Fix Frontier system slug: OpenACC/OpenMP, not just OpenACC
- Clarify stp as "field data arrays and I/O", not just "I/O"
- Document @:PROHIBIT, @:ASSERT, @:LOG error-checking macros
- Document @:PREFER_GPU unified memory macro
- Document m_checker*.fpp Fortran-side runtime validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 .claude/rules/common-pitfalls.md     |  2 +-
 .claude/rules/fortran-conventions.md | 15 ++++++++++++---
 .claude/rules/gpu-and-mpi.md         |  2 +-
 .claude/rules/parameter-system.md    |  8 +++++++-
 CLAUDE.md                            |  7 ++++---
 5 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/.claude/rules/common-pitfalls.md b/.claude/rules/common-pitfalls.md
index d25cb0041a..7defaba55c 100644
--- a/.claude/rules/common-pitfalls.md
+++ b/.claude/rules/common-pitfalls.md
@@ -36,7 +36,7 @@ Before submitting a PR:
 - [ ] `./mfc.sh precheck -j 8` (5 CI lint checks)
 - [ ] `./mfc.sh build -j 8` (compiles)
 - [ ] `./mfc.sh test --only <relevant> -j 8` (tests pass)
-- [ ] If adding parameters: all 3 locations updated
+- [ ] If adding parameters: all 4 locations updated
 - [ ] If modifying `src/common/`: all three targets tested
 - [ ] If changing output: golden files regenerated for affected tests
 - [ ] One logical change per commit
diff --git a/.claude/rules/fortran-conventions.md b/.claude/rules/fortran-conventions.md
index 90bf777dcc..c18df9b901 100644
--- a/.claude/rules/fortran-conventions.md
+++ b/.claude/rules/fortran-conventions.md
@@ -24,17 +24,26 @@ Every Fortran module follows this pattern:
 ## Forbidden Patterns
 
 Caught by `./mfc.sh precheck` (source lint step 4/5):
-- `dsqrt`, `dexp`, `dlog`, `dble`, `dabs` → use `sqrt`, `exp`, `log`, `real(..., wp)`
+- `dsqrt`, `dexp`, `dlog`, `dble`, `dabs`, `dcos`, `dsin`, `dtan`, etc. → use generic intrinsics
+- `1.0d0`, `2.5d-3` (Fortran `d` exponent literals) → use `1.0_wp`, `2.5e-3_wp`
+- `double precision` → use `real(wp)` or `real(stp)`
 - `real(8)`, `real(4)` → use `wp` or `stp` kind parameters
 - Raw `!$acc` or `!$omp` directives → use Fypp GPU_* macros from `parallel_macros.fpp`
+- Full list of forbidden patterns: `toolchain/bootstrap/precheck.sh`
 
 Enforced by convention/code review (not automated):
 - `goto`, `COMMON` blocks, global `save` variables
-- `stop`, `error stop` → use `call s_mpi_abort()`
+- `stop`, `error stop` → use `call s_mpi_abort()` or `@:PROHIBIT()`/`@:ASSERT()`
+
+## Error Checking Macros (from macros.fpp)
+- `@:PROHIBIT(condition, message)` — Runtime constraint check; aborts with file/line info
+- `@:ASSERT(predicate, message)` — Invariant assertion; aborts if predicate is false
+- `@:LOG(expr)` — Debug logging, active only in `MFC_DEBUG` builds
+- Fortran-side runtime validation also exists in `m_checker*.fpp` files using `@:PROHIBIT`
 
 ## Precision Types
 - `wp` (working precision): used for computation. Double by default.
-- `stp` (storage precision): used for I/O. Double by default.
+- `stp` (storage precision): used for field data arrays and I/O. Double by default.
 - In single-precision mode (`--single`): both become single.
 - In mixed-precision mode (`--mixed`): wp=double, stp=half.
 - MPI type matching: `mpi_p` must match `wp`, `mpi_io_p` must match `stp`.
diff --git a/.claude/rules/gpu-and-mpi.md b/.claude/rules/gpu-and-mpi.md
index 508de5b427..ad84067b67 100644
--- a/.claude/rules/gpu-and-mpi.md
+++ b/.claude/rules/gpu-and-mpi.md
@@ -49,7 +49,6 @@ Block macro usage:
   do k = 0, n; do j = 0, m
     ! loop body
   end do; end do
-  $:END_GPU_LOOP()
 #:endcall GPU_PARALLEL
 ```
 
@@ -59,6 +58,7 @@ The precheck source lint will catch raw directives and fail.
 ### Memory Management Macros (from macros.fpp)
 - `@:ALLOCATE(var1, var2, ...)` — Fortran allocate + `GPU_ENTER_DATA(create=...)`
 - `@:DEALLOCATE(var1, var2, ...)` — `GPU_EXIT_DATA(delete=...)` + Fortran deallocate
+- `@:PREFER_GPU(var1, var2, ...)` — NVIDIA unified memory page placement hint
 - Every `@:ALLOCATE` MUST have a matching `@:DEALLOCATE` in finalization
 - Conditional allocation MUST have conditional deallocation
 
diff --git a/.claude/rules/parameter-system.md b/.claude/rules/parameter-system.md
index 2cadc42ecb..d274e5967c 100644
--- a/.claude/rules/parameter-system.md
+++ b/.claude/rules/parameter-system.md
@@ -23,7 +23,7 @@ MFC has ~3,300 simulation parameters defined in Python and read by Fortran via n
    - Reads `&user_inputs` namelist
    - Each parameter must be declared in the namelist statement
 
-## Adding a New Parameter (3-location checklist)
+## Adding a New Parameter (4-location checklist)
 
 YOU MUST update all 4 locations. Missing any causes silent failures or compile errors.
 
@@ -43,6 +43,12 @@ YOU MUST update all 4 locations. Missing any causes silent failures or compile e
 - Create new cases with `./mfc.sh new <name>`
 - Search parameters with `./mfc.sh params <query>`
 
+## Fortran-Side Runtime Validation
+Each target has `m_checker*.fpp` files (e.g., `src/simulation/m_checker.fpp`,
+`src/common/m_checker_common.fpp`) containing runtime parameter validation using
+`@:PROHIBIT(condition, message)`. When adding parameters with physics constraints,
+add Fortran-side checks here in addition to `case_validator.py`.
+
 ## Analytical Initial Conditions
 String expressions in parameters become Fortran code via `case.py.__get_analytic_ic_fpp()`.
 These are compiled into the binary, so syntax errors cause build failures, not runtime errors.
diff --git a/CLAUDE.md b/CLAUDE.md
index 90a2a27ea0..312070f184 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -75,7 +75,7 @@ Supported systems and their slugs (full list in `toolchain/modules`):
 | Slug | System | GPU Backend | Example |
 |------|--------|-------------|---------|
 | `p` | GT Phoenix | OpenACC (nvfortran) | `source ./mfc.sh load -c p -m g` |
-| `f` | OLCF Frontier | OpenACC (Cray ftn) | `source ./mfc.sh load -c f -m g` |
+| `f` | OLCF Frontier | OpenACC/OpenMP (Cray ftn) | `source ./mfc.sh load -c f -m g` |
 | `tuo` | LLNL Tuolumne | OpenMP (Cray ftn) | `source ./mfc.sh load -c tuo -m g` |
 | `d` | NCSA Delta | OpenACC (nvfortran) | `source ./mfc.sh load -c d -m g` |
 | `b` | PSC Bridges2 | OpenACC (nvfortran) | `source ./mfc.sh load -c b -m g` |
@@ -132,7 +132,8 @@ NEVER use raw OpenACC/OpenMP pragmas (`!$acc`, `!$omp`). Use `GPU_*` Fypp macros
   Raw `#ifdef`/`#ifndef` preprocessor guards for feature/compiler/library gating ARE normal.
 NEVER use double-precision intrinsics: `dsqrt`, `dexp`, `dlog`, `dble`, `dabs`, `real(8)`, `real(4)`.
   Use generic intrinsics (`sqrt`, `exp`, `log`) and precision types (`wp`, `stp`).
-NEVER use `stop` or `error stop`. Use `call s_mpi_abort()`.
+NEVER use `d` exponent literals (`1.0d0`). Use `1.0_wp` instead.
+NEVER use `stop` or `error stop`. Use `call s_mpi_abort()` or `@:PROHIBIT()`/`@:ASSERT()`.
 NEVER use `goto`, `COMMON` blocks, or global `save` variables.
 
 Every `@:ALLOCATE(...)` MUST have a matching `@:DEALLOCATE(...)`.
@@ -153,7 +154,7 @@ Changes to `src/common/` affect ALL three executables. Test comprehensively.
 
 ## Precision System
 
-- `wp` = working precision (computation). `stp` = storage precision (I/O).
+- `wp` = working precision (computation). `stp` = storage precision (field data arrays and I/O).
 - Default: both double. Single mode: both single. Mixed: wp=double, stp=half.
 - MPI types must match: `mpi_p` ↔ `wp`, `mpi_io_p` ↔ `stp`.
 

From 7104b115b4f9d44f0dff5f776e42f5a873238733 Mon Sep 17 00:00:00 2001
From: Spencer Bryngelson <sbryngelson@gmail.com>
Date: Mon, 23 Feb 2026 20:21:13 -0500
Subject: [PATCH 7/7] Add field indexing, ghost cells, test system, analytical
 IC docs

- Fix parameter checklist: first 3 mandatory, #4 only if constraints exist
- Fix namelist format: &user_inputs ... &end/ (not &user_inputs / ... / &end)
- Update counts: ~3,400 params (was ~3,300), 560+ tests (was 500+)
- Add ghost cell allocation pattern (-buff_size:m+buff_size, idwint/idwbuff)
- Add field variable indexing system (cont_idx, mom_idx, E_idx, adv_idx, sys_size)
- Add test system details: programmatic generation, CaseGeneratorStack, UUID hashing
- Add scalar_field uses stp (not wp) to precision types
- Add analytical IC available variables (x, y, z, xc, yc, zc, lx, ly, lz, r, e, t)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 .claude/rules/common-pitfalls.md     | 31 +++++++++++++++++++++++-----
 .claude/rules/fortran-conventions.md |  5 +++++
 .claude/rules/parameter-system.md    | 18 +++++++++++++---
 CLAUDE.md                            | 12 +++++------
 4 files changed, 52 insertions(+), 14 deletions(-)

diff --git a/.claude/rules/common-pitfalls.md b/.claude/rules/common-pitfalls.md
index 7defaba55c..1e5f58aa6b 100644
--- a/.claude/rules/common-pitfalls.md
+++ b/.claude/rules/common-pitfalls.md
@@ -1,10 +1,28 @@
 # Common Pitfalls
 
-## Array Bounds
-- Arrays use non-unity lower bounds with ghost cells
+## Array Bounds & Ghost Cells
+- Grid dimensions: `m`, `n`, `p` (cells in x, y, z). 1D: n=p=0. 2D: p=0.
+- Interior domain: `0:m`, `0:n`, `0:p`
+- Buffer/ghost region: `-buff_size:m+buff_size` (similar for n, p in multi-D)
+- `buff_size` depends on WENO order and features (typically `2*weno_polyn + 2`)
+- Domain bounds: `idwint(1:3)` (interior `0:m`), `idwbuff(1:3)` (with ghost cells)
+- Cell-center coords: `x_cc(-buff_size:m+buff_size)`, `y_cc(...)`, `z_cc(...)`
+- Cell-boundary coords: `x_cb(-1-buff_size:m+buff_size)`
 - Riemann solver indexing: left state at `j`, right state at `j+1`
 - Off-by-one errors in ghost cell regions are a common source of bugs
 
+## Field Variable Indexing
+- Conserved variables: `q_cons_vf(1:sys_size)`. Primitive: `q_prim_vf(1:sys_size)`.
+- Index ranges depend on `model_eqns` and enabled features (set in `m_global_parameters.fpp`):
+  - `cont_idx` — continuity (partial densities, one per fluid)
+  - `mom_idx` — momentum components
+  - `E_idx` — total energy (scalar)
+  - `adv_idx` — volume fractions (advection equations)
+  - `bub_idx`, `stress_idx`, `xi_idx`, `species_idx`, `B_idx`, `c_idx` — optional
+- Shorthand scalars: `momxb`/`momxe`, `contxb`/`contxe`, `advxb`/`advxe`, etc.
+- `sys_size` = total number of conserved variables (computed at startup)
+- Changing `model_eqns` or enabling features changes ALL index positions
+
 ## Blast Radius
 - `src/common/` is shared by ALL three executables (pre_process, simulation, post_process)
 - Any change to common/ requires testing all three targets
@@ -23,12 +41,15 @@
 - Fypp macros must expand correctly for both GPU and CPU builds
 - GPU builds only work with nvfortran, Cray ftn, and AMD flang
 
-## Test Golden Files
-- Tests compare output against golden files in `tests/<hash>/golden.txt`
+## Test System
+- Tests are generated **programmatically** in `toolchain/mfc/test/cases.py`, not standalone files
+- Each test is a parameter modification on top of `BASE_CFG` defaults
+- Test UUID = CRC32 hash of the test's trace string; `./mfc.sh test -l` lists all
+- To add a test: modify `cases.py` using `CaseGeneratorStack` push/pop pattern
+- Golden files: `tests/<UUID>/golden.txt` — tolerance-based comparison, not exact match
 - If your change intentionally modifies output, regenerate golden files:
   `./mfc.sh test --generate --only <affected_tests> -j 8`
 - Do not regenerate ALL golden files unless you understand every output change
-- Golden file diffs are compared with tolerance, not exact match
 
 ## PR Checklist
 Before submitting a PR:
diff --git a/.claude/rules/fortran-conventions.md b/.claude/rules/fortran-conventions.md
index c18df9b901..d51dbf2048 100644
--- a/.claude/rules/fortran-conventions.md
+++ b/.claude/rules/fortran-conventions.md
@@ -50,6 +50,11 @@ Enforced by convention/code review (not automated):
 - Always use generic intrinsics: `sqrt` not `dsqrt`, `abs` not `dabs`.
 - Cast with `real(..., wp)` or `real(..., stp)`, never `dble(...)`.
 
+Key derived types (`m_derived_types.fpp`):
+- `scalar_field` — `real(stp), pointer :: sf(:,:,:)`. Uses `stp`, NOT `wp`.
+- `vector_field` — allocatable array of `scalar_field` components.
+- New field arrays MUST use `stp` for storage precision consistency.
+
 ## Size Guidelines (soft)
 - Subroutine: ≤500 lines
 - Helper routine: ≤150 lines
diff --git a/.claude/rules/parameter-system.md b/.claude/rules/parameter-system.md
index d274e5967c..28f020090a 100644
--- a/.claude/rules/parameter-system.md
+++ b/.claude/rules/parameter-system.md
@@ -1,7 +1,7 @@
 # Parameter System
 
 ## Overview
-MFC has ~3,300 simulation parameters defined in Python and read by Fortran via namelist files.
+MFC has ~3,400 simulation parameters defined in Python and read by Fortran via namelist files.
 
 ## Parameter Flow: Python → Fortran
 
@@ -17,7 +17,7 @@ MFC has ~3,300 simulation parameters defined in Python and read by Fortran via n
 
 3. **Input Generation**: `toolchain/mfc/run/input.py`
    - Python case dict → Fortran namelist `.inp` file
-   - Format: `&user_inputs / ... / &end`
+   - Format: `&user_inputs` ... `&end/`
 
 4. **Fortran Reading**: `src/*/m_start_up.fpp`
    - Reads `&user_inputs` namelist
@@ -25,7 +25,8 @@ MFC has ~3,300 simulation parameters defined in Python and read by Fortran via n
 
 ## Adding a New Parameter (4-location checklist)
 
-YOU MUST update all 4 locations. Missing any causes silent failures or compile errors.
+YOU MUST update the first 3 locations. Missing any causes silent failures or compile errors.
+Location 4 is required only if the parameter has physics constraints.
 
 1. **`toolchain/mfc/params/definitions.py`**: Add parameter with type, default, constraints
 2. **`src/*/m_global_parameters.fpp`**: Declare the Fortran variable in the relevant
@@ -52,3 +53,14 @@ add Fortran-side checks here in addition to `case_validator.py`.
 ## Analytical Initial Conditions
 String expressions in parameters become Fortran code via `case.py.__get_analytic_ic_fpp()`.
 These are compiled into the binary, so syntax errors cause build failures, not runtime errors.
+
+Available variables in analytical IC expressions:
+- `x`, `y`, `z` — cell-center coordinates (mapped to `x_cc(i)`, `y_cc(j)`, `z_cc(k)`)
+- `xc`, `yc`, `zc` — patch centroid coordinates
+- `lx`, `ly`, `lz` — patch lengths
+- `r` — patch radius; `eps`, `beta` — vortex parameters
+- `e` — Euler's number (2.71828...)
+- Standard Fortran math intrinsics available: `sin`, `cos`, `exp`, `sqrt`, `abs`, etc.
+- For moving immersed boundaries: `t` (simulation time) is also available
+
+Example: `'patch_icpp(1)%vel(2)': '(x - xc) * exp(-((x-xc)**2 + (y-yc)**2))'`
diff --git a/CLAUDE.md b/CLAUDE.md
index 312070f184..2b6a4a6d2d 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -29,7 +29,7 @@ All commands run from the repo root via `./mfc.sh`.
 ./mfc.sh run case.py -e batch -N 2 -n 4 -c phoenix -a ACCOUNT  # Batch submit on Phoenix
 
 # Testing
-./mfc.sh test -j 8                         # Run full test suite (500+ tests)
+./mfc.sh test -j 8                         # Run full test suite (560+ tests)
 ./mfc.sh test --only 1D -j 8              # Only 1D tests
 ./mfc.sh test --only 2D Bubbles -j 8      # Only 2D bubble tests
 ./mfc.sh test --only <UUID> -j 8          # Run one specific test by UUID
@@ -50,7 +50,7 @@ source ./mfc.sh load -c p -m c             # Load Phoenix CPU modules
 
 # Other
 ./mfc.sh validate case.py                  # Validate case file without running
-./mfc.sh params <query>                    # Search ~3300 case parameters
+./mfc.sh params <query>                    # Search ~3,400 case parameters
 ./mfc.sh clean                             # Remove build artifacts
 ./mfc.sh new <name>                        # Create new case from template
 ```
@@ -117,11 +117,11 @@ src/
   simulation/     # CFD solver (GPU-accelerated via OpenACC / OpenMP target offload)
   post_process/   # Data output and visualization
 toolchain/        # Python CLI, build system, testing, parameter management
-  mfc/params/definitions.py   # ~3300 parameter definitions (source of truth)
+  mfc/params/definitions.py   # ~3,400 parameter definitions (source of truth)
   mfc/case_validator.py       # Physics constraint validation
   mfc/test/                   # Test runner and case generation
 examples/         # Example simulation cases (case.py files)
-tests/            # 500+ regression test golden files
+tests/            # 560+ regression test golden files
 ```
 
 Source files are `.fpp` (Fortran + Fypp macros), preprocessed to `.f90` by CMake.
@@ -137,11 +137,11 @@ NEVER use `stop` or `error stop`. Use `call s_mpi_abort()` or `@:PROHIBIT()`/`@:
 NEVER use `goto`, `COMMON` blocks, or global `save` variables.
 
 Every `@:ALLOCATE(...)` MUST have a matching `@:DEALLOCATE(...)`.
-Every new parameter MUST be added in 4 places:
+Every new parameter MUST be added in at least 3 places (4 if it has constraints):
   1. `toolchain/mfc/params/definitions.py` (parameter definition)
   2. Fortran variable declaration in `src/*/m_global_parameters.fpp`
   3. Fortran namelist in `src/*/m_start_up.fpp` (namelist binding)
-  4. `toolchain/mfc/case_validator.py` (if constraints exist)
+  4. `toolchain/mfc/case_validator.py` (only if parameter has physics constraints)
 
 Changes to `src/common/` affect ALL three executables. Test comprehensively.