Fix case statement parsing and add real-world test cases#283
Merged
Conversation
The `separator_op` grammar rule matched a bare `;` which greedily consumed the first character of the `;;` (DSEMI) token inside case items when the command and `;;` were on the same line (e.g., `echo "hi" ;;`). Added a negative lookahead (`";" ~ !";"`) so `;` only matches when not followed by another `;`. Also adds real-world test cases from popular install scripts (pixi, rustup, nvm, docker, homebrew) to prevent regressions. https://claude.ai/code/session_01KeAvJYzHSHut73dHikHiwm
The grammar allows $(cmd) command substitution and $? exit status inside
parameter expansions like ${VAR:-$(fallback)}, but the parser's match
statement for PARAMETER_PENDING_WORD did not handle Rule::SUB_COMMAND or
Rule::EXIT_STATUS, causing "Unexpected rule" errors on real-world install
scripts.
https://claude.ai/code/session_01KeAvJYzHSHut73dHikHiwm
Two additional issues found when running the pixi install script:
1. PARAMETER_PENDING_WORD stops at `:`, so `${VAR:-https://example.com}`
only parsed `https` as the default value. Changed VAR_DEFAULT_VALUE,
VAR_ASSIGN_DEFAULT, VAR_ALTERNATE_VALUE, VAR_CHECK_UNSET, and
VAR_CHECK_SET to use PATTERN_PENDING_WORD (which allows `:`) instead
of PARAMETER_PENDING_WORD. VAR_SUBSTRING still uses
PARAMETER_PENDING_WORD since it needs `:` as a separator.
2. `set -eu` failed because `-u` (nounset) wasn't recognized. Added
ShellOptions::NoUnset variant and set command handling.
https://claude.ai/code/session_01KeAvJYzHSHut73dHikHiwm
…ORD in conditions Three features that unblock real-world install scripts: 1. Heredoc support (<<DELIM ... DELIM): - Pre-processes input to extract heredoc bodies before PEG parsing - Supports unquoted (with $VAR expansion) and quoted (<<'DELIM') heredocs - Supports <<- for tab stripping - Added IoFile::HereDoc variant and pipe-based execution 2. Condition negation ([ ! expr ] and [[ ! expr ]]): - Added condition_negation to grammar for [ ], [[ ]], and test - Added ConditionInner::Negation variant - Handles evaluation by inverting the inner condition result 3. UNQUOTED_PENDING_WORD in conditions: - Grammar allows bare words in conditions but parser rejected them - Now treats bare words as non-empty string checks (-n equivalent) With these fixes: - pixi install script: runs cleanly (stops at curl/wget check) - nvm install script: parses and runs - rustup install script: progresses further (blocked by assignment-only commands) https://claude.ai/code/session_01KeAvJYzHSHut73dHikHiwm
…on, case
Heredoc tests added:
- Empty heredoc, custom delimiter, special characters (< > | & ; ())
- Command substitution expansion in body
- Multiple heredocs in sequence
- Tab stripping with <<-
- Multiple variable expansions per line
- Double-quoted delimiter (no expansion)
Condition tests added:
- Negation with -n and -z: [ ! -n "" ], [ ! -z "hello" ]
- Negation with numeric ops: [ ! 5 -eq 3 ], [ ! 5 -gt 10 ]
- Negation in compound conditions: [ ! expr ] && [ expr ]
- test builtin with negation
- Bare word in [[ ]] (implicit -n check): [[ "notempty" ]], [[ "" ]]
Variable expansion tests added:
- Nested defaults: ${A:-${B:-deep_default}}
- Colons in alternate value and assign default
- Command substitution with pipes in default: ${VAR:-$(echo x | tr ...)}
Case statement tests added:
- Empty case body (just ;;)
- Case with only default pattern
- No-match case (no default)
- Inline with multiple commands: a) cmd1; cmd2 ;;
- Command substitution in case word
- Nested case statements
https://claude.ai/code/session_01KeAvJYzHSHut73dHikHiwm
Replace filesystem-based negation tests that used /tmp (which doesn't exist on Windows) with string and numeric comparison tests that work on all platforms. https://claude.ai/code/session_01KeAvJYzHSHut73dHikHiwm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes a grammar issue in case statement parsing and adds comprehensive test coverage for real-world case statement patterns.
Key Changes
separator_oprule ingrammar.pestto prevent;;(double semicolon) from being incorrectly parsed as two separate semicolon separators by adding a negative lookahead (!";")case.sh: Added basic pattern matching tests (simple strings, wildcards, tilde expansion)case_real_world.sh: Added 13 real-world case statement examples covering:Implementation Details
The grammar fix ensures that the
;;case statement terminator is properly recognized as a single token rather than being split into two separate semicolons, which was likely causing parsing errors in complex case statements.https://claude.ai/code/session_01KeAvJYzHSHut73dHikHiwm