Implement nom-based ESI parser with streaming support#43
Draft
Implement nom-based ESI parser with streaming support#43
Conversation
…ut esi:vars and recognize basic interpolation.
…little cleanup.
This merge brings in the new nom-based parser implementation up to commit f216023: Key changes: - Added nom dependency for modern parsing capabilities - Introduced new_parse.rs with comprehensive nom-based ESI parser - Added parser_types.rs with new Expr and Operator types for better type safety - Enhanced interpolation handling with process_interpolated_chars function - Improved error handling and debugging output - Fixed HTML tag closure issues in examples - Added support for expression comparisons (matches, matches_i operators) - Better integration between parser and interpreter with consistent types The new parser provides better performance, more robust parsing, and improved maintainability while maintaining full compatibility with existing ESI functionality.
… expressions and nested structures
…pdate related parsing logic
…a nested vector structure for improved handling of events
…nt handling - Introduced a request field in the Fragment struct to retain the original request. - Updated the Processor to utilize the provided fragment response processor. - Added tests to verify behavior with is_escaped_content configuration and response processing.
- Modify the `lower` function to return `Null` for `Value::Null` arguments. - Refactor `process_nom_chunk_to_output` to handle expression evaluation more robustly, including skipping output for `Null` values. - Replace the simple expression evaluator with a more comprehensive parser that supports additional operators and expressions. - Introduce parsing for interpolated strings and standalone ESI expressions. - Add support for logical, comparison, and negation operators in the parser. - Update `parser_types` to include new expression types and operators. - Enhance tests for expression parsing to cover new functionality.
…cess_chunk_to_output
- Introduced `WhenBranch` struct to represent branches in choose blocks. - Updated `Tag` enum to use `WhenBranch` and changed `Chunk` to `Element`. - Modified `Assign` tag to accept `Expr` instead of `String` for value. - Enhanced parsing logic to support interpolated expressions in assignments. - Added tests for long form assignments with interpolation and multiple variables. - Updated existing tests to reflect changes in parser structure and expression handling.
…`*chunk*` to `*element*`
…ess_include function for improved clarity and error management; update parser types to use string slices.
…sponding tests, cleaned up literals
…pport and corresponding tests
…eter evaluation Also a fix for `dsa="esi"` for alt for esi:includes
- Created a new `element_handler.rs` file to define the `ElementHandler` trait. - Implemented shared processing logic for both streaming (`DocumentHandler`) and function contexts (`FunctionHandler`). - Updated `expression.rs` and `lib.rs` to utilize the new trait for handling ESI elements. - Simplified element processing by consolidating common logic into default methods within the trait. - Ensured that context-specific behaviors for tags like `on_return`, `on_include`, `on_eval`, `on_try`, and `on_function` are clearly defined and handled appropriately.
All <esi:try> includes are dispatched into the same select() pool as bare includes, removing the Nx sequential penalty for consecutive try blocks. Try-block content reuses the main buf slots (no separate content_slots); a TryBlockTracker/AttemptTracker pair records which slots belong to each attempt and assembles the result when all resolve. - 3 new types: TryBlockTracker, AttemptTracker, SlotEntry - Request correlation keyed by (method, URL) - process_queue renamed from process_ready_queue_items for clarity
…figuration struct refactor(lib): Reduce cloning in ElementHandler::process() — pass IncludeAttributes by reference.
…t::Text→Content - Replace Expr::String(Option<String>) with Expr::String(Option<Bytes>) to eliminate allocations in the expression parser (Bytes' copy is ref count inc) - Rename Element::Text to Element::Content for clarity (vs Html passthrough) - Document Element and Expr enums
complex_document benchmark: ~15µs → ~12.5µs
- Updated tests in `streaming_behavior.rs` to utilize the `Parser` trait for improved clarity and consistency. - Changed instances of `tag` and `is_not` to use the `.parse()` method for better error handling with incomplete input. - Ensured that all relevant tests correctly assert the expected `Incomplete` results when parsing incomplete data.
minor style improvements
…ries, ensuring reference semantics
…ename and cleanup
- Clarified some documentation mistakes in README - Improved ESI expression evaluation by optimizing null value handling. - Refactored caching logic to utilize `Cow<str>` for better performance and reduced allocations. - Enhanced error handling and logging in cache configuration and request processing. - Updated various functions to use zero-copy techniques for string manipulation, improving efficiency. - Cleaned up code formatting and comments for better readability and maintainability.
…ce in parsing functions
… improved clarity and consistency
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a new parser utilizing the nom library for enhanced performance and maintainability. This implementation includes comprehensive support for ESI tags, improved error handling, and streaming processing capabilities. Refactor existing code to integrate the new parser while ensuring compatibility with current functionality. Extensive tests validate the new parser's behavior and performance.