Implement Output Formatter Framework with Trait-Based Architecture

[unclesp1d3r]

Context

StringyMcStringFace is a binary string extraction tool that analyzes executables and extracts meaningful strings with semantic classification and scoring. The tool needs a flexible output formatting system to support multiple output formats optimized for different use cases (interactive analysis, automation, YARA rule generation).

Currently, src/output/mod.rs exists but only contains a placeholder comment. The architecture is well-defined in docs/src/output-formats.md and docs/src/architecture.md, and the core data structures (FoundString, Encoding, Tag, etc.) are already implemented in src/types.rs.

Problem Statement

The tool requires a robust output formatting framework that:

• Supports multiple output formats (Human-readable tables, JSON Lines, YARA rules)
• Provides a consistent trait-based interface for formatters
• Allows easy extension for future formats (CSV, XML, Markdown)
• Handles edge cases (special character escaping, long string truncation, color coding)
• Integrates with the existing FoundString data structure
• Supports output customization (filtering, sorting, field selection)

Requirements (from Requirement 6.1)

1. Trait Design: Create a Formatter trait that defines the interface for all output formatters
2. Configuration: Implement output configuration options (format selection, color support, truncation limits)
3. Core Implementations: Provide initial implementations for:
   • Human-readable table format (default)
   • JSON Lines format (machine-readable)
   • YARA rule format (detection rules)
4. Error Handling: Proper error types for formatting failures
5. Testing: Unit tests for each formatter with edge cases
6. Documentation: Inline documentation for the trait and configuration options

Proposed Solution

1. Module Structure

src/output/
├── mod.rs           # Public API, Formatter trait, OutputConfig
├── human.rs         # HumanFormatter (interactive table view)
├── json.rs          # JsonFormatter (JSON Lines)
└── yara.rs          # YaraFormatter (YARA rules)

2. Trait Design

pub trait Formatter {
    /// Format a collection of strings for output
    fn format(&self, strings: &[FoundString], config: &OutputConfig) -> Result<String>;
    
    /// Format a single string (for streaming output)
    fn format_one(&self, string: &FoundString, config: &OutputConfig) -> Result<String>;
    
    /// Write header/preamble if needed
    fn write_header(&self, writer: &mut dyn Write, config: &OutputConfig) -> Result<()>;
    
    /// Write footer/postamble if needed
    fn write_footer(&self, writer: &mut dyn Write, config: &OutputConfig) -> Result<()>;
}

3. Configuration Structure

pub struct OutputConfig {
    pub format: OutputFormat,
    pub color: bool,
    pub truncate_length: Option<usize>,
    pub min_score: Option<i32>,
    pub sections_filter: Option<Vec<String>>,
    pub tags_filter: Option<Vec<Tag>>,
    pub max_results: Option<usize>,
}

pub enum OutputFormat {
    Human,
    Json,
    Yara,
}

4. Implementation Details

Human Formatter

• Use comfy-table or similar crate for table rendering
• Implement color coding based on score ranges (green: 80+, yellow: 60-79, red: <60)
• Handle truncation with "..." indicator for long strings
• Sort by score (descending) by default

JSON Formatter

• One JSON object per line (JSON Lines format)
• Leverage existing serde derives on FoundString
• No pretty-printing for pipeline compatibility
• Each object contains all fields from FoundString

YARA Formatter

• Proper escaping for YARA string syntax
• Group strings by semantic tag (URLs, GUIDs, paths, etc.)
• Include metadata (source file, generation timestamp)
• Add appropriate ascii/wide modifiers based on encoding
• Filter to high-confidence strings (score >= 80) by default

5. Testing Strategy

• Unit tests for each formatter with sample FoundString instances
• Edge cases:
  • Empty string collections
  • Strings with special characters (quotes, backslashes, newlines)
  • Very long strings (truncation)
  • UTF-16 encoded strings in YARA output
  • Null/missing fields (section, rva)
• Integration tests with real binary analysis output

6. Integration Points

• Called from main.rs after classification and ranking phases
• Receives sorted Vec<FoundString> from the analysis pipeline
• Configuration driven by CLI arguments (parsed by clap)
• Output to stdout by default, file redirection supported

Acceptance Criteria

• Formatter trait defined with documentation
• OutputConfig structure implemented with all options
• HumanFormatter implementation with color coding and tables
• JsonFormatter implementation producing valid JSON Lines
• YaraFormatter implementation with proper escaping and modifiers
• Unit tests achieving >80% code coverage for output module
• Integration test demonstrating all three formats
• Inline documentation for public APIs
• README updated with output format examples (reference existing docs/src/output-formats.md)

References

• Architecture Documentation: docs/src/architecture.md
• Output Format Specifications: docs/src/output-formats.md
• Core Data Structures: src/types.rs
• Related Issue: Task-ID stringy-analyzer/output-formatting-framework

Dependencies

• serde / serde_json (already in Cargo.toml)
• Consider adding: comfy-table or prettytable-rs for human-readable output
• Consider adding: colored or termcolor for color support

Estimated Effort

Medium complexity - requires trait design, three formatter implementations, and comprehensive testing. Estimated 1-2 days of focused development.

[unclesp1d3r]

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch details of issue #25
gh issue view 25 --repo EvilBit-Labs/StringyMcStringFace --json title,body,labels,assignees,milestone,state,author,createdAt,updatedAt

Length of output: 1364

---

🏁 Script executed:

#!/bin/bash
# Explore the project structure to understand the codebase
echo "=== Project Structure ==="
fd -t d -d 2 src

echo -e "\n=== Check if src/output exists ==="
ls -la src/output 2>&1 || echo "Directory does not exist yet"

echo -e "\n=== Look for existing output or formatting code ==="
rg -n "output|format" --type rust -g '!target/**' -C 2 | head -50

echo -e "\n=== Check main.rs or lib.rs for architecture ==="
cat src/main.rs 2>&1 || cat src/lib.rs 2>&1 || echo "Could not find main entry point"

echo -e "\n=== Look for existing traits ==="
rg -n "trait " --type rust -g '!target/**' -C 2 | head -30

echo -e "\n=== Check Cargo.toml for dependencies that might relate to output ==="
cat Cargo.toml | grep -A 20 "\[dependencies\]"

Length of output: 1695

---

🏁 Script executed:

#!/bin/bash
# Check current output module content
echo "=== Current src/output/mod.rs ==="
cat src/output/mod.rs

echo -e "\n=== Look for requirements or design docs ==="
fd -e md -e txt . | head -20
cat README.md 2>&1 | head -100

echo -e "\n=== Check docs directory ==="
fd . docs -e md -e txt

echo -e "\n=== Look for other modules to understand architecture ==="
ls -la src/

echo -e "\n=== Check if there are parser or analyzer modules ==="
fd . src -e rs

Length of output: 4561

---

🏁 Script executed:

#!/bin/bash
# Read the output formats documentation
echo "=== Output Formats Documentation ==="
cat docs/src/output-formats.md

echo -e "\n=== Architecture Documentation ==="
cat docs/src/architecture.md

echo -e "\n=== Check types.rs for data structures ==="
cat src/types.rs

echo -e "\n=== Check lib.rs ==="
cat src/lib.rs

Length of output: 17262

---

🏁 Script executed:

#!/bin/bash
# Update the issue with enhanced content

gh issue edit 25 --repo EvilBit-Labs/StringyMcStringFace \
  --title "Implement Output Formatter Framework with Trait-Based Architecture" \
  --body "## Context

StringyMcStringFace is a binary string extraction tool that analyzes executables and extracts meaningful strings with semantic classification and scoring. The tool needs a flexible output formatting system to support multiple output formats optimized for different use cases (interactive analysis, automation, YARA rule generation).

Currently, \`src/output/mod.rs\` exists but only contains a placeholder comment. The architecture is well-defined in \`docs/src/output-formats.md\` and \`docs/src/architecture.md\`, and the core data structures (\`FoundString\`, \`Encoding\`, \`Tag\`, etc.) are already implemented in \`src/types.rs\`.

## Problem Statement

The tool requires a robust output formatting framework that:
- Supports multiple output formats (Human-readable tables, JSON Lines, YARA rules)
- Provides a consistent trait-based interface for formatters
- Allows easy extension for future formats (CSV, XML, Markdown)
- Handles edge cases (special character escaping, long string truncation, color coding)
- Integrates with the existing \`FoundString\` data structure
- Supports output customization (filtering, sorting, field selection)

## Requirements (from Requirement 6.1)

1. **Trait Design**: Create a \`Formatter\` trait that defines the interface for all output formatters
2. **Configuration**: Implement output configuration options (format selection, color support, truncation limits)
3. **Core Implementations**: Provide initial implementations for:
   - Human-readable table format (default)
   - JSON Lines format (machine-readable)
   - YARA rule format (detection rules)
4. **Error Handling**: Proper error types for formatting failures
5. **Testing**: Unit tests for each formatter with edge cases
6. **Documentation**: Inline documentation for the trait and configuration options

## Proposed Solution

### 1. Module Structure

\`\`\`text
src/output/
├── mod.rs           # Public API, Formatter trait, OutputConfig
├── human.rs         # HumanFormatter (interactive table view)
├── json.rs          # JsonFormatter (JSON Lines)
└── yara.rs          # YaraFormatter (YARA rules)
\`\`\`

### 2. Trait Design

\`\`\`rust
pub trait Formatter {
    /// Format a collection of strings for output
    fn format(&self, strings: &[FoundString], config: &OutputConfig) -> Result<String>;
    
    /// Format a single string (for streaming output)
    fn format_one(&self, string: &FoundString, config: &OutputConfig) -> Result<String>;
    
    /// Write header/preamble if needed
    fn write_header(&self, writer: &mut dyn Write, config: &OutputConfig) -> Result<()>;
    
    /// Write footer/postamble if needed
    fn write_footer(&self, writer: &mut dyn Write, config: &OutputConfig) -> Result<()>;
}
\`\`\`

### 3. Configuration Structure

\`\`\`rust
pub struct OutputConfig {
    pub format: OutputFormat,
    pub color: bool,
    pub truncate_length: Option<usize>,
    pub min_score: Option<i32>,
    pub sections_filter: Option<Vec<String>>,
    pub tags_filter: Option<Vec<Tag>>,
    pub max_results: Option<usize>,
}

pub enum OutputFormat {
    Human,
    Json,
    Yara,
}
\`\`\`

### 4. Implementation Details

#### Human Formatter
- Use \`comfy-table\` or similar crate for table rendering
- Implement color coding based on score ranges (green: 80+, yellow: 60-79, red: <60)
- Handle truncation with \"...\" indicator for long strings
- Sort by score (descending) by default

#### JSON Formatter
- One JSON object per line (JSON Lines format)
- Leverage existing \`serde\` derives on \`FoundString\`
- No pretty-printing for pipeline compatibility
- Each object contains all fields from \`FoundString\`

#### YARA Formatter
- Proper escaping for YARA string syntax
- Group strings by semantic tag (URLs, GUIDs, paths, etc.)
- Include metadata (source file, generation timestamp)
- Add appropriate \`ascii\`/\`wide\` modifiers based on encoding
- Filter to high-confidence strings (score >= 80) by default

### 5. Testing Strategy

- Unit tests for each formatter with sample \`FoundString\` instances
- Edge cases:
  - Empty string collections
  - Strings with special characters (quotes, backslashes, newlines)
  - Very long strings (truncation)
  - UTF-16 encoded strings in YARA output
  - Null/missing fields (section, rva)
- Integration tests with real binary analysis output

### 6. Integration Points

- Called from \`main.rs\` after classification and ranking phases
- Receives sorted \`Vec<FoundString>\` from the analysis pipeline
- Configuration driven by CLI arguments (parsed by \`clap\`)
- Output to stdout by default, file redirection supported

## Acceptance Criteria

- [ ] \`Formatter\` trait defined with documentation
- [ ] \`OutputConfig\` structure implemented with all options
- [ ] \`HumanFormatter\` implementation with color coding and tables
- [ ] \`JsonFormatter\` implementation producing valid JSON Lines
- [ ] \`YaraFormatter\` implementation with proper escaping and modifiers
- [ ] Unit tests achieving >80% code coverage for output module
- [ ] Integration test demonstrating all three formats
- [ ] Inline documentation for public APIs
- [ ] README updated with output format examples (reference existing docs/src/output-formats.md)

## References

- Architecture Documentation: \`docs/src/architecture.md\`
- Output Format Specifications: \`docs/src/output-formats.md\`
- Core Data Structures: \`src/types.rs\`
- Related Issue: Task-ID stringy-analyzer/output-formatting-framework

## Dependencies

- \`serde\` / \`serde_json\` (already in Cargo.toml)
- Consider adding: \`comfy-table\` or \`prettytable-rs\` for human-readable output
- Consider adding: \`colored\` or \`termcolor\` for color support

## Estimated Effort

Medium complexity - requires trait design, three formatter implementations, and comprehensive testing. Estimated 1-2 days of focused development."

echo "Issue updated successfully!"

Length of output: 90

---

🏁 Script executed:

#!/bin/bash
# Check available labels in the repository
echo "=== Available labels in repository ==="
gh label list --repo EvilBit-Labs/StringyMcStringFace --json name,description --limit 100 | jq -r '.[] | "\(.name) - \(.description)"'

echo -e "\n=== Current labels on issue #25 ==="
gh issue view 25 --repo EvilBit-Labs/StringyMcStringFace --json labels | jq -r '.labels[] | .name'

Length of output: 703

---

🏁 Script executed:

#!/bin/bash
# Add documentation label since inline documentation is part of acceptance criteria
gh issue edit 25 --repo EvilBit-Labs/StringyMcStringFace --add-label "documentation"

echo "=== Updated labels on issue #25 ==="
gh issue view 25 --repo EvilBit-Labs/StringyMcStringFace --json labels | jq -r '.labels[] | .name' | sort

Length of output: 179

---

@unclesp1d3r I've successfully enhanced issue #25 with comprehensive context and implementation details. Here's what was updated:

Changes Made

1. Improved Title

• Before: "Output Formatting Framework"
• After: "Implement Output Formatter Framework with Trait-Based Architecture"

2. Enhanced Issue Body

The issue now includes:

• Context: Overview of StringyMcStringFace and the need for flexible output formatting
• Problem Statement: Clear articulation of the requirements for the output framework
• Detailed Requirements: Breakdown of Requirement 6.1 into 6 concrete sub-requirements
• Proposed Solution:
  • Module structure (mod.rs, human.rs, json.rs, yara.rs)
  • Trait design with Formatter interface
  • Configuration structure (OutputConfig, OutputFormat)
  • Implementation details for each formatter (Human table, JSON Lines, YARA rules)
  • Testing strategy with edge cases
  • Integration points with the main pipeline
• Acceptance Criteria: 9 checkboxes for tracking completion
• References: Links to architecture docs, output format specs, and core types
• Dependencies: Suggested crates (comfy-table, colored)
• Effort Estimate: Medium complexity, 1-2 days

3. Updated Labels

Added the documentation label since inline documentation is part of the acceptance criteria.

Final label set:

• area:analyzer
• documentation
• lang:rust
• needs:tests
• status:backlog
• type:enhancement

The issue is now well-structured and ready for implementation, with clear guidance on the trait-based architecture, configuration options, and the three core formatter implementations (Human, JSON, YARA). The proposed solution aligns with the existing architecture documentation and leverages the already-defined FoundString data structures.