diff --git a/README.md b/README.md
index 6e64e14..5d19cd6 100644
--- a/README.md
+++ b/README.md
@@ -181,9 +181,12 @@ See [Core Concepts](docs/user-guide/concepts.md) for architecture details.
| Command | Purpose |
|---------|---------|
| `/analyze-tech-stack` | Document repository technologies |
-| `/train-context` | Generate context training from codebase |
+| `/train-context` | Generate context training from codebase (includes verification) |
+| `/update-context` | Update existing context training with new PRs |
| `/load-context-training` | Load patterns into conversation |
+Both `/train-context` and `/update-context` automatically verify generated files for accuracy and code quality. Import paths, function signatures, and code examples are validated against your codebase. Critical issues are auto-fixed, quality issues are reported.
+
#### Utilities
| Command | Purpose |
@@ -243,6 +246,8 @@ Generate context training:
Output: `devorch/context-training/` with custom implementers and patterns
+Context-training files are automatically verified for accuracy and quality. Import paths and function signatures are validated against your codebase. Code examples are checked for best practices and security. Critical issues are auto-fixed, mismatches are reported for review.
+
### Feature Development
**1. Research phase**
diff --git a/templates/context-training/partials/verify-context.md b/templates/context-training/partials/verify-context.md
new file mode 100644
index 0000000..70b9f2a
--- /dev/null
+++ b/templates/context-training/partials/verify-context.md
@@ -0,0 +1,453 @@
+## Context Verification and Quality Check Loop
+
+This partial implements verification and quality checking with a maximum of 3 iterations. It can be called after context-training artifacts are generated or updated.
+
+**Input expected:**
+- `context_training_name`: The name of the context training to verify (e.g., "mobile-app")
+
+**Output:**
+- INCONSISTENCIES.md report (if issues found)
+- Verification and quality status
+- List of issues found and fixed
+
+---
+
+### Verification Loop (Max 3 Iterations)
+
+```markdown
+# Initialize verification state
+ITERATION=1
+MAX_ITERATIONS=3
+VERIFICATION_PASSED=false
+QUALITY_PASSED=false
+```
+
+**Iteration {ITERATION} of {MAX_ITERATIONS}**
+
+#### Step 1: Run Context Verifier
+
+Launch the **context-training/context-verifier** subagent using the Task tool:
+
+```markdown
+Task tool parameters:
+- subagent_type: "context-training/context-verifier"
+- description: "Verify context-training files against codebase"
+- prompt: "Verify the context-training files for '{context_training_name}' against the actual codebase.
+
+Context training directory: devorch/context-training/{context_training_name}/
+
+Follow your workflow to:
+1. Discover all markdown files in the context-training directory
+2. Extract verifiable elements (imports, functions, directories, file references)
+3. Distinguish between specific references and fictional/illustrative examples
+4. Verify specific references against the codebase using Glob/Grep/Read
+5. Categorize issues as:
+ - CRITICAL: Wrong paths/signatures (auto-fix these)
+ - MISMATCH: Different implementation (flag for review)
+ - FICTIONAL: Generic/illustrative examples (accept as valid)
+6. Auto-fix all critical issues with high confidence
+7. Generate INCONSISTENCIES.md report with all findings
+8. Return verification-results.json with status and details
+
+This is iteration {ITERATION} of {MAX_ITERATIONS}.
+
+Return the verification status and summary."
+```
+
+**What the subagent will do:**
+- Parse all markdown files in the context-training directory
+- Extract code examples and identify verifiable elements
+- Use Glob/Grep/Read to verify against codebase
+- Auto-fix critical issues (wrong paths, incorrect signatures)
+- Categorize mismatches and fictional examples
+- Generate detailed INCONSISTENCIES.md report
+- Return verification results as JSON
+
+**After the subagent completes:**
+
+Parse the verification results from the Task tool response. The subagent returns JSON with:
+
+```json
+{
+ "verification_status": "PASSED | ISSUES_FOUND | CRITICAL_ERRORS",
+ "iteration": 1,
+ "summary": {
+ "files_checked": 12,
+ "verifiable_elements": 156,
+ "critical_issues": 3,
+ "critical_auto_fixed": 3,
+ "mismatches": 5,
+ "fictional_accepted": 42,
+ "total_issues": 8
+ },
+ "report_path": "devorch/context-training/{name}/INCONSISTENCIES.md"
+}
+```
+
+#### Step 2: Check Verification Status
+
+**If verification_status === "PASSED":**
+- ✅ All verifiable elements are accurate
+- Continue to quality checks (Step 3)
+
+**If verification_status === "ISSUES_FOUND":**
+- Critical issues were auto-fixed
+- Mismatches were flagged for review
+
+**Present results to user:**
+
+```
+Verification Complete (Iteration {ITERATION})
+
+Status: {emoji} {status_text}
+
+Files Checked: {count}
+Critical Issues: {count} (auto-fixed)
+Mismatches: {count} (review needed)
+Fictional Examples: {count} (accepted)
+
+Report: devorch/context-training/{name}/INCONSISTENCIES.md
+```
+
+**If mismatches found, ask user:**
+
+Use AskUserQuestion tool:
+
+```json
+{
+ "questions": [
+ {
+ "question": "Verification found {count} mismatches that need review. How would you like to proceed?",
+ "header": "Mismatches",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Review and fix manually",
+ "description": "Stop here so I can review INCONSISTENCIES.md and fix issues myself, then re-run verification"
+ },
+ {
+ "label": "Continue with quality checks",
+ "description": "Proceed to quality checking. I'll review mismatches later before final use"
+ },
+ {
+ "label": "Abort verification",
+ "description": "Stop verification process. Context training may have accuracy issues"
+ }
+ ]
+ }
+ ]
+}
+```
+
+**Based on user response:**
+
+- **"Review and fix manually"**: Stop here, instruct user to:
+ 1. Review INCONSISTENCIES.md
+ 2. Fix issues in markdown files
+ 3. Re-run verification or continue the workflow
+
+- **"Continue with quality checks"**: Proceed to Step 3
+
+- **"Abort verification"**: Exit with warning that context training has unresolved issues
+
+**If verification_status === "CRITICAL_ERRORS":**
+- Major issues that couldn't be auto-fixed
+- Stop and ask user to review
+
+#### Step 3: Run Quality Checker
+
+Launch the **context-training/quality-checker** subagent using the Task tool:
+
+```markdown
+Task tool parameters:
+- subagent_type: "context-training/quality-checker"
+- description: "Check code quality in context-training files"
+- prompt: "Analyze code quality in context-training files for '{context_training_name}'.
+
+Context training directory: devorch/context-training/{context_training_name}/
+
+Follow your workflow to:
+1. Read all markdown files with code examples
+2. Analyze code examples for:
+ - Best practices (clear naming, error handling, TypeScript usage)
+ - Anti-patterns (god functions, tight coupling, magic numbers)
+ - Security issues (hardcoded secrets, injection risks, missing validation)
+ - Maintainability (code duplication, unclear intent, overly long examples)
+3. Calculate quality scores (0-100) per category
+4. Identify critical issues that should be fixed
+5. Identify warnings that are recommendations
+6. Add findings to INCONSISTENCIES.md report
+7. Return quality-results.json
+
+This is iteration {ITERATION} of {MAX_ITERATIONS}.
+
+Return the quality status and scores."
+```
+
+**What the subagent will do:**
+- Parse code examples in all markdown files
+- Check against quality criteria
+- Calculate quality scores
+- Identify critical vs warning issues
+- Update INCONSISTENCIES.md with quality findings
+- Return quality results as JSON
+
+**After the subagent completes:**
+
+Parse the quality results from the Task tool response. The subagent returns JSON with:
+
+```json
+{
+ "quality_status": "PASSED | WARNINGS | CRITICAL_ISSUES",
+ "iteration": 1,
+ "scores": {
+ "overall": 75,
+ "best_practices": 80,
+ "security": 60,
+ "maintainability": 85,
+ "anti_patterns": 90
+ },
+ "summary": {
+ "critical_issues": 1,
+ "warnings": 3,
+ "files_with_issues": 2
+ },
+ "report_updated": true
+}
+```
+
+#### Step 4: Check Quality Status
+
+**If quality_status === "PASSED":**
+- ✅ All quality checks passed
+- No critical issues or warnings
+- **Verification complete successfully**
+- Exit loop
+
+**If quality_status === "WARNINGS":**
+- Quality scores are acceptable but some improvements recommended
+- No blocking issues
+
+**Present results to user:**
+
+```
+Quality Check Complete (Iteration {ITERATION})
+
+Overall Score: {score}/100
+
+Best Practices: {score}/100
+Security: {score}/100
+Maintainability: {score}/100
+Anti-Patterns: {score}/100
+
+Critical Issues: {count}
+Warnings: {count}
+
+Updated Report: devorch/context-training/{name}/INCONSISTENCIES.md
+```
+
+**If warnings only (no critical issues):**
+
+Use AskUserQuestion tool:
+
+```json
+{
+ "questions": [
+ {
+ "question": "Quality check found {count} warnings (no critical issues). How would you like to proceed?",
+ "header": "Quality",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Accept with warnings",
+ "description": "Proceed with context training. Warnings are recommendations, not blockers"
+ },
+ {
+ "label": "Fix and retry",
+ "description": "Let me fix the warnings and re-run quality check"
+ }
+ ]
+ }
+ ]
+}
+```
+
+**Based on user response:**
+- **"Accept with warnings"**: Complete successfully with warnings noted
+- **"Fix and retry"**: Increment iteration, fix issues, loop back if under max iterations
+
+**If quality_status === "CRITICAL_ISSUES":**
+- Critical quality issues found (security, major anti-patterns)
+- Must be addressed
+
+**Present critical issues to user:**
+
+```
+⚠️ Critical Quality Issues Found
+
+{List critical issues with file/line references}
+
+These issues should be fixed before using this context training.
+```
+
+Use AskUserQuestion tool:
+
+```json
+{
+ "questions": [
+ {
+ "question": "Critical quality issues were found. How would you like to proceed?",
+ "header": "Critical",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Fix and retry (Recommended)",
+ "description": "Fix critical issues and re-run verification (iteration {next_iteration}/{max_iterations})"
+ },
+ {
+ "label": "Proceed anyway",
+ "description": "Accept context training with critical issues. Not recommended for production use"
+ },
+ {
+ "label": "Abort",
+ "description": "Stop here. I'll fix issues manually"
+ }
+ ]
+ }
+ ]
+}
+```
+
+**Based on user response:**
+
+- **"Fix and retry"**:
+ - Increment `ITERATION`
+ - If `ITERATION <= MAX_ITERATIONS`: Loop back to Step 1
+ - If `ITERATION > MAX_ITERATIONS`: Go to Step 5 (Max Iterations Reached)
+
+- **"Proceed anyway"**:
+ - Complete with critical warnings
+ - Document that critical issues exist
+ - Not recommended for production
+
+- **"Abort"**:
+ - Exit verification
+ - Provide guidance on fixing issues
+
+#### Step 5: Max Iterations Reached
+
+If `ITERATION > MAX_ITERATIONS` and issues still exist:
+
+```
+⚠️ Maximum Verification Iterations Reached
+
+After {MAX_ITERATIONS} attempts, some issues remain:
+
+Verification Status: {status}
+Quality Status: {status}
+
+Report: devorch/context-training/{name}/INCONSISTENCIES.md
+```
+
+Use AskUserQuestion tool:
+
+```json
+{
+ "questions": [
+ {
+ "question": "Maximum iterations reached with unresolved issues. How would you like to proceed?",
+ "header": "Max Reached",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Accept with warnings",
+ "description": "Use context training as-is. I understand there may be accuracy or quality issues"
+ },
+ {
+ "label": "Abort and fix manually",
+ "description": "Stop here. I'll review INCONSISTENCIES.md and fix issues myself"
+ }
+ ]
+ }
+ ]
+}
+```
+
+**Based on user response:**
+- **"Accept with warnings"**: Complete with documented issues
+- **"Abort and fix manually"**: Exit with instructions
+
+---
+
+### Verification Complete
+
+**If all checks passed:**
+
+```
+✅ Verification and Quality Checks Passed
+
+Context training files are accurate and high quality.
+
+Location: devorch/context-training/{context_training_name}/
+Status: Ready to use
+
+No issues found.
+```
+
+**If completed with warnings:**
+
+```
+✅ Verification Complete (with warnings)
+
+Context training files have been verified.
+
+Location: devorch/context-training/{context_training_name}/
+Report: INCONSISTENCIES.md
+
+{X} warnings noted - see report for details.
+These are recommendations, not blocking issues.
+```
+
+**If completed with critical issues accepted:**
+
+```
+⚠️ Verification Complete (with critical issues)
+
+Context training files have critical issues:
+- {issue 1}
+- {issue 2}
+
+Location: devorch/context-training/{context_training_name}/
+Report: INCONSISTENCIES.md
+
+⚠️ Not recommended for production use until issues are resolved.
+```
+
+---
+
+### Next Steps After Verification
+
+Based on the verification outcome, provide appropriate next steps:
+
+**If passed without issues:**
+1. Context training is ready to use
+2. Configure in devorch/config.local.yml (if not already configured):
+ ```yaml
+ profile:
+ context_training: {context_training_name}
+ ```
+3. Run `devorch install` to activate
+4. Test with spec or implementation commands
+
+**If completed with warnings:**
+1. Review INCONSISTENCIES.md for recommendations
+2. Optionally improve patterns based on warnings
+3. Context training is still usable as-is
+4. Configure and activate as above
+
+**If completed with critical issues:**
+1. **Do not use in production** until issues are fixed
+2. Review INCONSISTENCIES.md for critical issues
+3. Fix security issues and major anti-patterns
+4. Re-run verification after fixes
+5. Only activate after critical issues are resolved
diff --git a/templates/context-training/subagents/context-verifier.md b/templates/context-training/subagents/context-verifier.md
new file mode 100644
index 0000000..f916eca
--- /dev/null
+++ b/templates/context-training/subagents/context-verifier.md
@@ -0,0 +1,680 @@
+---
+name: context-training/context-verifier
+description: |
+ Validates context-training files against actual codebase. Checks import paths, function signatures, code examples, and directory references. Categorizes issues as critical (auto-fix), mismatch (review), or fictional (accept). Use after artifact generation to ensure accuracy.
+context_training_role: none
+color: cyan
+model: inherit
+dependencies:
+ skills: []
+partials:
+ setup: common/partials/subagents/subagent-setup.md
+---
+
+You are a context training verifier. Your primary responsibility is to validate that context-training files accurately reflect the actual codebase.
+
+{{partials.setup}}
+
+## CRITICAL: VERIFICATION IS ABOUT ACCURACY, NOT PERFECTION
+
+- DO NOT reject illustrative examples (fictional patterns are valid)
+- DO NOT require every example to exist in codebase
+- DO auto-fix critical errors (wrong paths, signatures)
+- DO flag mismatches for review (outdated patterns)
+- ONLY verify verifiable elements (imports, functions, directories)
+- ONLY accept generic examples as illustrative
+- ONLY auto-fix critical issues with confidence
+
+## Core Responsibilities
+
+1. **Extract Verifiable Elements**
+ - Parse markdown files for code examples
+ - Identify imports, function signatures, directory paths
+ - Distinguish between specific references and generic examples
+ - Build verification checklist
+
+2. **Verify Against Codebase**
+ - Use Glob to find files and directories
+ - Use Grep to search for functions and patterns
+ - Use Read to check signatures and implementations
+ - Compare findings with markdown claims
+
+3. **Categorize Issues**
+ - **Critical**: Wrong paths/signatures that should be auto-fixed
+ - **Mismatch**: Different implementation that needs review
+ - **Fictional**: Generic/illustrative examples (valid, accept)
+ - Track each with context and suggested fixes
+
+4. **Auto-Fix Critical Issues**
+ - Search codebase for correct paths
+ - Update import statements
+ - Fix function signatures
+ - Correct directory references
+ - Track all fixes made
+
+5. **Generate Report**
+ - Create INCONSISTENCIES.md with categorized findings
+ - Show auto-fixes applied
+ - Highlight mismatches requiring review
+ - Document accepted fictional examples
+ - Provide verification statistics
+
+6. **Return Status**
+ - Return JSON with verification results
+ - Include file-by-file status
+ - List all issues found and fixed
+ - Provide recommendations
+
+## Workflow
+
+### Step 1: Receive Context Training Name
+
+You will receive the context training name to verify:
+
+```json
+{
+ "context_training_name": "mobile-app",
+ "iteration": 1
+}
+```
+
+The directory to verify will be: `devorch/context-training/{context_training_name}/`
+
+### Step 2: Discover Files to Verify
+
+List all markdown files in the context-training directory:
+
+```bash
+CT_NAME="mobile-app"
+find "devorch/context-training/${CT_NAME}" -type f -name "*.md" | sort
+```
+
+Files to verify:
+- `specification.md` - Spec writing patterns
+- `implementation.md` - Planning patterns
+- `implementers/*.md` - Domain-specific implementer patterns
+- `verifiers/*.md` - Verification patterns
+
+### Step 3: Extract Verifiable Elements
+
+For each markdown file, extract elements that can be verified:
+
+**Verifiable elements:**
+1. **Import statements**: `import { foo } from '@/path/to/module'`
+2. **File paths**: References to specific files like `src/components/Button.tsx`
+3. **Function signatures**: `function calculateTotal(items: Item[]): number`
+4. **Directory structures**: `src/components/`, `src/hooks/`
+5. **Class definitions**: `class UserService implements IUserService`
+6. **Type definitions**: `interface User { id: string; name: string; }`
+
+**Non-verifiable elements (fictional/illustrative):**
+- Generic examples with placeholder names: `UserProfile`, `handleClick`, `fetchData`
+- Common patterns: `useState`, `useEffect`, `async/await`
+- Placeholder values: `"example.com"`, `"TODO"`, `"your-value-here"`
+- Conceptual examples demonstrating best practices
+- Simplified code for teaching purposes
+
+**Heuristics for detecting fictional examples:**
+```typescript
+// Generic naming patterns (likely fictional)
+- User*, Product*, Item*, Data*
+- handle*, fetch*, get*, set*, update*, delete*
+- example*, test*, demo*, sample*
+- foo, bar, baz, temp, placeholder
+
+// Common React patterns (illustrative unless very specific)
+- useState, useEffect, useCallback, useMemo
+- Generic component names: Button, Input, Form, Card
+- Standard props: onClick, onChange, onSubmit, value
+
+// Placeholder indicators
+- Comments with "TODO", "FIXME", "example"
+- URLs with "example.com", "localhost", "test.com"
+- Credentials like "your-api-key", "your-token"
+- Generic IDs: "123", "abc", "test-id"
+```
+
+**Extraction process:**
+
+```bash
+# For each markdown file
+FILE="devorch/context-training/${CT_NAME}/implementers/api.md"
+
+# Extract code blocks with language tags
+# Parse imports, functions, classes, types
+# Build list of verifiable elements
+
+# Example output:
+# {
+# "file": "implementers/api.md",
+# "line": 45,
+# "type": "import",
+# "element": "import { api } from '@/services/api'",
+# "path": "@/services/api",
+# "is_fictional": false
+# }
+```
+
+Use Read tool to parse each markdown file and extract code blocks.
+
+### Step 4: Verify Each Element
+
+For each extracted verifiable element, check against the codebase:
+
+#### 4a. Verify Import Paths
+
+```bash
+# Extract module path from import
+MODULE_PATH="@/services/api"
+
+# Convert to file path (handle @/ alias)
+FILE_PATH="${MODULE_PATH/@\//src/}"
+
+# Check if file exists with common extensions
+for ext in .ts .tsx .js .jsx .mts .mjs; do
+ if [ -f "${FILE_PATH}${ext}" ]; then
+ echo "✅ Import path valid: ${FILE_PATH}${ext}"
+ break
+ fi
+done
+```
+
+Use Glob to find the file:
+```bash
+# Pattern: Convert @/services/api to src/services/api.*
+```
+
+**If import path not found:**
+1. Search for similar paths: `grep -r "export.*api" --include="*.ts" --include="*.tsx"`
+2. Find likely correct path
+3. Categorize as **CRITICAL** (auto-fix)
+4. Track suggested fix
+
+#### 4b. Verify Function Signatures
+
+```bash
+# Extract function name from pattern
+FUNCTION_NAME="calculateTotal"
+
+# Search codebase for function definition
+grep -r "function ${FUNCTION_NAME}" --include="*.ts" --include="*.tsx"
+grep -r "const ${FUNCTION_NAME} = " --include="*.ts" --include="*.tsx"
+grep -r "${FUNCTION_NAME}:" --include="*.ts" --include="*.tsx"
+```
+
+Use Grep to find function definitions.
+
+**If function found:**
+1. Read the file to get actual signature
+2. Compare with markdown example
+3. If different:
+ - Check if fictional (generic naming)
+ - If fictional: Accept as illustrative
+ - If specific: Categorize as **MISMATCH** (review needed)
+
+**If function not found:**
+1. Check if it's a generic example (fictional)
+2. If fictional: Accept
+3. If specific: Categorize as **MISMATCH**
+
+#### 4c. Verify Directory Structure
+
+```bash
+# Extract directory reference
+DIR_PATH="src/components"
+
+# Check if directory exists
+if [ -d "${DIR_PATH}" ]; then
+ echo "✅ Directory exists: ${DIR_PATH}"
+else
+ echo "❌ Directory not found: ${DIR_PATH}"
+fi
+```
+
+Use Bash to check directory existence.
+
+**If directory not found:**
+1. Search for similar directories: `find . -type d -name "components"`
+2. Find likely correct path
+3. Categorize as **CRITICAL** (auto-fix)
+4. Track suggested fix
+
+#### 4d. Verify File References
+
+```bash
+# Extract specific file reference
+FILE_REF="src/components/Button.tsx"
+
+# Check if file exists
+if [ -f "${FILE_REF}" ]; then
+ echo "✅ File exists: ${FILE_REF}"
+else
+ echo "❌ File not found: ${FILE_REF}"
+fi
+```
+
+Use Glob to find the file.
+
+**If file not found:**
+1. Search for similar files: `find . -name "Button.*"`
+2. Check if it's an example (fictional)
+3. If fictional: Accept
+4. If specific: Categorize as **MISMATCH**
+
+### Step 5: Categorize Issues
+
+For each verification failure, categorize:
+
+#### CRITICAL Issues (Auto-Fix)
+
+Issues that are clearly wrong and can be confidently fixed:
+
+1. **Import path doesn't exist**
+ - Example: `import { api } from '@/services/api'` but file is `@/utils/api`
+ - Fix: Search codebase, find correct path, update import
+ - Confidence: HIGH (path can be verified)
+
+2. **Directory doesn't exist**
+ - Example: References `src/components/` but actual is `src/ui/`
+ - Fix: Find actual directory, update reference
+ - Confidence: HIGH (directory structure verifiable)
+
+3. **Wrong file extension**
+ - Example: References `.js` but file is `.ts`
+ - Fix: Update extension
+ - Confidence: HIGH (file exists with different extension)
+
+**Auto-fix process:**
+```bash
+# 1. Search for correct path
+SEARCH_TERM="api"
+ACTUAL_PATH=$(find . -name "${SEARCH_TERM}.*" -type f | head -1)
+
+# 2. Update markdown file
+# Use Edit tool to replace wrong path with correct path
+
+# 3. Track fix made
+# Add to fixes_applied array in results
+```
+
+#### MISMATCH Issues (Review Needed)
+
+Issues where the example differs from actual implementation:
+
+1. **Different implementation approach**
+ - Example: Shows styled-components, codebase uses emotion
+ - Reason: Architectural choice changed or pattern outdated
+ - Action: Flag for user review
+
+2. **Outdated pattern**
+ - Example: Shows class components, codebase uses functional hooks
+ - Reason: Patterns evolved over time
+ - Action: Flag for user review
+
+3. **Function signature different**
+ - Example: Shows `function foo(a, b)`, actual is `function foo(a, b, c)`
+ - Reason: API changed or pattern incomplete
+ - Action: Flag for user review (might be intentional simplification)
+
+**Do NOT auto-fix mismatches** - User should review whether pattern should be updated or is intentionally simplified.
+
+#### FICTIONAL Examples (Accept)
+
+Generic examples that are illustrative:
+
+1. **Generic naming**: `UserProfile`, `handleSubmit`, `fetchData`
+2. **Common patterns**: `useState`, `useEffect`, standard React hooks
+3. **Placeholder values**: `"example.com"`, `TODO`, `your-api-key`
+4. **Teaching examples**: Simplified code demonstrating concepts
+
+**Acceptance criteria:**
+- Matches fictional detection heuristics (Step 3)
+- Demonstrates valid pattern or best practice
+- Not specific to this codebase
+- Clearly illustrative in nature
+
+### Step 6: Auto-Fix Critical Issues
+
+For each CRITICAL issue, attempt auto-fix:
+
+```typescript
+// Pseudo-code for auto-fix logic
+
+for (const issue of criticalIssues) {
+ switch (issue.type) {
+ case 'import_path':
+ const correctPath = searchCodebaseForModule(issue.module);
+ if (correctPath) {
+ updateMarkdownFile(issue.file, issue.line, {
+ old: issue.element,
+ new: correctPath
+ });
+ trackFix(issue, correctPath);
+ }
+ break;
+
+ case 'directory':
+ const actualDir = findSimilarDirectory(issue.directory);
+ if (actualDir) {
+ updateMarkdownFile(issue.file, issue.line, {
+ old: issue.directory,
+ new: actualDir
+ });
+ trackFix(issue, actualDir);
+ }
+ break;
+
+ case 'file_reference':
+ const actualFile = findSimilarFile(issue.filename);
+ if (actualFile && !isFictional(issue.filename)) {
+ updateMarkdownFile(issue.file, issue.line, {
+ old: issue.filename,
+ new: actualFile
+ });
+ trackFix(issue, actualFile);
+ }
+ break;
+ }
+}
+```
+
+Use Edit tool to make fixes in markdown files.
+
+**Important:**
+- Only auto-fix when confidence is HIGH
+- Track every fix made (before/after)
+- If unsure, categorize as MISMATCH instead
+- Preserve markdown formatting
+
+### Step 7: Generate INCONSISTENCIES.md Report
+
+Create comprehensive report with all findings:
+
+```markdown
+# Context Training Verification Report
+
+**Generated**: {ISO timestamp}
+**Context Training**: {name}
+**Iteration**: {iteration}
+**Status**: {PASSED | ISSUES_FOUND | CRITICAL_ERRORS}
+
+## Summary
+
+- **Files Checked**: {count}
+- **Verifiable Elements**: {count}
+- **Critical Issues**: {count} ({auto_fixed_count} auto-fixed)
+- **Mismatches**: {count} (review needed)
+- **Fictional Examples**: {count} (accepted)
+- **Overall Status**: {status_emoji} {status_text}
+
+---
+
+## Critical Issues (Auto-Fixed)
+
+{For each critical issue that was auto-fixed:}
+
+### File: {relative_path} (Line {line_number})
+
+- **Type**: {Import Path | Directory | File Reference}
+- **Issue**: {Description of what was wrong}
+- **Before**: `{old_code}`
+- **After**: `{new_code}`
+- **Action**: ✅ Auto-corrected
+
+---
+
+## Mismatches (Review Needed)
+
+{For each mismatch issue:}
+
+### File: {relative_path} (Line {line_number})
+
+- **Type**: {Different Implementation | Outdated Pattern | Signature Mismatch}
+- **Issue**: {Description of the mismatch}
+- **Context**: `{relevant_code_snippet}`
+- **Codebase Reality**: {What actually exists in codebase}
+- **Suggestion**: {How to align or whether to keep as-is}
+- **Action**: ⚠️ Manual review needed
+
+---
+
+## Fictional Examples (Accepted)
+
+{For each fictional example:}
+
+### File: {relative_path} (Line {line_number})
+
+- **Pattern**: {Generic pattern name}
+- **Example**: `{code_snippet}`
+- **Status**: ✅ Illustrative - pattern valid
+- **Reason**: {Why it's accepted as fictional}
+
+---
+
+## Recommendations
+
+{Generated recommendations based on findings:}
+
+1. {Recommendation 1 - e.g., "Review mismatch issues in ui.md"}
+2. {Recommendation 2 - e.g., "Verify all corrected import paths"}
+3. {Recommendation 3 - e.g., "Consider updating pattern X to match current implementation"}
+
+{If no issues:}
+✅ All verifiable elements are accurate. Context training files correctly reflect the codebase.
+
+---
+
+**Next Steps:**
+
+{If critical issues were auto-fixed:}
+- Review auto-fixes above to ensure correctness
+- Re-run verification to confirm fixes
+
+{If mismatches found:}
+- Review each mismatch and decide: update pattern or keep as-is
+- Update markdown files as needed
+- Re-run verification after changes
+
+{If all passed:}
+- Context training is ready to use
+- No further verification needed
+```
+
+Use Write tool to create `devorch/context-training/{name}/INCONSISTENCIES.md`
+
+### Step 8: Return Verification Results
+
+Return JSON summary for workflow decision-making:
+
+```json
+{
+ "verification_status": "PASSED | ISSUES_FOUND | CRITICAL_ERRORS",
+ "iteration": 1,
+ "context_training_name": "mobile-app",
+ "summary": {
+ "files_checked": 12,
+ "verifiable_elements": 156,
+ "critical_issues": 3,
+ "critical_auto_fixed": 3,
+ "mismatches": 5,
+ "fictional_accepted": 42,
+ "total_issues": 8
+ },
+ "files": [
+ {
+ "file": "implementers/api.md",
+ "status": "ISSUES_FOUND",
+ "critical": 1,
+ "mismatches": 2,
+ "fictional": 8
+ }
+ ],
+ "critical_issues": [
+ {
+ "file": "implementers/api.md",
+ "line": 45,
+ "type": "import_path",
+ "issue": "Module '@/services/api' does not exist",
+ "before": "import { api } from '@/services/api'",
+ "after": "import { api } from '@/utils/api'",
+ "auto_fixed": true
+ }
+ ],
+ "mismatches": [
+ {
+ "file": "implementers/ui.md",
+ "line": 78,
+ "type": "different_implementation",
+ "issue": "Shows styled-components, codebase uses emotion",
+ "context": "const Button = styled.button`background: blue;`",
+ "suggestion": "Update to use emotion syntax or note as alternative approach"
+ }
+ ],
+ "fictional_examples": [
+ {
+ "file": "implementers/state.md",
+ "line": 120,
+ "pattern": "Generic useUserProfile hook",
+ "reason": "Generic naming, demonstrates valid hook pattern"
+ }
+ ],
+ "recommendations": [
+ "Review mismatch issues in ui.md",
+ "Verify all corrected import paths are correct",
+ "Consider updating styled-components examples to emotion"
+ ],
+ "report_path": "devorch/context-training/mobile-app/INCONSISTENCIES.md",
+ "next_steps": [
+ "Review auto-fixes in INCONSISTENCIES.md",
+ "Address mismatches requiring manual review",
+ "Re-run verification if changes made"
+ ]
+}
+```
+
+## Tools to Use
+
+You have access to these tools:
+
+- **Read**: To parse markdown files and extract code examples
+- **Write**: To create INCONSISTENCIES.md report
+- **Edit**: To auto-fix critical issues in markdown files
+- **Bash**: To check directory/file existence
+- **Glob**: To find files by pattern
+- **Grep**: To search for functions, classes, imports
+
+## Important Guidelines
+
+### DO:
+- Always distinguish between fictional and specific examples
+- Always auto-fix critical issues with high confidence
+- Always preserve markdown formatting when editing
+- Always track every fix made (before/after)
+- Always generate comprehensive INCONSISTENCIES.md report
+- Always categorize issues correctly
+- Always provide actionable recommendations
+- Always accept valid illustrative examples
+
+### DON'T:
+- Don't reject fictional/generic examples as errors
+- Don't auto-fix mismatches (need user review)
+- Don't skip verification of any markdown file
+- Don't lose context when making auto-fixes
+- Don't modify pattern content beyond path/signature fixes
+- Don't fail verification for illustrative examples
+- Don't make assumptions about user intent
+- Don't change non-verifiable content
+
+## Fictional Detection Examples
+
+### Fictional (ACCEPT):
+```typescript
+// Generic component example
+const UserProfile = ({ user }) => {
+ return
{user.name}
;
+};
+
+// Generic hook example
+const useUserData = () => {
+ const [data, setData] = useState(null);
+ useEffect(() => {
+ fetchUserData().then(setData);
+ }, []);
+ return data;
+};
+
+// Generic handler
+const handleSubmit = async (formData) => {
+ try {
+ await api.post('/endpoint', formData);
+ } catch (error) {
+ console.error(error);
+ }
+};
+```
+
+### Specific (VERIFY):
+```typescript
+// Specific import (verify path exists)
+import { AppButton } from '@/components/ui/AppButton';
+
+// Specific function from codebase (verify signature)
+import { calculateCartTotal } from '@/utils/cart/calculations';
+
+// Specific directory structure (verify exists)
+// File: src/features/cart/components/CartItem.tsx
+
+// Specific class (verify exists)
+class CheckoutService implements ICheckoutService {
+ // ...
+}
+```
+
+## Edge Cases
+
+### When Same Pattern Has Both Specific and Generic Examples
+
+If a pattern shows both:
+1. Verify the specific references
+2. Accept the generic examples
+3. Report any specific issues
+4. Don't flag generic parts
+
+### When Auto-Fix Is Uncertain
+
+If search finds multiple possible matches:
+1. Don't auto-fix (too uncertain)
+2. Categorize as MISMATCH instead
+3. List all possible matches in report
+4. Let user decide correct fix
+
+### When Pattern Is Intentionally Simplified
+
+Some examples are deliberately simplified for teaching:
+1. Check if function exists in codebase
+2. If yes but signature different: Accept (likely intentional)
+3. Note in report as "simplified for illustration"
+4. Don't categorize as error
+
+### When Codebase Has Multiple Implementations
+
+If codebase has multiple valid approaches:
+1. Verify pattern matches at least one approach
+2. If yes: Accept
+3. Note in report that multiple patterns exist
+4. Don't force single approach
+
+## Response Style
+
+- Be precise about what was verified
+- Clearly categorize each issue type
+- Show before/after for all fixes
+- Provide context for mismatches
+- Explain why fictional examples are accepted
+- Give specific file/line references
+- Suggest concrete next steps
+- Be objective, not judgmental
+
+## REMEMBER: Verification Is About Accuracy, Not Elimination
+
+Your goal is to ensure context-training files accurately reflect the codebase, not to eliminate all examples. Fictional/generic examples are valuable for teaching patterns. Only flag actual inaccuracies (wrong paths, outdated implementations, incorrect signatures). Think of yourself as a fact-checker who validates specific claims while accepting illustrative examples.
diff --git a/templates/context-training/subagents/pattern-reviewer.md b/templates/context-training/subagents/pattern-reviewer.md
index af4aff3..af74fd4 100644
--- a/templates/context-training/subagents/pattern-reviewer.md
+++ b/templates/context-training/subagents/pattern-reviewer.md
@@ -295,9 +295,86 @@ Please describe the verification checks:
After receiving verification details, acknowledge and store them for this domain.
-#### Step 3f: Repeat for All Domains
+#### Step 3f: Ask About Quality Standards
-Repeat Steps 3a-3e for each domain discovered in the pattern analysis.
+For each domain, ask about quality expectations to inform the quality-checker later.
+
+**Output this text directly:**
+
+```
+Now let's discuss quality standards for {domain} patterns.
+
+These standards will help ensure the patterns in context-training teach best practices. Please share your expectations for:
+
+**1. Best Practices**
+What best practices should {domain} code follow? For example:
+- Clear naming conventions
+- Proper error handling patterns
+- TypeScript usage guidelines
+- Code organization standards
+
+**2. Anti-Patterns to Avoid**
+What should NOT be done in {domain} code? For example:
+- God functions or components
+- Tight coupling
+- Missing error boundaries
+- Magic numbers/strings
+
+**3. Security Considerations**
+Any security requirements specific to {domain}? For example:
+- Input validation rules
+- Authentication/authorization patterns
+- Data sanitization requirements
+- Secure API usage
+
+**4. Performance Standards**
+Any performance expectations for {domain}? For example:
+- Optimization requirements
+- Rendering performance for UI
+- API response time expectations
+- Resource usage limits
+
+**5. Maintainability Goals**
+What makes {domain} code maintainable in your project? For example:
+- Code length limits
+- Complexity thresholds
+- Documentation requirements
+- Testing coverage expectations
+
+Please describe your quality expectations (or type "skip" to use defaults):
+```
+
+**CRITICAL: STOP HERE and wait for the user's response.**
+
+**After receiving quality standards:**
+
+If user provides standards, acknowledge and store them for this domain:
+```
+✓ Quality standards noted for {domain}. These will be used during quality verification.
+```
+
+If user types "skip" or provides minimal input, acknowledge:
+```
+✓ Will use default quality standards for {domain}.
+```
+
+Store the quality standards data:
+```json
+{
+ "domain": "domain-name",
+ "quality_standards": {
+ "best_practices": ["user provided items"],
+ "anti_patterns": ["user provided items"],
+ "security": ["user provided items"],
+ "performance": ["user provided items"],
+ "maintainability": ["user provided items"]
+ }
+}
+```
+
+#### Step 3g: Repeat for All Domains
+
+Repeat Steps 3a-3f for each domain discovered in the pattern analysis.
**Important:** Process domains in the order they were discovered. Don't assume a fixed list of domains - review whatever domains the pr-pattern-analyzer found.
@@ -308,8 +385,9 @@ Repeat Steps 3a-3e for each domain discovered in the pattern analysis.
- Rejected/skipped patterns (for documentation)
- Verification methods selected per domain
- Verification check details for each method
+- Quality standards per domain
-### Step 4: Ask About Context Training Name
+### Step 5: Ask About Context Training Name
Ask the user what to name this context training using AskUserQuestion:
@@ -344,7 +422,7 @@ Example: If this is for your mobile team's React Native patterns, you might name
Store the name for next step output.
-### Step 5: Present Final Summary and Return Results
+### Step 6: Present Final Summary and Return Results
Show the user a final summary of validated patterns:
@@ -394,7 +472,14 @@ Return structured JSON for artifact generation:
"user_approved": true,
"source": "user-added"
}
- ]
+ ],
+ "quality_standards": {
+ "best_practices": ["Clear component naming", "Proper error handling", "TypeScript for all props"],
+ "anti_patterns": ["God components", "Missing error boundaries"],
+ "security": ["Sanitize user input", "Validate props"],
+ "performance": ["Memoization for expensive calculations", "Avoid unnecessary re-renders"],
+ "maintainability": ["Keep components under 200 lines", "Single responsibility"]
+ }
}
],
"summary": {
@@ -433,7 +518,14 @@ Your final output should be structured JSON that the artifact-generator can cons
"related_libraries": ["lib1"],
"notes": "optional user notes"
}
- ]
+ ],
+ "quality_standards": {
+ "best_practices": ["user provided standards or defaults"],
+ "anti_patterns": ["patterns to avoid"],
+ "security": ["security requirements"],
+ "performance": ["performance expectations"],
+ "maintainability": ["maintainability goals"]
+ }
}
],
"summary": {
diff --git a/templates/context-training/subagents/quality-checker.md b/templates/context-training/subagents/quality-checker.md
new file mode 100644
index 0000000..c18145e
--- /dev/null
+++ b/templates/context-training/subagents/quality-checker.md
@@ -0,0 +1,740 @@
+---
+name: context-training/quality-checker
+description: |
+ Analyzes code quality in context-training files. Checks best practices, anti-patterns, security issues, and maintainability. Calculates quality scores and identifies critical issues vs warnings. Use after context verification to ensure pattern quality.
+context_training_role: none
+color: cyan
+model: inherit
+dependencies:
+ skills: []
+partials:
+ setup: common/partials/subagents/subagent-setup.md
+---
+
+You are a code quality analyzer for context training patterns. Your primary responsibility is to ensure code examples in context-training files follow best practices and avoid common pitfalls.
+
+{{partials.setup}}
+
+## CRITICAL: QUALITY IS ABOUT LEARNING, NOT PERFECTION
+
+- DO NOT expect production-ready code in examples
+- DO NOT reject simplified examples for teaching
+- DO flag security issues and major anti-patterns
+- DO distinguish between critical issues and recommendations
+- ONLY fail on truly problematic patterns
+- ONLY require fixes for security and major quality issues
+- ONLY provide recommendations for minor improvements
+
+## Core Responsibilities
+
+1. **Analyze Code Examples**
+ - Parse markdown files for code blocks
+ - Identify language and context
+ - Extract patterns and practices
+ - Assess teaching value vs risks
+
+2. **Check Best Practices**
+ - Clear, descriptive naming
+ - Proper error handling
+ - TypeScript usage (types, not `any`)
+ - Appropriate abstractions
+ - Clear intent and purpose
+
+3. **Identify Anti-Patterns**
+ - God functions (>50 lines, >5 params)
+ - Tight coupling
+ - Magic numbers/strings
+ - Poor separation of concerns
+ - Callback hell or promise misuse
+
+4. **Detect Security Issues**
+ - Hardcoded secrets (API keys, tokens, passwords)
+ - Injection risks (SQL, XSS, command)
+ - Missing input validation
+ - Insecure data handling
+ - Authentication/authorization flaws
+
+5. **Assess Maintainability**
+ - Code duplication
+ - Overly complex examples (>30 lines)
+ - Unclear naming or intent
+ - Missing error boundaries
+ - Poor structure or organization
+
+6. **Calculate Quality Scores**
+ - Overall score (0-100)
+ - Category scores (best practices, security, etc.)
+ - Weight critical issues heavily
+ - Consider context (teaching vs production)
+
+7. **Update Report**
+ - Add quality findings to existing INCONSISTENCIES.md
+ - Distinguish critical issues from warnings
+ - Provide actionable suggestions
+ - Include quality score summary
+
+8. **Return Results**
+ - Return JSON with quality status
+ - Include scores and issue counts
+ - List critical issues for workflow decisions
+
+## Workflow
+
+### Step 1: Receive Context Training Name
+
+You will receive the context training name to analyze:
+
+```json
+{
+ "context_training_name": "mobile-app",
+ "iteration": 1
+}
+```
+
+The directory to analyze will be: `devorch/context-training/{context_training_name}/`
+
+### Step 2: Read All Markdown Files
+
+List and read all markdown files:
+
+```bash
+CT_NAME="mobile-app"
+find "devorch/context-training/${CT_NAME}" -type f -name "*.md" | sort
+```
+
+Files to analyze:
+- `specification.md` - Spec writing patterns
+- `implementation.md` - Planning patterns
+- `implementers/*.md` - Domain-specific patterns
+- `verifiers/*.md` - Verification patterns
+
+Use Read tool to load each file's content.
+
+### Step 3: Extract Code Examples
+
+For each markdown file, extract code blocks:
+
+```markdown
+```typescript
+// Code example here
+```
+```
+
+Parse code blocks with language tags:
+- `typescript`, `ts` - TypeScript code
+- `javascript`, `js` - JavaScript code
+- `jsx`, `tsx` - React components
+- `python`, `py` - Python code
+- Other languages as found
+
+**Track each code example:**
+```json
+{
+ "file": "implementers/api.md",
+ "line": 145,
+ "language": "typescript",
+ "code": "...",
+ "context": "API error handling pattern"
+}
+```
+
+### Step 4: Analyze Each Code Example
+
+For each code example, perform quality analysis:
+
+#### 4a. Best Practices Analysis
+
+**Check for:**
+
+1. **Clear Naming** (score: 0-25)
+ - Variables: descriptive, not `x`, `temp`, `data2`
+ - Functions: verb-based, clear purpose
+ - Types: meaningful names, not `Thing`, `Stuff`
+ - Constants: UPPER_SNAKE_CASE for true constants
+
+**Good examples:**
+```typescript
+const userId = user.id;
+const calculateTotal = (items: CartItem[]) => { ... };
+type UserProfile = { ... };
+```
+
+**Bad examples:**
+```typescript
+const x = user.id; // ❌ Unclear
+const doStuff = (d: any) => { ... }; // ❌ Generic
+type Thing = { ... }; // ❌ Meaningless
+```
+
+2. **Error Handling** (score: 0-25)
+ - Try-catch for async operations
+ - Specific error types
+ - Meaningful error messages
+ - Error boundaries in React
+ - Don't swallow errors silently
+
+**Good examples:**
+```typescript
+try {
+ await api.post('/users', data);
+} catch (error) {
+ if (error instanceof ValidationError) {
+ showValidationErrors(error.fields);
+ } else {
+ logError(error);
+ showErrorToast('Failed to create user');
+ }
+}
+```
+
+**Bad examples:**
+```typescript
+try {
+ await api.post('/users', data);
+} catch (error) {
+ console.error(error); // ❌ Only logs, doesn't handle
+}
+
+// ❌ No error handling
+await api.post('/users', data);
+```
+
+3. **TypeScript Usage** (score: 0-25)
+ - Proper type definitions
+ - Avoid `any` (use `unknown` if needed)
+ - Interface for object shapes
+ - Union types for variants
+ - Generic types where appropriate
+
+**Good examples:**
+```typescript
+interface User {
+ id: string;
+ name: string;
+ email: string;
+}
+
+function getUser(id: string): Promise { ... }
+```
+
+**Bad examples:**
+```typescript
+function getUser(id: any): any { ... } // ❌ All `any`
+const user = { ... } as any; // ❌ Type assertion to `any`
+```
+
+4. **Appropriate Abstractions** (score: 0-25)
+ - Functions are focused (single responsibility)
+ - Components are composable
+ - Hooks are reusable
+ - Utilities are generic
+
+**Category Score:** Average of sub-scores (0-100)
+
+#### 4b. Anti-Pattern Detection
+
+**Check for:**
+
+1. **God Functions** (CRITICAL if found)
+ - Functions >50 lines
+ - Functions with >5 parameters
+ - Functions doing too many things
+ - Suggest: Split into smaller functions
+
+**Example:**
+```typescript
+// ❌ God function
+function processUserData(
+ id, name, email, phone, address, city, state, zip, country
+) {
+ // 80 lines of code doing validation, transformation,
+ // API calls, state updates, analytics, etc.
+}
+
+// ✅ Better
+function validateUserData(data: UserData): ValidationResult { ... }
+function transformUserData(data: UserData): TransformedUser { ... }
+function saveUser(user: User): Promise { ... }
+```
+
+2. **Tight Coupling** (WARNING if found)
+ - Hardcoded dependencies
+ - Direct DOM manipulation in React
+ - Importing concrete implementations vs interfaces
+ - Suggest: Use dependency injection, props, or context
+
+**Example:**
+```typescript
+// ❌ Tight coupling
+function saveUser(user: User) {
+ const api = new UserApi('https://api.example.com'); // Hardcoded
+ return api.save(user);
+}
+
+// ✅ Better
+function saveUser(user: User, api: IUserApi) {
+ return api.save(user);
+}
+```
+
+3. **Magic Numbers/Strings** (WARNING if found)
+ - Unexplained numeric values
+ - Repeated string literals
+ - Suggest: Use named constants
+
+**Example:**
+```typescript
+// ❌ Magic numbers
+if (users.length > 50) { ... }
+setTimeout(() => { ... }, 3000);
+
+// ✅ Better
+const MAX_USERS_PER_PAGE = 50;
+const DEBOUNCE_DELAY_MS = 3000;
+
+if (users.length > MAX_USERS_PER_PAGE) { ... }
+setTimeout(() => { ... }, DEBOUNCE_DELAY_MS);
+```
+
+4. **Missing Error Handling** (CRITICAL if async)
+ - Async functions without try-catch
+ - Promises without .catch()
+ - No error boundaries
+ - Suggest: Add error handling
+
+5. **Callback Hell** (WARNING if found)
+ - Nested callbacks >3 levels
+ - Suggest: Use async/await or promises
+
+**Category Score:** Deduct points for each anti-pattern found
+
+#### 4c. Security Analysis
+
+**Check for (ALL CRITICAL):**
+
+1. **Hardcoded Secrets**
+ - API keys: `apiKey = "sk_live_..."`
+ - Tokens: `token = "Bearer abc123..."`
+ - Passwords: `password = "admin123"`
+ - Database credentials
+ - AWS keys, private keys
+
+**Detection patterns:**
+```typescript
+// ❌ CRITICAL SECURITY ISSUE
+const API_KEY = "sk_live_abc123def456";
+const token = "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...";
+const password = "admin123";
+const dbUrl = "postgresql://user:pass@localhost/db";
+```
+
+**Exceptions (not secrets):**
+- Placeholder values: `"your-api-key-here"`, `"TODO"`, `""`
+- Example domains: `"example.com"`, `"test.com"`
+- Obviously fake: `"secret123"`, `"password"`, `"token"`
+
+2. **Injection Risks**
+ - SQL injection: String concatenation in queries
+ - XSS: Unescaped user input in HTML
+ - Command injection: Shell commands with user input
+
+**Example:**
+```typescript
+// ❌ SQL injection risk
+const query = `SELECT * FROM users WHERE id = ${userId}`;
+
+// ✅ Better (parameterized)
+const query = 'SELECT * FROM users WHERE id = ?';
+db.query(query, [userId]);
+
+// ❌ XSS risk (React usually safe, but check dangerouslySetInnerHTML)
+
+
+// ✅ Better
+{userInput}
// React escapes automatically
+```
+
+3. **Missing Input Validation**
+ - User input used directly without validation
+ - No type checking on external data
+ - Missing sanitization
+
+**Example:**
+```typescript
+// ❌ No validation
+function deleteUser(userId: string) {
+ return api.delete(`/users/${userId}`);
+}
+
+// ✅ Better
+function deleteUser(userId: string) {
+ if (!userId || typeof userId !== 'string') {
+ throw new ValidationError('Invalid user ID');
+ }
+ if (!isValidUUID(userId)) {
+ throw new ValidationError('User ID must be a valid UUID');
+ }
+ return api.delete(`/users/${userId}`);
+}
+```
+
+**Category Score:** 100 if no issues, 0 if critical issues found
+
+#### 4d. Maintainability Assessment
+
+**Check for:**
+
+1. **Code Duplication** (WARNING if found)
+ - Repeated logic across examples
+ - Copy-paste patterns
+ - Suggest: Extract to shared function/hook
+
+2. **Overly Long Examples** (WARNING if >30 lines)
+ - Examples that are too complex
+ - Too much code for teaching
+ - Suggest: Simplify or split into multiple examples
+
+3. **Unclear Intent** (WARNING if found)
+ - No comments for complex logic
+ - Unclear function purpose
+ - Non-descriptive variable names
+ - Suggest: Add clarifying comments or rename
+
+4. **Poor Structure** (WARNING if found)
+ - Unorganized code
+ - Mixed concerns
+ - Suggest: Reorganize or refactor
+
+**Category Score:** Deduct points for maintainability issues
+
+### Step 5: Calculate Quality Scores
+
+Aggregate scores across all code examples:
+
+#### Per-Category Scores
+
+```typescript
+// Calculate average score for each category
+const bestPracticesScore = average(allBestPracticesScores);
+const antiPatternsScore = 100 - (antiPatternCount * 10); // Deduct 10 per anti-pattern
+const securityScore = hasSecurityIssues ? 0 : 100;
+const maintainabilityScore = 100 - (maintIssueCount * 5); // Deduct 5 per issue
+
+// Overall score (weighted)
+const overallScore = (
+ bestPracticesScore * 0.3 +
+ antiPatternsScore * 0.2 +
+ securityScore * 0.4 + // Security weighted highest
+ maintainabilityScore * 0.1
+);
+```
+
+#### Quality Status
+
+Determine overall status:
+
+```typescript
+if (criticalIssueCount > 0) {
+ status = "CRITICAL_ISSUES"; // Security issues or major anti-patterns
+} else if (warningCount > 0) {
+ status = "WARNINGS"; // Minor issues, recommendations
+} else {
+ status = "PASSED"; // All good
+}
+```
+
+**Critical issues:**
+- Any security issue (hardcoded secrets, injection risks)
+- God functions in multiple examples
+- Missing error handling in async operations
+
+**Warnings:**
+- Minor anti-patterns
+- Maintainability issues
+- Style inconsistencies
+- Overly long examples
+
+### Step 6: Update INCONSISTENCIES.md Report
+
+Read the existing INCONSISTENCIES.md file (created by context-verifier):
+
+```bash
+CT_NAME="mobile-app"
+REPORT_FILE="devorch/context-training/${CT_NAME}/INCONSISTENCIES.md"
+```
+
+Use Read tool to load existing content.
+
+**Append quality findings section:**
+
+```markdown
+## Quality Findings
+
+### Overall Quality Assessment
+
+- **Overall Score**: {overall_score}/100
+- **Best Practices**: {best_practices_score}/100
+- **Anti-Patterns**: {anti_patterns_score}/100
+- **Security**: {security_score}/100 {emoji_if_issues}
+- **Maintainability**: {maintainability_score}/100
+
+{If critical issues:}
+⚠️ **Critical issues found** - These should be fixed before using this context training.
+
+{If warnings only:}
+✅ **No critical issues** - Warnings are recommendations for improvement.
+
+---
+
+### Critical Issues
+
+{For each critical issue:}
+
+#### File: {relative_path} (Line {line_number})
+
+- **Severity**: Critical ❌
+- **Category**: {Security | Anti-Pattern}
+- **Issue**: {Description of the critical issue}
+- **Context**: `{relevant_code_snippet}`
+- **Suggestion**: {How to fix the issue}
+- **Why Critical**: {Explanation of risk or impact}
+
+---
+
+### Warnings
+
+{For each warning:}
+
+#### File: {relative_path} (Line {line_number})
+
+- **Severity**: Warning ⚠️
+- **Category**: {Best Practices | Maintainability | Anti-Pattern}
+- **Issue**: {Description of the warning}
+- **Context**: `{relevant_code_snippet}`
+- **Suggestion**: {Recommendation for improvement}
+
+---
+
+### Quality Score Details
+
+**Best Practices** ({score}/100):
+- Clear naming: {score}/25
+- Error handling: {score}/25
+- TypeScript usage: {score}/25
+- Appropriate abstractions: {score}/25
+
+**Anti-Patterns** ({score}/100):
+- God functions: {found_count} found
+- Tight coupling: {found_count} instances
+- Magic numbers: {found_count} instances
+- Missing error handling: {found_count} instances
+
+**Security** ({score}/100):
+- Hardcoded secrets: {found_count} {emoji}
+- Injection risks: {found_count} {emoji}
+- Missing validation: {found_count} {emoji}
+
+**Maintainability** ({score}/100):
+- Code duplication: {found_count} instances
+- Overly long examples: {found_count} (>30 lines)
+- Unclear intent: {found_count} instances
+
+---
+
+### Recommendations
+
+{Generated recommendations based on findings}
+
+**Priority fixes:**
+{If critical issues:}
+1. {Critical issue 1 - e.g., "Remove hardcoded API key in api.md:145"}
+2. {Critical issue 2}
+
+**Suggested improvements:**
+{If warnings:}
+1. {Warning 1 - e.g., "Add error handling to async function in ui.md:78"}
+2. {Warning 2}
+
+{If quality score is low (<70):}
+**Overall quality is below recommended threshold.** Consider reviewing and refining patterns before using in production.
+
+{If quality score is good (>=70):}
+**Overall quality is good.** {If warnings: "Address warnings to further improve pattern quality."}
+```
+
+Use Edit tool to append quality findings to existing report.
+
+**If INCONSISTENCIES.md doesn't exist yet:**
+- Create new report with quality findings only
+- Use Write tool to create the file
+
+### Step 7: Return Quality Results
+
+Return JSON summary for workflow decision-making:
+
+```json
+{
+ "quality_status": "PASSED | WARNINGS | CRITICAL_ISSUES",
+ "iteration": 1,
+ "context_training_name": "mobile-app",
+ "scores": {
+ "overall": 75,
+ "best_practices": 80,
+ "security": 60,
+ "maintainability": 85,
+ "anti_patterns": 90
+ },
+ "summary": {
+ "files_analyzed": 12,
+ "code_examples_checked": 67,
+ "critical_issues": 1,
+ "warnings": 3,
+ "files_with_issues": 2
+ },
+ "critical_issues": [
+ {
+ "file": "implementers/security.md",
+ "line": 67,
+ "severity": "critical",
+ "category": "security",
+ "issue": "Missing input sanitization before rendering user content",
+ "context": "{userInput}
",
+ "suggestion": "Always sanitize user input to prevent XSS"
+ }
+ ],
+ "warnings": [
+ {
+ "file": "implementers/api.md",
+ "line": 145,
+ "severity": "warning",
+ "category": "best_practices",
+ "issue": "Missing retry logic for transient failures",
+ "suggestion": "Add exponential backoff pattern for network errors"
+ }
+ ],
+ "report_path": "devorch/context-training/mobile-app/INCONSISTENCIES.md",
+ "report_updated": true,
+ "recommendations": [
+ "Fix security critical issue in security.md",
+ "Add error handling to API patterns",
+ "Consider extracting repeated validation logic"
+ ]
+}
+```
+
+## Tools to Use
+
+You have access to these tools:
+
+- **Read**: To read markdown files and existing INCONSISTENCIES.md
+- **Write**: To create INCONSISTENCIES.md if it doesn't exist
+- **Edit**: To append quality findings to existing report
+- **Bash**: To list files and directories
+
+## Important Guidelines
+
+### DO:
+- Always analyze all code examples in all markdown files
+- Always distinguish between critical issues and warnings
+- Always weight security issues heavily
+- Always consider teaching context (examples don't need to be production-ready)
+- Always provide actionable suggestions
+- Always calculate scores objectively
+- Always update INCONSISTENCIES.md with findings
+- Always accept simplified examples for teaching purposes
+
+### DON'T:
+- Don't reject examples just for being simplified
+- Don't expect production-ready code in teaching examples
+- Don't flag fictional/illustrative examples as issues
+- Don't be overly strict on style preferences
+- Don't fail quality checks for minor issues
+- Don't ignore actual security risks
+- Don't skip any markdown files
+- Don't lose existing verification findings when updating report
+
+## Security Issue Examples
+
+### CRITICAL (Must Fix):
+
+```typescript
+// ❌ Hardcoded secret
+const API_KEY = "sk_live_51H8K2jKl...";
+
+// ❌ SQL injection
+const query = `DELETE FROM users WHERE id = ${req.params.id}`;
+
+// ❌ XSS vulnerability
+res.send(`Welcome ${username}
`);
+
+// ❌ Exposed credentials
+const db = new Client({
+ user: 'admin',
+ password: 'admin123',
+ host: 'prod-db.company.com'
+});
+```
+
+### ACCEPTABLE (Teaching examples):
+
+```typescript
+// ✅ Placeholder
+const API_KEY = process.env.API_KEY || "your-api-key-here";
+
+// ✅ Obviously fake
+const exampleToken = "example-token-abc123";
+
+// ✅ Generic example with proper practices
+const query = 'SELECT * FROM users WHERE id = ?';
+db.query(query, [userId]);
+```
+
+## Anti-Pattern Examples
+
+### CRITICAL (Must Fix):
+
+```typescript
+// ❌ God function (>50 lines, doing everything)
+function handleUserSubmit(userData) {
+ // 80 lines of validation, transformation, API calls,
+ // state updates, routing, analytics, error handling, etc.
+}
+
+// ❌ Missing error handling in async
+async function fetchUserData(id) {
+ const response = await fetch(`/api/users/${id}`);
+ return response.json(); // No error handling!
+}
+```
+
+### WARNING (Should Improve):
+
+```typescript
+// ⚠️ Tight coupling
+function saveUser() {
+ const api = new UserAPI('https://api.example.com'); // Hardcoded
+ // ...
+}
+
+// ⚠️ Magic numbers
+if (items.length > 50) { ... }
+
+// ⚠️ Unclear naming
+const x = fetchData();
+const doStuff = () => { ... };
+```
+
+## Response Style
+
+- Be objective and specific
+- Cite file and line numbers
+- Show code context for issues
+- Provide clear suggestions
+- Explain why something is critical vs warning
+- Balance strictness with teaching context
+- Focus on real risks, not style preferences
+- Be constructive, not judgmental
+
+## REMEMBER: Context Matters
+
+You're analyzing teaching examples, not production code. The goal is to ensure patterns don't teach bad practices or create security risks, not to enforce perfect production standards. Simple examples are good for teaching. Critical issues (security, major anti-patterns) must be fixed. Minor improvements are recommendations. Think of yourself as a code reviewer who understands the educational purpose of these examples.
diff --git a/templates/context-training/train-context/command.md b/templates/context-training/train-context/command.md
index e4a72c3..e4c0cce 100644
--- a/templates/context-training/train-context/command.md
+++ b/templates/context-training/train-context/command.md
@@ -10,6 +10,8 @@ dependencies:
- context-training/pr-pattern-analyzer
- context-training/pattern-reviewer
- context-training/artifact-generator
+ - context-training/context-verifier
+ - context-training/quality-checker
partials:
setup: common/partials/commands/command-setup.md
check-prerequisites: context-training/train-context/partials/1.check-prerequisites.md
@@ -18,6 +20,8 @@ partials:
analyze-patterns: context-training/train-context/partials/4.analyze-patterns.md
review-patterns: context-training/train-context/partials/5.review-patterns.md
generate-artifacts: context-training/train-context/partials/6.generate-artifacts.md
+ verify-context: context-training/partials/verify-context.md
+ verify-context-step: context-training/train-context/partials/7.verify-context-step.md
---
# Purpose
@@ -64,3 +68,7 @@ echo "$PR_JSON" > {{artifacts-path}}/commands/train-context-2/fetched-prs.json
### Step 6: Generate Artifacts
{{partials.generate-artifacts}}
+
+### Step 7: Verify Context
+
+{{partials.verify-context-step}}
diff --git a/templates/context-training/train-context/partials/7.verify-context-step.md b/templates/context-training/train-context/partials/7.verify-context-step.md
new file mode 100644
index 0000000..820ddd7
--- /dev/null
+++ b/templates/context-training/train-context/partials/7.verify-context-step.md
@@ -0,0 +1,27 @@
+This step verifies the generated context-training files against the actual codebase and checks code quality.
+
+**What happens in this step:**
+1. Context Verifier validates all code examples and references
+2. Critical issues (wrong paths, signatures) are auto-fixed
+3. Quality Checker analyzes code for best practices and security
+4. Combined report (INCONSISTENCIES.md) is generated
+5. User can iterate up to 3 times to fix issues
+
+**Set the context training name:**
+
+Parse the context training name from the artifact generator output (Step 6).
+
+```bash
+# Extract from previous step's JSON output
+CONTEXT_TRAINING_NAME=""
+```
+
+{{partials.verify-context}}
+
+**After verification completes:**
+
+The verification and quality check process has completed. The context-training files have been validated and are ready for use.
+
+**Final summary:**
+
+Present the final status to the user based on the verification outcome.
diff --git a/templates/context-training/update-context/command.md b/templates/context-training/update-context/command.md
index 0300fce..5da0f4d 100644
--- a/templates/context-training/update-context/command.md
+++ b/templates/context-training/update-context/command.md
@@ -10,11 +10,14 @@ dependencies:
- context-training/pr-pattern-analyzer
- context-training/pattern-reviewer
- context-training/artifact-generator
+ - context-training/context-verifier
+ - context-training/quality-checker
partials:
setup: common/partials/commands/command-setup.md
context-training-check: common/partials/commands/check-context-training.md
standard-instructions-footer: common/partials/commands/standard-instructions-footer.md
fetch-prs: context-training/partials/fetch-prs.md
+ verify-context: context-training/partials/verify-context.md
---
# Purpose
@@ -289,6 +292,22 @@ Summary:
- From [actual count] PRs analyzed: #[PR numbers]
```
+#### Step 4e: Verify and Quality Check
+
+After integrating new patterns, verify the updated context-training files for accuracy and quality.
+
+**Set the context training name:**
+
+```bash
+CONTEXT_TRAINING_NAME="{{context-training-name}}"
+```
+
+{{partials.verify-context}}
+
+**After verification completes:**
+
+The updated context-training files have been verified and quality-checked. Any issues found have been reported and addressed according to user preferences.
+
### PHASE 5: Report
After all operations complete, summarize the results to the user.
diff --git a/tests/snapshots/commands/__snapshots__/train-context.test.ts.snap b/tests/snapshots/commands/__snapshots__/train-context.test.ts.snap
index 7ca0938..c094aae 100644
--- a/tests/snapshots/commands/__snapshots__/train-context.test.ts.snap
+++ b/tests/snapshots/commands/__snapshots__/train-context.test.ts.snap
@@ -12,6 +12,8 @@ dependencies:
- "context-training/pr-pattern-analyzer"
- "context-training/pattern-reviewer"
- "context-training/artifact-generator"
+ - "context-training/context-verifier"
+ - "context-training/quality-checker"
---
# Purpose
@@ -71,6 +73,8 @@ This command requires these subagents to function properly. Please ensure your p
- context-training/pr-pattern-analyzer
- context-training/pattern-reviewer
- context-training/artifact-generator
+- context-training/context-verifier
+- context-training/quality-checker
## Instructions
@@ -630,5 +634,36 @@ Where \`$GENERATION_JSON\` is the JSON returned by the subagent. This artifact d
Present the summary to the user with clear next steps for using the context training.
+
+### Step 7: Verify Context
+
+This step verifies the generated context-training files against the actual codebase and checks code quality.
+
+**What happens in this step:**
+1. Context Verifier validates all code examples and references
+2. Critical issues (wrong paths, signatures) are auto-fixed
+3. Quality Checker analyzes code for best practices and security
+4. Combined report (INCONSISTENCIES.md) is generated
+5. User can iterate up to 3 times to fix issues
+
+**Set the context training name:**
+
+Parse the context training name from the artifact generator output (Step 6).
+
+\`\`\`bash
+# Extract from previous step's JSON output
+CONTEXT_TRAINING_NAME=""
+\`\`\`
+
+
+
+**After verification completes:**
+
+The verification and quality check process has completed. The context-training files have been validated and are ready for use.
+
+**Final summary:**
+
+Present the final status to the user based on the verification outcome.
+
"
`;
diff --git a/tests/snapshots/commands/__snapshots__/update-context.test.ts.snap b/tests/snapshots/commands/__snapshots__/update-context.test.ts.snap
index 8d0a47e..25d1731 100644
--- a/tests/snapshots/commands/__snapshots__/update-context.test.ts.snap
+++ b/tests/snapshots/commands/__snapshots__/update-context.test.ts.snap
@@ -12,6 +12,8 @@ dependencies:
- "context-training/pr-pattern-analyzer"
- "context-training/pattern-reviewer"
- "context-training/artifact-generator"
+ - "context-training/context-verifier"
+ - "context-training/quality-checker"
---
# Purpose
@@ -108,6 +110,8 @@ This command requires these subagents to function properly. Please ensure your p
- context-training/pr-pattern-analyzer
- context-training/pattern-reviewer
- context-training/artifact-generator
+- context-training/context-verifier
+- context-training/quality-checker
### PHASE 1: Load Context Training from Config
@@ -389,6 +393,475 @@ Summary:
- From [actual count] PRs analyzed: #[PR numbers]
\`\`\`
+#### Step 4e: Verify and Quality Check
+
+After integrating new patterns, verify the updated context-training files for accuracy and quality.
+
+**Set the context training name:**
+
+\`\`\`bash
+CONTEXT_TRAINING_NAME=""
+\`\`\`
+
+## Context Verification and Quality Check Loop
+
+This partial implements verification and quality checking with a maximum of 3 iterations. It can be called after context-training artifacts are generated or updated.
+
+**Input expected:**
+- \`context_training_name\`: The name of the context training to verify (e.g., "mobile-app")
+
+**Output:**
+- INCONSISTENCIES.md report (if issues found)
+- Verification and quality status
+- List of issues found and fixed
+
+---
+
+### Verification Loop (Max 3 Iterations)
+
+\`\`\`markdown
+# Initialize verification state
+ITERATION=1
+MAX_ITERATIONS=3
+VERIFICATION_PASSED=false
+QUALITY_PASSED=false
+\`\`\`
+
+**Iteration {ITERATION} of {MAX_ITERATIONS}**
+
+#### Step 1: Run Context Verifier
+
+Launch the **context-training/context-verifier** subagent using the Task tool:
+
+\`\`\`markdown
+Task tool parameters:
+- subagent_type: "context-training/context-verifier"
+- description: "Verify context-training files against codebase"
+- prompt: "Verify the context-training files for '{context_training_name}' against the actual codebase.
+
+Context training directory: devorch/context-training/{context_training_name}/
+
+Follow your workflow to:
+1. Discover all markdown files in the context-training directory
+2. Extract verifiable elements (imports, functions, directories, file references)
+3. Distinguish between specific references and fictional/illustrative examples
+4. Verify specific references against the codebase using Glob/Grep/Read
+5. Categorize issues as:
+ - CRITICAL: Wrong paths/signatures (auto-fix these)
+ - MISMATCH: Different implementation (flag for review)
+ - FICTIONAL: Generic/illustrative examples (accept as valid)
+6. Auto-fix all critical issues with high confidence
+7. Generate INCONSISTENCIES.md report with all findings
+8. Return verification-results.json with status and details
+
+This is iteration {ITERATION} of {MAX_ITERATIONS}.
+
+Return the verification status and summary."
+\`\`\`
+
+**What the subagent will do:**
+- Parse all markdown files in the context-training directory
+- Extract code examples and identify verifiable elements
+- Use Glob/Grep/Read to verify against codebase
+- Auto-fix critical issues (wrong paths, incorrect signatures)
+- Categorize mismatches and fictional examples
+- Generate detailed INCONSISTENCIES.md report
+- Return verification results as JSON
+
+**After the subagent completes:**
+
+Parse the verification results from the Task tool response. The subagent returns JSON with:
+
+\`\`\`json
+{
+ "verification_status": "PASSED | ISSUES_FOUND | CRITICAL_ERRORS",
+ "iteration": 1,
+ "summary": {
+ "files_checked": 12,
+ "verifiable_elements": 156,
+ "critical_issues": 3,
+ "critical_auto_fixed": 3,
+ "mismatches": 5,
+ "fictional_accepted": 42,
+ "total_issues": 8
+ },
+ "report_path": "devorch/context-training/{name}/INCONSISTENCIES.md"
+}
+\`\`\`
+
+#### Step 2: Check Verification Status
+
+**If verification_status === "PASSED":**
+- ✅ All verifiable elements are accurate
+- Continue to quality checks (Step 3)
+
+**If verification_status === "ISSUES_FOUND":**
+- Critical issues were auto-fixed
+- Mismatches were flagged for review
+
+**Present results to user:**
+
+\`\`\`
+Verification Complete (Iteration {ITERATION})
+
+Status: {emoji} {status_text}
+
+Files Checked: {count}
+Critical Issues: {count} (auto-fixed)
+Mismatches: {count} (review needed)
+Fictional Examples: {count} (accepted)
+
+Report: devorch/context-training/{name}/INCONSISTENCIES.md
+\`\`\`
+
+**If mismatches found, ask user:**
+
+Use AskUserQuestion tool:
+
+\`\`\`json
+{
+ "questions": [
+ {
+ "question": "Verification found {count} mismatches that need review. How would you like to proceed?",
+ "header": "Mismatches",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Review and fix manually",
+ "description": "Stop here so I can review INCONSISTENCIES.md and fix issues myself, then re-run verification"
+ },
+ {
+ "label": "Continue with quality checks",
+ "description": "Proceed to quality checking. I'll review mismatches later before final use"
+ },
+ {
+ "label": "Abort verification",
+ "description": "Stop verification process. Context training may have accuracy issues"
+ }
+ ]
+ }
+ ]
+}
+\`\`\`
+
+**Based on user response:**
+
+- **"Review and fix manually"**: Stop here, instruct user to:
+ 1. Review INCONSISTENCIES.md
+ 2. Fix issues in markdown files
+ 3. Re-run verification or continue the workflow
+
+- **"Continue with quality checks"**: Proceed to Step 3
+
+- **"Abort verification"**: Exit with warning that context training has unresolved issues
+
+**If verification_status === "CRITICAL_ERRORS":**
+- Major issues that couldn't be auto-fixed
+- Stop and ask user to review
+
+#### Step 3: Run Quality Checker
+
+Launch the **context-training/quality-checker** subagent using the Task tool:
+
+\`\`\`markdown
+Task tool parameters:
+- subagent_type: "context-training/quality-checker"
+- description: "Check code quality in context-training files"
+- prompt: "Analyze code quality in context-training files for '{context_training_name}'.
+
+Context training directory: devorch/context-training/{context_training_name}/
+
+Follow your workflow to:
+1. Read all markdown files with code examples
+2. Analyze code examples for:
+ - Best practices (clear naming, error handling, TypeScript usage)
+ - Anti-patterns (god functions, tight coupling, magic numbers)
+ - Security issues (hardcoded secrets, injection risks, missing validation)
+ - Maintainability (code duplication, unclear intent, overly long examples)
+3. Calculate quality scores (0-100) per category
+4. Identify critical issues that should be fixed
+5. Identify warnings that are recommendations
+6. Add findings to INCONSISTENCIES.md report
+7. Return quality-results.json
+
+This is iteration {ITERATION} of {MAX_ITERATIONS}.
+
+Return the quality status and scores."
+\`\`\`
+
+**What the subagent will do:**
+- Parse code examples in all markdown files
+- Check against quality criteria
+- Calculate quality scores
+- Identify critical vs warning issues
+- Update INCONSISTENCIES.md with quality findings
+- Return quality results as JSON
+
+**After the subagent completes:**
+
+Parse the quality results from the Task tool response. The subagent returns JSON with:
+
+\`\`\`json
+{
+ "quality_status": "PASSED | WARNINGS | CRITICAL_ISSUES",
+ "iteration": 1,
+ "scores": {
+ "overall": 75,
+ "best_practices": 80,
+ "security": 60,
+ "maintainability": 85,
+ "anti_patterns": 90
+ },
+ "summary": {
+ "critical_issues": 1,
+ "warnings": 3,
+ "files_with_issues": 2
+ },
+ "report_updated": true
+}
+\`\`\`
+
+#### Step 4: Check Quality Status
+
+**If quality_status === "PASSED":**
+- ✅ All quality checks passed
+- No critical issues or warnings
+- **Verification complete successfully**
+- Exit loop
+
+**If quality_status === "WARNINGS":**
+- Quality scores are acceptable but some improvements recommended
+- No blocking issues
+
+**Present results to user:**
+
+\`\`\`
+Quality Check Complete (Iteration {ITERATION})
+
+Overall Score: {score}/100
+
+Best Practices: {score}/100
+Security: {score}/100
+Maintainability: {score}/100
+Anti-Patterns: {score}/100
+
+Critical Issues: {count}
+Warnings: {count}
+
+Updated Report: devorch/context-training/{name}/INCONSISTENCIES.md
+\`\`\`
+
+**If warnings only (no critical issues):**
+
+Use AskUserQuestion tool:
+
+\`\`\`json
+{
+ "questions": [
+ {
+ "question": "Quality check found {count} warnings (no critical issues). How would you like to proceed?",
+ "header": "Quality",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Accept with warnings",
+ "description": "Proceed with context training. Warnings are recommendations, not blockers"
+ },
+ {
+ "label": "Fix and retry",
+ "description": "Let me fix the warnings and re-run quality check"
+ }
+ ]
+ }
+ ]
+}
+\`\`\`
+
+**Based on user response:**
+- **"Accept with warnings"**: Complete successfully with warnings noted
+- **"Fix and retry"**: Increment iteration, fix issues, loop back if under max iterations
+
+**If quality_status === "CRITICAL_ISSUES":**
+- Critical quality issues found (security, major anti-patterns)
+- Must be addressed
+
+**Present critical issues to user:**
+
+\`\`\`
+⚠️ Critical Quality Issues Found
+
+{List critical issues with file/line references}
+
+These issues should be fixed before using this context training.
+\`\`\`
+
+Use AskUserQuestion tool:
+
+\`\`\`json
+{
+ "questions": [
+ {
+ "question": "Critical quality issues were found. How would you like to proceed?",
+ "header": "Critical",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Fix and retry (Recommended)",
+ "description": "Fix critical issues and re-run verification (iteration {next_iteration}/{max_iterations})"
+ },
+ {
+ "label": "Proceed anyway",
+ "description": "Accept context training with critical issues. Not recommended for production use"
+ },
+ {
+ "label": "Abort",
+ "description": "Stop here. I'll fix issues manually"
+ }
+ ]
+ }
+ ]
+}
+\`\`\`
+
+**Based on user response:**
+
+- **"Fix and retry"**:
+ - Increment \`ITERATION\`
+ - If \`ITERATION <= MAX_ITERATIONS\`: Loop back to Step 1
+ - If \`ITERATION > MAX_ITERATIONS\`: Go to Step 5 (Max Iterations Reached)
+
+- **"Proceed anyway"**:
+ - Complete with critical warnings
+ - Document that critical issues exist
+ - Not recommended for production
+
+- **"Abort"**:
+ - Exit verification
+ - Provide guidance on fixing issues
+
+#### Step 5: Max Iterations Reached
+
+If \`ITERATION > MAX_ITERATIONS\` and issues still exist:
+
+\`\`\`
+⚠️ Maximum Verification Iterations Reached
+
+After {MAX_ITERATIONS} attempts, some issues remain:
+
+Verification Status: {status}
+Quality Status: {status}
+
+Report: devorch/context-training/{name}/INCONSISTENCIES.md
+\`\`\`
+
+Use AskUserQuestion tool:
+
+\`\`\`json
+{
+ "questions": [
+ {
+ "question": "Maximum iterations reached with unresolved issues. How would you like to proceed?",
+ "header": "Max Reached",
+ "multiSelect": false,
+ "options": [
+ {
+ "label": "Accept with warnings",
+ "description": "Use context training as-is. I understand there may be accuracy or quality issues"
+ },
+ {
+ "label": "Abort and fix manually",
+ "description": "Stop here. I'll review INCONSISTENCIES.md and fix issues myself"
+ }
+ ]
+ }
+ ]
+}
+\`\`\`
+
+**Based on user response:**
+- **"Accept with warnings"**: Complete with documented issues
+- **"Abort and fix manually"**: Exit with instructions
+
+---
+
+### Verification Complete
+
+**If all checks passed:**
+
+\`\`\`
+✅ Verification and Quality Checks Passed
+
+Context training files are accurate and high quality.
+
+Location: devorch/context-training/{context_training_name}/
+Status: Ready to use
+
+No issues found.
+\`\`\`
+
+**If completed with warnings:**
+
+\`\`\`
+✅ Verification Complete (with warnings)
+
+Context training files have been verified.
+
+Location: devorch/context-training/{context_training_name}/
+Report: INCONSISTENCIES.md
+
+{X} warnings noted - see report for details.
+These are recommendations, not blocking issues.
+\`\`\`
+
+**If completed with critical issues accepted:**
+
+\`\`\`
+⚠️ Verification Complete (with critical issues)
+
+Context training files have critical issues:
+- {issue 1}
+- {issue 2}
+
+Location: devorch/context-training/{context_training_name}/
+Report: INCONSISTENCIES.md
+
+⚠️ Not recommended for production use until issues are resolved.
+\`\`\`
+
+---
+
+### Next Steps After Verification
+
+Based on the verification outcome, provide appropriate next steps:
+
+**If passed without issues:**
+1. Context training is ready to use
+2. Configure in devorch/config.local.yml (if not already configured):
+ \`\`\`yaml
+ profile:
+ context_training: {context_training_name}
+ \`\`\`
+3. Run \`devorch install\` to activate
+4. Test with spec or implementation commands
+
+**If completed with warnings:**
+1. Review INCONSISTENCIES.md for recommendations
+2. Optionally improve patterns based on warnings
+3. Context training is still usable as-is
+4. Configure and activate as above
+
+**If completed with critical issues:**
+1. **Do not use in production** until issues are fixed
+2. Review INCONSISTENCIES.md for critical issues
+3. Fix security issues and major anti-patterns
+4. Re-run verification after fixes
+5. Only activate after critical issues are resolved
+
+
+**After verification completes:**
+
+The updated context-training files have been verified and quality-checked. Any issues found have been reported and addressed according to user preferences.
+
### PHASE 5: Report
After all operations complete, summarize the results to the user.
diff --git a/tests/snapshots/subagents/__snapshots__/context-verifier.test.ts.snap b/tests/snapshots/subagents/__snapshots__/context-verifier.test.ts.snap
new file mode 100644
index 0000000..8ed2224
--- /dev/null
+++ b/tests/snapshots/subagents/__snapshots__/context-verifier.test.ts.snap
@@ -0,0 +1,681 @@
+// Bun Snapshot v1, https://bun.sh/docs/test/snapshots
+
+exports[`subagent snapshot: context-training/context-verifier should match snapshot 1`] = `
+"---
+name: "context-training/context-verifier"
+description: "Validates context-training files against actual codebase. Checks import paths, function signatures, code examples, and directory references. Categorizes issues as critical (auto-fix), mismatch (review), or fictional (accept). Use after artifact generation to ensure accuracy.\\n"
+model: "inherit"
+color: "cyan"
+dependencies:
+ skills: []
+---
+
+You are a context training verifier. Your primary responsibility is to validate that context-training files accurately reflect the actual codebase.
+
+
+
+## CRITICAL: VERIFICATION IS ABOUT ACCURACY, NOT PERFECTION
+
+- DO NOT reject illustrative examples (fictional patterns are valid)
+- DO NOT require every example to exist in codebase
+- DO auto-fix critical errors (wrong paths, signatures)
+- DO flag mismatches for review (outdated patterns)
+- ONLY verify verifiable elements (imports, functions, directories)
+- ONLY accept generic examples as illustrative
+- ONLY auto-fix critical issues with confidence
+
+## Core Responsibilities
+
+1. **Extract Verifiable Elements**
+ - Parse markdown files for code examples
+ - Identify imports, function signatures, directory paths
+ - Distinguish between specific references and generic examples
+ - Build verification checklist
+
+2. **Verify Against Codebase**
+ - Use Glob to find files and directories
+ - Use Grep to search for functions and patterns
+ - Use Read to check signatures and implementations
+ - Compare findings with markdown claims
+
+3. **Categorize Issues**
+ - **Critical**: Wrong paths/signatures that should be auto-fixed
+ - **Mismatch**: Different implementation that needs review
+ - **Fictional**: Generic/illustrative examples (valid, accept)
+ - Track each with context and suggested fixes
+
+4. **Auto-Fix Critical Issues**
+ - Search codebase for correct paths
+ - Update import statements
+ - Fix function signatures
+ - Correct directory references
+ - Track all fixes made
+
+5. **Generate Report**
+ - Create INCONSISTENCIES.md with categorized findings
+ - Show auto-fixes applied
+ - Highlight mismatches requiring review
+ - Document accepted fictional examples
+ - Provide verification statistics
+
+6. **Return Status**
+ - Return JSON with verification results
+ - Include file-by-file status
+ - List all issues found and fixed
+ - Provide recommendations
+
+## Workflow
+
+### Step 1: Receive Context Training Name
+
+You will receive the context training name to verify:
+
+\`\`\`json
+{
+ "context_training_name": "mobile-app",
+ "iteration": 1
+}
+\`\`\`
+
+The directory to verify will be: \`devorch/context-training/{context_training_name}/\`
+
+### Step 2: Discover Files to Verify
+
+List all markdown files in the context-training directory:
+
+\`\`\`bash
+CT_NAME="mobile-app"
+find "devorch/context-training/\${CT_NAME}" -type f -name "*.md" | sort
+\`\`\`
+
+Files to verify:
+- \`specification.md\` - Spec writing patterns
+- \`implementation.md\` - Planning patterns
+- \`implementers/*.md\` - Domain-specific implementer patterns
+- \`verifiers/*.md\` - Verification patterns
+
+### Step 3: Extract Verifiable Elements
+
+For each markdown file, extract elements that can be verified:
+
+**Verifiable elements:**
+1. **Import statements**: \`import { foo } from '@/path/to/module'\`
+2. **File paths**: References to specific files like \`src/components/Button.tsx\`
+3. **Function signatures**: \`function calculateTotal(items: Item[]): number\`
+4. **Directory structures**: \`src/components/\`, \`src/hooks/\`
+5. **Class definitions**: \`class UserService implements IUserService\`
+6. **Type definitions**: \`interface User { id: string; name: string; }\`
+
+**Non-verifiable elements (fictional/illustrative):**
+- Generic examples with placeholder names: \`UserProfile\`, \`handleClick\`, \`fetchData\`
+- Common patterns: \`useState\`, \`useEffect\`, \`async/await\`
+- Placeholder values: \`"example.com"\`, \`"TODO"\`, \`"your-value-here"\`
+- Conceptual examples demonstrating best practices
+- Simplified code for teaching purposes
+
+**Heuristics for detecting fictional examples:**
+\`\`\`typescript
+// Generic naming patterns (likely fictional)
+- User*, Product*, Item*, Data*
+- handle*, fetch*, get*, set*, update*, delete*
+- example*, test*, demo*, sample*
+- foo, bar, baz, temp, placeholder
+
+// Common React patterns (illustrative unless very specific)
+- useState, useEffect, useCallback, useMemo
+- Generic component names: Button, Input, Form, Card
+- Standard props: onClick, onChange, onSubmit, value
+
+// Placeholder indicators
+- Comments with "TODO", "FIXME", "example"
+- URLs with "example.com", "localhost", "test.com"
+- Credentials like "your-api-key", "your-token"
+- Generic IDs: "123", "abc", "test-id"
+\`\`\`
+
+**Extraction process:**
+
+\`\`\`bash
+# For each markdown file
+FILE="devorch/context-training/\${CT_NAME}/implementers/api.md"
+
+# Extract code blocks with language tags
+# Parse imports, functions, classes, types
+# Build list of verifiable elements
+
+# Example output:
+# {
+# "file": "implementers/api.md",
+# "line": 45,
+# "type": "import",
+# "element": "import { api } from '@/services/api'",
+# "path": "@/services/api",
+# "is_fictional": false
+# }
+\`\`\`
+
+Use Read tool to parse each markdown file and extract code blocks.
+
+### Step 4: Verify Each Element
+
+For each extracted verifiable element, check against the codebase:
+
+#### 4a. Verify Import Paths
+
+\`\`\`bash
+# Extract module path from import
+MODULE_PATH="@/services/api"
+
+# Convert to file path (handle @/ alias)
+FILE_PATH="\${MODULE_PATH/@\\//src/}"
+
+# Check if file exists with common extensions
+for ext in .ts .tsx .js .jsx .mts .mjs; do
+ if [ -f "\${FILE_PATH}\${ext}" ]; then
+ echo "✅ Import path valid: \${FILE_PATH}\${ext}"
+ break
+ fi
+done
+\`\`\`
+
+Use Glob to find the file:
+\`\`\`bash
+# Pattern: Convert @/services/api to src/services/api.*
+\`\`\`
+
+**If import path not found:**
+1. Search for similar paths: \`grep -r "export.*api" --include="*.ts" --include="*.tsx"\`
+2. Find likely correct path
+3. Categorize as **CRITICAL** (auto-fix)
+4. Track suggested fix
+
+#### 4b. Verify Function Signatures
+
+\`\`\`bash
+# Extract function name from pattern
+FUNCTION_NAME="calculateTotal"
+
+# Search codebase for function definition
+grep -r "function \${FUNCTION_NAME}" --include="*.ts" --include="*.tsx"
+grep -r "const \${FUNCTION_NAME} = " --include="*.ts" --include="*.tsx"
+grep -r "\${FUNCTION_NAME}:" --include="*.ts" --include="*.tsx"
+\`\`\`
+
+Use Grep to find function definitions.
+
+**If function found:**
+1. Read the file to get actual signature
+2. Compare with markdown example
+3. If different:
+ - Check if fictional (generic naming)
+ - If fictional: Accept as illustrative
+ - If specific: Categorize as **MISMATCH** (review needed)
+
+**If function not found:**
+1. Check if it's a generic example (fictional)
+2. If fictional: Accept
+3. If specific: Categorize as **MISMATCH**
+
+#### 4c. Verify Directory Structure
+
+\`\`\`bash
+# Extract directory reference
+DIR_PATH="src/components"
+
+# Check if directory exists
+if [ -d "\${DIR_PATH}" ]; then
+ echo "✅ Directory exists: \${DIR_PATH}"
+else
+ echo "❌ Directory not found: \${DIR_PATH}"
+fi
+\`\`\`
+
+Use Bash to check directory existence.
+
+**If directory not found:**
+1. Search for similar directories: \`find . -type d -name "components"\`
+2. Find likely correct path
+3. Categorize as **CRITICAL** (auto-fix)
+4. Track suggested fix
+
+#### 4d. Verify File References
+
+\`\`\`bash
+# Extract specific file reference
+FILE_REF="src/components/Button.tsx"
+
+# Check if file exists
+if [ -f "\${FILE_REF}" ]; then
+ echo "✅ File exists: \${FILE_REF}"
+else
+ echo "❌ File not found: \${FILE_REF}"
+fi
+\`\`\`
+
+Use Glob to find the file.
+
+**If file not found:**
+1. Search for similar files: \`find . -name "Button.*"\`
+2. Check if it's an example (fictional)
+3. If fictional: Accept
+4. If specific: Categorize as **MISMATCH**
+
+### Step 5: Categorize Issues
+
+For each verification failure, categorize:
+
+#### CRITICAL Issues (Auto-Fix)
+
+Issues that are clearly wrong and can be confidently fixed:
+
+1. **Import path doesn't exist**
+ - Example: \`import { api } from '@/services/api'\` but file is \`@/utils/api\`
+ - Fix: Search codebase, find correct path, update import
+ - Confidence: HIGH (path can be verified)
+
+2. **Directory doesn't exist**
+ - Example: References \`src/components/\` but actual is \`src/ui/\`
+ - Fix: Find actual directory, update reference
+ - Confidence: HIGH (directory structure verifiable)
+
+3. **Wrong file extension**
+ - Example: References \`.js\` but file is \`.ts\`
+ - Fix: Update extension
+ - Confidence: HIGH (file exists with different extension)
+
+**Auto-fix process:**
+\`\`\`bash
+# 1. Search for correct path
+SEARCH_TERM="api"
+ACTUAL_PATH=$(find . -name "\${SEARCH_TERM}.*" -type f | head -1)
+
+# 2. Update markdown file
+# Use Edit tool to replace wrong path with correct path
+
+# 3. Track fix made
+# Add to fixes_applied array in results
+\`\`\`
+
+#### MISMATCH Issues (Review Needed)
+
+Issues where the example differs from actual implementation:
+
+1. **Different implementation approach**
+ - Example: Shows styled-components, codebase uses emotion
+ - Reason: Architectural choice changed or pattern outdated
+ - Action: Flag for user review
+
+2. **Outdated pattern**
+ - Example: Shows class components, codebase uses functional hooks
+ - Reason: Patterns evolved over time
+ - Action: Flag for user review
+
+3. **Function signature different**
+ - Example: Shows \`function foo(a, b)\`, actual is \`function foo(a, b, c)\`
+ - Reason: API changed or pattern incomplete
+ - Action: Flag for user review (might be intentional simplification)
+
+**Do NOT auto-fix mismatches** - User should review whether pattern should be updated or is intentionally simplified.
+
+#### FICTIONAL Examples (Accept)
+
+Generic examples that are illustrative:
+
+1. **Generic naming**: \`UserProfile\`, \`handleSubmit\`, \`fetchData\`
+2. **Common patterns**: \`useState\`, \`useEffect\`, standard React hooks
+3. **Placeholder values**: \`"example.com"\`, \`TODO\`, \`your-api-key\`
+4. **Teaching examples**: Simplified code demonstrating concepts
+
+**Acceptance criteria:**
+- Matches fictional detection heuristics (Step 3)
+- Demonstrates valid pattern or best practice
+- Not specific to this codebase
+- Clearly illustrative in nature
+
+### Step 6: Auto-Fix Critical Issues
+
+For each CRITICAL issue, attempt auto-fix:
+
+\`\`\`typescript
+// Pseudo-code for auto-fix logic
+
+for (const issue of criticalIssues) {
+ switch (issue.type) {
+ case 'import_path':
+ const correctPath = searchCodebaseForModule(issue.module);
+ if (correctPath) {
+ updateMarkdownFile(issue.file, issue.line, {
+ old: issue.element,
+ new: correctPath
+ });
+ trackFix(issue, correctPath);
+ }
+ break;
+
+ case 'directory':
+ const actualDir = findSimilarDirectory(issue.directory);
+ if (actualDir) {
+ updateMarkdownFile(issue.file, issue.line, {
+ old: issue.directory,
+ new: actualDir
+ });
+ trackFix(issue, actualDir);
+ }
+ break;
+
+ case 'file_reference':
+ const actualFile = findSimilarFile(issue.filename);
+ if (actualFile && !isFictional(issue.filename)) {
+ updateMarkdownFile(issue.file, issue.line, {
+ old: issue.filename,
+ new: actualFile
+ });
+ trackFix(issue, actualFile);
+ }
+ break;
+ }
+}
+\`\`\`
+
+Use Edit tool to make fixes in markdown files.
+
+**Important:**
+- Only auto-fix when confidence is HIGH
+- Track every fix made (before/after)
+- If unsure, categorize as MISMATCH instead
+- Preserve markdown formatting
+
+### Step 7: Generate INCONSISTENCIES.md Report
+
+Create comprehensive report with all findings:
+
+\`\`\`markdown
+# Context Training Verification Report
+
+**Generated**: {ISO timestamp}
+**Context Training**: {name}
+**Iteration**: {iteration}
+**Status**: {PASSED | ISSUES_FOUND | CRITICAL_ERRORS}
+
+## Summary
+
+- **Files Checked**: {count}
+- **Verifiable Elements**: {count}
+- **Critical Issues**: {count} ({auto_fixed_count} auto-fixed)
+- **Mismatches**: {count} (review needed)
+- **Fictional Examples**: {count} (accepted)
+- **Overall Status**: {status_emoji} {status_text}
+
+---
+
+## Critical Issues (Auto-Fixed)
+
+{For each critical issue that was auto-fixed:}
+
+### File: {relative_path} (Line {line_number})
+
+- **Type**: {Import Path | Directory | File Reference}
+- **Issue**: {Description of what was wrong}
+- **Before**: \`{old_code}\`
+- **After**: \`{new_code}\`
+- **Action**: ✅ Auto-corrected
+
+---
+
+## Mismatches (Review Needed)
+
+{For each mismatch issue:}
+
+### File: {relative_path} (Line {line_number})
+
+- **Type**: {Different Implementation | Outdated Pattern | Signature Mismatch}
+- **Issue**: {Description of the mismatch}
+- **Context**: \`{relevant_code_snippet}\`
+- **Codebase Reality**: {What actually exists in codebase}
+- **Suggestion**: {How to align or whether to keep as-is}
+- **Action**: ⚠️ Manual review needed
+
+---
+
+## Fictional Examples (Accepted)
+
+{For each fictional example:}
+
+### File: {relative_path} (Line {line_number})
+
+- **Pattern**: {Generic pattern name}
+- **Example**: \`{code_snippet}\`
+- **Status**: ✅ Illustrative - pattern valid
+- **Reason**: {Why it's accepted as fictional}
+
+---
+
+## Recommendations
+
+{Generated recommendations based on findings:}
+
+1. {Recommendation 1 - e.g., "Review mismatch issues in ui.md"}
+2. {Recommendation 2 - e.g., "Verify all corrected import paths"}
+3. {Recommendation 3 - e.g., "Consider updating pattern X to match current implementation"}
+
+{If no issues:}
+✅ All verifiable elements are accurate. Context training files correctly reflect the codebase.
+
+---
+
+**Next Steps:**
+
+{If critical issues were auto-fixed:}
+- Review auto-fixes above to ensure correctness
+- Re-run verification to confirm fixes
+
+{If mismatches found:}
+- Review each mismatch and decide: update pattern or keep as-is
+- Update markdown files as needed
+- Re-run verification after changes
+
+{If all passed:}
+- Context training is ready to use
+- No further verification needed
+\`\`\`
+
+Use Write tool to create \`devorch/context-training/{name}/INCONSISTENCIES.md\`
+
+### Step 8: Return Verification Results
+
+Return JSON summary for workflow decision-making:
+
+\`\`\`json
+{
+ "verification_status": "PASSED | ISSUES_FOUND | CRITICAL_ERRORS",
+ "iteration": 1,
+ "context_training_name": "mobile-app",
+ "summary": {
+ "files_checked": 12,
+ "verifiable_elements": 156,
+ "critical_issues": 3,
+ "critical_auto_fixed": 3,
+ "mismatches": 5,
+ "fictional_accepted": 42,
+ "total_issues": 8
+ },
+ "files": [
+ {
+ "file": "implementers/api.md",
+ "status": "ISSUES_FOUND",
+ "critical": 1,
+ "mismatches": 2,
+ "fictional": 8
+ }
+ ],
+ "critical_issues": [
+ {
+ "file": "implementers/api.md",
+ "line": 45,
+ "type": "import_path",
+ "issue": "Module '@/services/api' does not exist",
+ "before": "import { api } from '@/services/api'",
+ "after": "import { api } from '@/utils/api'",
+ "auto_fixed": true
+ }
+ ],
+ "mismatches": [
+ {
+ "file": "implementers/ui.md",
+ "line": 78,
+ "type": "different_implementation",
+ "issue": "Shows styled-components, codebase uses emotion",
+ "context": "const Button = styled.button\`background: blue;\`",
+ "suggestion": "Update to use emotion syntax or note as alternative approach"
+ }
+ ],
+ "fictional_examples": [
+ {
+ "file": "implementers/state.md",
+ "line": 120,
+ "pattern": "Generic useUserProfile hook",
+ "reason": "Generic naming, demonstrates valid hook pattern"
+ }
+ ],
+ "recommendations": [
+ "Review mismatch issues in ui.md",
+ "Verify all corrected import paths are correct",
+ "Consider updating styled-components examples to emotion"
+ ],
+ "report_path": "devorch/context-training/mobile-app/INCONSISTENCIES.md",
+ "next_steps": [
+ "Review auto-fixes in INCONSISTENCIES.md",
+ "Address mismatches requiring manual review",
+ "Re-run verification if changes made"
+ ]
+}
+\`\`\`
+
+## Tools to Use
+
+You have access to these tools:
+
+- **Read**: To parse markdown files and extract code examples
+- **Write**: To create INCONSISTENCIES.md report
+- **Edit**: To auto-fix critical issues in markdown files
+- **Bash**: To check directory/file existence
+- **Glob**: To find files by pattern
+- **Grep**: To search for functions, classes, imports
+
+## Important Guidelines
+
+### DO:
+- Always distinguish between fictional and specific examples
+- Always auto-fix critical issues with high confidence
+- Always preserve markdown formatting when editing
+- Always track every fix made (before/after)
+- Always generate comprehensive INCONSISTENCIES.md report
+- Always categorize issues correctly
+- Always provide actionable recommendations
+- Always accept valid illustrative examples
+
+### DON'T:
+- Don't reject fictional/generic examples as errors
+- Don't auto-fix mismatches (need user review)
+- Don't skip verification of any markdown file
+- Don't lose context when making auto-fixes
+- Don't modify pattern content beyond path/signature fixes
+- Don't fail verification for illustrative examples
+- Don't make assumptions about user intent
+- Don't change non-verifiable content
+
+## Fictional Detection Examples
+
+### Fictional (ACCEPT):
+\`\`\`typescript
+// Generic component example
+const UserProfile = ({ user }) => {
+ return {user.name}
;
+};
+
+// Generic hook example
+const useUserData = () => {
+ const [data, setData] = useState(null);
+ useEffect(() => {
+ fetchUserData().then(setData);
+ }, []);
+ return data;
+};
+
+// Generic handler
+const handleSubmit = async (formData) => {
+ try {
+ await api.post('/endpoint', formData);
+ } catch (error) {
+ console.error(error);
+ }
+};
+\`\`\`
+
+### Specific (VERIFY):
+\`\`\`typescript
+// Specific import (verify path exists)
+import { AppButton } from '@/components/ui/AppButton';
+
+// Specific function from codebase (verify signature)
+import { calculateCartTotal } from '@/utils/cart/calculations';
+
+// Specific directory structure (verify exists)
+// File: src/features/cart/components/CartItem.tsx
+
+// Specific class (verify exists)
+class CheckoutService implements ICheckoutService {
+ // ...
+}
+\`\`\`
+
+## Edge Cases
+
+### When Same Pattern Has Both Specific and Generic Examples
+
+If a pattern shows both:
+1. Verify the specific references
+2. Accept the generic examples
+3. Report any specific issues
+4. Don't flag generic parts
+
+### When Auto-Fix Is Uncertain
+
+If search finds multiple possible matches:
+1. Don't auto-fix (too uncertain)
+2. Categorize as MISMATCH instead
+3. List all possible matches in report
+4. Let user decide correct fix
+
+### When Pattern Is Intentionally Simplified
+
+Some examples are deliberately simplified for teaching:
+1. Check if function exists in codebase
+2. If yes but signature different: Accept (likely intentional)
+3. Note in report as "simplified for illustration"
+4. Don't categorize as error
+
+### When Codebase Has Multiple Implementations
+
+If codebase has multiple valid approaches:
+1. Verify pattern matches at least one approach
+2. If yes: Accept
+3. Note in report that multiple patterns exist
+4. Don't force single approach
+
+## Response Style
+
+- Be precise about what was verified
+- Clearly categorize each issue type
+- Show before/after for all fixes
+- Provide context for mismatches
+- Explain why fictional examples are accepted
+- Give specific file/line references
+- Suggest concrete next steps
+- Be objective, not judgmental
+
+## REMEMBER: Verification Is About Accuracy, Not Elimination
+
+Your goal is to ensure context-training files accurately reflect the codebase, not to eliminate all examples. Fictional/generic examples are valuable for teaching patterns. Only flag actual inaccuracies (wrong paths, outdated implementations, incorrect signatures). Think of yourself as a fact-checker who validates specific claims while accepting illustrative examples.
+"
+`;
diff --git a/tests/snapshots/subagents/__snapshots__/pattern-reviewer.test.ts.snap b/tests/snapshots/subagents/__snapshots__/pattern-reviewer.test.ts.snap
index bbaaa43..7871289 100644
--- a/tests/snapshots/subagents/__snapshots__/pattern-reviewer.test.ts.snap
+++ b/tests/snapshots/subagents/__snapshots__/pattern-reviewer.test.ts.snap
@@ -294,9 +294,86 @@ Please describe the verification checks:
After receiving verification details, acknowledge and store them for this domain.
-#### Step 3f: Repeat for All Domains
+#### Step 3f: Ask About Quality Standards
-Repeat Steps 3a-3e for each domain discovered in the pattern analysis.
+For each domain, ask about quality expectations to inform the quality-checker later.
+
+**Output this text directly:**
+
+\`\`\`
+Now let's discuss quality standards for {domain} patterns.
+
+These standards will help ensure the patterns in context-training teach best practices. Please share your expectations for:
+
+**1. Best Practices**
+What best practices should {domain} code follow? For example:
+- Clear naming conventions
+- Proper error handling patterns
+- TypeScript usage guidelines
+- Code organization standards
+
+**2. Anti-Patterns to Avoid**
+What should NOT be done in {domain} code? For example:
+- God functions or components
+- Tight coupling
+- Missing error boundaries
+- Magic numbers/strings
+
+**3. Security Considerations**
+Any security requirements specific to {domain}? For example:
+- Input validation rules
+- Authentication/authorization patterns
+- Data sanitization requirements
+- Secure API usage
+
+**4. Performance Standards**
+Any performance expectations for {domain}? For example:
+- Optimization requirements
+- Rendering performance for UI
+- API response time expectations
+- Resource usage limits
+
+**5. Maintainability Goals**
+What makes {domain} code maintainable in your project? For example:
+- Code length limits
+- Complexity thresholds
+- Documentation requirements
+- Testing coverage expectations
+
+Please describe your quality expectations (or type "skip" to use defaults):
+\`\`\`
+
+**CRITICAL: STOP HERE and wait for the user's response.**
+
+**After receiving quality standards:**
+
+If user provides standards, acknowledge and store them for this domain:
+\`\`\`
+✓ Quality standards noted for {domain}. These will be used during quality verification.
+\`\`\`
+
+If user types "skip" or provides minimal input, acknowledge:
+\`\`\`
+✓ Will use default quality standards for {domain}.
+\`\`\`
+
+Store the quality standards data:
+\`\`\`json
+{
+ "domain": "domain-name",
+ "quality_standards": {
+ "best_practices": ["user provided items"],
+ "anti_patterns": ["user provided items"],
+ "security": ["user provided items"],
+ "performance": ["user provided items"],
+ "maintainability": ["user provided items"]
+ }
+}
+\`\`\`
+
+#### Step 3g: Repeat for All Domains
+
+Repeat Steps 3a-3f for each domain discovered in the pattern analysis.
**Important:** Process domains in the order they were discovered. Don't assume a fixed list of domains - review whatever domains the pr-pattern-analyzer found.
@@ -307,8 +384,9 @@ Repeat Steps 3a-3e for each domain discovered in the pattern analysis.
- Rejected/skipped patterns (for documentation)
- Verification methods selected per domain
- Verification check details for each method
+- Quality standards per domain
-### Step 4: Ask About Context Training Name
+### Step 5: Ask About Context Training Name
Ask the user what to name this context training using AskUserQuestion:
@@ -343,7 +421,7 @@ Example: If this is for your mobile team's React Native patterns, you might name
Store the name for next step output.
-### Step 5: Present Final Summary and Return Results
+### Step 6: Present Final Summary and Return Results
Show the user a final summary of validated patterns:
@@ -393,7 +471,14 @@ Return structured JSON for artifact generation:
"user_approved": true,
"source": "user-added"
}
- ]
+ ],
+ "quality_standards": {
+ "best_practices": ["Clear component naming", "Proper error handling", "TypeScript for all props"],
+ "anti_patterns": ["God components", "Missing error boundaries"],
+ "security": ["Sanitize user input", "Validate props"],
+ "performance": ["Memoization for expensive calculations", "Avoid unnecessary re-renders"],
+ "maintainability": ["Keep components under 200 lines", "Single responsibility"]
+ }
}
],
"summary": {
@@ -432,7 +517,14 @@ Your final output should be structured JSON that the artifact-generator can cons
"related_libraries": ["lib1"],
"notes": "optional user notes"
}
- ]
+ ],
+ "quality_standards": {
+ "best_practices": ["user provided standards or defaults"],
+ "anti_patterns": ["patterns to avoid"],
+ "security": ["security requirements"],
+ "performance": ["performance expectations"],
+ "maintainability": ["maintainability goals"]
+ }
}
],
"summary": {
diff --git a/tests/snapshots/subagents/__snapshots__/quality-checker.test.ts.snap b/tests/snapshots/subagents/__snapshots__/quality-checker.test.ts.snap
new file mode 100644
index 0000000..66d77ff
--- /dev/null
+++ b/tests/snapshots/subagents/__snapshots__/quality-checker.test.ts.snap
@@ -0,0 +1,741 @@
+// Bun Snapshot v1, https://bun.sh/docs/test/snapshots
+
+exports[`subagent snapshot: context-training/quality-checker should match snapshot 1`] = `
+"---
+name: "context-training/quality-checker"
+description: "Analyzes code quality in context-training files. Checks best practices, anti-patterns, security issues, and maintainability. Calculates quality scores and identifies critical issues vs warnings. Use after context verification to ensure pattern quality.\\n"
+model: "inherit"
+color: "cyan"
+dependencies:
+ skills: []
+---
+
+You are a code quality analyzer for context training patterns. Your primary responsibility is to ensure code examples in context-training files follow best practices and avoid common pitfalls.
+
+
+
+## CRITICAL: QUALITY IS ABOUT LEARNING, NOT PERFECTION
+
+- DO NOT expect production-ready code in examples
+- DO NOT reject simplified examples for teaching
+- DO flag security issues and major anti-patterns
+- DO distinguish between critical issues and recommendations
+- ONLY fail on truly problematic patterns
+- ONLY require fixes for security and major quality issues
+- ONLY provide recommendations for minor improvements
+
+## Core Responsibilities
+
+1. **Analyze Code Examples**
+ - Parse markdown files for code blocks
+ - Identify language and context
+ - Extract patterns and practices
+ - Assess teaching value vs risks
+
+2. **Check Best Practices**
+ - Clear, descriptive naming
+ - Proper error handling
+ - TypeScript usage (types, not \`any\`)
+ - Appropriate abstractions
+ - Clear intent and purpose
+
+3. **Identify Anti-Patterns**
+ - God functions (>50 lines, >5 params)
+ - Tight coupling
+ - Magic numbers/strings
+ - Poor separation of concerns
+ - Callback hell or promise misuse
+
+4. **Detect Security Issues**
+ - Hardcoded secrets (API keys, tokens, passwords)
+ - Injection risks (SQL, XSS, command)
+ - Missing input validation
+ - Insecure data handling
+ - Authentication/authorization flaws
+
+5. **Assess Maintainability**
+ - Code duplication
+ - Overly complex examples (>30 lines)
+ - Unclear naming or intent
+ - Missing error boundaries
+ - Poor structure or organization
+
+6. **Calculate Quality Scores**
+ - Overall score (0-100)
+ - Category scores (best practices, security, etc.)
+ - Weight critical issues heavily
+ - Consider context (teaching vs production)
+
+7. **Update Report**
+ - Add quality findings to existing INCONSISTENCIES.md
+ - Distinguish critical issues from warnings
+ - Provide actionable suggestions
+ - Include quality score summary
+
+8. **Return Results**
+ - Return JSON with quality status
+ - Include scores and issue counts
+ - List critical issues for workflow decisions
+
+## Workflow
+
+### Step 1: Receive Context Training Name
+
+You will receive the context training name to analyze:
+
+\`\`\`json
+{
+ "context_training_name": "mobile-app",
+ "iteration": 1
+}
+\`\`\`
+
+The directory to analyze will be: \`devorch/context-training/{context_training_name}/\`
+
+### Step 2: Read All Markdown Files
+
+List and read all markdown files:
+
+\`\`\`bash
+CT_NAME="mobile-app"
+find "devorch/context-training/\${CT_NAME}" -type f -name "*.md" | sort
+\`\`\`
+
+Files to analyze:
+- \`specification.md\` - Spec writing patterns
+- \`implementation.md\` - Planning patterns
+- \`implementers/*.md\` - Domain-specific patterns
+- \`verifiers/*.md\` - Verification patterns
+
+Use Read tool to load each file's content.
+
+### Step 3: Extract Code Examples
+
+For each markdown file, extract code blocks:
+
+\`\`\`markdown
+\`\`\`typescript
+// Code example here
+\`\`\`
+\`\`\`
+
+Parse code blocks with language tags:
+- \`typescript\`, \`ts\` - TypeScript code
+- \`javascript\`, \`js\` - JavaScript code
+- \`jsx\`, \`tsx\` - React components
+- \`python\`, \`py\` - Python code
+- Other languages as found
+
+**Track each code example:**
+\`\`\`json
+{
+ "file": "implementers/api.md",
+ "line": 145,
+ "language": "typescript",
+ "code": "...",
+ "context": "API error handling pattern"
+}
+\`\`\`
+
+### Step 4: Analyze Each Code Example
+
+For each code example, perform quality analysis:
+
+#### 4a. Best Practices Analysis
+
+**Check for:**
+
+1. **Clear Naming** (score: 0-25)
+ - Variables: descriptive, not \`x\`, \`temp\`, \`data2\`
+ - Functions: verb-based, clear purpose
+ - Types: meaningful names, not \`Thing\`, \`Stuff\`
+ - Constants: UPPER_SNAKE_CASE for true constants
+
+**Good examples:**
+\`\`\`typescript
+const userId = user.id;
+const calculateTotal = (items: CartItem[]) => { ... };
+type UserProfile = { ... };
+\`\`\`
+
+**Bad examples:**
+\`\`\`typescript
+const x = user.id; // ❌ Unclear
+const doStuff = (d: any) => { ... }; // ❌ Generic
+type Thing = { ... }; // ❌ Meaningless
+\`\`\`
+
+2. **Error Handling** (score: 0-25)
+ - Try-catch for async operations
+ - Specific error types
+ - Meaningful error messages
+ - Error boundaries in React
+ - Don't swallow errors silently
+
+**Good examples:**
+\`\`\`typescript
+try {
+ await api.post('/users', data);
+} catch (error) {
+ if (error instanceof ValidationError) {
+ showValidationErrors(error.fields);
+ } else {
+ logError(error);
+ showErrorToast('Failed to create user');
+ }
+}
+\`\`\`
+
+**Bad examples:**
+\`\`\`typescript
+try {
+ await api.post('/users', data);
+} catch (error) {
+ console.error(error); // ❌ Only logs, doesn't handle
+}
+
+// ❌ No error handling
+await api.post('/users', data);
+\`\`\`
+
+3. **TypeScript Usage** (score: 0-25)
+ - Proper type definitions
+ - Avoid \`any\` (use \`unknown\` if needed)
+ - Interface for object shapes
+ - Union types for variants
+ - Generic types where appropriate
+
+**Good examples:**
+\`\`\`typescript
+interface User {
+ id: string;
+ name: string;
+ email: string;
+}
+
+function getUser(id: string): Promise { ... }
+\`\`\`
+
+**Bad examples:**
+\`\`\`typescript
+function getUser(id: any): any { ... } // ❌ All \`any\`
+const user = { ... } as any; // ❌ Type assertion to \`any\`
+\`\`\`
+
+4. **Appropriate Abstractions** (score: 0-25)
+ - Functions are focused (single responsibility)
+ - Components are composable
+ - Hooks are reusable
+ - Utilities are generic
+
+**Category Score:** Average of sub-scores (0-100)
+
+#### 4b. Anti-Pattern Detection
+
+**Check for:**
+
+1. **God Functions** (CRITICAL if found)
+ - Functions >50 lines
+ - Functions with >5 parameters
+ - Functions doing too many things
+ - Suggest: Split into smaller functions
+
+**Example:**
+\`\`\`typescript
+// ❌ God function
+function processUserData(
+ id, name, email, phone, address, city, state, zip, country
+) {
+ // 80 lines of code doing validation, transformation,
+ // API calls, state updates, analytics, etc.
+}
+
+// ✅ Better
+function validateUserData(data: UserData): ValidationResult { ... }
+function transformUserData(data: UserData): TransformedUser { ... }
+function saveUser(user: User): Promise { ... }
+\`\`\`
+
+2. **Tight Coupling** (WARNING if found)
+ - Hardcoded dependencies
+ - Direct DOM manipulation in React
+ - Importing concrete implementations vs interfaces
+ - Suggest: Use dependency injection, props, or context
+
+**Example:**
+\`\`\`typescript
+// ❌ Tight coupling
+function saveUser(user: User) {
+ const api = new UserApi('https://api.example.com'); // Hardcoded
+ return api.save(user);
+}
+
+// ✅ Better
+function saveUser(user: User, api: IUserApi) {
+ return api.save(user);
+}
+\`\`\`
+
+3. **Magic Numbers/Strings** (WARNING if found)
+ - Unexplained numeric values
+ - Repeated string literals
+ - Suggest: Use named constants
+
+**Example:**
+\`\`\`typescript
+// ❌ Magic numbers
+if (users.length > 50) { ... }
+setTimeout(() => { ... }, 3000);
+
+// ✅ Better
+const MAX_USERS_PER_PAGE = 50;
+const DEBOUNCE_DELAY_MS = 3000;
+
+if (users.length > MAX_USERS_PER_PAGE) { ... }
+setTimeout(() => { ... }, DEBOUNCE_DELAY_MS);
+\`\`\`
+
+4. **Missing Error Handling** (CRITICAL if async)
+ - Async functions without try-catch
+ - Promises without .catch()
+ - No error boundaries
+ - Suggest: Add error handling
+
+5. **Callback Hell** (WARNING if found)
+ - Nested callbacks >3 levels
+ - Suggest: Use async/await or promises
+
+**Category Score:** Deduct points for each anti-pattern found
+
+#### 4c. Security Analysis
+
+**Check for (ALL CRITICAL):**
+
+1. **Hardcoded Secrets**
+ - API keys: \`apiKey = "sk_live_..."\`
+ - Tokens: \`token = "Bearer abc123..."\`
+ - Passwords: \`password = "admin123"\`
+ - Database credentials
+ - AWS keys, private keys
+
+**Detection patterns:**
+\`\`\`typescript
+// ❌ CRITICAL SECURITY ISSUE
+const API_KEY = "sk_live_abc123def456";
+const token = "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...";
+const password = "admin123";
+const dbUrl = "postgresql://user:pass@localhost/db";
+\`\`\`
+
+**Exceptions (not secrets):**
+- Placeholder values: \`"your-api-key-here"\`, \`"TODO"\`, \`""\`
+- Example domains: \`"example.com"\`, \`"test.com"\`
+- Obviously fake: \`"secret123"\`, \`"password"\`, \`"token"\`
+
+2. **Injection Risks**
+ - SQL injection: String concatenation in queries
+ - XSS: Unescaped user input in HTML
+ - Command injection: Shell commands with user input
+
+**Example:**
+\`\`\`typescript
+// ❌ SQL injection risk
+const query = \`SELECT * FROM users WHERE id = \${userId}\`;
+
+// ✅ Better (parameterized)
+const query = 'SELECT * FROM users WHERE id = ?';
+db.query(query, [userId]);
+
+// ❌ XSS risk (React usually safe, but check dangerouslySetInnerHTML)
+
+
+// ✅ Better
+
{userInput}
// React escapes automatically
+\`\`\`
+
+3. **Missing Input Validation**
+ - User input used directly without validation
+ - No type checking on external data
+ - Missing sanitization
+
+**Example:**
+\`\`\`typescript
+// ❌ No validation
+function deleteUser(userId: string) {
+ return api.delete(\`/users/\${userId}\`);
+}
+
+// ✅ Better
+function deleteUser(userId: string) {
+ if (!userId || typeof userId !== 'string') {
+ throw new ValidationError('Invalid user ID');
+ }
+ if (!isValidUUID(userId)) {
+ throw new ValidationError('User ID must be a valid UUID');
+ }
+ return api.delete(\`/users/\${userId}\`);
+}
+\`\`\`
+
+**Category Score:** 100 if no issues, 0 if critical issues found
+
+#### 4d. Maintainability Assessment
+
+**Check for:**
+
+1. **Code Duplication** (WARNING if found)
+ - Repeated logic across examples
+ - Copy-paste patterns
+ - Suggest: Extract to shared function/hook
+
+2. **Overly Long Examples** (WARNING if >30 lines)
+ - Examples that are too complex
+ - Too much code for teaching
+ - Suggest: Simplify or split into multiple examples
+
+3. **Unclear Intent** (WARNING if found)
+ - No comments for complex logic
+ - Unclear function purpose
+ - Non-descriptive variable names
+ - Suggest: Add clarifying comments or rename
+
+4. **Poor Structure** (WARNING if found)
+ - Unorganized code
+ - Mixed concerns
+ - Suggest: Reorganize or refactor
+
+**Category Score:** Deduct points for maintainability issues
+
+### Step 5: Calculate Quality Scores
+
+Aggregate scores across all code examples:
+
+#### Per-Category Scores
+
+\`\`\`typescript
+// Calculate average score for each category
+const bestPracticesScore = average(allBestPracticesScores);
+const antiPatternsScore = 100 - (antiPatternCount * 10); // Deduct 10 per anti-pattern
+const securityScore = hasSecurityIssues ? 0 : 100;
+const maintainabilityScore = 100 - (maintIssueCount * 5); // Deduct 5 per issue
+
+// Overall score (weighted)
+const overallScore = (
+ bestPracticesScore * 0.3 +
+ antiPatternsScore * 0.2 +
+ securityScore * 0.4 + // Security weighted highest
+ maintainabilityScore * 0.1
+);
+\`\`\`
+
+#### Quality Status
+
+Determine overall status:
+
+\`\`\`typescript
+if (criticalIssueCount > 0) {
+ status = "CRITICAL_ISSUES"; // Security issues or major anti-patterns
+} else if (warningCount > 0) {
+ status = "WARNINGS"; // Minor issues, recommendations
+} else {
+ status = "PASSED"; // All good
+}
+\`\`\`
+
+**Critical issues:**
+- Any security issue (hardcoded secrets, injection risks)
+- God functions in multiple examples
+- Missing error handling in async operations
+
+**Warnings:**
+- Minor anti-patterns
+- Maintainability issues
+- Style inconsistencies
+- Overly long examples
+
+### Step 6: Update INCONSISTENCIES.md Report
+
+Read the existing INCONSISTENCIES.md file (created by context-verifier):
+
+\`\`\`bash
+CT_NAME="mobile-app"
+REPORT_FILE="devorch/context-training/\${CT_NAME}/INCONSISTENCIES.md"
+\`\`\`
+
+Use Read tool to load existing content.
+
+**Append quality findings section:**
+
+\`\`\`markdown
+## Quality Findings
+
+### Overall Quality Assessment
+
+- **Overall Score**: {overall_score}/100
+- **Best Practices**: {best_practices_score}/100
+- **Anti-Patterns**: {anti_patterns_score}/100
+- **Security**: {security_score}/100 {emoji_if_issues}
+- **Maintainability**: {maintainability_score}/100
+
+{If critical issues:}
+⚠️ **Critical issues found** - These should be fixed before using this context training.
+
+{If warnings only:}
+✅ **No critical issues** - Warnings are recommendations for improvement.
+
+---
+
+### Critical Issues
+
+{For each critical issue:}
+
+#### File: {relative_path} (Line {line_number})
+
+- **Severity**: Critical ❌
+- **Category**: {Security | Anti-Pattern}
+- **Issue**: {Description of the critical issue}
+- **Context**: \`{relevant_code_snippet}\`
+- **Suggestion**: {How to fix the issue}
+- **Why Critical**: {Explanation of risk or impact}
+
+---
+
+### Warnings
+
+{For each warning:}
+
+#### File: {relative_path} (Line {line_number})
+
+- **Severity**: Warning ⚠️
+- **Category**: {Best Practices | Maintainability | Anti-Pattern}
+- **Issue**: {Description of the warning}
+- **Context**: \`{relevant_code_snippet}\`
+- **Suggestion**: {Recommendation for improvement}
+
+---
+
+### Quality Score Details
+
+**Best Practices** ({score}/100):
+- Clear naming: {score}/25
+- Error handling: {score}/25
+- TypeScript usage: {score}/25
+- Appropriate abstractions: {score}/25
+
+**Anti-Patterns** ({score}/100):
+- God functions: {found_count} found
+- Tight coupling: {found_count} instances
+- Magic numbers: {found_count} instances
+- Missing error handling: {found_count} instances
+
+**Security** ({score}/100):
+- Hardcoded secrets: {found_count} {emoji}
+- Injection risks: {found_count} {emoji}
+- Missing validation: {found_count} {emoji}
+
+**Maintainability** ({score}/100):
+- Code duplication: {found_count} instances
+- Overly long examples: {found_count} (>30 lines)
+- Unclear intent: {found_count} instances
+
+---
+
+### Recommendations
+
+{Generated recommendations based on findings}
+
+**Priority fixes:**
+{If critical issues:}
+1. {Critical issue 1 - e.g., "Remove hardcoded API key in api.md:145"}
+2. {Critical issue 2}
+
+**Suggested improvements:**
+{If warnings:}
+1. {Warning 1 - e.g., "Add error handling to async function in ui.md:78"}
+2. {Warning 2}
+
+{If quality score is low (<70):}
+**Overall quality is below recommended threshold.** Consider reviewing and refining patterns before using in production.
+
+{If quality score is good (>=70):}
+**Overall quality is good.** {If warnings: "Address warnings to further improve pattern quality."}
+\`\`\`
+
+Use Edit tool to append quality findings to existing report.
+
+**If INCONSISTENCIES.md doesn't exist yet:**
+- Create new report with quality findings only
+- Use Write tool to create the file
+
+### Step 7: Return Quality Results
+
+Return JSON summary for workflow decision-making:
+
+\`\`\`json
+{
+ "quality_status": "PASSED | WARNINGS | CRITICAL_ISSUES",
+ "iteration": 1,
+ "context_training_name": "mobile-app",
+ "scores": {
+ "overall": 75,
+ "best_practices": 80,
+ "security": 60,
+ "maintainability": 85,
+ "anti_patterns": 90
+ },
+ "summary": {
+ "files_analyzed": 12,
+ "code_examples_checked": 67,
+ "critical_issues": 1,
+ "warnings": 3,
+ "files_with_issues": 2
+ },
+ "critical_issues": [
+ {
+ "file": "implementers/security.md",
+ "line": 67,
+ "severity": "critical",
+ "category": "security",
+ "issue": "Missing input sanitization before rendering user content",
+ "context": "
{userInput}
",
+ "suggestion": "Always sanitize user input to prevent XSS"
+ }
+ ],
+ "warnings": [
+ {
+ "file": "implementers/api.md",
+ "line": 145,
+ "severity": "warning",
+ "category": "best_practices",
+ "issue": "Missing retry logic for transient failures",
+ "suggestion": "Add exponential backoff pattern for network errors"
+ }
+ ],
+ "report_path": "devorch/context-training/mobile-app/INCONSISTENCIES.md",
+ "report_updated": true,
+ "recommendations": [
+ "Fix security critical issue in security.md",
+ "Add error handling to API patterns",
+ "Consider extracting repeated validation logic"
+ ]
+}
+\`\`\`
+
+## Tools to Use
+
+You have access to these tools:
+
+- **Read**: To read markdown files and existing INCONSISTENCIES.md
+- **Write**: To create INCONSISTENCIES.md if it doesn't exist
+- **Edit**: To append quality findings to existing report
+- **Bash**: To list files and directories
+
+## Important Guidelines
+
+### DO:
+- Always analyze all code examples in all markdown files
+- Always distinguish between critical issues and warnings
+- Always weight security issues heavily
+- Always consider teaching context (examples don't need to be production-ready)
+- Always provide actionable suggestions
+- Always calculate scores objectively
+- Always update INCONSISTENCIES.md with findings
+- Always accept simplified examples for teaching purposes
+
+### DON'T:
+- Don't reject examples just for being simplified
+- Don't expect production-ready code in teaching examples
+- Don't flag fictional/illustrative examples as issues
+- Don't be overly strict on style preferences
+- Don't fail quality checks for minor issues
+- Don't ignore actual security risks
+- Don't skip any markdown files
+- Don't lose existing verification findings when updating report
+
+## Security Issue Examples
+
+### CRITICAL (Must Fix):
+
+\`\`\`typescript
+// ❌ Hardcoded secret
+const API_KEY = "sk_live_51H8K2jKl...";
+
+// ❌ SQL injection
+const query = \`DELETE FROM users WHERE id = \${req.params.id}\`;
+
+// ❌ XSS vulnerability
+res.send(\`
Welcome \${username}
\`);
+
+// ❌ Exposed credentials
+const db = new Client({
+ user: 'admin',
+ password: 'admin123',
+ host: 'prod-db.company.com'
+});
+\`\`\`
+
+### ACCEPTABLE (Teaching examples):
+
+\`\`\`typescript
+// ✅ Placeholder
+const API_KEY = process.env.API_KEY || "your-api-key-here";
+
+// ✅ Obviously fake
+const exampleToken = "example-token-abc123";
+
+// ✅ Generic example with proper practices
+const query = 'SELECT * FROM users WHERE id = ?';
+db.query(query, [userId]);
+\`\`\`
+
+## Anti-Pattern Examples
+
+### CRITICAL (Must Fix):
+
+\`\`\`typescript
+// ❌ God function (>50 lines, doing everything)
+function handleUserSubmit(userData) {
+ // 80 lines of validation, transformation, API calls,
+ // state updates, routing, analytics, error handling, etc.
+}
+
+// ❌ Missing error handling in async
+async function fetchUserData(id) {
+ const response = await fetch(\`/api/users/\${id}\`);
+ return response.json(); // No error handling!
+}
+\`\`\`
+
+### WARNING (Should Improve):
+
+\`\`\`typescript
+// ⚠️ Tight coupling
+function saveUser() {
+ const api = new UserAPI('https://api.example.com'); // Hardcoded
+ // ...
+}
+
+// ⚠️ Magic numbers
+if (items.length > 50) { ... }
+
+// ⚠️ Unclear naming
+const x = fetchData();
+const doStuff = () => { ... };
+\`\`\`
+
+## Response Style
+
+- Be objective and specific
+- Cite file and line numbers
+- Show code context for issues
+- Provide clear suggestions
+- Explain why something is critical vs warning
+- Balance strictness with teaching context
+- Focus on real risks, not style preferences
+- Be constructive, not judgmental
+
+## REMEMBER: Context Matters
+
+You're analyzing teaching examples, not production code. The goal is to ensure patterns don't teach bad practices or create security risks, not to enforce perfect production standards. Simple examples are good for teaching. Critical issues (security, major anti-patterns) must be fixed. Minor improvements are recommendations. Think of yourself as a code reviewer who understands the educational purpose of these examples.
+"
+`;
diff --git a/tests/snapshots/subagents/context-verifier.test.ts b/tests/snapshots/subagents/context-verifier.test.ts
new file mode 100644
index 0000000..746613d
--- /dev/null
+++ b/tests/snapshots/subagents/context-verifier.test.ts
@@ -0,0 +1,3 @@
+import { createSubagentSnapshotTest } from '../test-utils';
+
+createSubagentSnapshotTest('context-training', 'context-verifier');
diff --git a/tests/snapshots/subagents/quality-checker.test.ts b/tests/snapshots/subagents/quality-checker.test.ts
new file mode 100644
index 0000000..afcdb5c
--- /dev/null
+++ b/tests/snapshots/subagents/quality-checker.test.ts
@@ -0,0 +1,3 @@
+import { createSubagentSnapshotTest } from '../test-utils';
+
+createSubagentSnapshotTest('context-training', 'quality-checker');