[Schema Consistency] Schema Consistency Check - 2026-03-04

[github-actions[bot]]

Automated schema consistency analysis run for §22664165419 on 2026-03-04 using Strategy 20: Behavioral Change Archaeology + Documentation Cross-Reference Audit.

This strategy reads pending changeset files to surface recently-landed behavioral changes, then cross-references the code against all documentation files to find stale or contradictory entries.

Summary

• Inconsistencies Found: 5
• Categories Analyzed: Schema, Parser/Compiler, Documentation, Workflows
• Strategy Used: Strategy 20 — Behavioral Change Archaeology + Documentation Cross-Reference Audit
• New Strategy: Yes

Severity	Count	Items
Critical	1	reaction: status comment docs contradiction
High	2	Top-level bots: schema, safe-outputs.md global fields
Medium	2	features.copilot-requests, on.status-comment in frontmatter.md

---

Critical Issues

1. triggers.md contradicts glossary.md on reaction: + status comments

Changeset: minor-decouple-status-comment.md — "Decouple the status comment from the ordinary ai-reaction emoji so they must each be enabled explicitly (e.g., add status-comment: true if you still need the started/completed comment)."

The problem: Two documentation files now directly contradict each other:

• docs/src/content/docs/reference/triggers.md:317 — WRONG (stale):

  "The reaction is added to the triggering item. For issues/PRs, a comment with the workflow run link is created. For comment events in command workflows, the comment is edited to include the run link."

• docs/src/content/docs/reference/command-triggers.md:176 — WRONG (stale):

  "Command workflows automatically add the 'eyes' (👀) emoji reaction to triggering comments and edit them with workflow run links."

• docs/src/content/docs/reference/glossary.md:102 — CORRECT:

  "Must be explicitly enabled — it is not automatically bundled with ai-reaction."

Code confirms the decoupling (pkg/workflow/compiler_activation_job.go:141-142):

// Add comment with workflow run link if status comments are explicitly enabled
if data.StatusComment != nil && *data.StatusComment {

And actions/setup/js/add_reaction.cjs:11 confirms:

"This script only adds reactions - it does NOT create comments."

Impact: Users who add reaction: "eyes" expecting the status comment behaviour they read about in triggers.md will see only the emoji, with no workflow run link comment. This was a breaking behaviour change with no update to the primary documentation page.

Fix: Update triggers.md and command-triggers.md to remove the claim that reactions automatically create status comments, and add a note directing users to status-comment: true.

---

High Priority Issues

Issue 2: Top-level bots: in schema not marked as deprecated

Schema location: pkg/parser/schemas/main_workflow_schema.json — properties.bots

The problem: The schema defines bots as a top-level property with no deprecation marker:

"bots": {
  "type": "array",
  "description": "Allow list of bot identifiers that can trigger the workflow even if they don't meet the required role permissions.",
  ...
}

However, the compiler only reads from on.bots (pkg/workflow/role_checks.go:142-158):

func (c *Compiler) extractBots(frontmatter map[string]any) []string {
    // Check on.bots
    if onValue, exists := frontmatter["on"]; exists {
        if onMap, ok := onValue.(map[string]any); ok {
            if botsValue, hasBots := onMap["bots"]; hasBots {
                ...
            }
        }
    }
    // No bots specified, return empty array
    return []string{}
}

A codemod (pkg/cli/codemod_bots.go) migrates top-level bots: → on.bots:. The documentation correctly mentions this (docs/.../frontmatter.md:287):

"Run gh aw fix workflow.md --write to automatically migrate top-level bots: to on.bots: using the built-in codemod."

But the schema itself does not mark the top-level bots: as deprecated. Users relying on schema validation alone (without running gh aw compile) would see no warning about the field being ignored.

Impact: Workflows using top-level bots: will silently compile without bot filtering, as the field is ignored.

Fix: Add a deprecation note to the top-level bots: schema description, e.g.:

"description": "DEPRECATED: Use 'on.bots' instead. Will be silently ignored — run 'gh aw fix' to migrate."

Issue 3: safe-outputs.md missing documentation for global configuration fields

The primary safe-outputs.md reference page covers all safe-output handler types but does not document the global configuration fields that apply to the safe-outputs job itself. These fields are in both the schema (properties.safe-outputs.properties) and the compiler (pkg/workflow/compiler_types.go) but only appear in frontmatter-full.md:

Field	Schema	Code	safe-outputs.md	frontmatter-full.md
id-token	✅	✅ frontmatter_types.go:390	❌	✅ line 4448
runs-on	✅	✅ compiler_types.go:491	❌	✅
steps	✅	✅ compiler_types.go:497	❌	✅
activation-comments	✅	✅ safe_outputs_config.go:461	❌	✅ line 4426

The schema for id-token describes a non-trivial feature (OIDC auto-detection override), but users reading safe-outputs.md will never encounter it.

Fix: Add a "Global Configuration" section to safe-outputs.md covering id-token, runs-on, steps, activation-comments, group-reports, and max-bot-mentions.

---

Medium Priority Issues

Issue 4: features.copilot-requests implemented but undocumented

Changeset: patch-add-copilot-requests-feature.md — "Documented the features.copilot-requests feature flag so GitHub Actions token authentication, threat detection permissions, and the Copilot CLI execution environment honor the Copilot requests flow."

The changeset claims the feature was documented, but a search through all reference docs returns zero hits for copilot-requests:

grep -rn "copilot-requests" docs/src/content/docs/  # → no results

The feature is implemented in the code:

• pkg/constants/constants.go:632: CopilotRequestsFeatureFlag FeatureFlag = "copilot-requests"
• Used in pkg/workflow/copilot_engine_execution.go:214 to inject COPILOT_GITHUB_TOKEN, enable S2STOKENS=true, and add copilot-requests: write permissions
• Used in pkg/workflow/tools.go:196 to configure the GitHub tools token chain

The schema (features.additionalProperties: true) allows any feature flag, so schema validation does not help here.

Impact: Users of the Copilot engine who want to use the managed Copilot requests token flow have no documentation to follow.

Fix: Add documentation for features.copilot-requests to docs/src/content/docs/reference/engines.md or a dedicated features reference.

Issue 5: on.status-comment absent from frontmatter.md

The on.status-comment field (schema: properties.on.oneOf[1].properties.status-comment) appears in:

• frontmatter-full.md:683 ✅
• glossary.md:102 ✅
• All 10+ smoke test workflows (e.g., .github/workflows/smoke-agent.md:8) ✅

But it does not appear in the primary frontmatter.md reference. The on: triggers section of frontmatter.md documents reaction:, stop-after:, skip-bots:, roles:, etc., but omits status-comment:.

Given the behaviour change in minor-decouple-status-comment (now requiring explicit opt-in), this field needs prominent documentation in the primary reference.

Fix: Add a ### Status Comment (\on.status-comment:`)section tofrontmatter.md, adjacent to the reaction:` section, explaining that status comments (workflow run link comments) must now be explicitly opted into.

---

Recommendations

1. Immediately update triggers.md — remove the sentence "For issues/PRs, a comment with the workflow run link is created" and replace with a note that status-comment: true must be set explicitly. Also update command-triggers.md accordingly. This is the highest-impact change since it directly contradicts the glossary.

2. Add a deprecation notice to the top-level bots: schema field — the description should mention that the field is ignored by the compiler and point users to on.bots.

3. Add a "Global Configuration" section to safe-outputs.md — covering id-token, runs-on, steps, and activation-comments with the same depth as the handler sections.

4. Document features.copilot-requests — add a section to engines.md explaining when and how to use this feature flag for Copilot engine workflows.

5. Add status-comment to frontmatter.md — given the decoupling change, users need to find this field in the primary reference without reading the full reference file.

---

Strategy Performance

• Strategy Used: Strategy 20 — Behavioral Change Archaeology + Documentation Cross-Reference Audit
• Findings: 5 (1 critical, 2 high, 2 medium)
• Effectiveness: HIGH — reading changeset files directly surfaced the critical status-comment decoupling that had not been propagated to documentation
• Should Reuse: Yes — this strategy is particularly effective when pending changesets are present in .changeset/

Next Steps

• Fix docs/src/content/docs/reference/triggers.md — remove false claim about automatic status comments
• Fix docs/src/content/docs/reference/command-triggers.md — update reactions section
• Update pkg/parser/schemas/main_workflow_schema.json — add deprecation to top-level bots: description
• Expand docs/src/content/docs/reference/safe-outputs.md — add global config fields section
• Document features.copilot-requests in engine reference docs
• Add on.status-comment to docs/src/content/docs/reference/frontmatter.md

References:

• §22664165419

AI generated by Schema Consistency Checker · history

• expires on Mar 5, 2026, 10:08 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-05T10:08:20.471Z.

Closed by Workflow