Skip to content

Adding Copilot skills+instructions+agent for specialized compiler review (trained on historical feedback here)#19447

Merged
T-Gro merged 6 commits intomainfrom
feat/expert-review
Mar 18, 2026
Merged

Adding Copilot skills+instructions+agent for specialized compiler review (trained on historical feedback here)#19447
T-Gro merged 6 commits intomainfrom
feat/expert-review

Conversation

@T-Gro
Copy link
Member

@T-Gro T-Gro commented Mar 18, 2026

No description provided.

T-Gro and others added 6 commits March 17, 2026 15:40
…y extraction

Generated from analysis of 695 review comments (2,102 effective weighted)
across 4 expert reviewers spanning 10 years of dotnet/fsharp and
fsharp/fslang-design review history.

Artifacts:
- .github/agents/expert-reviewer.md: 15-dimension review agent with
  5-wave workflow and folder hotspot mapping
- .github/skills/expert-review/SKILL.md: Slim invocation pointer with
  dimension selection guide and self-review checklist
- .github/instructions/ExpertReview.instructions.md: Top 10 compiler
  review rules scoped to src/Compiler/**
- .github/agents/extraction-pipeline.md: Reusable pipeline agent for
  future reviewer expertise extraction

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Skill name: gerund form (reviewing-compiler-prs)
- Description: third person, keyword-rich, specific triggers
- Removed redundant prose (Claude already knows)
- Added cross-references to complementary skills (review-council,
  hypothesis-driven-debugging, ilverify-failure, vsintegration-ide-debugging)
- Slim pointer pattern: skill is overview, agent is source of truth

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The review-council skill (generic multi-model review) and expert-review
skill (F#-specific dimensions) had 0.55 trigger overlap. Merged into
a single reviewing-compiler-prs skill that combines:
- F#-specific 15 dimensions with file-based selection
- Multi-model 3-phase methodology (briefing, parallel assessment,
  consolidation with dedup/false-positive filtering)
- Self-review checklist

Deleted .github/skills/review-council/ — its methodology is now
embedded in the merged skill's Multi-Model Review Methodology section.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Description: natural third-person prose, no keyword spam list
- Removed redundant opening paragraph (restated description)
- Trimmed Multi-Model section: removed briefing pack steps Claude
  already knows; kept only unique content (assessment gates,
  consolidation rules)
- Removed redundant 'Before requesting...' preamble
- 75 -> 54 lines (28% reduction, no content loss)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Agent (expert-reviewer.md):
- Dim 1: test layer selection guide (Typecheck/SyntaxTree/EmittedIL/
  compileAndRun/Service/Core), happy/negative/interaction paths,
  assertion quality
- Dim 12: pipelines over nesting, active patterns for complex
  conditions, nesting depth limit (2 levels), wide over deep
- Dim 13: ad-hoc if/then band-aid detection, higher-order pattern
  extraction, search for existing utilities before reimplementing,
  respect intentional deviations in isolated projects

Skill (SKILL.md):
- Claims coverage check (orphan claims/changes/partial impls)
- 'Correct convention' false positive filter
- 'Unexplained != wrong' — doc gap not defect
- 'Agents love nitpicks' — deprioritize
- Minimum viable council = 2 models

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… from instructions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Contributor

✅ No release notes required

@T-Gro T-Gro merged commit 4a2c294 into main Mar 18, 2026
10 checks passed

---

### 8. IDE Performance & Reactor Efficiency
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Now that's a name I've not heard in a long time"

Just a nitpick, but I wonder if the term Reactor wouldn't be confusing both to LLMs and human users of this repo. It's been replaced long time ago. Search for "reactor" returns FSharpProjectOptionsReactor which is something entirely different.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you are right, I should get rid of it - its a risk of confusion (or lost tokens spend searching for what it is), while not gaining anything.

But it surprisingly came up when using 10y of PR feedback and issue comments in this repo as training data :))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The training obviously overfitted on case of historical PRs receiving many comments, this is one of the examples.

(i.e. PR with 40 review comments is over-repsented in my training over one that was created without much negative feedback and merged it)

Will do another round.

@T-Gro T-Gro deleted the feat/expert-review branch March 18, 2026 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants