Sync .github directory with azure-sdk-tools for PR 15062#7058
Open
Sync .github directory with azure-sdk-tools for PR 15062#7058
Conversation
Rename 5 shared skills to use the azsdk-common- prefix so the sync-.github.yml pipeline can pattern-match and distribute them to all subscribed Azure SDK language repos. Renames: generate-sdk-locally → azsdk-common-generate-sdk-locally apiview-feedback-resolution → azsdk-common-apiview-feedback-resolution pipeline-troubleshooting → azsdk-common-pipeline-troubleshooting prepare-release-plan → azsdk-common-prepare-release-plan sdk-release → azsdk-common-sdk-release Updated: - SKILL.md name: fields in all 5 skills - All eval.yaml and trigger_tests.yaml skill/name references - .github/skills/README.md links and naming convention docs - sensei SKILL.md example reference - 8 instruction files in eng/common/instructions/azsdk-tools/ with Related Skill cross-references pointing to new skill paths - copilot/sdk-release.instructions.md with skill cross-reference Relates to #15049 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Delete 7 instruction files whose content is now fully covered by skills: - local-sdk-workflow.instructions.md → azsdk-common-generate-sdk-locally - create-release-plan.instructions.md → azsdk-common-prepare-release-plan - sdk-details-in-release-plan.instructions.md → azsdk-common-prepare-release-plan - verify-namespace-approval.instructions.md → azsdk-common-prepare-release-plan - check-package-readiness.instructions.md → azsdk-common-sdk-release - check-package-validation.instructions.md → azsdk-common-generate-sdk-locally - copilot/sdk-release.instructions.md → azsdk-common-sdk-release Skills updated with all unique details from instruction files: - generate-sdk-locally: added commit checkpoints, language selection, config file identification, SDK project path, metadata update tools - prepare-release-plan: added critical LLM instructions (NextSteps field), Release Planner Tool link, package name extraction rules per language, valid/invalid examples table, JSON structure, namespace approval flow - sdk-release: added readiness check details, interaction flow pattern, do-not-ask-for-PR/plan guard, case-sensitive language note typespec-to-sdk orchestrator updated to reference skills directly. Removed all related_skill frontmatter and blockquotes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous commit incorrectly rewrote 3 SKILL.md files from scratch. This restores the original content authored by the tools team, with only the name: field updated to use the azsdk-common- prefix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
azure-sdk
added
EngSys
Contributor
There was a problem hiding this comment.
Pull request overview
Syncs this repo’s .github/skills content with azure-sdk-tools (PR 15062) by adding a set of shared “azsdk-common-*” Copilot skill definitions, along with task specs, fixtures, references, and evaluation suites used to validate triggering/behavior.
Changes:
- Added five shared skills: SDK release, prepare release plan, pipeline troubleshooting, local SDK generation/customization, and APIView feedback resolution.
- Added task YAMLs (happy-path / edge / negative) and supporting references/fixtures for each skill.
- Added eval configurations and trigger/anti-trigger tests for automated validation.
Reviewed changes
Copilot reviewed 60 out of 60 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/skills/azsdk-common-sdk-release/SKILL.md | New skill definition for checking readiness and triggering SDK release pipelines |
| .github/skills/azsdk-common-sdk-release/tasks/release-basic-001.yaml | Happy-path task for full release flow |
| .github/skills/azsdk-common-sdk-release/tasks/release-trigger-001.yaml | Task focused on triggering the release pipeline |
| .github/skills/azsdk-common-sdk-release/tasks/release-readiness-001.yaml | Task focused on readiness check only |
| .github/skills/azsdk-common-sdk-release/tasks/release-negative-001.yaml | Negative task to ensure non-release prompts don’t trigger release actions |
| .github/skills/azsdk-common-sdk-release/eval.yaml | Eval suite definition (top-level) for the release skill |
| .github/skills/azsdk-common-sdk-release/evals/eval.yaml | Eval suite definition (under evals/) for the release skill |
| .github/skills/azsdk-common-sdk-release/evals/trigger_tests.yaml | Trigger/anti-trigger prompt set for the release skill |
| .github/skills/azsdk-common-sdk-release/evals/tasks/basic-trigger.yaml | Eval task validating release skill triggers on release requests |
| .github/skills/azsdk-common-sdk-release/evals/tasks/anti-trigger.yaml | Eval task validating release skill does not trigger on unrelated requests |
| .github/skills/azsdk-common-prepare-release-plan/SKILL.md | New skill definition for creating/updating/linking release plans |
| .github/skills/azsdk-common-prepare-release-plan/tasks/basic-usage.yaml | Task for creating a new release plan |
| .github/skills/azsdk-common-prepare-release-plan/tasks/link-sdk-prs.yaml | Task for linking SDK PRs to an existing release plan |
| .github/skills/azsdk-common-prepare-release-plan/tasks/edge-case.yaml | Task for validating/correcting package name formats |
| .github/skills/azsdk-common-prepare-release-plan/tasks/should-not-trigger.yaml | Negative task to avoid triggering on “release package now” prompts |
| .github/skills/azsdk-common-prepare-release-plan/references/release-plan-details.md | Detailed reference steps for gathering required info and updating plans |
| .github/skills/azsdk-common-prepare-release-plan/fixtures/tspconfig.yaml | Fixture TypeSpec config used by the release-plan skill tests |
| .github/skills/azsdk-common-prepare-release-plan/eval.yaml | Eval suite definition (top-level) for the release-plan skill |
| .github/skills/azsdk-common-prepare-release-plan/evals/eval.yaml | Eval suite definition (under evals/) for the release-plan skill |
| .github/skills/azsdk-common-prepare-release-plan/evals/trigger_tests.yaml | Trigger/anti-trigger prompt set for the release-plan skill |
| .github/skills/azsdk-common-prepare-release-plan/evals/tasks/basic-trigger.yaml | Eval task validating release-plan skill triggers appropriately |
| .github/skills/azsdk-common-prepare-release-plan/evals/tasks/anti-trigger.yaml | Eval task validating release-plan skill does not trigger on unrelated prompts |
| .github/skills/azsdk-common-pipeline-troubleshooting/SKILL.md | New skill definition for diagnosing CI/pipeline failures |
| .github/skills/azsdk-common-pipeline-troubleshooting/tasks/basic-usage.yaml | Task for analyzing a pipeline failure and calling analysis tooling |
| .github/skills/azsdk-common-pipeline-troubleshooting/tasks/local-reproduction.yaml | Task for guiding local reproduction of pipeline failures |
| .github/skills/azsdk-common-pipeline-troubleshooting/tasks/edge-case.yaml | Task for multi-language pipeline failures |
| .github/skills/azsdk-common-pipeline-troubleshooting/tasks/should-not-trigger.yaml | Negative task to avoid triggering for TypeSpec authoring questions |
| .github/skills/azsdk-common-pipeline-troubleshooting/references/failure-patterns.md | Reference guide of common pipeline failure patterns and fixes |
| .github/skills/azsdk-common-pipeline-troubleshooting/eval.yaml | Eval suite definition (top-level) for the pipeline troubleshooting skill |
| .github/skills/azsdk-common-pipeline-troubleshooting/evals/eval.yaml | Eval suite definition (under evals/) for the pipeline troubleshooting skill |
| .github/skills/azsdk-common-pipeline-troubleshooting/evals/trigger_tests.yaml | Trigger/anti-trigger prompt set for the pipeline troubleshooting skill |
| .github/skills/azsdk-common-pipeline-troubleshooting/evals/tasks/basic-trigger.yaml | Eval task validating pipeline troubleshooting skill triggers on pipeline prompts |
| .github/skills/azsdk-common-pipeline-troubleshooting/evals/tasks/anti-trigger.yaml | Eval task validating pipeline troubleshooting skill does not trigger on unrelated prompts |
| .github/skills/azsdk-common-generate-sdk-locally/SKILL.md | New skill definition for local SDK generation/build/test + customization |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/basic-usage.yaml | Task for local Python SDK generation flow |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/full-workflow.yaml | Task covering end-to-end local generation/build/check/test workflow |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/edge-case.yaml | Task for handling build failures via customization |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/customization-workflow.yaml | Task for resolving duplicate-field conflicts after regeneration |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/breaking-changes.yaml | Task for resolving breaking-change-driven build failures |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/analyzer-errors.yaml | Task for fixing .NET analyzer errors via customization |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/rename-client.yaml | Task for renaming client/model via TypeSpec customization |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/hide-operation.yaml | Task for hiding internal operations from public SDK API |
| .github/skills/azsdk-common-generate-sdk-locally/tasks/anti-trigger.yaml | Negative task to avoid triggering for pipeline-based generation requests |
| .github/skills/azsdk-common-generate-sdk-locally/references/customization-workflow.md | Reference describing the two-phase customization workflow |
| .github/skills/azsdk-common-generate-sdk-locally/references/sdk-repos.md | Reference mapping languages to SDK repos and config paths |
| .github/skills/azsdk-common-generate-sdk-locally/fixtures/tspconfig.yaml | Fixture TypeSpec config for local generation scenarios |
| .github/skills/azsdk-common-generate-sdk-locally/eval.yaml | Eval suite definition for the local generation skill |
| .github/skills/azsdk-common-apiview-feedback-resolution/SKILL.md | New skill definition for retrieving/categorizing/resolving APIView feedback |
| .github/skills/azsdk-common-apiview-feedback-resolution/tasks/basic-usage.yaml | Task ensuring APIView comments retrieval triggers tool usage |
| .github/skills/azsdk-common-apiview-feedback-resolution/tasks/no-feedback.yaml | Task for handling “no APIView comments found” scenarios |
| .github/skills/azsdk-common-apiview-feedback-resolution/tasks/edge-case.yaml | Task for feedback requiring TypeSpec changes |
| .github/skills/azsdk-common-apiview-feedback-resolution/tasks/should-not-trigger.yaml | Negative task to avoid triggering for TypeSpec project creation prompts |
| .github/skills/azsdk-common-apiview-feedback-resolution/references/feedback-resolution-steps.md | Reference steps for categorizing/resolving/delegating APIView feedback |
| .github/skills/azsdk-common-apiview-feedback-resolution/fixtures/apiview-comment.json | Fixture APIView comment payload for tests |
| .github/skills/azsdk-common-apiview-feedback-resolution/eval.yaml | Eval suite definition (top-level) for APIView feedback resolution |
| .github/skills/azsdk-common-apiview-feedback-resolution/evals/eval.yaml | Eval suite definition (under evals/) for APIView feedback resolution |
| .github/skills/azsdk-common-apiview-feedback-resolution/evals/trigger_tests.yaml | Trigger/anti-trigger prompt set for APIView feedback resolution |
| .github/skills/azsdk-common-apiview-feedback-resolution/evals/tasks/basic-trigger.yaml | Eval task validating the skill triggers on APIView prompts |
| .github/skills/azsdk-common-apiview-feedback-resolution/evals/tasks/url-trigger.yaml | Eval task validating URL-only APIView prompts trigger the skill |
| .github/skills/azsdk-common-apiview-feedback-resolution/evals/tasks/anti-trigger.yaml | Eval task validating the skill does not trigger on unrelated prompts |
Comment on lines
+14
to
+29
| ## MCP Tools | ||
|
|
||
| | Tool | Purpose | | ||
| |------|---------| | ||
| | `azure-sdk-mcp:azsdk_package_generate_code` | Generate SDK from TypeSpec | | ||
| | `azure-sdk-mcp:azsdk_package_build_code` | Build package | | ||
| | `azure-sdk-mcp:azsdk_package_run_check` | Validate package | | ||
| | `azure-sdk-mcp:azsdk_package_run_tests` | Run tests | | ||
| | `azure-sdk-mcp:azsdk_customized_code_update` | Apply TypeSpec and code customizations to resolve build errors, breaking changes, or SDK modification requests (includes regeneration and build internally) | | ||
|
|
||
| **Prerequisites:** azure-sdk-mcp server must be running. Without MCP, use `npx tsp-client` CLI. | ||
|
|
||
| ## Steps | ||
|
|
||
| 1. **Verify** — Run `azure-sdk-mcp:azsdk_verify_setup` to confirm environment. | ||
| 2. **Generate** — Run `azure-sdk-mcp:azsdk_package_generate_code` with `tspconfig.yaml` or `tsp-location.yaml` path. |
There was a problem hiding this comment.
The MCP tool list table omits azure-sdk-mcp:azsdk_verify_setup, but the steps and troubleshooting sections instruct using it. This makes the documentation internally inconsistent; add the tool to the table (and consider including it in the INVOKES summary if that field is used for tool discovery).
Comment on lines
+8
to
+25
| compatibility: | ||
| requires: "azure-sdk-mcp server, local azure-sdk-for-{language} clone, language build tools" | ||
| --- | ||
|
|
||
| # Generate SDK Locally | ||
|
|
||
| ## MCP Tools | ||
|
|
||
| | Tool | Purpose | | ||
| |------|---------| | ||
| | `azure-sdk-mcp:azsdk_package_generate_code` | Generate SDK from TypeSpec | | ||
| | `azure-sdk-mcp:azsdk_package_build_code` | Build package | | ||
| | `azure-sdk-mcp:azsdk_package_run_check` | Validate package | | ||
| | `azure-sdk-mcp:azsdk_package_run_tests` | Run tests | | ||
| | `azure-sdk-mcp:azsdk_customized_code_update` | Apply TypeSpec and code customizations to resolve build errors, breaking changes, or SDK modification requests (includes regeneration and build internally) | | ||
|
|
||
| **Prerequisites:** azure-sdk-mcp server must be running. Without MCP, use `npx tsp-client` CLI. | ||
|
|
There was a problem hiding this comment.
The compatibility frontmatter says this skill "requires" azure-sdk-mcp server, but the prerequisites/steps describe a CLI fallback (npx tsp-client) when MCP is unavailable. Clarify whether MCP is truly required or supported-but-optional, so users and any automation relying on this metadata behave correctly.
| config: | ||
| regex_not_match: | ||
| - (?i)error occurred | ||
| - (?i)failed |
There was a problem hiding this comment.
The eval grader forbids the substring "failed" (case-insensitive). Several tasks explicitly involve build failures; a correct response will often include words like "failed" or "failure", causing false negatives. Narrow this to truly fatal/system errors (similar to the other eval configs) or scope it to specific phrases you want to forbid.
Suggested change
| - (?i)failed | |
| - (?i)command failed unexpectedly |
Comment on lines
+17
to
+24
| Identify languages from `tspconfig.yaml` emitter configuration: | ||
|
|
||
| - Map emitter names to languages (.NET, Java, Python, JavaScript, Go) | ||
| - Extract package names per language | ||
| - **Validate package names:** | ||
| - JavaScript: Must start with `@azure/` | ||
| - Go: Must start with `sdk/` | ||
| - Run `azsdk_update_sdk_details_in_release_plan` with language/package JSON |
There was a problem hiding this comment.
This reference says Go package names must start with sdk/, but the included fixture uses a full Go module path (github.com/Azure/azure-sdk-for-go/sdk/...) which will never start with sdk/. Clarify whether validation should check for /sdk/ in the module path, or whether the extracted value should be repo-relative (e.g., sdk/...).
Comment on lines
+6
to
+23
| distribution: shared | ||
| description: "Analyze and resolve APIView review feedback on Azure SDK PRs. **UTILITY SKILL**. USE FOR: APIView comments, API review feedback, SDK API surface changes. DO NOT USE FOR: general code review, non-APIView feedback. INVOKES: azure-sdk-mcp:azsdk_apiview_get_comments, azure-sdk-mcp:azsdk_typespec_customized_code_update." | ||
| compatibility: | ||
| requires: "azure-sdk-mcp server, SDK pull request with APIView review link" | ||
| --- | ||
|
|
||
| # APIView Feedback Resolution | ||
|
|
||
| **Prerequisites:** azure-sdk-mcp server required; no CLI fallback. Without MCP, this skill cannot retrieve APIView comments or apply TypeSpec changes. Connect the `azure-sdk-mcp` server before use. | ||
|
|
||
| ## MCP Tools | ||
|
|
||
| | Tool | Purpose | | ||
| |------|---------| | ||
| | `azure-sdk-mcp:azsdk_apiview_get_comments` | Get APIView comments | | ||
| | `azure-sdk-mcp:azsdk_typespec_customized_code_update` | Apply TypeSpec changes locally | | ||
| | `azure-sdk-mcp:azsdk_typespec_delegate_apiview_feedback` | Delegate to CCA pipeline | | ||
| | `azure-sdk-mcp:azsdk_run_typespec_validation` | Validate TypeSpec | |
There was a problem hiding this comment.
azure-sdk-mcp:azsdk_typespec_customized_code_update is referenced here, but the related local generation skill uses azure-sdk-mcp:azsdk_customized_code_update. If these are meant to be the same tool, this inconsistent naming will prevent the skill from invoking the intended MCP capability; please confirm the correct tool name and use it consistently throughout the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync .github directory with azure-sdk-tools for PR Azure/azure-sdk-tools#15062 See eng/common workflow