fix: restore lost custom sections and prevent future overwrite regresion (#869)

fix: restore lost custom sections and prevent future overwrite regression

Two bugs in the toolkit-docs-generator caused hand-authored custom
sections (documentationChunks, customImports, subPages) to be silently
wiped on every CI run for affected toolkits:

Bug 1 (Critical): buildFallbackToolkit in previous-output.ts hardcoded
empty arrays for documentationChunks, customImports, and subPages instead
of carrying them forward from the raw record. This was triggered whenever
MergedToolkitSchema.safeParse() failed — which happened for any toolkit
whose chunks used location/type values missing from the schema enum
("before_available_tools", "after_available_tools", "custom_section",
"section"). Once wiped, the preservation logic had nothing to restore,
so every subsequent CI run preserved the empty arrays.

Bug 2 (High): buildMergeErrorResult in data-merger.ts also hardcoded
empty arrays, meaning any exception during mergeToolkitEntry would wipe
custom sections from the output file.

Root cause: DocumentationChunkLocationSchema and DocumentationChunkTypeSchema
in the generator were out of sync with the frontend types, which already
supported the broader set of values. MergedToolkitSchema.subPages was also
declared as z.array(z.string()) while jira and zoom had rich objects.

Fixes:
- Sync DocumentationChunkLocationSchema: add before_available_tools,
  after_available_tools, custom_section
- Sync DocumentationChunkTypeSchema: add section
- Add header and priority optional fields to DocumentationChunkSchema
- Fix MergedToolkitSchema.subPages to accept string | Record union
- buildFallbackToolkit now reads documentationChunks/customImports/subPages
  from raw record instead of hardcoding empty arrays
- buildMergeErrorResult now accepts previousToolkit and uses its custom
  sections instead of hardcoding empty arrays
- Restore 17 affected toolkit JSON files from git history (github,
  brightdata, clickup, googlecalendar, googledocs, googlefinance,
  googleflights, googlehotels, googlejobs, googlemaps, googlenews,
  googleslides, intercomapi, jira, linear, zohobooksapi, zoom)
- Add 4 tests covering the fallback preservation and error-result
  preservation paths

All 17 toolkits now parse STRICT (no fallback) in the CI round-trip.

Made-with: Cursor

Author: jottakka

app/_components/toolkit-docs/types/index.ts

@@ -18,7 +18,8 @@ export type DocumentationChunkType =
   | "code"
   | "warning"
   | "info"
-  | "tip";
+  | "tip"
+  | "section";
 
 /**
  * Location where the chunk should be injected
@@ -343,8 +344,12 @@ export type ToolkitData = {
   documentationChunks?: DocumentationChunk[];
   /** Custom imports for MDX */
   customImports: string[];
-  /** Sub-pages that exist for this toolkit */
-  subPages: string[];
+  /**
+   * Sub-pages that exist for this toolkit.
+   * Each entry is either a string (legacy slug) or a rich object with
+   * { type, content, relativePath } for inline MDX sub-page content.
+   */
+  subPages: (string | Record<string, unknown>)[];
   /** Optional pip package name override */
   pipPackageName?: string;
   /** Generation timestamp */

toolkit-docs-generator/data/toolkits/brightdata.json

@@ -408,9 +408,17 @@
       }
     }
   ],
-  "documentationChunks": [],
+  "documentationChunks": [
+    {
+      "type": "section",
+      "location": "custom_section",
+      "position": "after",
+      "content": "## Secrets\n\nThis tool requires the following secrets:\n\n- `BRIGHTDATA_API_KEY`\n- `BRIGHTDATA_ZONE`\n\n### Auth\n\nThe Arcade Bright Data MCP Server uses [Bright Data](https://brightdata.com/) to access proxy networks and web scraping infrastructure.\n\n**Global Environment Variables:**\n\n- `BRIGHTDATA_API_KEY`: Your Bright Data API key. You can generate this from your [Bright Data dashboard](https://brightdata.com/cp/zones) under Account Settings → API Access.\n\n- `BRIGHTDATA_ZONE`: Your Bright Data zone name (e.g., `residential_proxy1`). This is the zone identifier you created in your Bright Data dashboard under Proxies & Scraping Infrastructure → Zones.\n\n**How to get your credentials:**\n\n1. **API Key**: Navigate to your [Bright Data Control Panel](https://brightdata.com/cp) → Settings → API Access → Generate API Token\n2. **Zone**: Go to Zones section in your dashboard, find your zone name in the format shown in the zone username: `brd-customer-{customer_id}-zone-{zone_name}`\n\nFor more details, see the [Bright Data API Documentation](https://docs.brightdata.com/api-reference).",
+      "header": "## Secrets"
+    }
+  ],
   "customImports": [],
   "subPages": [],
   "generatedAt": "2026-03-01T11:12:08.063Z",
   "summary": "Bright Data provides a developer toolkit for large-scale web search, crawling, and scraping, enabling reliable extraction of pages and structured data without getting blocked. It supports search queries, content-to-Markdown conversion, and configurable data feeds across many site types.\n\nDesigned for integration into data pipelines and analytics workflows with parameterized feeds and output formats.\n\n**Capabilities**\n- Scale-resistant crawling and scraping with anti-blocking behavior for sustained collection.\n- Flexible search engine queries with advanced parameters across major engines.\n- Transform pages into clean Markdown and emit structured JSON feeds for profiles, products, reviews, listings, and media.\n- Configurable extraction parameters for batching, pagination, and media handling.\n\n**Secrets**\n- API key (BRIGHTDATA_API_KEY) and zone token (BRIGHTDATA_ZONE). Example values: BRIGHTDATA_API_KEY=sk_..., BRIGHTDATA_ZONE=zone123."
-}
+}

toolkit-docs-generator/data/toolkits/clickup.json

@@ -2688,9 +2688,31 @@
       }
     }
   ],
-  "documentationChunks": [],
+  "documentationChunks": [
+    {
+      "type": "section",
+      "location": "custom_section",
+      "position": "after",
+      "content": "## TaskPriority\n\n- **URGENT**: `URGENT`\n- **HIGH**: `HIGH`\n- **NORMAL**: `NORMAL`\n- **LOW**: `LOW`",
+      "header": "## TaskPriority"
+    },
+    {
+      "type": "section",
+      "location": "custom_section",
+      "position": "after",
+      "content": "## TaskOrderBy\n\n- **CREATED**: `created`\n- **UPDATED**: `updated`\n- **DUE_DATE**: `due_date`",
+      "header": "## TaskOrderBy"
+    },
+    {
+      "type": "section",
+      "location": "custom_section",
+      "position": "after",
+      "content": "## CommentResolution\n\n- **SET_AS_RESOLVED**: `resolved`\n- **SET_AS_UNRESOLVED**: `unresolved`",
+      "header": "## CommentResolution"
+    }
+  ],
   "customImports": [],
   "subPages": [],
   "generatedAt": "2026-02-26T20:45:10.379Z",
   "summary": "The Arcade.dev Toolkit for ClickUp empowers developers to efficiently manage tasks, comments, and workspace structures within ClickUp through seamless API interactions. \n\n**Capabilities**\n- Create, update, and retrieve tasks and comments, enabling direct engagement with team tasks.\n- Conduct fuzzy searches for lists, folders, and members to enhance navigability.\n- Retrieve insights about workspaces and configure task statuses dynamically.\n- Manage team assignments effectively, facilitating team collaboration.\n- Execute structured queries for tasks based on specified criteria to optimize task management.\n\n**OAuth**  \n- Provider: ClickUp  \n- Note: Utilizes OAuth2 authentication for secure access.\n\n**Secrets**  \n- No secrets required for integration."
-}
+}

toolkit-docs-generator/data/toolkits/github.json

@@ -5727,9 +5727,45 @@
       }
     }
   ],
-  "documentationChunks": [],
-  "customImports": [],
+  "documentationChunks": [
+    {
+      "type": "warning",
+      "location": "description",
+      "position": "after",
+      "content": "<Callout type=\"warning\">\n  **Critical**: This MCP Server is built for **GitHub Apps**, not OAuth Apps.\n  \n  You **must** create a GitHub App (not an OAuth App) to use this server properly. \n  \n  👉 [Complete GitHub App Setup Guide](/references/auth-providers/github)\n</Callout>"
+    },
+    {
+      "type": "info",
+      "location": "description",
+      "position": "after",
+      "content": "<Callout type=\"info\">\n  **Configuration Required**: All tools require secrets to be configured in Arcade Dashboard. See [Secrets Setup](#secrets-setup) below.\n</Callout>"
+    },
+    {
+      "type": "info",
+      "location": "before_available_tools",
+      "position": "after",
+      "content": "## GitHub Enterprise Support\n\n<Callout type=\"info\">\n  This MCP Server fully supports **GitHub Enterprise Server 2.22+**\n</Callout>\n\n**Default Configuration:**\n- If no `GITHUB_SERVER_URL` is configured, the default is `https://api.github.com` (GitHub.com)\n- All tools work with GitHub.com out of the box\n\n**For GitHub Enterprise Server:**\n\n1. Create your GitHub App on your Enterprise instance (not github.com)\n2. Configure the `GITHUB_SERVER_URL` secret in Arcade Dashboard (see [Secrets Setup](#secrets-setup) below)\n3. Use your Enterprise server's API endpoint\n\n**Example Enterprise Server URLs:**\n- `https://github.yourcompany.com/api/v3`\n- `https://enterprise.yourorg.com/api/v3`\n- `https://git.company.internal/api/v3`\n\n<Callout>\n  **Note**: GitHub Enterprise Server uses the `/api/v3` path after the hostname. GitHub.com uses `https://api.github.com` (no `/api/v3` suffix).\n</Callout>\n\n---",
+      "header": "## GitHub Enterprise Support"
+    },
+    {
+      "type": "section",
+      "location": "before_available_tools",
+      "position": "after",
+      "content": "## GitHub App Permissions Summary\n\nWhen creating your GitHub App, you'll need to grant specific permissions. Here's a quick reference of which tools require which permissions:\n\n### Repository Permissions\n\n| Permission | Level | Required For |\n|------------|-------|--------------|\n| **Contents** | Read | All repository and pull request tools, getting file contents |\n| **Contents** | Write | Creating/updating files, creating branches, merging PRs |\n| **Issues** | Read & Write | Issue management, PR assignments, managing labels (Issues) |\n| **Pull requests** | Read & Write | Pull request management, reviews, managing labels (PRs) |\n| **Metadata** | Read | All tools (automatically granted) |\n| **Statuses** | Read | `CheckPullRequestMergeStatus` |\n\n### Organization Permissions\n\n| Permission | Level | Required For |\n|------------|-------|--------------|\n| **Members** | Read | Projects, collaborators, org repos, user search |\n| **Projects** | Read & Write | All Projects V2 tools |\n\n### User Permissions\n\n| Permission | Level | Required For |\n|------------|-------|--------------|\n| **Read user profile** | Read | User context tools, review workload |\n| **Act on behalf of user** | Enabled | `SetStarred` (starring repositories) |\n\n### Tools by Permission Requirements\n\n<details>\n<summary><b>Basic Repository Access</b> (Contents Read + Metadata)</summary>\n\n- `GetRepository`\n- `CountStargazers`\n- `ListStargazers`\n- `ListRepositoryActivities`\n- `GetFileContents`\n\n</details>\n\n<details>\n<summary><b>Repository Write</b> (Contents Write + Metadata)</summary>\n\n- `CreateBranch`\n- `CreateOrUpdateFile`\n- `UpdateFileLines`\n\n</details>\n\n<details>\n<summary><b>Issue Management</b> (Contents Read + Issues + Metadata)</summary>\n\n- `CreateIssue`\n- `UpdateIssue`\n- `GetIssue`\n- `ListIssues`\n- `CreateIssueComment`\n- `ListRepositoryLabels`\n- `ManageLabels` (for issues)\n\n</details>\n\n<details>\n<summary><b>Pull Request Read</b> (Contents + Pull requests Read + Metadata)</summary>\n\n- `ListPullRequests`\n- `GetPullRequest`\n- `ListPullRequestCommits`\n- `ListReviewCommentsOnPullRequest`\n- `CheckPullRequestMergeStatus` (+ Statuses)\n\n</details>\n\n<details>\n<summary><b>Pull Request Write</b> (Contents Read + Pull requests Write + Metadata)</summary>\n\n- `UpdatePullRequest`\n- `CreatePullRequest`\n- `SubmitPullRequestReview`\n- `ManagePullRequest`\n- `ManagePullRequestReviewers`\n- `CreateReviewComment`\n- `CreateReplyForReviewComment`\n- `ResolveReviewThread`\n- `ManageLabels` (for pull requests)\n- `MergePullRequest` (+ Contents Write)\n\n</details>\n\n<details>\n<summary><b>Organization Tools</b> (Contents + Metadata + Members)</summary>\n\n- `ListOrgRepositories`\n- `SearchMyRepos`\n- `ListRepositoryCollaborators`\n- `AssignPullRequestUser` (+ Issues Write)\n\n</details>\n\n<details>\n<summary><b>Projects V2</b> (Contents + Metadata + Projects + Members)</summary>\n\n- `ListProjects`\n- `ListProjectItems`\n- `SearchProjectItem`\n- `ListProjectFields`\n- `UpdateProjectItem` (Projects Write)\n\n</details>\n\n<details>\n<summary><b>User Context</b> (Contents + Metadata + Read user profile)</summary>\n\n- `WhoAmI` (+ Members)\n- `GetUserRecentActivity`\n- `GetUserOpenItems`\n- `GetReviewWorkload` (+ Pull requests Read)\n\n</details>\n\n<details>\n<summary><b>⚠️ Special: Notifications</b> (Requires Classic PAT)</summary>\n\n- `GetNotificationSummary`\n- `ListNotifications`\n\n**Note**: GitHub Apps cannot access notifications API. Requires classic Personal Access Token with `notifications` scope.\n\n</details>\n\n---",
+      "header": "## GitHub App Permissions Summary"
+    },
+    {
+      "type": "warning",
+      "location": "custom_section",
+      "position": "after",
+      "content": "## Configuration & Setup\n\n### Authentication\n\n<Callout type=\"warning\">\n  **Critical**: This MCP Server uses **GitHub Apps** authentication, not OAuth Apps.\n  \n  You **must** create a GitHub App to use this server. OAuth Apps are not supported.\n</Callout>\n\nThe Arcade GitHub MCP Server uses the [GitHub auth provider](/references/auth-providers/github) to connect to users' GitHub accounts.\n\n**For Arcade Cloud:**\n- No configuration needed\n- Your users will see `Arcade` as the requesting application\n- All tools work out of the box\n\n**For Self-Hosted:**\n- You must [create your own GitHub App](/references/auth-providers/github#creating-a-github-app)\n- [Configure the GitHub auth provider](/references/auth-providers/github#configuring-github-auth-in-arcade) with your app credentials\n- Your users will see your application name\n\n<Callout type=\"info\">\n  **New to GitHub Apps?** Read [Why Arcade Uses GitHub Apps](/references/auth-providers/github#why-arcade-uses-github-apps-not-oauth-apps) \n  to understand the security and compliance benefits.\n</Callout>\n\n### Secrets Setup\n\nAll tools require secrets to be configured in Arcade Dashboard.\n\n**Steps:**\n\n1. Go to [Arcade Dashboard](https://api.arcade.dev/dashboard)\n2. Navigate to **Secrets** in the left sidebar\n3. Click **Add Secret**\n4. Add the following secrets:\n\n| Secret Name | Value | Required For |\n|-------------|-------|--------------|\n| `GITHUB_SERVER_URL` | `https://api.github.com` (default for GitHub.com) | All tools |\n| `GITHUB_CLASSIC_PERSONAL_ACCESS_TOKEN` | Classic PAT with `notifications` scope | Notifications tools only |\n\n<Callout type=\"info\">\n  **Default**: If `GITHUB_SERVER_URL` is not configured, it defaults to `https://api.github.com` (GitHub.com)\n  \n  **GitHub Enterprise Users**: Set `GITHUB_SERVER_URL` to your Enterprise server's API endpoint (e.g., `https://github.yourcompany.com/api/v3`). Note that Enterprise uses `/api/v3` path. See [GitHub Enterprise Support](#github-enterprise-support) for details.\n</Callout>\n\n<Callout type=\"warning\">\n  **For Notifications Tools**: Create a classic Personal Access Token at [github.com/settings/tokens](https://github.com/settings/tokens) with only the `notifications` scope checked. GitHub Apps cannot access the notifications API.\n</Callout>\n\n---",
+      "header": "## Configuration & Setup"
+    }
+  ],
+  "customImports": [
+    "import { Callout, Tabs } from \"nextra/components\";"
+  ],
   "subPages": [],
   "generatedAt": "2026-03-04T11:17:26.984Z",
   "summary": "Arcade.dev's GitHub toolkit exposes programmatic access to repositories, pull requests, issues, reviews, and project data. It enables automated repo/file edits, PR lifecycle handling, issue/project workflows, and user/activity discovery.\n\n**Capabilities**\n- Repository and file management: create branches and files, update file blocks, and read file contents or ranges.\n- Pull request and review workflows: create/update/merge PRs, manage reviewers, post and resolve review comments, and check merge status.\n- Issue, label, and project automation: create/update issues and comments, manage labels, and list/search Project V2 items and fields.\n- Search, auditing, and user context: list collaborators, stargazers, recent activity, and search/repos accessible to the authenticated user.\n\n**OAuth**\nProvider: GitHub\nScopes: None (no predeclared scopes; runtime consent required)\n\n**Secrets**\nGITHUB_SERVER_URL — secret type unknown; typically a GitHub Enterprise base URL (e.g., https://github.example.com). Store endpoints and tokens in a secure secrets manager."
-}
+}

toolkit-docs-generator/data/toolkits/googlecalendar.json

@@ -892,9 +892,26 @@
       }
     }
   ],
-  "documentationChunks": [],
-  "customImports": [],
+  "documentationChunks": [
+    {
+      "type": "markdown",
+      "location": "auth",
+      "position": "after",
+      "content": "The Arcade Google Calendar MCP Server uses the [Google auth provider](/references/auth-providers/google) to connect to users' Google accounts.\n---",
+      "header": "## Auth"
+    },
+    {
+      "type": "section",
+      "location": "custom_section",
+      "position": "after",
+      "content": "## UpdateGoogleMeetOptions\n\n- **`NONE`**: No action is taken.\n- **`ADD`**: Add the Google Meet link to the event.\n- **`REMOVE`**: Remove the Google Meet link from the event.\n\n<ToolFooter pipPackageName=\"arcade_google_calendar\" />",
+      "header": "## UpdateGoogleMeetOptions"
+    }
+  ],
+  "customImports": [
+    "import ScopePicker from \"@/app/_components/scope-picker\";"
+  ],
   "subPages": [],
   "generatedAt": "2026-02-26T20:45:10.392Z",
   "summary": "Arcade.dev offers a toolkit for Google Calendar that streamlines event management and calendar interaction through powerful APIs. This toolkit enables developers to effectively create, update, and manage calendar events while accessing user calendars.\n\n**Capabilities:**\n- Create and delete events seamlessly.\n- Fetch user calendars and events within specified date ranges.\n- Identify time slots when all participants are available.\n- Retrieve user profile information from Google services.\n\n**OAuth:**\n- Provider: Google\n- Scopes: `https://www.googleapis.com/auth/calendar.events`, `https://www.googleapis.com/auth/calendar.readonly`, `https://www.googleapis.com/auth/userinfo.email`, `https://www.googleapis.com/auth/userinfo.profile`."
-}
+}

toolkit-docs-generator/data/toolkits/googledocs.json

@@ -1156,9 +1156,32 @@
       }
     }
   ],
-  "documentationChunks": [],
-  "customImports": [],
+  "documentationChunks": [
+    {
+      "type": "warning",
+      "location": "description",
+      "position": "after",
+      "content": "<Callout type=\"warning\">\n  This Toolkit is not available in Arcade Cloud. You can use these tools with a\n  [self-hosted](/guides/deployment-hosting/configure-engine) instance of Arcade.\n</Callout>"
+    },
+    {
+      "type": "section",
+      "location": "custom_section",
+      "position": "after",
+      "content": "## Tab Support\n\nGoogle Docs supports hierarchical tabs within documents. The Google Docs tools provide comprehensive support for working with tabs:\n\n- **Tab Metadata**: `GetDocumentMetadata` returns hierarchical tab structures with approximate character and word counts for each tab\n- **Tab Content**: `GetDocumentAsDocMD` and `SearchAndRetrieveDocuments` include all tab content in their output\n- **Tab Filtering**: `GetDocumentAsDocMD` supports filtering to retrieve content from a specific tab using the `tab_id` parameter\n\nTabs are represented with the following structure:\n- Each tab has a unique `tabId`, `title`, `index`, and `nestingLevel`\n- Tabs can be nested up to 3 levels deep (parent → child → grandchild)\n- Tab metadata includes approximate character and word counts for each tab's content\n\n---",
+      "header": "## Tab Support"
+    },
+    {
+      "type": "markdown",
+      "location": "auth",
+      "position": "after",
+      "content": "The Arcade Google Docs MCP Server uses the [Google auth provider](/references/auth-providers/google) to connect to users' Google accounts.\n---",
+      "header": "## Auth"
+    }
+  ],
+  "customImports": [
+    "import ScopePicker from \"@/app/_components/scope-picker\";"
+  ],
   "subPages": [],
   "generatedAt": "2026-02-26T20:45:10.393Z",
   "summary": "Arcade.dev Google Docs toolkit lets LLM-driven apps create, read, edit, comment on, and search Google Docs while accessing document metadata and user profile information.\n\n**Capabilities**\n- Create and populate new documents or append and edit document body content programmatically.\n- Retrieve structured document content (DocMD) and metadata including tab structure and character counts for robust context-aware workflows.\n- Search Drive for documents, list comments, and produce file-picker URLs to recover from permission or missing-file errors.\n- Access authenticated user profile and environment info to tailor behavior and permissions.\n\n**OAuth**\nProvider: google\nScopes: https://www.googleapis.com/auth/drive.file, https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile\n\n**Secrets**\nType: api_key\nExample: OPENAI_API_KEY (used for LLM or external API calls; store in environment variables or a secret manager)."
-}
+}