Skip to content

TESTING.md

Latest commit

 

History

History
189 lines (154 loc) · 5.58 KB

TESTING.md

File metadata and controls

189 lines (154 loc) · 5.58 KB

Testing MCP TypeScript Type Injection

This document explains how to test the TypeScript type injection feature for MCP tools.

What Was Implemented

The system now provides complete MCP integration following the MCP specification:

1. Tools ✅

  • execute_code - Execute JavaScript/TypeScript with MCP tool access
  • list_runtimes - Show available runtimes and capabilities
  • get_runtime_capabilities - Get detailed runtime information
  • runtime_health_check - Verify runtime operational status

2. Resources ✅

  • mcp://types/declarations - TypeScript declarations for all MCP tools
    • Read this resource BEFORE writing code to get type information
    • Contains full TypeScript interfaces, function signatures, and TSDoc comments

3. Prompts ✅

  • mcp-tool-example - Template for calling MCP tools with error handling
  • async-handler - Async function template with logging
  • mcp-batch-operations - Template for parallel MCP operations

4. Runtime Type Injection ✅

  • Auto-generates TypeScript declarations from MCP schemas on startup
  • Injects types into TypeScript-aware runtimes (Bun, Deno) during execution
  • Provides full IDE-like type safety and code completion

How It Works

1. Type Generation

When the MCP server starts with CODEMODE_ENABLE_MCP_INTEGRATION=true:

  • Discovers all connected MCP servers and their tool schemas
  • Converts JSON Schema to TypeScript interfaces
  • Generates generated/mcp.d.ts with all type definitions

2. Runtime Injection

When executing code with TypeScript-aware runtimes:

  • System checks capabilities.supportsTypeScript
  • If true, prepends TypeScript declarations to the code
  • Runtime can now type-check and provide completion

3. Type-Safe Tool Access

User code gets full TypeScript support:

// IDE will show autocomplete and type checking
const memory = await mcp.automem.store_memory({
  content: "Test",
  tags: ["demo"],
  importance: 0.8  // IDE knows this must be a number
});

// Hyphenated names work too
const result = await mcp["sequential-thinking"].sequentialthinking({
  thought: "...",
  nextThoughtNeeded: true,
  thoughtNumber: 1,
  totalThoughts: 5
});

Testing via Claude Code

The best way to test is through the Claude Code MCP tool:

Test 1: Basic Type Injection

Ask Claude Code to:
"Use the codemode-unified execute_code tool with bun runtime to test MCP types:

await mcp.automem.store_memory({
  content: 'Test from Claude',
  importance: 0.9
})
"

Test 2: Type Error Detection

Ask Claude Code to:
"Execute this code with bun and show any type errors:

await mcp.automem.store_memory({
  content: 'Test',
  importance: 'high'  // Should error: string not assignable to number
})
"

Test 3: Hyphenated Namespaces

Ask Claude Code to:
"Use sequential-thinking via the MCP proxy in bun runtime:

await mcp['sequential-thinking'].sequentialthinking({
  thought: 'Testing types',
  nextThoughtNeeded: false,
  thoughtNumber: 1,
  totalThoughts: 1
})
"

Manual Testing

Start MCP Server with Types Enabled

cd services/codemode-unified
CODEMODE_ENABLE_MCP_INTEGRATION=true npx tsx src/mcp-server.ts

Output should show:

🔧 Generating TypeScript declarations for MCP tools...
   File: ./generated/mcp.d.ts
   Servers: 4
   Tools: 25
✅ TypeScript declarations generated: ./generated/mcp.d.ts

Verify Generated Types

cat generated/mcp.d.ts | head -100

Should show TypeScript interfaces like:

declare global {
  const mcp: {
    automem: {
      store_memory(args: AutomemStoreMemoryArgs): Promise<MCPToolResult>;
      // ...
    };
    "sequential-thinking": {
      sequentialthinking(args: SequentialThinkingSequentialthinkingArgs): Promise<MCPToolResult>;
    };
    // ...
  };
}

Expected Results

✅ Success Indicators

  • Generated mcp.d.ts file exists and contains TypeScript declarations
  • Bun/Deno runtimes receive type definitions in injected code
  • Type errors are caught during execution (not just ignored)
  • IDE-like type checking works in supported runtimes

❌ Failure Indicators

  • No mcp.d.ts generated (MCP integration disabled or no servers connected)
  • TypeScript errors in generated declarations (syntax issues, malformed types)
  • Runtime doesn't receive type definitions (check supportsTypeScript capability)
  • Type violations not detected (runtime not enforcing types)

Architecture Notes

Type Injection Flow

MCP Server Startup
  → CodeGenService.generateDeclarations()
  → generated/mcp.d.ts created

Execute Code (Bun/Deno)
  → Check capabilities.supportsTypeScript
  → loadTypeScriptDeclarations()
  → Prepend to user code
  → Runtime executes with full type information

Files Modified

  • src/codegen/declaration-generator.ts - Generates TypeScript declarations
  • src/mcp-server.ts - Loads and injects declarations into runtime
  • generated/mcp.d.ts - Auto-generated type definitions

Key Design Decisions

  1. On-Demand Generation: Types generated once at startup, cached for performance
  2. Runtime Conditional: Only inject types for TypeScript-aware runtimes
  3. Global Declaration: Use declare global for ergonomic access without imports
  4. Hyphen Handling: Quote identifiers with hyphens for valid TypeScript syntax

Future Enhancements

Potential improvements:

  • Watch Mode: Regenerate types when MCP servers change
  • VSCode Integration: Provide types to VSCode extension for local development
  • Type Validation: Pre-validate user code before runtime execution
  • Incremental Updates: Update types when servers connect/disconnect dynamically