Conduit is a type-safe Swift framework for working with cloud and on-device language models through one API.

English | Español | 日本語 | 中文

---

What it gives you

• Fast local inference: tuned for Apple Silicon with first-class MLX support.
• Type-safe structured output: Swift 6 macros validate your generated shapes at compile time.
• One surface for multiple providers: switching between Claude, GPT-4o, and local models is a small config change.
• Actor-based providers: the concurrency model stays explicit and thread-safe.

---

Performance

Conduit is tuned for Apple Silicon and local model throughput. The chart below shows the kind of token rates you can expect on recent M-series hardware.

M1 M2 Max M3 Max Tokens/sec

Benchmark: Token throughput on M3 Max (Llama 3.1 8B, 4-bit Quantized)

---

Quick Start

The default path is intentionally small:

import Conduit

// 1. Initialize your provider
let app = Conduit(.anthropic(apiKey: "sk-ant-..."))

// 2. Create a session with modern async/await
let session = try app.session(model: .anthropic("claude-opus-4-6"))

// 3. Run and get type-safe results
let response = try await session.run("Explain the benefits of Swift Actors.")
print(response)

Provider Swap in One Line

Moving from cloud to local mostly means swapping the provider initializer:

// From Cloud...
let cloud = Conduit(.openAI(apiKey: "sk-..."))

// ...to Local Apple Silicon (MLX)
let local = Conduit(.mlx())
let localSession = try local.session(model: .mlxLocal("/Users/me/models/Llama-3.2-1B-Instruct-4bit"))

---

Documentation

The docs cover the main pieces without much ceremony:

• Getting Started: installation and your first generation.
• Structured Output: type-safe JSON with @Generable.
• Tool Calling: extending LLMs with native Swift functions.
• Streaming: real-time token streaming with AsyncSequence.
• Architecture: how Conduit is put together.

---

License

Conduit is released under the MIT License. See LICENSE.