rusk sdk v2 init

[samuel100]

Add Rust SDK v2

Introduces the Foundry Local Rust SDK (sdk_v2/rust), updated Rust samples (samples/rust), and CI pipeline support.

SDK (sdk_v2/rust)

A safe, async Rust wrapper over the Foundry Local Core native library, providing:

• FoundryLocalManager — singleton entry point that initializes the native engine, exposes the model catalog, and manages the local web service lifecycle.
• Catalog — async model discovery with a 6-hour TTL cache. Lookup by alias or variant ID, with helpful error messages listing available alternatives on miss.
• Model / ModelVariant — full model lifecycle: download (with streaming progress), load, unload, cache inspection, and removal.
• ChatClient — OpenAI-compatible chat completions (non-streaming and streaming via futures_core::Stream). Supports temperature, top-p/top-k, response format (text, JSON, JSON schema, Lark grammar), and tool calling.
• AudioClient — OpenAI-compatible audio transcription (non-streaming and streaming). Accepts impl AsRef<Path> for file paths.
• Error handling — thiserror-based FoundryLocalError enum with discriminated variants (LibraryLoad, CommandExecution, ModelOperation, Validation, etc.) and a crate-wide Result<T> alias.
• FFI bridge (detail/core_interop.rs) — dynamically loads the native .dll/.so/.dylib via libloading, with platform-aware memory deallocation and a trampoline pattern for streaming callbacks. Async methods use tokio::task::spawn_blocking to keep the runtime unblocked.
• Build script (build.rs) — automatically downloads NuGet packages (Core, ORT, GenAI) at compile time, extracts platform-specific native libraries, and caches them across builds. Supports winml and nightly feature flags.

Re-exports async-openai request/response types for convenience so callers don't need a direct dependency.

Samples (samples/rust)

Replaces the old hello-foundry-local sample with four focused examples:

• native-chat-completions — basic sync + streaming chat completion
• tool-calling-foundry-local — multi-turn tool calling with streaming chunk assembly
• audio-transcription-example — audio file transcription (sync and streaming)
• foundry-local-webserver — starts the local web service and makes requests via the OpenAI-compatible HTTP endpoint

Each sample has its own Cargo.toml, README.md, and is included in the workspace.

CI (.github/workflows)

• build-rust-steps.yml — reusable composite action for Rust SDK build + test steps.
• foundry-local-sdk-build.yml — workflow that triggers on sdk_v2/rust/** and samples/rust/** changes.

Testing

• Integration tests mirror the JavaScript SDK test suite structure: manager_test, catalog_test, model_test, model_load_manager_test, chat_client_test, audio_client_test.
• All tests are #[ignore] by default (require native library at runtime) and can be enabled in CI with --include-ignored.
• Shared test utilities (tests/common/mod.rs) provide config, model aliases, and helper functions.
• cargo clippy --all-targets --all-features passes with zero code warnings.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview, Comment	Mar 13, 2026 5:11pm

[prathikr]

awesome first iteration! some things to look over but great start

[Copilot]

Pull request overview

Adds a new Rust SDK v2 under sdk_v2/rust that wraps the Foundry Local Core native library (FFI + async APIs), alongside refreshed Rust samples and GitHub Actions CI support for building/testing the Rust SDK.

Changes:

• Introduces the foundry-local-sdk Rust crate (manager/catalog/model lifecycle + OpenAI-compatible chat/audio clients).
• Adds Rust integration tests and shared test utilities.
• Updates Rust samples workspace and CI workflows for Rust build/test packaging (including WinML option on Windows).

Reviewed changes

Copilot reviewed 45 out of 45 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
sdk_v2/rust/tests/integration.rs	Single integration test binary covering manager/catalog/model/chat/audio/web service flows
sdk_v2/rust/tests/common/mod.rs	Shared test config/utilities (test-data-shared pathing, tool definition helper)
sdk_v2/rust/src/types.rs	Shared SDK types (ModelInfo, response format, tool choice, etc.)
sdk_v2/rust/src/openai/mod.rs	Exposes OpenAI-compatible client modules
sdk_v2/rust/src/openai/chat_client.rs	Chat client + streaming wrapper and settings serialization
sdk_v2/rust/src/openai/audio_client.rs	Audio transcription client + streaming wrapper and settings serialization
sdk_v2/rust/src/model_variant.rs	Model variant operations (download/load/unload/path/cache) + client factories
sdk_v2/rust/src/model.rs	Alias-level model wrapper with variant selection logic
sdk_v2/rust/src/lib.rs	Crate root exports and re-exports of OpenAI request/response types
sdk_v2/rust/src/foundry_local_manager.rs	Singleton manager initialization, catalog access, web service lifecycle
sdk_v2/rust/src/error.rs	FoundryLocalError + crate-wide Result<T> alias
sdk_v2/rust/src/detail/model_load_manager.rs	Load/unload/list logic via HTTP when external service is configured, else via core
sdk_v2/rust/src/detail/mod.rs	Internal module wiring / re-exports
sdk_v2/rust/src/detail/core_interop.rs	FFI bridge: dynamic library loading, command execution, streaming callback trampoline
sdk_v2/rust/src/configuration.rs	User config -> native parameter map conversion + unit tests
sdk_v2/rust/src/catalog.rs	Model discovery + TTL cache + alias/id lookup helpers
sdk_v2/rust/examples/tool_calling.rs	Tool-calling example using streaming tool_calls assembly
sdk_v2/rust/examples/interactive_chat.rs	Interactive streaming chat example
sdk_v2/rust/examples/chat_completion.rs	Basic non-streaming + streaming chat completion example
sdk_v2/rust/build.rs	Build-time NuGet download/extract of native libs + RID selection
sdk_v2/rust/README.md	Rust SDK README (installation, features, platform support, quick start)
sdk_v2/rust/GENERATE-DOCS.md	Rust docs generation instructions (cargo doc)
sdk_v2/rust/Cargo.toml	New Rust crate manifest (deps/features/examples)
sdk_v2/rust/.rustfmt.toml	Rustfmt settings
sdk_v2/rust/.clippy.toml	Clippy config (MSRV)
samples/rust/tool-calling-foundry-local/src/main.rs	Updated tool-calling sample using SDK v2
samples/rust/tool-calling-foundry-local/README.md	Tool-calling sample instructions
samples/rust/tool-calling-foundry-local/Cargo.toml	Tool-calling sample manifest
samples/rust/native-chat-completions/src/main.rs	Native (non-HTTP) chat completions sample
samples/rust/native-chat-completions/README.md	Native chat sample instructions
samples/rust/native-chat-completions/Cargo.toml	Native chat sample manifest
samples/rust/foundry-local-webserver/src/main.rs	Web service start + REST streaming sample
samples/rust/foundry-local-webserver/README.md	Web service sample instructions
samples/rust/foundry-local-webserver/Cargo.toml	Web service sample manifest
samples/rust/audio-transcription-example/src/main.rs	Audio transcription (sync + streaming) sample
samples/rust/audio-transcription-example/README.md	Audio transcription sample instructions
samples/rust/audio-transcription-example/Cargo.toml	Audio transcription sample manifest
samples/rust/README.md	Samples index updated to new set of examples
samples/rust/Cargo.toml	Workspace members updated (replaces old hello sample)
samples/rust/hello-foundry-local/src/main.rs	Removed legacy hello sample
samples/rust/hello-foundry-local/README.md	Removed legacy hello sample README
samples/rust/hello-foundry-local/Cargo.toml	Removed legacy hello sample manifest
.github/workflows/rustfmt.yml	Removed legacy rustfmt workflow targeting old paths
.github/workflows/foundry-local-sdk-build.yml	Adds Rust build jobs (win/winml/ubuntu/macos)
.github/workflows/build-rust-steps.yml	New reusable workflow for Rust fmt/clippy/build/test/package/artifacts

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

cargo test --test '*' is not a valid way to run all integration tests; --test expects a concrete test target name, so this will typically fail with "no test target named '*'" (or run nothing). To execute all integration tests, use cargo test --tests (or list the specific integration test target like --test integration).

Suggested change

	run: cargo test --test '*' ${{ env.CARGO_FEATURES }} -- --test-threads=1 --nocapture
	run: cargo test ${{ env.CARGO_FEATURES }} --tests -- --test-threads=1 --nocapture

[samuel100]

Fixed. Changed cargo test --test '*' to cargo test --tests. Also added --include-ignored to run ignored integration tests in CI.

[Copilot]

This test name mentions an "invalid callback", but the body only passes an empty messages array (same as the preceding test) and doesn't exercise any callback validation. This makes the intent unclear and duplicates coverage. Consider removing this test or rewriting it to actually validate the intended error condition.

[samuel100]

Fixed. Removed the duplicate should_throw_when_completing_streaming_chat_with_invalid_callback test.

[Copilot]

progress is a &str; using {progress:.1}% applies string precision and will truncate the progress string to 1 character. If you want to show the reported progress value, print {progress} directly (or parse to a number and then format with one decimal).

Suggested change

	print!("\r {progress:.1}%");
	print!("\r {progress}%");

[samuel100]

Fixed. Changed {progress:.1}% to {progress}% in all 4 sample apps.

[Copilot]

For audio transcription streaming, this adds stream: true into the OpenAI request payload. The JS and C# SDKs implement streaming via the native callback without adding a stream field to the audio_transcribe request (see sdk_v2/js/src/openai/audioClient.ts and sdk_v2/cs/src/OpenAI/AudioClient.cs), so this is an API-shape inconsistency that may cause request validation differences across SDKs. Consider removing the stream field (or aligning all SDKs / documenting why Rust needs it).

[samuel100]

Fixed. Removed stream: true from the audio transcription streaming request. Streaming is now handled purely via the native callback, matching JS/C# SDKs.

[Copilot]

streaming_trampoline invokes a Rust closure directly from an extern "C" callback. If the user callback panics, the unwind would cross the FFI boundary, which is undefined behavior (and may abort or corrupt state). Please wrap the callback invocation in std::panic::catch_unwind and convert any panic into a captured error (or abort the stream safely) so no panic can unwind past the C ABI boundary.

[samuel100]

Fixed. Wrapped streaming_trampoline body in std::panic::catch_unwind(AssertUnwindSafe(...)) to prevent UB from panics crossing the FFI boundary.

[Copilot]

These "with_temperature" transcription tests set temperature(0.0), which is the same value used in the non-"with_temperature" variants. As written, they don't actually verify that changing temperature is accepted/applied. Consider using a non-default temperature (or asserting the request/behavior difference) so the test meaningfully covers the setting.

[samuel100]

Fixed. Changed temperature from 0.0 to 0.5 in the with_temperature transcription test.

[Copilot]

This streaming "with_temperature" transcription test sets temperature(0.0), matching the other streaming test. That means it doesn't actually validate that temperature changes are supported. Consider using a different temperature value (or asserting on request metadata) to make this test cover the intended behavior.

[samuel100]

Fixed. Changed temperature from 0.0 to 0.5 in the streaming with_temperature transcription test.

[Copilot]

FoundryLocalManager::create is intended to be a singleton initializer, but the current pattern (INSTANCE.get() check followed by initialization and then INSTANCE.set) is not atomic. If two threads call create concurrently, both can run initialize and build a catalog before one loses the set race, which can reintroduce the "already initialized" failure this singleton is meant to prevent. Consider using OnceLock::get_or_try_init (or equivalent) so the initialization closure runs at most once and only one thread executes the native initialize call.

[samuel100]

Fixed. Replaced non-atomic get()+set() pattern with Once + OnceLock to ensure initialization runs exactly once (stable Rust compatible — get_or_try_init is nightly-only).

[Copilot]

In this progress callback, progress is a &str, and the format specifier {progress:.1} applies string precision (truncation), so it will only print the first character of the progress string. If the intent is to display the full progress (or a numeric percentage), print {progress} as-is or parse to a number before formatting.

[samuel100]

Fixed. Changed {progress:.1} to {progress}%.

[Copilot]

Here progress is a &str, and {progress:.1}% will truncate the string to a single character (string precision) rather than formatting a numeric percentage. Consider printing {progress} as-is, or parsing it to a numeric type before applying float formatting.

Suggested change

	print!("\rDownloading model... {progress:.1}%");
	print!("\rDownloading model... {progress}%");

[samuel100]

Fixed. Changed {progress:.1}% to {progress}%.

[samuel100]

Additional fix: WinAppSDK Bootstrap auto-detection

During testing, discovered that the Rust SDK was failing to load WinML/OpenVINO execution providers because the native core's WinAppSDK bootstrapper was never being triggered.

Root cause: The Rust SDK pre-loaded Microsoft.WindowsAppRuntime.Bootstrap.dll but never set Bootstrap = true in the config params passed to the native initialize command. Without this param, the native core skips the WinAppSDK bootstrapper, which means plug-in EPs (like OpenVINO) are unavailable.

Fix: CoreInterop::new now auto-detects the Bootstrap DLL next to the resolved core library and sets Bootstrap = true in config params before initialize is called. This matches the JS SDK behavior (see sdk_v2/js/src/detail/coreInterop.ts:56-63).

[nenad1002]

Ah, this is swallowed by tokio, if you use Claude ask it to return an error out, then change the method execute_command_async to return Result<String, FoundryLocalError>.

This will force you to return Result<(), FoundryLocalError> or similar on above funs.

load() should return Error if there is an error.

[samuel100]

I've restructured this so the error can no longer be swallowed.

Rather than just fixing the consumer side, I moved the error propagation into execute_command_streaming_channel itself. The channel now carries Result<String> items instead of plain String, and after the FFI call completes, any error from the native core response buffer is sent as the final channel item:

   tokio::task::spawn_blocking(move || {
       let tx_chunk = tx.clone();
       let result = this.execute_command_streaming(
           &command,
           params.as_ref(),
           move |chunk: &str| {
               let _ = tx_chunk.send(Ok(chunk.to_owned()));
           },
       );

       // Surface any error through the channel so it
       // cannot be silently swallowed.
       if let Err(e) = result {
           let _ = tx.send(Err(e));
       }
   });

This eliminates the JoinHandle<Result<String>> from the return type entirely — it's now impossible for consumers to accidentally drop the error. ChatCompletionStream and AudioTranscriptionStream are much simpler as a result:
no handle field, no close() method, and poll_next just forwards Ok/Err items from the channel. Net ~100 lines deleted.

[nenad1002]

So as already suggested, let's go with builder pattern, it's idiomatic in Rust, and we already break the contract on this method in C++ anyway.

[nenad1002]

Also btw, where is logger? Did you port logger?

[nenad1002]

Cause in C++, we allow custom logger:
explicit FoundryLocalManager(Configuration configuration, ILogger* logger = nullptr);

[nenad1002]

Afaik, this shouldn't return anything, why do you return listening URLs?
GetUrls() method returns the list of urls unless if the API changed recently, and you pass urls through configuration.

Either way, returning urls can probably be [&str], cause you set them beforehand an FL Manager and is owning. You are here doing unefficient clone to return this as FL manager has to own the urls to move them in the first place.

Write to Claude:
urls can almost certainly be made owned by the manager, can you make GetUrls method that returns [&str] and make StartServer not to return anything except Result that might be needed?

[nenad1002]

This will use build pattern in future, so examples would slightly change to be idiomatic

[nenad1002]

ListModels**
This is the method we don't want to change name. Vec<Arc> is probably correct, I don't think we can do better.

[nenad1002]

Should return ModelVariant, not String, unless again, my API is out of date.

[nenad1002]

@prathikr , can you confirm this?

[nenad1002]

The internal alias in error.rs is fine for our own code, but remove pub use self::error::Result from lib.rs. Users should use Result<T, FoundryLocalError> explicitly. Otherwise we shadow standard Result.

Let's be very explicit in public API. Not Result<()>, but Result<(), FoundryLocalError>

[nenad1002]

Result<(), FoundryLocalError>

[nenad1002]

Result<(), FoundryLocalError>

[nenad1002]

Let's make this select_variant(&self, id: &str)

And write to AI to use interior mutability to achieve this. Selected variant is not really a state we need to expose mutability for. This goes for both Model & ModelVariant.

[nenad1002]

You are leaking internal impl with Debug here. Debug gives ability to print contents of the object.

Consider implementing custom impl fmt::Debug for ModelVariant

And only returning info. Same for Model.

[nenad1002]

Dont return serde_json in public API,

do use string or something like this

__pub segments: Option<Vec<TranscriptionSegment>>

pub words: Option<Vec<TranscriptionWord>>__

[nenad1002]

should be u64 (measuring in bytes) or if you want to keep f64, then have check in place so it cannot go negative

[nenad1002]

Same here, u64

[nenad1002]

u64

[nenad1002]

Again, @prathikr can give feedback here, but I don't think this method is public