feat: support ggml backend select and enhance the build system

[Cyberhan123]

This PR requires merging with previous PR : #1184

[Copilot]

Pull request overview

This PR adds runtime GGML backend device selection (including multi-backend setups and RPC devices) and refactors the CMake build/install packaging to better support these new backend workflows.

Changes:

• Introduces named-backend initialization helpers and new C API utilities to list available devices and register RPC devices.
• Extends sd_ctx_params_t to allow selecting devices per model component (diffusion/CLIP/VAE/ControlNet/etc.) and updates model/component construction accordingly.
• Restructures the CMake build into subdirectories and adds install artifacts (pkg-config + CMake package config), plus updates CLI/server docs and flags.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/z_image.hpp	Switches backend-specific numeric workarounds from compile-time macros to runtime backend detection.
src/qwen_image.hpp	Same as above (runtime Vulkan precision workaround).
src/common_block.hpp	Same as above (runtime Vulkan precision workaround).
src/util.h	Adds sd_backend_is() declaration and includes ggml backend header.
src/util.cpp	Implements sd_backend_is().
src/ggml_extend.hpp	Adds named-backend init helpers; extends LoRA adapter interfaces to be backend-aware; updates cast/sync helpers.
src/lora.hpp	Passes backend through LoRA weight-diff/cast paths so Vulkan-specific casting can be applied correctly.
src/conditioner.hpp	Enables multiple backends for multi-encoder conditioners; adds per-encoder backend accessors.
src/stable-diffusion.cpp	Implements per-component backend initialization, RPC device registration, and backend/device listing APIs; refactors backend usage throughout initialization and LoRA flows.
src/upscaler.cpp	Adds backend device selection for upscaler initialization via named backend.
src/model.cpp	Removes backend-specific includes (moving toward backend-agnostic compilation).
include/stable-diffusion.h	Extends sd_ctx_params_t with per-component device strings; extends upscaler API; adds device listing + RPC registration APIs.
examples/common/common.hpp	Adds CLI/server flags and wiring for per-component backend devices; adjusts option parsing flow control.
examples/cli/main.cpp	Adds --rpc and --list-devices; passes selected upscaler backend device into the upscaler API.
examples/cli/README.md	Documents new CLI flags and backend device selection options.
examples/server/README.md	Documents new server flags and backend device selection options; removes CPU-only component toggles.
examples/CMakeLists.txt	Adjusts examples build flags/includes and subdirectories.
docs/rpc.md	New documentation for building/using RPC server setups with this project.
src/CMakeLists.txt	New src-level library target definition.
cmake/common.cmake	New shared compile-flag helpers (warnings/sanitizers).
cmake/build-info.cmake	New build metadata extraction (git commit/count, compiler, target).
cmake/sd.pc.in	Adds pkg-config template for installation.
cmake/sd-config.cmake.in	Adds CMake package config template for installation.
CMakeLists.txt	Major build refactor: module path, shared/static defaults, ggml option transitions, add src/ subdir build, and install/package generation.

Comments suppressed due to low confidence (1)

CMakeLists.txt:160

• The if(GGML_SYCL) block calls target_compile_options(${SD_LIB} ...) before ${SD_LIB} is created (the target is defined later in src/CMakeLists.txt). This will fail configuration when GGML_SYCL=ON. Move the SYCL-specific target_compile_options() call to after add_subdirectory(src) (or apply the flags via variables/target properties inside src/CMakeLists.txt).

# Is this needed?
if(GGML_SYCL)
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-narrowing -fsycl")
    # disable fast-math on host, see:
    # https://www.intel.com/content/www/us/en/docs/cpp-compiler/developer-guide-reference/2021-10/fp-model-fp.html
    if (WIN32)
        set(SYCL_COMPILE_OPTIONS /fp:precise)
    else()
        set(SYCL_COMPILE_OPTIONS -fp-model=precise)
    endif()
    message("-- Turn off fast-math for host in SYCL backend")
    target_compile_options(${SD_LIB} PRIVATE ${SYCL_COMPILE_OPTIONS})
endif()

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.