Add Backpack support to Stack

[philippedev101]

Addresses #2540.

This PR adds support for GHC's Backpack module system, the feature request that has been open since August 2016. After this change, Stack can build projects that use signatures, mixins, and cross-package mixin linking, which previously only worked under cabal-install.

Background

Backpack lets you write a library against an abstract interface (a signature) and have the consumer decide which concrete implementation to plug in. The compiler recompiles the library for each implementation, so there's no runtime overhead. When the signature and the implementation live in the same package (using sub-libraries), this is "private Backpack" and has always worked in Stack. The hard part is cross-package Backpack, where the signature lives in one package and the implementation in another. This requires the build tool to create extra instantiation build steps, which Stack didn't do before.

What changed

The work breaks down into three layers:

Per-component build plan. The build plan is now keyed by ComponentKey (package name + component) instead of just package name. This was a prerequisite: Backpack instantiation tasks need their own entries in the plan alongside the regular library task for the same package. This also lets Stack detect intra-package sub-library dependencies (the Backpack pattern) and avoid splitting those packages into independent component tasks, which would break the build order that Cabal expects.

Cross-package instantiation. When a consumer package uses mixins to depend on an indefinite package, Stack now scans the consumer's dependencies, resolves which modules fill which signatures, and creates CInst (component instantiation) tasks. Each CInst task runs Setup configure --instantiate-with=Sig=impl-pkg:Module followed by Setup build in its own inst-<hash> build directory. The hash is derived from the signature-to-implementation mapping, so different instantiations of the same package get different build artifacts.

This handles the full range of Backpack patterns: default renaming (name identity), explicit ModuleRenaming, HidingRenaming (propagating unfilled holes), multiple instantiations of the same indefinite package with different implementations, transitive chains where an indefinite package depends on another indefinite package, sub-library signatures and implementations, and indefinite packages from Hackage or snapshots (not just local packages). CInst tasks also get their own config cache entries, precompiled cache support, and haddock generation.

Documentation and changelog. A new topic page at doc/topics/backpack.md introduces what Backpack is, walks through its features (signatures, mixin linking, renaming, multiple instantiations, sub-libraries, reexported modules, Template Haskell restrictions), and then explains how Stack supports it, including what the build output looks like and what the limitations are. The page is linked from mkdocs.yml under Topics, and cross-referenced from the package description tutorial (mentioning the signatures field) and the multi-package projects tutorial (mentioning Backpack as a use case for multi-package setups). A major changes entry has been added to ChangeLog.md under the unreleased section.

Testing

103 unit tests in ConstructPlanSpec cover the instantiation logic: signature resolution, renaming, hiding, deduplication, transitive chains, multiple instantiations, sub-library mixins, ADRFound (installed) packages, config cache round-trips, and various warning/error paths.

8 integration tests build real multi-package Backpack projects end-to-end: private Backpack, sub-library dependencies, cross-package with default renaming, explicit renaming, sub-library mixins, transitive chains, and multiple instantiations.

The full existing test suite (814 tests) continues to pass. The two pre-existing Stack.Config/loadConfig failures are environmental (TLS certificates) and unrelated.

What's not covered

requires hiding that actually hides signatures does not create a partial instantiation. Cabal requires a closing substitution, so the hidden signatures propagate as holes to the consumer. Template Haskell in indefinite packages doesn't work, but that's a GHC limitation that affects cabal-install equally.

[mpilgrem]

@philippedev101, thanks! There is a lot to read but, in advance of that, in implementing component-based builds, did you encounter, and overcome, the performance issue that was blocking @theobat making progress at:

• Component-based builds #6356
• Component-based builds #6356 (comment)

[philippedev101]

@mpilgrem Thanks for the pointer to #6356 and @theobat's work. After going through the discussion there and the branches on their fork in some detail: short answer, this PR takes a different architectural approach that sidesteps the performance problem, at the cost of not solving the broader set of issues that full per-component builds would address.

How the two approaches differ

@theobat's approach was to make Stack build each component of a package as a separate Setup configure / Setup build cycle. So for a package with a library and an executable, Stack would call the Setup process twice instead of once. This is what cabal-install does and it's the "correct" long-term direction, but as @theobat found, it introduces a 30-40% overhead on integration tests because each subprocess invocation has a fixed cost that adds up.

The approach in this PR is narrower. The build plan is keyed per-component using a ComponentKey type (package name + component), but the actual Setup invocations for regular packages are unchanged: one configure, one build, same as before. The only additional subprocess calls happen for Backpack instantiation tasks (CInst), and those only exist when a project actually uses cross-package Backpack. Projects that don't use Backpack see zero additional subprocess invocations and no performance difference.

What this does not solve

Full per-component execution would address several long-standing issues that this approach leaves untouched:

Unnecessary recompilation when switching between stack build and stack test (#2800). Right now, if you build just the library and then run tests, Stack reconfigures the entire package with test dependencies included. This causes the library to be unregistered and recompiled from scratch even though nothing in it changed. Per-component execution would configure and build the test suite separately, leaving the library alone.

Rebuilding components that aren't dependencies of the target (#6569). If you ask for stack build foo:exe:my-exe but a sub-library that the exe doesn't depend on has changed, Stack still rebuilds because it configures and builds the whole package. Per-component execution would only touch the components in the actual dependency chain.

Parallel builds within a package (#4391). A package with five independent executables currently builds them sequentially (Cabal handles the ordering internally). Per-component execution would let Stack schedule them in parallel, the same way it parallelizes across packages.

Component-level cycle resolution (#2583). When package A's library depends on package B's library, and B's test suite depends on A's library, that looks like a cycle at the package level. At the component level it's not: build A:lib, then B:lib, then both test suites. Per-component execution could handle this.

Where this PR sits

There are roughly three points on the spectrum:

1. No Backpack support at all (current Stack).
2. Backpack support that works correctly with no performance penalty for non-Backpack projects, but doesn't tackle the broader per-component issues (this PR).
3. Full per-component builds that solve all of the above plus Backpack, but with the subprocess overhead problem that needs to be addressed first.

This PR lands at point 2. It gets cross-package Backpack working today without regressing anything for existing users.

Transitioning to full per-component builds later

The ComponentKey plan infrastructure that this PR introduces is a step toward point 3. The plan is already split per-component, dependency edges between components within the same package are already tracked (intraPackageDeps), and the action scheduler already generates one action per component key. The remaining work to reach full per-component execution would be:

• Per-component Setup configure: thread the component name from ComponentKey into the Setup configure call so it targets a single component. Cabal has supported this since 2.0 and the types are already in place.
• Per-component config cache: key the config cache by ComponentKey instead of PackageIdentifier, so that configuring the test suite doesn't invalidate the library's cache. This PR already does this for CInst tasks (ConfigCacheTypeInstantiation), the same pattern extends to regular components.
• Per-component dependency flags: scope the --dependency flags passed to Setup configure to just the dependencies of the component being built, rather than the union of all component dependencies.
• Per-component Task creation: currently multiple ComponentKey entries in the plan share the same Task object. For true per-component execution, each component would have its own Task with its own dependency sets.

None of these changes require rethinking the plan-level architecture. They're about threading component information through the execution layer. And they're independent of the subprocess performance question, which could be tackled separately (the in-process Distribution.Simple.defaultMainArgs approach discussed in #6356 looks promising for build-type: Simple packages).