error: unknown debugging option: `dual-proc-macros` when running cross tests

[mati865]

I'm working on tests for dynamic musl (#58575) and since #58013 I'm hitting:

Building stage0 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-musl)
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc - --crate-name ___ --print=file-names -C target-feature=-crt-static --target x86_64-unknown-linux-musl --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro` (exit code: 1)
--- stderr
error: unknown debugging option: `dual-proc-macros`

[RUSTC-TIMING] ___ test:false 0.011

command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-musl" "-Zdual-proc-macros" "-j" "16" "--release" "--locked" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101
failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --host x86_64-unknown-linux-musl --target x86_64-unknown-linux-musl

Error comes from:

rust/src/bootstrap/bin/rustc.rs

Line 115 in f2ef283

cmd.arg("-Zdual-proc-macros");

The configuration is very unusual because the builder is x86_64-unknown-linux-gnu and both the host and the target are set to x86_64-unknown-linux-musl.

Full log from rebuild: https://gist.github.com/mati865/147c64df0f093117f730c78b66299d86

cc @Zoxc

[Zoxc]

Are you overriding the rustc version used to compile?

[Zoxc]

Oh wait, that's trying to cross compile using the stage0 compiler, when it should be using the stage1 compiler.

[ForceBru]

The same happens when I try to build for armv7-apple-ios:

I tried to run the following command:

../x.py build -i  --stage 1 -j 4 --target armv7-apple-ios --host armv7-apple-ios src/libstd

Got the following output:

Updating only changed submodules
Submodules updated in 0.11 seconds
    Finished dev [unoptimized] target(s) in 2.28s
Building stage0 std artifacts (x86_64-apple-darwin -> armv7-apple-ios)
   Compiling core v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libcore)
   Compiling libc v0.2.46
   Compiling compiler_builtins v0.1.5
   Compiling unwind v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libunwind)
   Compiling backtrace-sys v0.1.27
   Compiling std v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libstd)
   Compiling rustc-std-workspace-core v1.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/tools/rustc-std-workspace-core)
   Compiling alloc v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/liballoc)
   Compiling panic_abort v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libpanic_abort)
   Compiling rustc-demangle v0.1.10
   Compiling panic_unwind v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libpanic_unwind)
warning: dropping unsupported crate type `dylib` for target `armv7-apple-ios`2: std                                                                                                                                                       

    Finished release [optimized] target(s) in 1m 00s
Copying stage0 std from stage0 (x86_64-apple-darwin -> x86_64-apple-darwin / armv7-apple-ios)
Building stage0 test artifacts (x86_64-apple-darwin -> armv7-apple-ios)
   Compiling proc_macro v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libproc_macro)
   Compiling getopts v0.2.17
   Compiling term v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libterm)
warning: dropping unsupported crate type `dylib` for target `armv7-apple-ios`4: proc_macro, getopts, term                                                                                                                                 

   Compiling test v0.0.0 (/Users/forcebru/Desktop/Test/RUST/rust/src/libtest)
warning: dropping unsupported crate type `dylib` for target `armv7-apple-ios`4: test                                                                                                                                                      

    Finished release [optimized] target(s) in 17.29s
Copying stage0 test from stage0 (x86_64-apple-darwin -> x86_64-apple-darwin / armv7-apple-ios)
Building stage0 std artifacts (x86_64-apple-darwin -> x86_64-apple-darwin)
    Finished release [optimized] target(s) in 1.28s
Copying stage0 std from stage0 (x86_64-apple-darwin -> x86_64-apple-darwin / x86_64-apple-darwin)
Building stage0 test artifacts (x86_64-apple-darwin -> x86_64-apple-darwin)
    Finished release [optimized] target(s) in 0.50s
Copying stage0 test from stage0 (x86_64-apple-darwin -> x86_64-apple-darwin / x86_64-apple-darwin)
Building stage0 compiler artifacts (x86_64-apple-darwin -> armv7-apple-ios)
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/Users/forcebru/Desktop/Test/RUST/rust/build_macos/build/bootstrap/debug/rustc - --crate-name ___ --print=file-names --target armv7-apple-ios --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro` (exit code: 1)
--- stderr
error: unknown debugging option: `dual-proc-macros`


command did not execute successfully: "/Users/forcebru/Desktop/Test/RUST/rust/build_macos/build/x86_64-apple-darwin/stage0/bin/cargo" "build" "--target" "armv7-apple-ios" "-Zdual-proc-macros" "-j" "4" "--release" "--features" "" "--manifest-path" "/Users/forcebru/Desktop/Test/RUST/rust/src/rustc/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101
failed to run: /Users/forcebru/Desktop/Test/RUST/rust/build_macos/build/bootstrap/debug/bootstrap build -i --stage 1 -j 4 --target armv7-apple-ios --host armv7-apple-ios src/libstd
Build completed unsuccessfully in 0:01:25

Building only the libraries with ../x.py --target=armv7-apple-ios check works fine.

[mati865]

@Zoxc I came up with simple workaround:

diff --git a/src/bootstrap/bin/rustc.rs b/src/bootstrap/bin/rustc.rs
index ca86aeb810..b884d815bf 100644
--- a/src/bootstrap/bin/rustc.rs
+++ b/src/bootstrap/bin/rustc.rs
@@ -111,7 +111,7 @@ fn main() {
 
         // Link crates to the proc macro crate for the target, but use a host proc macro crate
         // to actually run the macros
-        if env::var_os("RUST_DUAL_PROC_MACROS").is_some() {
+        if env::var_os("RUST_DUAL_PROC_MACROS").is_some() && stage != "0" {
             cmd.arg("-Zdual-proc-macros");
         }

I've tried to make it use stage1 but I couldn't make bootstrap successful. In case you want to see where stage0 is coming from here is part of the log with some debug info:

Assembling stage2 compiler (x86_64-unknown-linux-musl)
[src/bootstrap/compile.rs:352] Std{compiler, target,} = Std {
    target: "x86_64-unknown-linux-musl",
    compiler: Compiler {
        stage: 2,
        host: "x86_64-unknown-linux-musl"
    }
}
Uplifting stage1 std (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-musl)
Copying stage2 std from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-musl / x86_64-unknown-linux-musl)
[src/bootstrap/compile.rs:965] Rustc{compiler: build_compiler, target: target_compiler.host,} = Rustc {
    target: "x86_64-unknown-linux-musl",
    compiler: Compiler {
        stage: 0,
        host: "x86_64-unknown-linux-gnu"
    }
}
[src/bootstrap/compile.rs:477] Test{compiler, target,} = Test {
    target: "x86_64-unknown-linux-musl",
    compiler: Compiler {
        stage: 0,
        host: "x86_64-unknown-linux-gnu"
    }
}
[src/bootstrap/compile.rs:352] Std{compiler, target,} = Std {
    target: "x86_64-unknown-linux-musl",
    compiler: Compiler {
        stage: 0,
        host: "x86_64-unknown-linux-gnu"
    }
}
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-musl)

Full log: https://gist.github.com/mati865/579d833afab7e0c77adbdb584f5ce869
Line numbers match with: https://github.com/rust-lang/rust/blob/0c8700b9d50a1e3d31f7b6c0956df555279ac441/src/bootstrap/compile.rs

Assemble ensures itself recursively until it hits stage0 in this line:

rust/src/bootstrap/compile.rs

Lines 957 to 958 in 0c8700b

	let build_compiler =
	builder.compiler(target_compiler.stage - 1, builder.config.build);

Then it ensures there is Rustc for stage0:

rust/src/bootstrap/compile.rs

Lines 965 to 968 in 0c8700b

	builder.ensure(Rustc {
	compiler: build_compiler,
	target: target_compiler.host,
	});

And then it builds other stage0 stuff until it hit's the error.

[mati865]

During tests I'm getting:

/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps/rustc_macros-5dcf607750efd869: error while loading shared libraries: libstd-d301c78d889cf39d.so: cannot open shared object file: No such file or directory

Maybe this workaround isn't enough.

[Zoxc]

@mati865 I'd like to know what causes Assemble to be used. Can you run it with verbose output? (x.py -vv)

[mati865]

@Zoxc every time compiler method is called it uses Assembly:

rust/src/bootstrap/builder.rs

Lines 570 to 574 in 0f88167

	pub fn compiler(&self, stage: u32, host: Interned<String>) -> Compiler {
	self.ensure(compile::Assemble {
	target_compiler: Compiler { stage, host },
	})
	}

I'm in the middle of work on compiletest so I cannot run it right now. Will add new comment later.

[Zoxc]

@mati865 The question is, what is trying to obtain a stage 1 musl compiler?

[mati865]

https://gist.github.com/mati865/39b6162632dbcde4564024726bd155b7
Interesting part:

Copying stage2 std from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-musl / x86_64-unknown-linux-musl)
          c Libdir { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-musl" }, target: "x86_64-unknown-linux-musl" }
          > Libdir { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-musl" }, target: "x86_64-unknown-linux-gnu" }
            c Sysroot { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-musl" } }
          < Libdir { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-musl" }, target: "x86_64-unknown-linux-gnu" }
          > Assemble { target_compiler: Compiler { stage: 1, host: "x86_64-unknown-linux-musl" } }
            c Assemble { target_compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
            > Rustc { target: "x86_64-unknown-linux-musl", compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
              > Test { target: "x86_64-unknown-linux-musl", compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
                > Std { target: "x86_64-unknown-linux-musl", compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
                  > StartupObjects { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-musl" }
                  < StartupObjects { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-musl" }
                  > Libdir { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-musl" }
                    c Sysroot { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
                  < Libdir { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-musl" }
                  c Libdir { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu" }
                  c Sysroot { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-musl)

[mati865]

@Zoxc it goes like that:

rust/src/bootstrap/compile.rs

Line 237 in 0c8700b

builder.cargo(target_compiler, Mode::ToolStd, target, "clean");

⬇️

rust/src/bootstrap/builder.rs

Line 730 in 0c8700b

self.compiler(1, compiler.host)

⬇️

rust/src/bootstrap/compile.rs

Line 931 in 0c8700b

fn run(self, builder: &Builder<'_>) -> Compiler {

[Zoxc]

The builder.cargo(target_compiler part doesn't make sense, as we can't actually run the target_compiler, so trying to run cargo with it won't work.

@alexcrichton Do you know what is going on here?

[mati865]

First introduced by cec6816 as part of #43630

[alexcrichton]

Can you elaborate a bit more on what's going on here? There's good deal of context here and I'm not sure how much is relevant

[mati865]

I'll try to summarize it:
I'm patching up musl host tests and encountered this error after recent bootstrap changes in #58013. With this commit: mati865@4a53632 running ./src/ci/docker/run.sh dist-x86_64-musl will fail with error: unknown debugging option: dual-proc-macros because bootstrap builds stage0 musl compiler. It's because of this line:

rust/src/bootstrap/compile.rs

Line 237 in 0c8700b

builder.cargo(target_compiler, Mode::ToolStd, target, "clean");

builder::cargo calls builder::compiler(1, "musl") which runs Assembly:

rust/src/bootstrap/builder.rs

Lines 570 to 574 in 0c8700b

	pub fn compiler(&self, stage: u32, host: Interned<String>) -> Compiler {
	self.ensure(compile::Assemble {
	target_compiler: Compiler { stage, host },
	})
	}

Then compile::Assemble runs recursively down to stage0:

rust/src/bootstrap/compile.rs

Line 958 in 0c8700b

builder.compiler(target_compiler.stage - 1, builder.config.build);

I'm trying to avoid it with mati865@efd45e9 but it's still wip and fails in the tests.

EDIT: The error from my hack is:

Testing rustc_macros stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-musl)
running: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-musl" "-Zdual-proc-macros" "-j" "16" "-v" "--release" "--locked" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "-p" "rustc_macros" "--"
       Fresh unicode-xid v0.1.0
       Fresh proc-macro2 v0.4.24
       Fresh quote v0.6.10
       Fresh syn v0.15.22
       Fresh synstructure v0.10.1
       Fresh rustc_macros v0.1.0 (/checkout/src/librustc_macros)
    Finished release [optimized] target(s) in 0.23s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps/rustc_macros-5dcf607750efd869`
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps/rustc_macros-5dcf607750efd869: error while loading shared libraries: libstd-d301c78d889cf39d.so: cannot open shared object file: No such file or directory
error: test failed, to rerun pass '--lib'

EDIT2:

$ find obj/build -name "libstd-d301c78d889cf39d.so" 
obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d301c78d889cf39d.so
obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libstd-d301c78d889cf39d.so
obj/build/x86_64-unknown-linux-gnu/stage2/lib/libstd-d301c78d889cf39d.so