nix build on recent master fails

[vincenzopalazzo]

configure: error: cannot import Python module "distutils".
Please check your Python installation. The error was:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'distutils'
make: *** [external/Makefile:88: external/build-x86_64-unknown-linux-gnu/libwally-core-build/src/secp256k1/libsecp256k1.la] Error 1
make: *** Waiting for unfinished jobs....
yes
checking whether C compiler accepts -mavx512f... yes
checking for AVX512F instructions set... yes
checking whether C compiler accepts -mavx512f... (cached) yes

[antagonistica]

I also ran into an error when building in a nix shell under NixOS 24.05:

python3 -m grpc_tools.protoc -I cln-grpc/proto cln-grpc/proto/node.proto --python_out=contrib/pyln-grpc-proto/pyln/grpc/ --grpc_python_out=contrib/pyln-grpc-proto/pyln/grpc/ --experimental_allow_proto3_optional
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/user/.cache/pypoetry/virtualenvs/cln-meta-project-eKO3ra09-py3.11/lib/python3.11/site-packages/grpc_tools/protoc.py", line 20, in <module>
    from grpc_tools import _protoc_compiler
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory
make: *** [Makefile:394: contrib/pyln-grpc-proto/pyln/grpc/primitives_pb2.py] Error 1
make: *** Waiting for unfinished jobs....

I tried around with installing a lot of different nix and python packages, but wasn't able to get rid of this error. The closest I've come was to add the missing python package mako and remove duplicates in the nix-shell command:

diff --git a/doc/getting-started/getting-started/installation.md b/doc/getting-started/getting-started/installation.md
index fba2a3e8b..6a187165d 100644
--- a/doc/getting-started/getting-started/installation.md
+++ b/doc/getting-started/getting-started/installation.md
@@ -314,8 +314,8 @@ Finally, build `c-lightning`:
 Use nix-shell launch a shell with a full Core Lightning dev environment:
 
 ```shell
-nix-shell -Q -p gdb sqlite autoconf git clang libtool sqlite autoconf \
-autogen automake gmp zlib gettext libsodium poetry 'python3.withPackages (p: [p.bitcoinlib])' \
+nix-shell -Q -p gdb autoconf git clang libtool sqlite \
+autogen automake gmp zlib gettext libsodium poetry 'python3.withPackages (p: [p.bitcoinlib p.mako])' \
 valgrind --run "./configure && poetry shell"
 poetry install
 make

My guess is that I need to manually add a library to python in the shell along the lines of this post, but that's where my nix abilities end.

[vincenzopalazzo]

Still need to look into it, but thanks for this diff will be really helpful

[IngwiePhoenix]

Running into this error on Ubuntu 24.04 LTS:

root@coinboi /o/lightning ((v25.09.3)) [2]# make -j(nproc)
CC: cc -DCLN_NEXT_VERSION="v25.09" -DPKGLIBEXECDIR="/usr/local/libexec/c-lightning" -DBINDIR="/usr/local/bin" -DPLUGINDIR="/usr/local/libexec/c-lightning/plugins" -DCCAN_TAL_NEVER_RETURN_NULL=1 -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -Wno-maybe-uninitialized -Wshadow=local -std=gnu11 -g -fstack-protector-strong -Og -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/build-aarch64-linux-gnu/libbacktrace-build -I . -I/usr/local/include       -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DCOMPAT_V0121=1  -c -o
LD: cc   -Og  config.vars  -Lexternal/build-aarch64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -L/usr/local/include -lm -lsqlite3   -lsodium  -lz -o
python3 -m grpc_tools.protoc -I cln-grpc/proto cln-grpc/proto/node.proto --python_out=contrib/pyln-grpc-proto/pyln/grpc/ --grpc_python_out=contrib/pyln-grpc-proto/pyln/grpc/ --experimental_allow_proto3_optional
/usr/lib/python3/dist-packages/grpc_tools/protoc.py:17: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
Missing value for flag: --experimental_allow_proto3_optional
make: *** [Makefile:412: contrib/pyln-grpc-proto/pyln/grpc/primitives_pb2.py] Error 1
make: *** Waiting for unfinished jobs....

Anything I can patch to fix the build?

[ekzyis]

Is this still an issue? I was able to run nix build after nix develop on 13608c5 + #8950.

I tried to reproduce #7515 (comment) with the instructions for nix-shell in getting-started/installation.md but it already failed at autogen because the command wasn't found. I think a common issue with nix-shell is that which dependencies are pulled depends on which channel + commit you're on. Flakes fix this by pinning the dependencies in a lock file.

Additionally, since these instructions were last updated 2 years ago, and a9ffa37 mentioned moving to a flake.nix, I think this nix-shell approach is outdated and flakes via nix develop as mentioned here should be used.

---

update: oh, I missed the \ to combine the lines

But if I naively just paste the complete nix-shell command including the \, I get this error:

error: attribute 'bitcoinlib' missing

$ nix-shell -Q -p gdb sqlite autoconf git clang libtool sqlite autoconf \
autogen automake gmp zlib gettext libsodium poetry 'python3.withPackages (p: [p.bitcoinlib])' \
valgrind --run "./configure && poetry shell"
error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:37:12:
           36|
           37|   strict = derivationStrict drvAttrs;
             |            ^
           38|

       … while evaluating derivation 'shell'
         whose name attribute is located at /nix/store/z9g260zjw29c18xgn57q4yark9is9qfg-nixos-25.11/nixos/pkgs/stdenv/generic/make-derivation.nix:541:13

       … while evaluating attribute 'buildInputs' of derivation 'shell'
         at /nix/store/z9g260zjw29c18xgn57q4yark9is9qfg-nixos-25.11/nixos/pkgs/stdenv/generic/make-derivation.nix:593:13:
          592|             depsHostHost = elemAt (elemAt dependencies 1) 0;
          593|             buildInputs = elemAt (elemAt dependencies 1) 1;
             |             ^
          594|             depsTargetTarget = elemAt (elemAt dependencies 2) 0;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'bitcoinlib' missing
       at «string»:1:268:
            1| {...}@args: with import <nixpkgs> args; (pkgs.runCommandCC or pkgs.runCommand) "shell" { buildInputs = [ (gdb) (sqlite) (autoconf) (git) (clang) (libtool) (sqlite) (autoconf) (autogen) (automake) (gmp) (zlib) (gettext) (libsodium) (poetry) (python3.withPackages (p: [p.bitcoinlib])) (valgrind) ]; } ""
             |

So I still think the flake is now meant to be used ¯\_(ツ)_/¯

---

update: I got the same error as #7515 (comment) on 13608c5 when running nix develop + nix-shell -p openssl uv + uv run pytest tests/:

ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

$ uv run pytest tests/
============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.13.8, pytest-9.0.2, pluggy-1.6.0
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Run ID: 292866066351513600, server: None
rootdir: /home/ekzyis/prog/c-lightning/tests
configfile: pytest.ini
plugins: xdist-3.8.0, timeout-2.4.0, rerunfailures-16.1, custom-exit-code-0.3.0, benchmark-5.2.3, test-groups-1.2.1, trackflaky-0.1.0
collected 981 items / 1 error

=================================================================================================================== ERRORS ====================================================================================================================
_______________________________________________________________________________________________________ ERROR collecting test_cln_rs.py _______________________________________________________________________________________________________
ImportError while importing test module '/home/ekzyis/prog/c-lightning/tests/test_cln_rs.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/nix/store/cfapjd2rvqrpry4grb0kljnp8bvnvfxz-python3-3.13.8/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_cln_rs.py:3: in <module>
    from pyln import grpc as clnpb
contrib/pyln-grpc-proto/pyln/grpc/__init__.py:3: in <module>
    from pyln.grpc.node_pb2_grpc import *  # noqa: F401,F403
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
contrib/pyln-grpc-proto/pyln/grpc/node_pb2_grpc.py:3: in <module>
    import grpc
.venv/lib/python3.13/site-packages/grpc/__init__.py:22: in <module>
    from grpc import _compression
.venv/lib/python3.13/site-packages/grpc/_compression.py:20: in <module>
    from grpc._cython import cygrpc
E   ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory
=========================================================================================================== short test summary info ===========================================================================================================
ERROR tests/test_cln_rs.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================== 1 error in 1.57s ===============================================================================================================

This patch fixed it for me:

diff --git a/nix/shells.nix b/nix/shells.nix
index bb3588c28..14b57c337 100644
--- a/nix/shells.nix
+++ b/nix/shells.nix
@@ -9,7 +9,13 @@
     }:
     {
       devShells = {
-        default = pkgs.mkShell { inputsFrom = [ config.packages.default ]; };
+        default = pkgs.mkShell {
+          inputsFrom = [ config.packages.default ];
+          buildInputs = [ pkgs.stdenv.cc.cc.lib ];
+          shellHook = ''
+            export LD_LIBRARY_PATH="${pkgs.stdenv.cc.cc.lib}/lib''${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH"
+          '';
+        };
         postgres = pkgs.mkShell { inputsFrom = [ config.packages.cln-postgres ]; };
         rust = pkgs.craneLib.devShell {
           checks = {