python tensorrt_lean import fails with "No module named 'tensorrt_bindings'"

[rcode6]

Description

When attempting to import tensorrt_lean, I get ModuleNotFoundError: No module named 'tensorrt_bindings'. In order to get the import to succeed, I have to also run pip install tensorrt-cu12-bindings tensorrt-cu12-libs, which seems to basically be the full tensorrt package again.

after installing with pip install tensorrt-lean-cu12:

import tensorrt_lean
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import tensorrt_lean
  File "/usr/local/lib/python3.13/site-packages/tensorrt_lean/__init__.py", line 18, in <module>
    from tensorrt_bindings import *
ModuleNotFoundError: No module named 'tensorrt_bindings'

if i run pip install tensorrt-cu12-bindings and try again:

import tensorrt_lean
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import tensorrt_lean
  File "/usr/local/lib/python3.13/site-packages/tensorrt_lean/__init__.py", line 18, in <module>
    from tensorrt_bindings import *
  File "/usr/local/lib/python3.13/site-packages/tensorrt_bindings/__init__.py", line 96, in <module>
    from .tensorrt import *
ImportError: libnvinfer.so.10: cannot open shared object file: No such file or directory

if i run pip install tensorrt-cu12-libs, it succeeds, but the image size is the same as using the full tensorrt package.

Environment

TensorRT Version: 10.12.0.36

NVIDIA GPU: RTX 4000 SFF Ada

NVIDIA Driver Version: 550.127.05

CUDA Version: 12.4

CUDNN Version:

Operating System: ubuntu22.04

Python Version (if applicable): 3.13

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version): Container, nvidia/cuda:12.4.1-runtime-ubuntu22.04

Relevant Files

Model link:

Steps To Reproduce

1. make sure system doesn't have tensorrt package installed
2. pip install tensorrt-lean-cu12
3. import tensorrt_lean

Commands or scripts:

# pip install tensorrt-lean-cu12
Collecting tensorrt-lean-cu12
  Downloading tensorrt_lean_cu12-10.12.0.36.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tensorrt_lean_cu12_libs==10.12.0.36 (from tensorrt-lean-cu12)
  Downloading tensorrt_lean_cu12_libs-10.12.0.36.tar.gz (716 bytes)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tensorrt_lean_cu12_bindings==10.12.0.36 (from tensorrt-lean-cu12)
  Downloading tensorrt_lean_cu12_bindings-10.12.0.36-cp313-none-manylinux_2_28_x86_64.whl.metadata (612 bytes)
Requirement already satisfied: nvidia-cuda-runtime-cu12 in /usr/local/lib/python3.13/site-packages (from tensorrt_lean_cu12_libs==10.12.0.36->tensorrt-lean-cu12) (12.9.79)
Downloading tensorrt_lean_cu12_bindings-10.12.0.36-cp313-none-manylinux_2_28_x86_64.whl (725 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 725.3/725.3 kB 16.4 MB/s eta 0:00:00
Building wheels for collected packages: tensorrt-lean-cu12, tensorrt_lean_cu12_libs
  Building wheel for tensorrt-lean-cu12 (pyproject.toml) ... done
  Created wheel for tensorrt-lean-cu12: filename=tensorrt_lean_cu12-10.12.0.36-py2.py3-none-any.whl size=17665 sha256=d2d2a7ffb76d956a58a6455cc655582d7c2b7cdeceac97e9bc2b1d44d41ca802
  Stored in directory: /root/.cache/pip/wheels/c4/d9/08/87dfe3d6d5087b5576700f79505252ff15534bf00687992d7c
  Building wheel for tensorrt_lean_cu12_libs (pyproject.toml) ... done
  Created wheel for tensorrt_lean_cu12_libs: filename=tensorrt_lean_cu12_libs-10.12.0.36-py2.py3-none-manylinux_2_28_x86_64.whl size=51756596 sha256=894fdb8bbcf355ca3b6502e10f25891a773a091666ec2b3e5ce351442f687142
  Stored in directory: /root/.cache/pip/wheels/d0/b9/cc/91b9e8657b054304ed9f60f44a6eb3f219d8b479b0806ba574
Successfully built tensorrt-lean-cu12 tensorrt_lean_cu12_libs
Installing collected packages: tensorrt_lean_cu12_bindings, tensorrt_lean_cu12_libs, tensorrt-lean-cu12
Successfully installed tensorrt-lean-cu12-10.12.0.36 tensorrt_lean_cu12_bindings-10.12.0.36 tensorrt_lean_cu12_libs-10.12.0.36

import tensorrt_lean
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import tensorrt_lean
  File "/usr/local/lib/python3.13/site-packages/tensorrt_lean/__init__.py", line 18, in <module>
    from tensorrt_bindings import *
ModuleNotFoundError: No module named 'tensorrt_bindings'

Have you tried the latest release?:

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

[asfiyab-nvidia]

@rcode6 thanks for reporting this. I have filed an internal bug to address this issue

[spkgyk]

I have the same problem. The documentation on how to actually use the lean runtime is incredibly sparse. Can we have a full demo from export to the runtime deps (including installation!) Thanks

[ameliefroessl]

hi @asfiyab-nvidia , are there any updates on the internal bug that you filed for this?

[asfiyab-nvidia]

@ameliefroessl . I don't see this issue with TensorRT 10.16.0.72. Can you verify on your end and update if you're seeing an error?