cbor2 5.8.0 breaks jadepy serial compatibility by delaying small CBOR replies until serial timeout

[1-21gigasats]

Disclaimer

Issue was found and fixed by LLM, but I manually verified both possible solutions.

Summary

jadepy serial RPCs can block until the configured serial timeout expires, even when the full Jade reply has already arrived.

On Arch Linux, this regression is reproducible with the system package python-cbor2 5.8.0-2 and disappears again after downgrading system-wide to python-cbor2 5.7.1.

The underlying issue appears to be that jadepy exposes the serial transport as a file-like object for cbor2.load(self), while JadeSerialImpl.read(n) forwards large read requests directly to pyserial.Serial.read(n). When cbor2 requests a large read, pyserial waits for either the full requested size or timeout, which delays delivery of much smaller Jade replies until timeout expiry.

Impact

This currently makes Jade effectively unusable without a system-wide cbor2 downgrade on at least some rolling-release distributions, with Arch Linux confirmed.

In practice this breaks downstream integrations such as Electrum's Jade support unless the user either:

• applies a local hotfix to the vendored jadepy copy
• downgrades the system python-cbor2 package to 5.7.1

Symptoms

In the affected environment, RPC latency tracks the configured serial timeout exactly:

• timeout=1 -> reply arrives after about 1s
• timeout=3 -> reply arrives after about 3s
• timeout=10 -> reply arrives after about 10s

This was reproducible with:

• ping()
• get_version_info()
• Electrum's Jade initialization path, especially the follow-up add_entropy() call after reconnect

Example test script:

from time import perf_counter
from jadepy import JadeAPI

with JadeAPI.create_serial("/dev/ttyACM0", timeout=10) as jade:
    t0 = perf_counter()
    print(jade.get_version_info())
    print(perf_counter() - t0)

The device responds correctly, but only after the timeout interval.

Environment observed

• Arch Linux system packages
• Python 3.14.3
• python-pyserial 3.5-8
• python-cbor2 5.8.0-2
• electrum 4.7.1-1
• Jade detected at /dev/ttyACM0

Also relevant:

• downgrading from Arch python-cbor2 5.8.0-2 to 5.7.1 avoids the issue without any Jade-side hotfix
• the current Electrum AppImage did not show the problem
• this repository already pins cbor2==5.7.1 and mentions an existing cbor2 issue in requirements.txt

This suggests the bug is exposed by newer cbor2 behavior, but the actual compatibility problem is in jadepy serial adapter semantics.

Root cause analysis

The problematic chain is:

1. jadepy decodes replies with cbor.load(self)
2. the file-like object delegates reads to JadeInterface.read(n)
3. the serial backend delegates that unchanged to JadeSerialImpl.read(n)
4. JadeSerialImpl.read(n) currently does:

def read(self, n):
    assert self.ser is not None
    return self.ser.read(n)

5. in affected environments, cbor2.load(self) asks for large chunks, observed in logs as read(4096)
6. pyserial.Serial.read(4096) waits for either 4096 bytes or timeout
7. Jade replies are much smaller than 4096 bytes
8. result: replies are delivered only when timeout expires

This does not appear to be primarily an Electrum bug. Electrum only makes the issue very visible because it:

• first connects with timeout=1 for probing
• then reconnects with the default timeout
• then calls add_entropy()

So the second phase can appear to hang for the full default timeout.

Evidence from downgrade and downstream hotfix

Two independent mitigations were tested and both resolved the issue.

1. System-wide cbor2 downgrade

Downgrading the Arch package from:

• python-cbor2 5.8.0-2

to:

• python-cbor2 5.7.1-2

avoided the problem without any local Jade patch.

2. Local jadepy serial hotfix

A local hotfix that changed the vendored jade_serial.read() implementation to avoid blindly reading the full requested size also fixed the issue immediately.

Hotfix logic:

def read(self, n):
    assert self.ser is not None

    waiting = getattr(self.ser, "in_waiting", 0) or 0
    if waiting > 0:
        return self.ser.read(min(n, waiting))
    return self.ser.read(1)

After this change:

• Electrum detected Jade normally
• RPCs returned promptly
• the scanning problem disappeared

Taken together, this strongly suggests that cbor2 5.8.0 exposes a transport-layer compatibility bug in jadepy rather than introducing a Jade-specific functional regression by itself.

Proposed fix

Fix jadepy serial transport so that its read(n) implementation does not block waiting for the full requested size when used as a file-like source for cbor2.

A transport-level fix in jadepy/jade_serial.py seems like the right place.

Possible approach:

• if bytes are already waiting, read up to min(n, in_waiting)
• otherwise read a small amount, for example 1, to preserve blocking semantics without forcing exact-size waits on large n

This would make jadepy robust against decoder implementations that request large read sizes.

Suggested regression test

Add a regression test covering the interaction between:

• cbor.load(self)
• JadeInterface.read(n)
• serial transport semantics

The test should simulate a small CBOR reply and a decoder path that requests a large read size, and verify that the reply does not wait for the serial timeout.

Downstream follow-up

Electrum vendors a copy of jadepy under:

electrum/plugins/jade/jadepy/

After this is fixed upstream in Jade, the vendored copy in Electrum should also be updated so downstream users get the fix.

[jgriffiths]

Hi, Please see commit c5147a0 which references the upstream cbor2 issue agronholm/cbor2#272

The upstream package was completely broken for many users, not just Jade, and should not be part of any distro release. When a new release is available we can bump the dependency in requirements.txt, until then the distro should downgrade.

I do not want to start adding workarounds for third party issues unless absolutely required.

[1-21gigasats]

Thanks jgriffiths, I actually didn't notice the pinning in requirements.txt, this explains it very well.

So the fix is already applied in cbor2 master and we just need to wait for a release and arch linux users need to downgrade in the meanwhile. Agreed, in this case a fix/workaround doesn't make any sense here.

I would suggest leaving this issue open as long the cbor upstream release isn't available so rolling release distro users could take notice.

Btw, the optional dependencies for jade were added to electrum:

https://gitlab.archlinux.org/archlinux/packaging/packages/electrum/-/commit/fbb1009471f92dee02b79b7480b63dbb736eed36

[jgriffiths]

I would suggest leaving this issue open as long the cbor upstream release isn't available so rolling release distro users could take notice.

Ack. I think the upstream release won't happen imminently unfortunately since theres a rewrite of the internal impl underway.

[1-21gigasats]

cbor2 5.9.0 was released containing the fix https://github.com/agronholm/cbor2/releases/tag/5.9.0 - this one is working as expected with Electrum.

I'll leave this ticket open to reconsider pinning this version in requirements.txt.

[jgriffiths]

@1-21gigasats thanks for the update, I'll test this with 1.0.40 (in beta now) and update for the final release if everything looks good.