test/fixtures_rsa/RSA_KEY_2048: make key SP800-56b compatible

[huwcbjones]

These keys cannot be loaded with FIPS ensuring SP800-56b compliance, however the keys can be fixed as follows:

pkey = load_func(pem, password, unsafe_skip_rsa_key_validation=True)
assert isinstance(pkey, rsa.RSAPrivateKey)

minval = 2 ** (pkey.key_size // 2)
numbers = pkey.private_numbers()

p_1 = numbers.p - 1
q_1 = numbers.q - 1
lambda_n = (p_1 * q_1) // math.gcd(p_1, q_1)

if numbers.d >= lambda_n:
    d_modln = numbers.d % lambda_n
    numbers = rsa.RSAPrivateNumbers(
        numbers.p,
        numbers.q,
        d_modln,
        d_modln % p_1,
        d_modln % q_1,
        numbers.iqmp,
        numbers.public_numbers,
    )
assert minval < numbers.d < lambda_n
pkey = numbers.private_key()
# write out new pkey in the required format, or extract the numbers

And to prove that the keys fail to load.

cryptography$ FIPS_FORCE_ENABLE=1 python3
Python 3.13.5 (main, Mar  9 2026, 23:19:52) [GCC 14.2.0] on linux
>>> from tests.hazmat.primitives.fixtures_rsa import RSA_KEY_2048
>>> from cryptography.hazmat.primitives import serialization
>>> print(RSA_KEY_2048.private_key().private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8, serialization.NoEncryption()).decode())
 ValueError: Invalid private key
>>>
cryptography$ python3
Python 3.13.5 (main, Mar  9 2026, 23:19:52) [GCC 14.2.0] on linux
>>> from tests.hazmat.primitives.fixtures_rsa import RSA_KEY_2048
>>> from cryptography.hazmat.primitives import serialization
>>> f = open("bad-key.pem", "wb")
>>> f.write(RSA_KEY_2048.private_key().private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8, serialization.NoEncryption()))
1704
cryptography$ openssl pkey -in bad-key.pem -check -noout
Key is valid
cryptography$ FIPS_FORCE_ENABLE=1 openssl pkey -in bad-key.pem -check -noout
Key is invalid
20F054B7FFFF0000:error:020000AB:rsa routines:ossl_rsa_sp800_56b_check_keypair:invalid keypair:crypto/rsa/rsa_sp800_56b_check.c:430:

See also:

• pexip/os-python-cryptography@trixie...huw/fips
• pexip/os-python-cryptography-vectors@78b1b1e

[reaperhulk]

I think we need to understand why this isn't failing on our existing FIPS builders.

[huwcbjones]

I think we need to understand why this isn't failing on our existing FIPS builders.

Our environment will always be special, but especially so as we've just migrated to OpenSSL 3.5.5, but our FIPS module is fixed at 3.0.10 because it's on the CMVP list. We're hoping to get a new module based on 3.5.x at some point, but the when is very indeterminate.

As such, the behaviour under the FIPS_FORCE_ENABLE is from our patch (we use /etc/fips/enabled as a flag to force FIPS validated crypto on) here: https://github.com/pexip/os-openssl/blob/trixie/debian/patches/fips-build.patch.

So I would assume we're doing something different (more strict?) to what's happening in CI.

Either way, those keys aren't SP800-56b compliant!