Skip to content

test_pkcs12.rb: Fix test failures with OpenSSL 4.0.#1019

Draft
junaruga wants to merge 5 commits intoruby:masterfrom
junaruga:wip/openssl-4.0-compatibility
Draft

test_pkcs12.rb: Fix test failures with OpenSSL 4.0.#1019
junaruga wants to merge 5 commits intoruby:masterfrom
junaruga:wip/openssl-4.0-compatibility

Conversation

@junaruga
Copy link
Member

This PR is based on the #1005, fixing the following test failures with OpenSSL master case. I don't intend to merge this PR. It's just show the result of the CI. I added the 5th commit on the #1005.

Use AES-256-CBC for OpenSSL to avoid errors on the error stack with OpenSSL 4.0. LibreSSL and AWS-LC don't support AES-256-CBC well, so use PBE-SHA1-3DES for them.

I haven't checked which commit of the openssl/openssl master branch caused the error.

This commit fixes the following test failures.
https://github.com/ruby/openssl/actions/runs/23093927984/job/67083113067?pr=1005

2) Failure: test_create_no_pass(OpenSSL::TestPKCS12)
/home/runner/work/openssl/openssl/test/openssl/utils.rb:140:in `teardown'
<[]> expected but was
<["error:0308010C:digital envelope routines::unsupported"]>

diff:
? ["error:0308010C:digital envelope routines::unsupported"]

However, I see other kind of test failure. I cannot reproduce this failure on my local, even when using the same commit of the OpenSSL openssl/openssl@5f084f8.

https://github.com/junaruga/ruby-openssl/actions/runs/23562223843/job/68604792678#step:11:590

1) Error: test_private_encoding_encrypted(OpenSSL::TestPKeyRSA): OpenSSL::PKey::PKeyError: i2d_PKCS8PrivateKey_bio: encrypt error
/home/runner/work/ruby-openssl/ruby-openssl/test/openssl/test_pkey_rsa.rb:465:in `private_to_der'
/home/runner/work/ruby-openssl/ruby-openssl/test/openssl/test_pkey_rsa.rb:465:in `test_private_encoding_encrypted'
     462: 
     463:   def test_private_encoding_encrypted
     464:     rsa = Fixtures.pkey("rsa2048")
  => 465:     encoded = rsa.private_to_der("aes-128-cbc", "abcdef")
     466:     asn1 = OpenSSL::ASN1.decode(encoded) # PKCS #8 EncryptedPrivateKeyInfo
     467:     assert_kind_of OpenSSL::ASN1::Sequence, asn1
     468:     assert_equal 2, asn1.value.size
Error: OpenSSL::PKey::PKeyError: i2d_PKCS8PrivateKey_bio: encrypt error
/home/runner/work/ruby-openssl/ruby-openssl/test/openssl/test_pkey_rsa.rb:465:in `private_to_der'
/home/runner/work/ruby-openssl/ruby-openssl/test/openssl/test_pkey_rsa.rb:465:in `test_private_encoding_encrypted'

rhenium and others added 5 commits March 16, 2026 22:11
OpenSSL's master branch is changing functions to return const pointers
where the returned objects are not meant to be modified by the caller.

Update ossl_*_new() to take const pointers accordingly. Unfortunately,
*_dup() in older versions of OpenSSL and in LibreSSL/AWS-LC take
non-const pointers, so const casts are required.
ossl_ec_new() was removed in commit 94aeab2 (pkey: simplify
ossl_pkey_new(), 2017-03-16), but it forgot to remove the declaration
while doing so.
ASN1_STRING has been made opaque in OpenSSL's master branch. Use the
new accessor functions instead of accessing fields directly.

Other uses of ASN1_STRING fields were already updated in
<ruby#978>. This patch converts the
remaining ones, which require the new functions added in OpenSSL 4.0
and were not available at that time.
OpenSSL master added support for RFC 7919 groups in TLS 1.2. They are
preferred over SSLContext#tmp_dh= or #tmp_dh_callback= values if the
client advertises them in the supported_groups extension.
Use AES-256-CBC for OpenSSL to avoid errors on the error stack with OpenSSL
4.0. LibreSSL and AWS-LC don't support AES-256-CBC well, so use PBE-SHA1-3DES
for them.

This commit fixes the following test failures.
https://github.com/ruby/openssl/actions/runs/23093927984/job/67083113067?pr=1005

```
2) Failure: test_create_no_pass(OpenSSL::TestPKCS12)
/home/runner/work/openssl/openssl/test/openssl/utils.rb:140:in `teardown'
<[]> expected but was
<["error:0308010C:digital envelope routines::unsupported"]>

diff:
? ["error:0308010C:digital envelope routines::unsupported"]
```
@junaruga
Copy link
Member Author

Here is the error.

https://github.com/ruby/openssl/actions/runs/23562768731/job/68606686661?pr=1019#step:11:590

1) Error: test_private_encoding_encrypted(OpenSSL::TestPKeyRSA): OpenSSL::PKey::PKeyError: i2d_PKCS8PrivateKey_bio: encrypt error
/home/runner/work/openssl/openssl/test/openssl/test_pkey_rsa.rb:465:in `private_to_der'
/home/runner/work/openssl/openssl/test/openssl/test_pkey_rsa.rb:465:in `test_private_encoding_encrypted'
     462: 
     463:   def test_private_encoding_encrypted
     464:     rsa = Fixtures.pkey("rsa2048")
  => 465:     encoded = rsa.private_to_der("aes-128-cbc", "abcdef")
     466:     asn1 = OpenSSL::ASN1.decode(encoded) # PKCS #8 EncryptedPrivateKeyInfo
     467:     assert_kind_of OpenSSL::ASN1::Sequence, asn1
     468:     assert_equal 2, asn1.value.size
Error: OpenSSL::PKey::PKeyError: i2d_PKCS8PrivateKey_bio: encrypt error
/home/runner/work/openssl/openssl/test/openssl/test_pkey_rsa.rb:465:in `private_to_der'
/home/runner/work/openssl/openssl/test/openssl/test_pkey_rsa.rb:465:in `test_private_encoding_encrypted'

@junaruga
Copy link
Member Author

I got the following CI log when I ran without the 5th commit (my commit) now. Interestingly there is no error about Error: test_private_encoding_encrypted(OpenSSL::TestPKeyRSA): OpenSSL::PKey::PKeyError: i2d_PKCS8PrivateKey_bio: encrypt error in the log. That means the 5th commit modifying test/openssl/test_pkcs12.rb caused the test_private_encoding_encrypted error in the test/openssl/test_pkey_rsa.rb. It is weird. Do I need to clean something up in the teardown method in the test/openssl/test_pkcs12.rb?

https://github.com/junaruga/ruby-openssl/actions/runs/23566877802/job/68620449742

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants