Skip to content

fix(public_key): correctly strip DER leading zero in RSA key_size()#236

Open
sumleo wants to merge 1 commit intorusticata:masterfrom
sumleo:fix/rsa-key-size
Open

fix(public_key): correctly strip DER leading zero in RSA key_size()#236
sumleo wants to merge 1 commit intorusticata:masterfrom
sumleo:fix/rsa-key-size

Conversation

@sumleo
Copy link

@sumleo sumleo commented Feb 15, 2026

Summary

  • RSAPublicKey::key_size() incorrectly calculated the key size by checking if the first byte's MSB was unset (modulus[0] & 0x80 == 0) and unconditionally stripping the first byte
  • This produced incorrect results when the modulus had no DER leading zero padding (i.e., when the MSB of the actual modulus value happens to be 0)
  • The fix checks specifically for a 0x00 padding byte before stripping it, and returns the key size for all non-empty modulus values (not just those with MSB unset)

Test plan

  • Full test suite passes with cargo test --all-features

Copy link
Collaborator

@cpu cpu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this merits test coverage. I'd also prefer one commit instead of one with the change and one addressing a linting finding in the previous change.

}

#[cfg(test)]
mod tests {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this is better than nothing, but it seems conceivable you could generate real keys with the required properties and test them end-to-end through x509-parser as an integration test instead.

The previous implementation unconditionally stripped the first byte of
the modulus when the MSB was clear, which is incorrect. DER encoding
only prepends a 0x00 byte when the MSB of the actual value is set.

This fix checks specifically for a leading 0x00 padding byte and only
strips that, correctly computing the key size for all RSA keys.

Add both unit tests for edge cases and integration tests that parse
real DER certificates with 1024, 2048, and 4096-bit RSA keys to
verify key_size() end-to-end.
@sumleo sumleo force-pushed the fix/rsa-key-size branch from 7f76b92 to fd49368 Compare March 10, 2026 01:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants