Skip to content

Avoid undefined behavior in be32decode#6284

Merged
fingolfin merged 1 commit intogap-system:masterfrom
jamesjer:master
Mar 25, 2026
Merged

Avoid undefined behavior in be32decode#6284
fingolfin merged 1 commit intogap-system:masterfrom
jamesjer:master

Conversation

@jamesjer
Copy link
Contributor

When GAP is built with UBSAN, the undefined behavior sanitizer, the sanitizer emits this warning:

src/sha256.c:98:30: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'

The affected code in be32decode performs bitwise operations on values of type UInt1. Section 6.3.1.1 of the C standard requires that such 8-bit values be promoted to type int prior to performing the operations. The promotion introduces a sign bit and leads to a left shift that cannot be represented in the int type. This change keeps the values unsigned at all times to avoid undefined left shifts.

Text for release notes

see title

Further details

I am not claiming that the current code produces incorrect results. Most likely it produces correct results on all supported architectures with current compilers. However, undefined behavior can lead to problems in the future if a new compiler optimization assumes that undefined behavior never occurs.

@fingolfin fingolfin added topic: kernel release notes: not needed PRs introducing changes that are wholly irrelevant to the release notes labels Mar 25, 2026
Copy link
Member

@fingolfin fingolfin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@fingolfin fingolfin merged commit 7fc4d46 into gap-system:master Mar 25, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release notes: not needed PRs introducing changes that are wholly irrelevant to the release notes topic: kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants