Skip to content

Add support for bytes key btree indices#4733

Open
Centril wants to merge 12 commits intomasterfrom
centril/range-compat-bytes-key
Open

Add support for bytes key btree indices#4733
Centril wants to merge 12 commits intomasterfrom
centril/range-compat-bytes-key

Conversation

@Centril
Copy link
Copy Markdown
Contributor

@Centril Centril commented Mar 31, 2026

Description of Changes

Add support for btree indices where the keys are encoded byte strings for e.g., multi-column indices of no-unbounded-types (arrays and strings) that aren't floats.

The main interesting stuff in this PR is in bytes_key.rs which defines RangeCompatBytesKey, a type that is derived from BytesKey, by converting little-endian encoded integers to big-endian. Signed integers are now also supported, but floats are not. table_index/mod.rs also includes a bunch of interesting stuff.

API and ABI breaking changes

Technically this fixes pre-existing bugs in the handling of Excluded ranges for multi-col indices.

Expected complexity level and risk

2?

Testing

  • A proptest order_in_bsatn_is_preserved is now adjusted and enabled to exercise the ordering of RangeCompatBytesKey.
  • A proptest btree_multi_col_range_scans_work is added to check the behavior of range scans on multi-col indices.

Copy link
Copy Markdown
Collaborator

@joshua-spacetime joshua-spacetime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think more testing is needed for this patch.

Comment on lines +239 to +243
pub(super) fn from_bsatn_prefix_and_endpoint(
prefix: &[u8],
prefix_types: &[ProductTypeElement],
endpoint: &[u8],
range_type: &AlgebraicType,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefix is the fixed (non-range) part of the scan right? But the entire range key itself could still be a prefix of the index? In which case there's an implicit suffix with MIN and MAX bounds that we're omitting which is not correct.

Currently I believe if you have a 3-column index on (u64, u64, u64), prefix=5 and endpoint=1 effectively becomes the range [(5,1,0), (5,1,0)].

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be fixed now in the last commit. There were also some pre-existing bugs on master that I fixed. A proptest is also included for all of this now.

@Centril Centril force-pushed the centril/range-compat-bytes-key branch from 0367d60 to 58644fe Compare April 8, 2026 17:16
@Centril Centril changed the base branch from master to centril/table-index-iter April 8, 2026 17:16
Base automatically changed from centril/table-index-iter to master April 8, 2026 21:48
@Centril Centril force-pushed the centril/range-compat-bytes-key branch from 58644fe to e649f64 Compare April 9, 2026 12:12
@Centril Centril requested a review from joshua-spacetime April 9, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants