SparseArray slices can serialize with non-zero patch offsets, but SparseVTable::build asserts offset == 0

[Jason8Ni]

What happened?

Summary

There appears to be an internal contract mismatch in Vortex sparse encoding:

• slicing a SparseArray preserves a non-zero patch offset
• serialized sparse metadata keeps that offset
• deserialization then asserts that sparse patches must start at offset 0

That makes some sliced sparse arrays unreadable after round-trip.

Relevant Code

SparseArray::slice() delegates to Patches::slice():

• vortex-sparse/src/ops.rs
• vortex-array/src/patches.rs

Patches::slice() preserves the logical offset:

Some(Self {
    array_len: range.len(),
    offset: range.start + self.offset(),
    indices,
    values,
    chunk_offsets,
    offset_within_chunk,
})

But SparseVTable::build() asserts offset must be 0:

assert_eq!(
    metadata.0.patches.offset(),
    0,
    "Patches must start at offset 0"
);

Observed Behavior

At read time we hit panics like:

assertion `left == right` failed: Patches must start at offset 0
  left: 16384
 right: 0

Steps to reproduce

1. create a sparse array
   • DictArray
   • whose codes child is a SparseArray
   • the dict array is sliced with a non-zero row offset
   • the sliced value is serialized and read back
2. slice it with a non-zero start
3. serialize it
4. deserialize it

Environment

• Vortex version: 0.58.0
• rustc: 1.94.0 (4a4ef493e 2026-03-02)
• cargo: 1.94.0 (85eff7c80 2026-01-15)

Additional context

No response

[robert3005]

This is indeed a bug, however, we never intended for sliced arrays to be serialisable. The current writer will always normalise the arrays so that they have 0 offset. What kind of process do you have that you end up wanting to serialise these arrays?

[Jason8Ni]

We have a storage flush path where we build a large StructArray in memory, then chunk it by row and serialize each chunk separately to keep file segments bounded. One of the struct fields is a DictArray, and its codes child can be sparse, so row-chunking the parent struct ends up serializing sliced sparse-backed dict arrays.

The flow is basically:

1. build a large StructArray
2. slice it by row into chunks
3. serialize each chunk

We assumed that was a valid use of the array API + writer surface, since the chunks are ordinary arrays from the caller’s point of view.