Context:
From review discussion in #3802 (comment), @dcherian suggested adding an append(self, sizes: Iterable[int]) method to VaryingDimension to allow explicitly extending chunk edges.
Current behavior:
VaryingDimension.resize(new_extent) handles growth by appending a single chunk covering the gap:
# [10, 10, 10] resized from 30 to 45 produces [10, 10, 10, 15]
This works but gives the caller no control over how the new region is partitioned.
Proposed behavior:
# Explicit chunk extension
dim = VaryingDimension([10, 10, 10], extent=30)
dim = dim.append([10, 5]) # -> edges=(10, 10, 10, 10, 5), extent=45
# Would raise on FixedDimension (regular grids don't have explicit edges)
Open questions:
- Should this live on
VaryingDimension, ChunkGrid, or Array?
- Should
FixedDimension.append raise, or convert to VaryingDimension?
- How does this interact with metadata serialization? Appending chunks to a regular grid would change the serialization format from
"regular" to "rectilinear".
- Should
Array.resize() accept an optional chunks parameter that delegates to this? (Also noted in the design doc's resize section)
Related:
shall we also add append(self, sizes: Iterable[int]) to allow a different mode of extension (would fail for the FixedDimension).
Originally posted by @dcherian in #3802 (comment)
Context:
From review discussion in #3802 (comment), @dcherian suggested adding an
append(self, sizes: Iterable[int])method toVaryingDimensionto allow explicitly extending chunk edges.Current behavior:
VaryingDimension.resize(new_extent)handles growth by appending a single chunk covering the gap:# [10, 10, 10] resized from 30 to 45 produces [10, 10, 10, 15]This works but gives the caller no control over how the new region is partitioned.
Proposed behavior:
Open questions:
VaryingDimension,ChunkGrid, orArray?FixedDimension.appendraise, or convert toVaryingDimension?"regular"to"rectilinear".Array.resize()accept an optionalchunksparameter that delegates to this? (Also noted in the design doc's resize section)Related:
Originally posted by @dcherian in #3802 (comment)