Skip to content

Promql cpp functions#272

Open
cherep58 wants to merge 65 commits intoppfrom
promql_cpp
Open

Promql cpp functions#272
cherep58 wants to merge 65 commits intoppfrom
promql_cpp

Conversation

@cherep58
Copy link
Copy Markdown
Collaborator

@cherep58 cherep58 commented Mar 24, 2026

Summary

Implement C++ pushdown optimizations for PromQL functions at the storage layer. When the query engine provides SelectHints.Func, the storage can now reduce data before it reaches the Go PromQL engine — returning only the samples needed for the specific function instead of the full series.

New decorator iterators

  • Aggregation: MinOverTimeIterator, MaxOverTimeIterator, LastOverTimeIterator, SumOverTimeIterator (via OverTimeFuncIterator template with pluggable handlers)
  • Counter/rate family: RateIterator (rate, increase), IRateIterator (irate, idelta)
  • Transform: DeltaIterator, ChangesIterator, ResetsIterator
  • Downsampling: DownsamplingDecodeIterator — interval-based sample reduction

Refactored decode iterator infrastructure

  • CRTP-based DecodeIteratorTrait with seek(), seek_to(), invalidate(), set() methods
  • SeekResult enum for composable seek logic (kUpdateSample, kNext, kStop, kUpdateSampleNextAndStop)
  • UniversalDecodeIterator extended with seek(), seek_to(), invalidate(), set() that dispatch through std::visit
  • Outer DecodeIterator variant wrapping all iterator types for the Go binding layer

Function dispatch

  • gperf-generated perfect hash (FunctionNamesHash) for O(1) function name lookup
  • create_decode_iterator() dispatches on SelectHints.Func to construct the appropriate iterator

Go bridge changes

  • SelectHints passed through cppbridge via ABI-compatible GenericSelectHints<Go::String, Go::SliceView>
  • downsamplingMs parameter added to Query() and ChunkRecoder paths
  • DataStorageSerializedDataIterator control block simplified: direct Timestamp()/Value() accessors replacing raw field access
  • Namespace change: entrypoint::headentrypoint::series_data

Testing

  • Parametric C++ unit tests for each new iterator (boundary conditions, StaleNaN handling, time interval filtering, reset scenarios)
  • Go integration tests for all dispatched PromQL functions (downsampling, min/max/last/sum_over_time, rate, increase, changes, delta, irate, idelta, resets)

Copy link
Copy Markdown
Collaborator

@vporoshok vporoshok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall solid work — well-structured decorator iterators, good CRTP-based seek infrastructure, thorough test coverage. Two bugs and a few minor items below.

Comment thread pp/entrypoint/series_data/decode_iterator.h
Comment thread pp/series_data/decoder/decorator/over_time_func_iterator.h
Comment thread pp/entrypoint/series_data/decode_iterator.h Outdated
Comment thread pp/go/storage/block/writer.go
@cherep58 cherep58 marked this pull request as draft March 27, 2026 14:29
@cherep58 cherep58 marked this pull request as ready for review April 3, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants