fix: log panel flickering in Grafana 12.3+ with newLogsPanel virtualization#1
Merged
hatemosphere merged 7 commits intomainfrom Mar 23, 2026
Merged
Conversation
…to fix log panel flickering
The plugin used the deprecated `getDataProvider()` API which created its own
Observable and immediately emitted `{ state: Loading, data: [] }` on every
subscription. This cleared log panel data before results arrived, causing
visible flickering/ghosting of log rows in Grafana Explore and dashboards.
Migrated to `getSupplementaryRequest()` which returns a plain DataQueryRequest,
letting Grafana's framework handle the Observable lifecycle. This matches how
Grafana's built-in Elasticsearch and Loki datasources work.
Also fixed a broken array spread in processResponse.ts where the comma operator
inside the spread caused data links to not be properly appended to existing ones.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous approach created a synthetic `$qw_message` field and injected it into the dataframe. This extra field caused Grafana's virtualized log list to miscalculate row heights for multiline/wrapped log entries, resulting in rows overlapping each other. Now follows the same approach as Grafana's built-in Elasticsearch datasource: reorder dataFrame.fields so the configured logMessageField is the first string field. Grafana's parseLegacyLogsFrame picks the body via getFirstFieldOfType(FieldType.string), so field ordering is sufficient. For multiple comma-separated message fields, the first field's values are replaced with concatenated content and extra source fields are removed, keeping the field count unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Go backend's sortPropNames had the shouldSortLogMessageField parameter but never used it, causing fields to be sorted purely alphabetically. This meant the configured logMessageField (e.g. "message") ended up after other string fields like "enr" alphabetically. Grafana's parseLegacyLogsFrame picks the log body via getFirstFieldOfType(FieldType.string), so when "enr" was first, it became the body field. The virtualizer then measured "enr" values (short) for row height instead of the actual "message" content (long), causing all rows to get 50px height regardless of wrapped message length — resulting in rows overlapping each other. This matches the fix in Grafana's built-in Elasticsearch datasource (response_utils.go) which correctly prepends logMessageField before the alphabetical sort. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Go backend's sortPropNames now correctly sorts logMessageField to first position, so the frontend reorderMessageField is unnecessary. Removed it along with the original $qw_message synthetic field injection. Kept: data links spread fix, data links application logic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Grafana's virtualized log panel (newLogsPanel) uses _source for body text measurement when calculating row heights with displayed fields. Without _source, the virtualizer measures only the short message field value but renders all displayed field values inline, causing height underestimation and row overlap. Serialize each document as a JSON string in _source, matching what the built-in Elasticsearch datasource returns. This ensures the virtualizer has the full document text for accurate height calculation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The _source field caused Grafana's handleOverflow/resetAfterIndex to enter an infinite loop (13k+ calls/sec), making the panel flicker continuously. The overflow correction never converges because _source JSON length varies per row. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Grafana's virtualized log panel (newLogsPanel) uses LogRowModel.uid as
cache key for row height measurements. The uid is derived from the frame's
"id" field: uid = "${refId}_${idValue}". Without an "id" field, all rows
get uid "A_null" and share one cache entry.
When rows have different heights but share a cache key, the overflow
correction (handleOverflow → storeLogLineSize → resetAfterIndex) enters
an infinite loop: row 1 stores height 50, row 7 reads 50 but needs 660,
corrects to 660, row 1 reads 660 but needs 50, corrects to 50, repeat.
This causes 13k+ resetAfterIndex calls/sec = visible flickering.
The fix: generate a unique "id" field per row (matching ES behavior where
each hit has _id). Grafana's parseLegacyLogsFrame excludes the "id" field
from extraFields (line 49/63), so it won't appear as a displayed field.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
hatemosphere
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes log panel flickering/row overlap in Grafana 12.3+ caused by three issues in how the plugin returns data frames to Grafana's new virtualized log panel (
newLogsPanel, GA since 12.3).Root cause
Grafana 12.3 introduced a virtualized log list that caches row heights by
LogRowModel.uid. The uid is${refId}_${idField.values[row]}. Without anidfield, all rows shared uidA_null, causing the height cache to oscillate between different row heights in an infiniteresetAfterIndexloop (13k+ calls/sec = visible flickering).Additionally,
sortPropNamesignored theshouldSortLogMessageFieldparameter, sologMessageFieldwas sorted alphabetically instead of being placed first. Grafana picks the log body viagetFirstFieldOfType(FieldType.string), so wrong field order = wrong body field = wrong height measurement.Changes
pkg/quickwit/response_parser.go:idfield per log row (sequential index as fallback when doc has noid), matching Elasticsearch behavior where each hit has_id/id. Grafana'sparseLegacyLogsFrameuses this forLogRowModel.uid, which the virtualizer uses as height cache key.sortPropNamesto actually sortlogMessageFieldto first position whenshouldSortLogMessageFieldis true, matching the Grafana built-in Elasticsearch datasource implementation.src/datasource/processResponse.ts:$qw_messagefield injection — now redundant since Go backend correctly sortslogMessageFieldfirst.[...(a || [], b)](comma operator, drops existing links) →[...(a || []), ...b].src/datasource/supplementaryQueries.ts:getDataProvider()togetSupplementaryRequest()API. The old code created an Observable that emitted{data: []}on every subscription. Matches how Grafana's built-in ES and Loki datasources work.Test plan
resetAfterIndexcalls vs 13k+/sec for QW)idfield → unique uids → no more infinite loop → no flickeringmessagefield correctly sorted to first position in frameUpstream PR: quickwit-oss#174