Implement Grafana Query Builder

[archef2000]

Add support for all operations of LogsQL in the Grafana Query Builder.
@Loori-R

#48

[Loori-R]

Tested on Grafana 11.3.1.

An error occurs when adding filters:

1. Open Explorer
2. Go to Operations → Filters → Words
   This issue happens not only with Words, but also with other filters.

[archef2000]

I am currently on 12.0.2 and everything is working will test version 11

[archef2000]

The problem exists up to including v 11.4.6 and is fixed in 11.5.0.
The source problem is that the custom Field Editors get undefined as props

[Loori-R]

It looks like the issue is with the Combobox component - it doesn't work correctly with older versions of Grafana.
The Select component (which was its alternative) is now considered deprecated.

I think we should consider raising the minimum supported Grafana version.
What do you think, @dmitryk-dk, @hagen1778?

[Loori-R]

@archef2000, Could you please take a look at the tests and fix them?
Here’s the link to the failing workflow:
https://github.com/VictoriaMetrics/victorialogs-datasource/actions/runs/16075336610/job/45477811316?pr=329

[dk]

It looks like the issue is with the Combobox component - it doesn't work correctly with older versions of Grafana.
The Select component (which was its alternative) is now considered deprecated.

I think we should consider raising the minimum supported Grafana version.
What do you think, @dk, @roman?

In my experience query builders are fairly hard to use manually, - click click click hundred times, - I prefer plain old text for that. But what do I know, I am probably a wrong dk :)

[Loori-R]

In my experience query builders are fairly hard to use manually, - click click click hundred times, - I prefer plain old text for that. But what do I know, I am probably a wrong dk :)

Oops, looks like I tagged the wrong person 😅
Thanks a lot for sharing your thoughts - and sorry for the ping!

[dmitryk-dk]

It looks like the issue is with the Combobox component - it doesn't work correctly with older versions of Grafana. The Select component (which was its alternative) is now considered deprecated.

I think we should consider raising the minimum supported Grafana version. What do you think, @dmitryk-dk, @hagen1778?

As for me, to up the version is ok, but we need to test everything properly

We could have another option to use, use the old Combobox, but I am not sure it would work in the newest versions.

11.3.1 as @Loori-R mentioned, is not an old version and many users are using versions 10-11.

[Loori-R]

• There is no "old" Combobox - it was introduced as a new component and is currently marked as Alpha. View source

• The Select component is marked as Deprecated. View source

As an option, we could use Select for now since it’s more stable, and plan to migrate to Combobox in the future.
Guide: Migrating from Select to Combobox

[dmitryk-dk]

• There is no "old" Combobox - it was introduced as a new component and is currently marked as Alpha. View source
• The Select component is marked as Deprecated. View source

As an option, we could use Select for now since it’s more stable, and plan to migrate to Combobox in the future. Guide: Migrating from Select to Combobox

I think using Select instead of Combobox is a better idea. We do not need to change the supported version, and we do not need to use the Alpha component.

[archef2000]

@Loori-R Can comfirm that with the Select component everything works.

[Loori-R]

@Loori-R Can comfirm that with the Select component everything works.

@archef2000 Would you mind replacing Combobox with Select?

[archef2000]

Already on it

[Loori-R]

Already on it

It looks like there are still some Combobox components in the code:
https://github.com/VictoriaMetrics/victorialogs-datasource/pull/329/files#diff-35151418f87b2fad4cd2f8e9b4515cd9685e1c922d771a348463efc8672b15ecR12-R22

[archef2000]

I am not finished yet and will make a commit once all are changed and tested

[archef2000]

This query should give me the value types of ClientPort right?
/select/logsql/field_values?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats+%7C+uniq+type&start=1751839200000&end=1752011999999&field=type

query: _msg:* | uniq by ClientPort | block_stats | uniq type
field: type

[Loori-R]

This query should give me the value types of ClientPort right? /select/logsql/field_values?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats+%7C+uniq+type&start=1751839200000&end=1752011999999&field=type

query: _msg:* | uniq by ClientPort | block_stats | uniq type field: type

Not sure, but it seems like uniq by ClientPort and uniq type might overlap.
@dmitryk-dk, could you please help clarify?

[Loori-R]

Seems like the query won’t return ClientPort types — it just gets unique type values from logs matching the full query as a text filter, because /field_values doesn’t run the pipeline logic.

[dmitryk-dk]

_msg:* | uniq by ClientPort | block_stats | uniq type

Sorry guys missed you question

query _msg:* | uniq by ClientPort | block_stats will return

https://play-vmlogs.victoriametrics.com/select/vmui/#/?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats&g0.range_input=5m&g0.end_input=2025-07-09T11%3A56%3A01&g0.relative_time=last_5_minutes

field: ClientPorttype: constvalues_bytes: 0bloom_bytes: 0dict_items: 0dict_bytes: 0rows: 1part_path: inmemory

the next filter will get only type so the query like
_msg:* | uniq by ClientPort | block_stats | uniq type will return only type: const
https://play-vmlogs.victoriametrics.com/select/vmui/#/?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats+%7C+uniq+type&g0.range_input=5m&g0.end_input=2025-07-09T11%3A55%3A21&g0.relative_time=last_5_minutes

[archef2000]

I was in the assumption that /field_values will return the values of the field that are available after the query. Just noticed it in the value_type operation, but then I will have to change the logic for all. Should I change the getFieldList function with an additional type?

[Loori-R]

I don't quite understand what you're trying to achieve.

For the expression:
_msg:* | uniq by ClientPort | block_stats | uniq type

/fields_values can only be used for _msg (I'm not sure how usable it is for _msg, but still).
To substitute values for uniq, you need to use /field_names

[archef2000]

For the value_type operation I want to get the availabe value types of the field specified. The field_values is explained so that I get all field values of the specified field from the result of the query. The query gives me the correct result, but the endpoint seems to ignore the query.

[Loori-R]

Maybe we could just hardcode the supported value types (like const, dict, string, int64, float64, etc.) and allow users to enter a custom type if needed - that might be a simpler and more flexible solution.

[archef2000]

The stats word is only prepended when the stats pipe is used.

[AndrewChubatiuk]

can stats pipe be excluded on a stage, when no filters were selected yet?
i mean in general query builder should help to build query for those who is not familiar with logsql, but it confuses instead by allowing to select building blocks, which produce query that 100% fails

[archef2000]

No as the OperationList component provided by grafana/ui just asks the queryModeller for all operation categories without any context.

[szibis]

Quick question how far we are with releasing this version of datasource officially?

[arturminchukov]

Quick question how far we are with releasing this version of datasource officially?

@szibis Right now I don't have the exact ETA for releasing this feature. If I have any updates, I will let you know.

[AndrewChubatiuk]

@szibis
in a current state query builder doesn't simplify procedure of building logsql expressions.
we've evaluated it internally and despite logsql syntax knowledge some people failed to build successful query during the first attempt to use it and only reverse conversion from raw expression to builder mode helped to understand why it failed. In my opinion builder should not allow to select options that produce absolutely invalid query. For example here value query.expr should impact a subset of categories that is passed to QueryModeller, which are allowed to choose on the next step (don't allow to select pipes, when filter is not set, don't allow setting filter, when pipes are set, etc)

@szibis Have you tried it already?

[archef2000]

The QueryModeller is pretty heavy to create and it would need more than just this change as there are many places where a full QueryModeller is used. Then there is the problem that nearly all operations that are placed result in a query that fails. You first need to fill it with information so that also requires knowledge. It also means limiting advanced users that might create a query out of order. And what happens when I drag a pipe to the start? The error code of vl-select is very informative and should be very easy to understand. There is always a learning curve of a new tool and after the first failed query it should be clear what is allowed.

[AndrewChubatiuk]

understand your concerns since it's a huge feature and it took significant time to implement, but i cannot agree with your statement regarding "learning curve". Builder is an intuitive tool, which should require no learning at all. by the way have you checked loki builder? it has required non-draggable filter, reusing this approach in current PR could improve UX a lot

[szibis]

@szibis in a current state query builder doesn't simplify procedure of building logsql expressions. we've evaluated it internally and despite logsql syntax knowledge some people failed to build successful query during the first attempt to use it and only reverse conversion from raw expression to builder mode helped to understand why it failed. In my opinion builder should not allow to select options that produce absolutely invalid query. For example here value query.expr should impact a subset of categories that is passed to QueryModeller, which are allowed to choose on the next step (don't allow to select pipes, when filter is not set, don't allow setting filter, when pipes are set, etc)

@szibis Have you tried it already?

I can Try to test it but artifacts expired now from last build

For What I am expecting is to simplify Engineers life.
Current Datasource with beta builder is not really stable and not working properly not mentioning zero feature only simple filter.
Yes people can use source query but datasources especially for people moving from another storage to VictoriaLogs should be able to "click-ops" easy simple things not even looking into docs or putting them via source section.
Source section is for more advance users or users need more advance things to do.
I think datasource builder in first place should focus on log management - filtering eg. metadata, deep dive to find any specific event when looking for root causes or just easy and simple build charts graphs from Logs in Dashboards.

[AndrewChubatiuk]

/build

[AndrewChubatiuk]

@szibis
retriggering artifact build, it should be available soon again

[github-actions]

## 🚀 Build Started

**PR:** #329
**Source:** Fork (`archef2000/victorialogs-datasource`)
**Triggered by:** @AndrewChubatiuk

[⏳ View build progress](https://github.com/VictoriaMetrics/victorialogs-datasource/actions/runs/18691164593)

[github-actions]

✅ Preview Build Completed!

Version: v0.21.0-pr329-20251021-165007-55f7fbe
PR: #329
Source: Fork (archef2000/victorialogs-datasource)
Commit: 55f7fbe9b0b28cadd15274035f8d74d45d69ab24
Triggered by: @AndrewChubatiuk

📦 Build Artifacts

• 📦 victoriametrics-logs-datasource-heads-main-0-g55f7fbe.tar.gz (67.43 MB) 🔐 Signed ✓ SHA1

• 📦 victoriametrics-logs-datasource-heads-main-0-g55f7fbe.zip (67.42 MB) 🔐 Signed ✓ SHA1

  📥 Download

  ⬇️ Download from Actions Run

  Artifact name: victoriametrics-logs-datasource-pr329-20251021-165007

  🔐 Attestation

  Build provenance attestation: View details

  🧪 How to test this build

  1. Download the artifact from the Actions run link above
  2. Unzip the downloaded file
  3. Copy the plugin to your Grafana plugins directory:

     # Local Grafana
     unzip victoriametrics-logs-datasource-*.zip
     cp -r victoriametrics-logs-datasource /var/lib/grafana/plugins/
     
     # Docker Grafana
     docker cp victoriametrics-logs-datasource grafana:/var/lib/grafana/plugins/

  4. Restart Grafana:

     # Systemd
     sudo systemctl restart grafana-server
     
     # Docker
     docker restart grafana

  5. Verify the plugin in Grafana: Configuration → Plugins
  📋 Build Details

  • Workflow: PR Build on Command
  • Branch: main
  • Is Fork: ✅ Yes
  • Retention: 14 days

  ---

  ⚠️ Note: This is a preview build for testing only. Artifacts will be automatically deleted after 14 days.

[archef2000]

@AndrewChubatiuk Where would that non draggable filter be? I can't seem to find it.

[AndrewChubatiuk]

[archef2000]

Oh that. Will try to implement it.

[AndrewChubatiuk]

thank you for your effort!

[Loori-R]

LGTM! Left a couple of minor readability/cleanup notes

[arturminchukov]

@archef2000
Tested it and have some notices and bugs.

Bugs

1. If we select _stream or _stream_id in label filters, we will get an error. Is it possible to filter fieldnames in this list? Or do we need the filter labels at all?

2. If we select a field name and then select a value, and then select other fieldname, we still have the previous fieldvalues. I think it would be more better to reset these values, if it possible.

Screen.Recording.2025-12-18.at.19.10.36.mov

3. cpu load 100% if using a regexp filter

Screen.Recording.2025-12-18.at.12.04.54.mov

4. In multiselect filters, it is not comfortable if you need to select several values, you have to click every time

Screen.Recording.2025-12-18.at.13.18.46.mov

5. Looks like Result fields in some Stats category should be just text input not a selector

What I suggest

I want to suggest moving these label filters to operations at the top of the list, like a quick action. Something like on screenshot:

Could we add predefined and pinned(you cannot remove it) first operation as stream filter?

It will allow users build a effective fast executable queries. It will be nice if possible to detect if user didn't use the stream filter and we can hide {} in result query (I think like button hide is working now).

[archef2000]

Fixed 1.
2. is out of my control as I just get an update of the new labels
3. Regexp is just a text input without any logic Operations.tsx#L3608 and I can't replicate it
4. also out of my control
5. for what stats would you suggest that?

This quick action is sadly not possible as the only state I have is the string query
Stream filter maybe as default for a new query?

[arturminchukov]

After discussing with the team, we decided to wait until the autocomplete API is ready. We think that, based on this API, it will be possible to build a query builder that helps users build queries. After it, we will return to the query builder.

[archef2000]

Wouldn't it be easier to make that a seperate PR it would mainly be the ‎editorHelper.ts‎ file that needs changing.