Valkey Search Now Delivers Expanded, Rich Search and Aggregations

[KarthikSubbarao]

Description

This PR adds the blog content for the upcoming valkey-search 1.2 release.

In the blog, we cover the search and aggregation capabilities added since 1.0 as well as diving into use cases where different search operators are helpful.

The content was put together by the Authors listed in the PR.

Issues Resolved

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

[makubo-aws]

please add the blog tag and add this to the blog project board.

[KarthikSubbarao]

please add the blog tag and add this to the blog project board.

@makubo-aws , I checked and I don't have the permission to do either of these

[madolson]

please add the blog tag and add this to the blog project board.

We typically just use the issue in the board, but the PR should close the issue.

@KarthikSubbarao Reminder, can you split up the blog so each sentence is on its own line. Otherwise suggesting comments is very difficult.

[KarthikSubbarao]

please add the blog tag and add this to the blog project board.

We typically just use the issue in the board, but the PR should close the issue.

@KarthikSubbarao Reminder, can you split up the blog so each sentence is on its own line. Otherwise suggesting comments is very difficult.

Yes, I will push the change in a few minutes

[KarthikSubbarao]

@madolson , Done

[KarthikSubbarao]

@cnuthalapati, can you check the title and desc here? Both of them sound like titles to me. We can pick one, and then add a short description

[cnuthalapati]

Title -> Valkey Search Now Delivers Expanded, Rich Search and Aggregations

Description -> Valkey Search enables filtering across text, tags, and numeric data and analyzing results with aggregations.

[KarthikSubbarao]

Millions of queries per second depends on how many nodes are in the cluster, the query string, command options, and the dataset. Some query operations can reach 50-60K RPS per node (this is from one of the perf scenarios). It can possibly be higher if we try to find the most optimal query. But it can also be much lower (hundreds / thousands)

[cnuthalapati]

We are interested in reporting at the numbers achievable for exact match tag queries which are the most used queries by an overwhelming majority. Based on our discussion, customers that require that scale of a million search RPS are able to achieve it with the right configuration. No change needed

[cnuthalapati]

Title -> Valkey Search Now Delivers Expanded, Rich Search and Aggregations

Description -> Valkey Search enables filtering across text, tags, and numeric data and analyzing results with aggregations.

[cnuthalapati]

We are interested in reporting at the numbers achievable for exact match tag queries which are the most used queries by an overwhelming majority. Based on our discussion, customers that require that scale of a million search RPS are able to achieve it with the right configuration. No change needed

[cnuthalapati]

[madolson]

This reads a lot like a how to and not much like a blog. It reads a lot more like technical documentation.

[madolson]

Should aggregations be capitalized?

[KarthikSubbarao]

@BCathcart , not sure if this is true, since a key is ingested by one worker thread. Do you want to suggest wording this differently? We can update it

[BCathcart]

Suggestion:

Index updates are parallelized at the key level: a mutated key is processed on a single thread. The response to the client request triggering the mutation is only unblocked after the mutation has been applied to the indexes.

[KarthikSubbarao]

@cnuthalapati , this is good clarification. But this seems like a section we want to cover clearly in our documentation. We already have a WIP section for this here: https://github.com/valkey-io/valkey-doc/blob/c0060260432a6a700482c33fd2cfa036df42b1a5/topics/search.md?plain=1#L92

Let me know if you agree, or if you think it is important to cover here

[cnuthalapati]

We should retain an explanation of how indexing is parallelized without going into the level of detail outlined in the documentation. We can simplify to -

"Index updates are processed by background worker threads, and index changes become visible only after the update completes, at which point the client is unblocked."

[madolson]

Some of these comments were AI generated, I asked it to prompt me before publishing but it published anyways. They're wordy but directionally correct.

[madolson]

Is this needed? Just dive in to how to get started.

[stockholmux]

Yeah. Cut it.

[madolson]

The opening reads like a feature spec sheet rather than a blog post. It lists capabilities without establishing why the reader should care or what problem this solves. The first sentence packs ~5 claims into 30+ words, and the next several paragraphs continue listing features without any narrative thread.

Compare to the vector search blog which opens with a clear value proposition and context, or the Valkey 9.0 blog which leads with the story of what changed and why.

Suggested rewrite:

Until now, Valkey Search focused on vector similarity. This enabled a wide range of workloads such as semantic search and AI workloads, but if you needed to filter products by price range, match a category tag exactly, or search descriptions by keyword, you had to build that yourself.

That changes with the new 1.2 release of valkey-search. You can now combine full-text search, exact-match tags, numeric ranges, and vector similarity in a single query, then analyze results with server-side aggregations with the low latency you expect from Valkey.

In this post, we'll walk through what's new, show how it works with concrete examples, and explore the use cases these capabilities unlock.

This establishes: (1) what existed before (context), (2) what's new (the news), (3) what you'll learn (reader expectation). The current opening skips straight to (2) without (1), which means readers who aren't already familiar with Valkey Search have no frame of reference.

[madolson]

The current structure front-loads implementation details (threading, consistency, scaling) before showing the reader what they can actually do. Most readers want to see capabilities first, then understand how it works.

Current structure:

1. Feature dump intro
2. Real-time Search with Multi-threading (implementation detail)
3. Consistency Your Application Needs (implementation detail)
4. Scale to Terabytes... (implementation detail)
5. Searching Your Valkey Data — 4 subsections (~60% of the blog)
6. Aggregations
7. Getting Started

Suggested restructure:

1. Opening hook — what's new, why it matters, what was the gap before
2. What You Can Search — brief overview of the 4 query types, condensed into one section
3. Show Me — one end-to-end example (e.g., e-commerce product search) that naturally uses text, tag, numeric, and hybrid queries with actual FT.SEARCH commands
4. Aggregations — with a concrete FT.AGGREGATE example
5. Under the Hood — condense threading, consistency, and scaling into one shorter section. Link to docs for the deep dive.
6. Getting Started — with actual commands and links

The "Searching Your Valkey Data" section is the biggest issue — it's essentially four mini-documentation pages stacked together, each following the same template (definition → capabilities → numbered use-case list). Consider condensing the four query type descriptions into a single overview paragraph, then showing ONE compelling example that naturally combines them. Link to the Valkey Search docs for the full reference.

[madolson]

How do we defend the millions of QPS? It feels like that is not true unless we quantify it more.

[stockholmux]

This sentence needs revision, I can't grok it after multiple reads.

There is a weird dependency between ' using more CPUs', then leverage multithreading.

[madolson]

Might be me, but I would prefer, "Read after write consistency" or something. The title feels cute, but not very clear.

[stockholmux]

I totally agree. Every time I see the term 'consistency' unqualified part of me dies.

[madolson]

This section uses a lot of passive voice, consider revising.

• "replicas can be used to offload reads" -> "you can offload reads to replicas"
• "The index is only exposed after..." -> "Valkey Search exposes the index only after..."

[madolson]

The photo path points to karthiksubbarao.jpg, not a photo for Chaitanya. This needs to be updated to the correct photo, or a placeholder should be used.

Same issue exists in bcathcart.md.

[madolson]

GROUPBY, REDUCE, COUNT, SUM, AVG, APPLY, FILTER, SORTBY, LIMIT are all-caps here. "Aggregations" is sometimes capitalized as a proper noun and sometimes not. Please pick one convention and be consistent throughout. I'd suggest lowercase "aggregations" when used as a general concept, and only capitalize when referring to a specific command or feature name.

[madolson]

Also, lets use https://hub.docker.com/r/valkey/valkey-bundle in the getting started.

[stockholmux]

Lot of work needed here.

1. I can't decide who this blog post is for. It covers a lot of technical ground but doesn't provide links or examples.
2. It feels overly long for the information it actually contains.
3. Generally, there is a lot of telling without showing.

[stockholmux]

I've had this argument before but 'microseconds' feels like puffery.

Show don't tell.

[madolson]

I did suggest this. I think it's a fun title, but only if we can back up the contents materially. I'm not honestly convinced we can search terabytes in microsends?

[stockholmux]

The repo consistently uses Valkey 'dash' search, you're using Valkey 'space' search.

[stockholmux]

'search your Valkey data' is a weird phrase. It sounds like you're searching data about Valkey, not data in Valkey.

[stockholmux]

'supports' is overloaded and used frequently in the blog post. Rephrase.

[stockholmux]

Don't use the term 'consistent' - it has special meaning in databases. Please avoid.

[stockholmux]

What does 'risk action on stale data' really mean?

[stockholmux]

Why are you calling out terrabytes here? Will it not work on gigabytes? Is there something special that Valkey Search does to enable this?

[stockholmux]

are groupby, reduce, etc a reserved words? Should they be in an inline code block

[stockholmux]

For that matter, why aren't they linked to docs?

[stockholmux]

Could you provide a more concrete example?

[stockholmux]

Yeah. Cut it.

[stockholmux]

Where would I find out how to do this?

[stockholmux]

Links?

[cnuthalapati]

Thank you @stockholmux @madolson, We are incorporating you feedback. See current changes here:
KarthikSubbarao@2dcda46
KarthikSubbarao@86145c6

[madolson]

Looks OK to me.

[stockholmux]

Much improved. Mostly minor changes.

[stockholmux]

When does this go away? I think you should give a hint that this is temporary and until when.

[madolson]

This will go away when 9.1 launches.