Q2 2026 roadmap planning

[adp2201]

Q2 2026 roadmap planning

Hi all,

Q2 is coming up, so it seems like a good time to start a fresh roadmap discussion building on the Q1 thread:

• Q1 roadmap: [DISCUSSION] DataFusion Road Map: Q1 2026 #18494

The goals here are simple:

1. Identify what should carry forward from Q1 into Q2
2. Identify what new projects we think should be prioritized in Q2

Likely carry-forward themes

From the Q1 discussion, these seem like the clearest themes to continue:

• Performance

  • ClickBench / Parquet leaderboard
  • Parquet pushdown / late materialization
  • planning speed
  • sort / join / spilling improvements

• Developer experience / coordination

  • CI reliability
  • contributor workflow
  • better coordination across active efforts

• Extensibility / SQL surface

  • extension types / UDTs
  • VARIANT
  • lambda syntax / array functionality
  • async UDF / UDTF work
  • datetime / timezone improvements

Some concrete Q2 candidates

A few open epics / proposal epics that seem especially relevant:

• Improve query planning speed: [EPIC] Improve query planning speed #19795
• ClickBench Parquet leaderboard: [EPIC] Make DataFusion the top of the ClickBench Parquet leaderboard #18489
• Improve NUMA-awareness: [EPIC] Improve NUMA-awareness #20902
• Sort-merge join execution improvements: [EPIC] Sort-merge join (SMJ) execution improvements #20910
• Support VARIANT: [EPIC] Support VARIANT type for unstructured data #16116
• Support Extension Types / UDTs: Support Extension Types / User Defined Types in DataFusion #12644
• Improve developer / CI speed: [EPIC] A collection of items to improve developer / CI speed  #13813
• Dynamic filtering: [Epic] Dynamic filtering related items #15512
• Statistics improvements: Epic: Statistics improvements #8227

Questions

• Which Q1 themes should clearly carry into Q2?
• Which open epics feel most important right now?
• What important Q2 projects are missing?
• Which efforts need more implementation help, benchmarking, or design work?

If you already plan to work on something in Q2, please comment with the issue / epic and the kind of help you’d welcome from others.

[2010YOUY01]

Thank you for opening this discussion!

This is an advanced predicate pruning feature that I believe is important in the long term. I’d like to help implement it, and would appreciate any help with reviewing.

• feat: Prune complex/nested predicates via statistics propagation #19609

Also for testing coverage, @sdf-jkl and I are working on a SQL fuzzing framework for DataFusion:

• Rewrite datafusion-sqlancer in Rust #14535
• https://github.com/2010YOUY01/datafusion-fuzzer

We hope to complete the initial version soon and start deploying it in CI.

[sdf-jkl]

I plan to work on:

• Pushing VARIANT forward
  • [EPIC][Parquet] Finalize Variant Type support in Parquet arrow-rs#8480
  • Variant integration in DataFusion epic: Initial implementation of this crate datafusion-contrib/datafusion-variant#2 with @friendlymatthew. We have some good first issues
• Abovementioned Rewrite datafusion-sqlancer in Rust #14535
• ALP encoding support for Parquet reader [Parquet] Prototype ALP encoding arrow-rs#8748

[LiaCastaneda]

👋 Hello! We’re very interested in having DataFusion support lambda functions. There has been an ongoing epic for this, and some work has already started toward a basic e2e implementation:

• EPIC - Add support for lambda/higher order functions #14205
• First PR – [RFC] Add lambda support and array_transform udf #18921

It feels like we’re currently missing consensus on the approach to follow. We’d be happy to help -- whether it's reviewing, contributing, or anything else that would move this forward.

Also, @benbellick from substrait-io has added lambda support in Substrait, so it seems the remaining big piece is the function implementation.

cc @gstvg as the one driving this effort!

[benbellick]

From the @substrait-io side, I am happy to help however I can, including leading the implementation of the substrait consumer / producer once consensus is reached on the approach for lambdas. Excited to see this one through!

[asolimando]

Hello! I'd like to continue helping out on "statistics improvements", broadly tracked by #8227, as mentioned in the main description.

I can contribute with PRs and reviews from other active contributors on the topic (@jonathanc-n, @buraksenn, @gene-bordegaray, @kumarUjjawal and I might be forgetting someone!), it would be great if some committer could help us get contributions over the merge line.

cc: (no commitment needed and no pressure, just wanted to make sure you are aware)

• @paleolimbot as you have shown interest in the aforementioned epic
• @xudong963 as you have been very active in the area, notably with Implement partition_statistics API for more operators #15873 and recent PRs around stats
• @gabotechs as you have worked and helped with stats already

[kumarUjjawal]

Sound good! I would be happy to help with this. We should make sure all the relevant issue related to statistics improvements are tracked by #8227.

[jonathanc-n]

Do we think we should revamp the #8227 ticket for statistics improvements?

[kumarUjjawal]

Do you have any particular direction for this?

[paleolimbot]

I'd be happy to help with statistics generally...my primary interest is in ensuring they can represent geometry statistics in a way that doesn't obliterate them when propagating through DataFusion internals. I know @adriangb is interested in making them lazier and also able to represent nested types (not sure the "lazier" bit is captured in an issue yet).

[gene-bordegaray]

Hello! I would like to continue working on statistics improvements as @asolimando mentioned.

Along with this we have found RepartitionExec to be expensive in some cases so diving in on this and making some improvements here is another goal I have. I have not filed an issue or EPIC for this but have been making micro-benches to isolate behavior under different workloads but once done think it will be a good effort for others to collaborate on.