Enhanced Queries (Joins & Aggregations) in AWS SDK Java v2#6813
Draft
anasatirbasa wants to merge 1 commit intoaws:masterfrom
Draft
Enhanced Queries (Joins & Aggregations) in AWS SDK Java v2#6813anasatirbasa wants to merge 1 commit intoaws:masterfrom
anasatirbasa wants to merge 1 commit intoaws:masterfrom
Conversation
c3d0f3b to
373439e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Implements the Enhanced Query API for the DynamoDB Enhanced Client, enabling SQL-like join, aggregation, filtering, ordering, and projection capabilities on top of DynamoDB.
Initial ticket: #491
Modifications
New query API surface (
query/package)QueryExpressionBuilder-- fluent builder for constructing enhanced query specifications (single-table, join, aggregation, filtering, ordering, projection, execution mode, limit).QueryExpressionSpec-- immutable query specification produced by the builder.Condition/ConditionEvaluator-- in-memory condition tree (eq, gt, gte, lt, lte, between, contains, beginsWith, and, or, not, group) with recursive evaluation, dot-path nested attribute support, and combined base+joined map view for post-join filters.EnhancedQueryResult/DefaultEnhancedQueryResult-- sync iterable result set.EnhancedQueryRow-- single result row withgetItem(alias)andgetAggregate(outputName).EnhancedQueryLatencyReport-- timing breakdown (base query, joined lookups, in-memory processing, total).AggregateSpec/OrderBySpec-- aggregation and sort specifications.AggregationFunction(COUNT, SUM, AVG, MIN, MAX),JoinType(INNER, LEFT, RIGHT, FULL),ExecutionMode(STRICT_KEY_ONLY, ALLOW_SCAN),SortDirection(ASC, DESC).Query engine (
query/engine/package)QueryExpressionEngine/QueryExpressionAsyncEngine-- sync and async execution interfaces.DefaultQueryExpressionEngine/DefaultQueryExpressionAsyncEngine-- full sync and async implementations handling base-only, join, and join+aggregation paths with parallel joined-table lookups (200-thread executor).QueryEngineSupport-- shared helpers for aggregation bucketing, sorting, and group key construction.JoinedTableObjectMapSyncFetcher/JoinedTableObjectMapAsyncFetcher-- joined-table lookup strategy (PK query, GSI query, or scan fallback based on ExecutionMode).JoinRowAliases-- alias map helpers for base/joined item assembly.AttributeValueConversion-- DynamoDBAttributeValueto Java object conversion.Client integration
DynamoDbEnhancedClient/DynamoDbEnhancedAsyncClient-- addedenhancedQuery(spec)andenhancedQuery(spec, latencyConsumer)default methods.DefaultDynamoDbEnhancedClient/DefaultDynamoDbEnhancedAsyncClient-- wired engine construction andenhancedQuerydelegation.Documentation
package-info.java-- comprehensive Javadoc for the query package (concepts, filter phases, execution modes, tooling scripts).ENHANCED_QUERY_GUIDE.md-- 23-section playbook covering setup, class reference, query examples with expected output, ExecutionMode deep-dive, condition system deep-dive (leaf/composite types, operator semantics, tree structure, dot-path resolution, 10 input/output examples), execution flow diagrams, build validation, performance tips, complete example, and benchmark guide.EC2_BENCHMARK_INSTRUCTIONS.md-- local and EC2 benchmark setup guide.Tooling scripts
run-enhanced-query-benchmark-local.sh-- local benchmark runner (DynamoDB Local, 1000x1000 dataset, CSV output).run-enhanced-query-tests-and-print-timing.sh-- functional test runner with timing output.Testing
Unit tests:
ConditionTest-- all condition types (eq, gt, gte, lt, lte, between, contains, beginsWith), combinators (and, or, not, group), dot-path nested attribute resolution, null handling,resolveAttributeedge cases.QueryExpressionBuilderTest-- build validation (groupBy without aggregate, filterBase/filterJoined without join, incomplete join config), all builder methods, deprecated method aliases.EnhancedQueryRowTest-- builder, getItem, getAggregate, empty defaults.EnhancedQueryResultTest-- iteration over wrapped rows.EnhancedQueryLatencyReportTest-- constructor and accessor verification.BuildValidationTest-- builder shape validation error messages.Functional tests (DynamoDB Local, 1000 customers x 1000 orders):
EnhancedQueryJoinSyncTest-- base-only query, INNER/LEFT/RIGHT/FULL joins, filterBase, filterJoined, tree conditions, ExecutionMode (STRICT_KEY_ONLY vs ALLOW_SCAN), scan limit, latency report assertions.EnhancedQueryJoinAsyncTest-- async equivalents of all sync join tests.EnhancedQueryAggregationSyncTest-- GROUP BY + COUNT/SUM/MIN/MAX/AVG, joined aggregation with filters, orderByAggregate, limit, execution mode behavior.EnhancedQueryAggregationAsyncTest-- async equivalents of all sync aggregation tests.NestedAttributeFilteringTest-- dot-path filters (address.city, address.state, address.zip), between, contains, beginsWith, AND/OR combinations, missing path handling.Benchmark runner:
EnhancedQueryBenchmarkRunner-- 5 scenarios (baseOnly_keyCondition, joinInner_c1, aggregation_groupByCount_c1, aggregation_groupBySum_c1, joinLeft_c1_limit50), warmup + measured iterations, console table and CSV output.Types of changes
Checklist
mvn installsucceedsscripts/new-changescript and following the instructions. Commit the new file created by the script in.changes/next-releasewith your changes.License