Skip to content

Update test framework: add granular tests, remove unstandardized tests, and document known variations#537

Open
ryan-syed wants to merge 13 commits intomicrosoft:masterfrom
ryan-syed:dev/ryan-syed/UpdateTestFramework
Open

Update test framework: add granular tests, remove unstandardized tests, and document known variations#537
ryan-syed wants to merge 13 commits intomicrosoft:masterfrom
ryan-syed:dev/ryan-syed/UpdateTestFramework

Conversation

@ryan-syed
Copy link
Contributor

Summary

Updates the PQ SDK Test Framework with granular tests, removes tests that need further
standardization, and adds documentation for known test variations across connectors.

Changes

Test updates

  • Added granular individual tests in Standard folder (replacing monolithic test files)
  • Updated Sanity tests and outputs
  • Updated schema and schema-related documentation

Tests removed (18 tests, pending standardization)

Removed tests that fail across multiple connectors due to test design issues, not connector bugs.
These will be re-added after standardization in the private repo:

  • Timestamp precision (6): EndOf* tests coupled to database-specific sub-second precision
  • Week boundary (4): StartOfWeek/EndOfWeek/DayOfWeek missing explicit Day.Sunday parameter
  • Trig functions (7): Number.RoundDown at 5 digits still differs across backends
  • NumberFunctions (1): Combined 16-operation test with no rounding

Documentation

  • Rewrote "Known Test Variations" section in PQSDKTestSuites.md with 4 categories:
    1. Datatype Conversion (M-specific type casting)
    2. Floating-Point and Arithmetic Precision
    3. DateTime Arithmetic and Duration Operations
    4. Text Edge Cases
  • Added guidance distinguishing genuine backend variations from connector bugs
  • Updated TestFilters example and workflow recommendations

Test runner

  • Fixed constructor in test runner script
  • Updated test runner script

KazimMir and others added 13 commits June 11, 2025 09:45
… Value functions and other scenarios

- Implemented GroupByWithListAverage query to calculate average LocationID by Borough.
- Added output for GroupByWithListAverage to validate results.
- Created GroupByWithListMinMax query to find max and min values for Zone and LocationID by Borough.
- Added output for GroupByWithListMinMax to validate results.
- Developed GroupByWithListMinMaxDateTime query to find max and min pickup datetime by PULocationID.
- Added output for GroupByWithListMinMaxDateTime to validate results.
- Implemented GroupByWithTableRunCount query to count occurrences of Zones.
- Added output for GroupByWithTableRunCount to validate results.
- Created ValueFunctions query to test various mathematical operations on passenger_count and total_amount.
- Added output for ValueFunctions to validate results.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants