FIX: Cursor.describe invalid data #355

dlevy-msft-sql · 2025-12-01T17:19:35Z

Work Item / Issue Reference

GitHub Issue: #352

Summary

This PR fixes cursor.description to properly return SQL type codes per DB-API 2.0 specification, while maintaining backwards compatibility with libraries like pandas that compare against Python types.

Key Changes

New SQLTypeCode class - A dual-compatible type code that compares equal to both:
- SQL type integers (e.g., desc[1] == -9) for DB-API 2.0 compliance
- Python types (e.g., desc[1] == str) for pandas/library compatibility
SQL Server spatial type support - Added handling for:
- geography (e.g., GPS coordinates, regions)
- geometry (e.g., shapes, planar coordinates)
- hierarchyid (e.g., org charts, tree structures)
ODBC 3.x date/time type codes - Added support for:
- SQL_TYPE_DATE (91)
- SQL_TYPE_TIME (92)
- SQL_TYPE_TIMESTAMP (93)
- SQL_TYPE_TIMESTAMP_WITH_TIMEZONE (95)
Hash/equality contract fix - Made SQLTypeCode unhashable since __eq__ compares to multiple types with different hash values

Files Changed

File	Changes
`mssql_python/cursor.py`	Added `SQLTypeCode` class, updated `_initialize_description()`
`mssql_python/constants.py`	Added `SQL_SS_UDT`, `SQL_SS_TIME2`, `SQL_SS_XML` constants
`mssql_python/pybind/ddbc_bindings.cpp`	Added C++ handling for spatial types
`mssql_python/__init__.py`	Exported `SQLTypeCode`
`mssql_python/mssql_python.pyi`	Added type stubs for `SQLTypeCode`
`tests/test_002_types.py`	Added `SQLTypeCode` unit tests
`tests/test_004_cursor.py`	Added spatial type tests, thread safety tests

Usage Examples

SQLTypeCode - Dual Compatibility

cursor.execute("SELECT id, name FROM users")
desc = cursor.description

# Style 1: Compare with Python types (pandas compatibility)
if desc[0][1] == int:
    print("Integer column")

# Style 2: Compare with SQL type codes (DB-API 2.0)
if desc[0][1] == 4:  # SQL_INTEGER
    print("Integer column")

# Get raw SQL type code
type_code = int(desc[0][1])  # Returns 4 for SQL_INTEGER

Spatial Types

# Geography (GPS coordinates)
cursor.execute("""
    INSERT INTO locations (geo) 
    VALUES (geography::STGeomFromText('POINT(-122.349 47.651)', 4326))
""")

# Fetch as binary
cursor.execute("SELECT geo FROM locations")
geo_bytes = cursor.fetchone()[0]  # Returns bytes

# Or fetch as WKT text
cursor.execute("SELECT geo.STAsText() FROM locations")
wkt = cursor.fetchone()[0]  # Returns "POINT (-122.349 47.651)"

Testing

All existing tests pass
Added 40+ new tests for spatial types (geography, geometry, hierarchyid)
Added thread safety tests for _column_metadata
Added SQLTypeCode unit tests
Coverage: 98% diff coverage, 76% overall

Copilot Review Comments Addressed

Comment	Resolution
Missing ODBC 3.x date/time codes	Added 91, 92, 93, 95 to `_type_map`
Hash/eq contract violation	Set `__hash__ = None` (unhashable)
Geography executemany test incomplete	Test now inserts and verifies `geo_col`

Breaking Changes

None. The SQLTypeCode class maintains full backwards compatibility:

cursor.description[i][1] == str still works ✅
cursor.description[i][1] == -9 also works ✅

…ins an LOB

Copilot

Pull request overview

This PR fixes issue #352 by implementing two main improvements: (1) correcting cursor.description to return SQL type codes (integers) instead of Python type objects, ensuring DB-API 2.0 specification compliance, and (2) adding support for SQL Server spatial data types (geography, geometry, hierarchyid) by handling the SQL_SS_UDT type code (-151).

Key changes:

Fixed cursor.description[i][1] to return SQL type integer codes (e.g., 4, -9) instead of Python types (int, str) per DB-API 2.0 spec
Added SQL_SS_UDT (-151) support for SQL Server User-Defined Types including spatial data types
Updated output converter lookup to use SQL type codes consistently throughout the codebase

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
mssql_python/cursor.py	Changed `cursor.description` to return SQL type codes instead of Python types; added `_column_metadata` storage; updated `_build_converter_map` to use SQL type codes; added mappings for UDT, XML, DATETIME2, SMALLDATETIME types
mssql_python/constants.py	Added `SQL_SS_UDT = -151` constant for SQL Server User-Defined Types
mssql_python/pybind/ddbc_bindings.cpp	Added C++ constants for SQL_SS_UDT, SQL_DATETIME2, SQL_SMALLDATETIME; implemented UDT handling in SQLGetData_wrap, FetchBatchData, FetchMany_wrap, and FetchAll_wrap for LOB streaming
tests/test_004_cursor.py	Added `lob_wvarchar_column` to test schema; updated `test_cursor_description` to expect SQL type codes; added comprehensive geography type test suite (14 new tests); separated LOB and non-LOB fetch tests; fixed output converter test for UTF-16LE encoding
tests/test_003_connection.py	Simplified converter integration tests to use SQL type constants directly instead of dynamic type detection

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/test_004_cursor.py

mssql_python/constants.py

mssql_python/cursor.py

…ins an LOB

…ython into Issue-352

github-actions · 2026-01-05T19:27:13Z

📊 Code Coverage Report

🔥 Diff Coverage 92%	🎯 Overall Coverage 77%	📈 Total Lines Covered: `5555` out of `7212` 📁 Project: `mssql-python`

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

mssql_python/init.py (100%)
mssql_python/connection.py (88.2%): Missing lines 50,1020
mssql_python/constants.py (100%)
mssql_python/cursor.py (90.4%): Missing lines 134,152,1126,1473,2541
mssql_python/pybind/ddbc_bindings.cpp (100%)

Summary

Total: 99 lines
Missing: 7 lines
Coverage: 92%

mssql_python/connection.py

Lines 46-54

Lines 1016-1024

mssql_python/cursor.py

Lines 130-138

Lines 148-156

Lines 1122-1130

Lines 1469-1477

Lines 2537-2545

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.hpp: 58.8%
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.cpp: 69.5%
mssql_python.pybind.ddbc_bindings.h: 69.7%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.row.py: 79.5%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.__init__.py: 84.9%
mssql_python.cursor.py: 85.2%

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

mssql_python/cursor.py

Copilot

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/connection.py

mssql_python/cursor.py

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/connection.py

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/connection.py

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

mssql_python/connection.py:952

The add_output_converter docstring says non-NULL values passed to converters “will be a bytes object”, but the Row conversion logic only encodes str to UTF-16LE bytes; other types (e.g., int, decimal.Decimal, datetime) are passed through as their native Python objects. Please update the docstring to reflect the actual value types converters may receive (bytes for strings/binary, native objects otherwise).

        Args:
            sqltype (int, SQLTypeCode, or type): The integer SQL type value to convert, which can be
                          one of the defined standard constants (e.g. SQL_VARCHAR) or a
                          database-specific value (e.g. -151 for the SQL Server 2008
                          geometry data type). Also accepts SQLTypeCode objects (from
                          cursor.description) or Python types (e.g., str, int) for
                          backward compatibility.
            func (callable): The converter function which will be called with a single parameter,
                            the value, and should return the converted value. If the value is NULL
                            then the parameter passed to the function will be None, otherwise it
                            will be a bytes object.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/connection.py

mssql_python/pybind/ddbc_bindings.cpp

tests/test_002_types.py

…UDT against SQL_NO_TOTAL, update hash test

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

saurabh500 · 2026-02-01T19:46:23Z

mssql_python/mssql_python.pyi

            Tuple[
                str,
-                Any,
+                Union[SQLTypeCode, type],


Is SQLTypeCode going to be a public type?

saurabh500 · 2026-02-01T20:00:17Z

mssql_python/cursor.py

+    # Instances are intentionally unhashable because __eq__ allows
+    # comparisons to both Python types and integer SQL codes, and
+    # there is no single hash value that can be consistent with both.
+    __hash__ = None


Since the hash of this type is underterministic, due to dual type support, I understand the rationale behind setting the hash function to None.

While this will make any hashing operation in python error out (including using this item as a key in the dictionary), would it make sense to make the error a little better and guide the code author with the approach?

def __hash__(self): raise TypeError( f"SQLTypeCode is unhashable. Use int(type_code) or type_code.type_code " f"as a dict key instead. Example: {{int(desc[1]): handler}}" )

saurabh500 · 2026-02-02T06:08:34Z

tests/test_004_cursor.py

+
+        cursor.execute(
+            "INSERT INTO #pytest_geography_text (geo_col) VALUES (geography::STGeomFromText(?, 4326));",
+            POINT_WKT,


Do we need to test binary parameters too ? I am not sure how one would use those! But in case you had a usecase in mind.

saurabh500 · 2026-02-02T06:09:25Z

tests/test_004_cursor.py

        db_connection.commit()


+# ==================== GEOGRAPHY TYPE TESTS ====================


This test file has become quite large. Can we consider adding a new test file called test_00N_geography.py ?

saurabh500

Looks pretty good to me.

Do we want to consider doing something about testing with the stable release of libraries which are using the driver?

saurabh500

Looks pretty good to me.

dlevy-msft-sql added 4 commits November 24, 2025 20:05

Fix for FetchMany(number of rows) ignores batch size when table conta…

c403899

…ins an LOB

First round of code review fixes

f96d14c

Fix test_fetchone

e135e40

Fix for issue 352

38966d0

Copilot AI review requested due to automatic review settings December 1, 2025 17:19

Copilot started reviewing on behalf of dlevy-msft-sql December 1, 2025 17:20 View session

Copilot finished reviewing on behalf of dlevy-msft-sql December 1, 2025 17:23

Copilot AI reviewed Dec 1, 2025

View reviewed changes

tests/test_004_cursor.py Outdated Show resolved Hide resolved

tests/test_004_cursor.py Outdated Show resolved Hide resolved

mssql_python/constants.py Outdated Show resolved Hide resolved

mssql_python/cursor.py Outdated Show resolved Hide resolved

dlevy-msft-sql added bug Something isn't working pr-size: medium Moderate update size labels Dec 1, 2025

dlevy-msft-sql added 6 commits December 1, 2025 11:31

Apply fixes from copilot review

3f52dd7

Fix for FetchMany(number of rows) ignores batch size when table conta…

38b041b

…ins an LOB

First round of code review fixes

139994a

Fix test_fetchone

6fd823e

Fix for issue 352

a7d9b8e

Apply fixes from copilot review

646ef04

dlevy-msft-sql force-pushed the Issue-352 branch from 3f52dd7 to 646ef04 Compare December 9, 2025 20:56

dlevy-msft-sql added 2 commits January 5, 2026 13:01

Merge branch 'Issue-352' of https://github.com/dlevy-msft-sql/mssql-p…

8018872

…ython into Issue-352

Merge branch 'main' into Issue-352

67268e5

dlevy-msft-sql added 4 commits January 7, 2026 22:49

Merge branch 'main' into Issue-352

d335ed0

Merge branch 'main' into Issue-352

2dc93f3

Merge branch 'main' into Issue-352

eee5256

Merge branch 'microsoft:main' into Issue-352

d289a9b

dlevy-msft-sql added pr-size: small Minimal code update pr-size: medium Moderate update size and removed pr-size: medium Moderate update size labels Jan 16, 2026

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Outdated Show resolved Hide resolved

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Outdated Show resolved Hide resolved

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Show resolved Hide resolved

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 02:37 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

mssql_python/connection.py Show resolved Hide resolved

mssql_python/cursor.py Outdated Show resolved Hide resolved

dlevy-msft-sql added 2 commits February 1, 2026 20:48

style: Fix black formatting in test_004_cursor.py

deb4242

fix: Only forward int sqltypes to native conn, use SQL_WVARCHAR constant

2b6af43

dlevy-msft-sql requested a review from Copilot February 2, 2026 02:56

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 02:56 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

mssql_python/connection.py Show resolved Hide resolved

fix: Widen add/get/remove_output_converter annotations to include type

5ce031a

dlevy-msft-sql requested a review from Copilot February 2, 2026 03:11

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 03:11 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

mssql_python/connection.py Show resolved Hide resolved

fix: Only forward int sqltypes to native remove_output_converter

73f47ae

dlevy-msft-sql requested a review from Copilot February 2, 2026 03:24

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 03:25 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

docs: Fix add_output_converter docstring to reflect actual value types

496c3d1

dlevy-msft-sql requested a review from Copilot February 2, 2026 03:39

dlevy-msft-sql removed the pr-size: small Minimal code update label Feb 2, 2026

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 03:39 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

mssql_python/connection.py Outdated Show resolved Hide resolved

mssql_python/pybind/ddbc_bindings.cpp Outdated Show resolved Hide resolved

tests/test_002_types.py Outdated Show resolved Hide resolved

fix: Add python_type fallback for get_output_converter, guard SQL_SS_…

0dfa606

…UDT against SQL_NO_TOTAL, update hash test

dlevy-msft-sql requested a review from Copilot February 2, 2026 03:54

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 03:55 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

dlevy-msft-sql marked this pull request as ready for review February 2, 2026 04:03

saurabh500 reviewed Feb 2, 2026

View reviewed changes

saurabh500 approved these changes Feb 2, 2026

View reviewed changes

		db_connection.commit()


		# ==================== GEOGRAPHY TYPE TESTS ====================

FIX: Cursor.describe invalid data #355

Are you sure you want to change the base?

FIX: Cursor.describe invalid data #355

Conversation

dlevy-msft-sql commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Work Item / Issue Reference

Summary

Key Changes

Files Changed

Usage Examples

SQLTypeCode - Dual Compatibility

Spatial Types

Testing

Copilot Review Comments Addressed

Breaking Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Code Coverage Report

🔥 Diff Coverage

92%

🎯 Overall Coverage

77%

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

Summary

mssql_python/connection.py

mssql_python/cursor.py

📋 Files Needing Attention

🔗 Quick Links

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

saurabh500 Feb 1, 2026

Choose a reason for hiding this comment

Uh oh!

saurabh500 Feb 1, 2026

Choose a reason for hiding this comment

dlevy-msft-sql commented Dec 1, 2025 •

edited

Loading

github-actions bot commented Jan 5, 2026 •

edited

Loading