From cf743d171c48d3720a08ff06f23d49c8edc55c0f Mon Sep 17 00:00:00 2001 From: "Alina (Xi) Li" Date: Thu, 8 Jan 2026 12:01:58 -0800 Subject: [PATCH 1/7] Add Arrow Flight SQL ODBC driver Co-authored-by: alinalibq Co-authored-by: rscales Co-authored-by: justing-bq Co-authored-by: vic-tsang ODBC commits squashed into 1 to make rebasing to main easier. Add initial framework for odbc dll - Add ARROW_FLIGHT_SQL_ODBC option. If we set `ARROW_FLIGHT_SQL_ODBC=ON`, the flightsql odbc folder will be built - Add odbc api layer for entry_point.cc - builds odbc dll file, with ODBC APIs exported in odbc.def Address James' comments Fix `odbcabstraction` build errors and partially fix `flightsql_odbc` errors Fix boost-variant not found error - Adding dependencies from odbc/vcpkg.json to cpp/vcpkg.json - Fix whereami.cc and .h dependency; ported lates code Update whereami.cc - use `long` instead of `int64`. Fixed namespace issues. - PR CI fix: Add `parquet-testing` back Partial build fix for `flight_sql` folder - Replaced `namespace arrow` and `namespace odbcabstraction` with `using namespace ...` - fix flight_sql_connection.cc Fix `util::nullopt` to use `std::nullopt` - fix std::optional - fix BufferReader - Fix GetSchema - fix json_converter.cc - partial fix configuration.h - partial fix get_info_cache.cc - Fix winsock build error - Comment out `flight_sql` files that cannot build - Comment out configuration and unit tests - Comment out get info cache and system trust store Create initial odbc tests folder Implement SQLAllocEnv Fix cmake build Implement SQLFreeEnv Fix rest of build errors from `flightsql-odbc` - Fix get info errors - Fix for configuration window - added odbcinst library - Fix system trust store - unit test fixes - Add dependency of ARROW_COMPUTE. `arrow/compute/api.h` is used in `flight_sql`. Adding `ARROW_COMPUTE=ON` during build fixed run time unit tests failures. Implement SQLAllocConnect and SQLFreeConnect Fix build issue from static flight sql driver Lint and code style fixes Re-add deleted submodule parquet-testing clang-format lint fix cpplint lint fix Exclude whereami in rat exclude list C++/CLI lint fix Update parquet-testing to match commit from `main` Address Kou's comments ODBC directory lint fixes Catching the lint fixes outside of `flightsql-odbc` code Fix build warnings that get treated as error Implement SQLSetEnvAttr and SQLGetEnvAttr Implement use of ExecuteWithDiagnostics Doxygen Error Fixes and Address comments from Kou and James Address comments from Kou - Updates License.txt - Update cmake toolchain - Move whereami to `vendored` - Use string_view instead of NOLINT std::string Remove `whereami.cc` from arrow util build We are building whereami.cc as part of odbc Fix include headers to replace <> with "" Address comments from James Implement SQLGetDiagField SQLDriverConnect, SQLConnect and SQLDisconnect Implement stubs for SQLGetInfo, SQLGetDiagField and SQLGetDiagRec Separate RegisterDsn and UnregisterDsn from windows build Update code to save driver value from connection string Add ReadMes for ODBC and tests Fix test issues with string_view Address code reviews Update entry_points.cc to fix build issue Remove Dremio references Use emplace properly Address comment from Rob and add SQLDisconnect test case Add odbc.def and cmd file to rat_exclude Nit - remove duplicate lines Accidentally committed the change during git rebase Nit - remove usage of nullptr DSN window implementation Add licenses to `.cmd` and `.def` files Implement SQLGetDiagRec Build ODBC in Windows Workflow Tests are skipped for now. Updates for SQLGetDiagFieldW Enable Driver Logging Add todo to update logging system later Add logs Implement ODBC API with debug logging Enable mock test * Add todo for noauth validation * mock server with token auth Add tests * run same test with both modes * Enable ODBC tests in workflow * Switch current test cases to use FlightSQLODBCTestBase So the tests can be skipped when `TEST_CONNECT_STR` is not set. * Change tests to run on both mock and remote modes Wrap usage of TEST_CONNECT_STR where possible * Rename test fixtures and make connection string functions virtual * Fix lint issue * Attempt to enable ODBC build on Windows platforms * Attempt to fix clang64 and MinGW errors * Attempt to register ODBC * Address James' comments Use constant string for token * use ServerMiddleware to validate token Fix connection issues to DBT Labs PopulateCallOptions before making a connection Fix dsn window bug with advance properties Fix seg fault issue from empty string Implement SQLAllocStmt Follow-up DBT Labs connection fix Implement SQLGetDiag Rec and Field for statement Unicode support for DSN ODBC APIs * Let compiler append `W` to ODBC APIs where applicable. Initialize Kernel functions As per changes from #46261, we need to initialize Kernel library explicitly to get the functions registered Implement SQLSetConnectAttr and SQLGetConnectAttr ODBC Test Segfault Fix * Move `connection_test.cc` to last in `CMakeLists.txt`, this resolves the seg fault. The seg fault was not reproducible on my local environment, even when I use the msys2 shell. * Rename test executable to flight_sql_odbc_test SQLExecDirect implementation * SQLGetStmtAttr stub implementation * Fix missing break statement in SQLGetDiagRec * Run ShutdownProtobufLibrary as part of test environment * Add comment for GH-46889 * Pass call options to executed prepared statement `call_options_` contains the authentication token which is needed Implement SQLGetInfo SQLGetInfo Workflow Fixes SQLFetch & SQLGetData Implementation SQLGetStmtAttr stub implementation Stub call for SQLGetStmtAttr Enable statement handle allocation Implement SQLExecDirect - test hang issue Update odbc_api.cc Run `ShutdownProtobufLibrary` after all ODBC tests Resolves test hanging problem Fix missing break statement in SQLGetDiagRec Add tests Lint fixes Run ShutdownProtobufLibrary as part of test environment Add debug messages Draft code Remove drafts Add comment Add comment for GH-46889 Update statement test headers Pass call options to executed prepared statement `call_options_` contains the authentication token which is needed SQLFetch & SQLGetData initial impl EVERYTHING before this commit is for SQLExecDirect SQLFetch and SQLGetData TestSQLExecDirectSimpleQuery test Make GetData() return SQLRETURN `TestSQLExecDirectDataQuery` for remote and mock servers Remove unneeded test case Add more data types * Add SQL_GUID in getCTypeForSQLType * Add data types in query * Add checks for columns 1 to 25 * Update statement_test.cc Add varbinary test * Implement SQLNumResultCols Will leave tests for later * Implement SQLRowCount * Implement SQLMoreResults * Work on date retrieval fix The `TestSQLExecDirectDataQuery` checks for date is not working. * Fix date retrieval error by importing fix from Paul * imported fix from https://github.com/dremio/flightsql-odbc/commit/d44d862bd07f05328f5fcfd68a8f40357aa072fa, an Apache-Licensed repository * Add more checks for get data * Add more tests * Continue work on float truncation test * Disable float truncation test * Address comments from James - do not return errors for invalid rowCountPtr and columnCountPtr - add static cast for columnCount - Add tests with SQL_C_DEFAULT as target type - Add test checks for indicator - Fix SQL_NUMERIC, SQL_BIT, SQL_FLOAT target type * Fix build issues in CI * More fix for CI * Add test for invalid buffer length SQLPrepare & SQLExecute Implementation * Add tests * Address comments from Rob Code Style fixes * Move ifdefs outside of test cases * use `stmt` * Move logs to first line * fix test for `SQL_DRIVER_HSTMT` * Use camel case for tests Implement SQLGetStmtAttr and SQLSetStmtAttr SQLBindCol Implementation * SQL_UNBIND implementation + tests * Add tests for `SQL_ATTR_ROW_ARRAY_SIZE` * Add check for `SQL_ATTR_ROWS_FETCHED_PTR` Enable TestCloseConnectionWithOpenStatement test case SQLFetchScroll Implementation * Add checks for SQL_ATTR_ROWS_FETCHED_PTR * SQLFetchScroll is supposed to return the number of rows fetched using SQL_ATTR_ROWS_FETCHED_PTR SQLExtendedFetch Implementation * Implement rowCountPtr and rowStatusArray for SQLExtendedFetch * Add tests for SQLExtendedFetch * `SQLExtendedFetch` doesn't return `SQL_SUCCESS_WITH_INFO` for error state 22002. * Add tests for `SQL_ROWSET_SIZE` * Address comments from James GH-46584 Iterate endpoint * use suggestion from James for one-liner change to return `Status::OK` directly * fix for iterating endpoints, it is based on JDBC's fix for the same bug * save value of `FlightClientOptions` * Use `TestFlightServer` Add `arrow_flight_sql_shared` to enable usages for `TestFlightServer` * Fix build errors from missing `gmock` * Add checks for array data Update flight_sql_stream_chunk_buffer_test.cc Add `FlightStreamChunkBufferTest` unit test Draft test with `FlightSQLODBCMockEndpointTestBase` * Reference GH-47117 and Clean up code * Add driver and DSN to built-in properties to not pass driver/dsn as attributes to server * Allow `=` in values inside connection strings, fixes connectivity problems * Address comments from Rob SQLColumns Implementation * Initial impl for SQLColumns * Add test for SQLColumns * Add columns test with all supported column types - mock server doesn't support schema * Address comments from James - Test different data type return value for SQLColumns - Add todo comment for SQLDescribeCol tests --------- Co-authored-by: rscales Fix bug of DSN not read properly * Fix reading DSN bug Use a simpler, more robust way to load the DSN SQLColAttribute implementation * SQLColAttribute initial impl * Switch to use `GetAttributeSQLWCHAR` to be unicode-compatible * Add SQLColAttribute tests and fix bugs in `flightsql-odbc` impl * Fixed bug with incorrect column count. It was returning column count + 1. * Fixed bug with incorrect numPrecRadix. It was returning SQL_NO_TOTAL (maps to -4) for non-numeric columns, but the correct value is 0. * Fixed bug with unsigned column to return true for unsigned columns (non-numeric columns or unsigned numeric columns) and false for signed columns (numeric columns) * Fixed bug with incorrect non-concise data type return value for date time type types. The correct return is SQL_DATETIME for all date time types * Address comments from James * use const for `ConvertToWString` * add nullptr check in odbc_api.cc layer * Add support for `SQL_COLUMN_LENGTH`, `SQL_COLUMN_PRECISION`, and `SQL_COLUMN_SCALE`. The driver will return the same values for ODBC2 and ODBC3. * Add tests for ODBC v2 SQLColAttributes values * I converted tests that can be run on remote servers where possible * Address comment from Rob Implement SQLTables Use C++ 20 standard for ODBC * Add support for building with C++20 * Upgrade spdlogs version as an attempt to fix build issues * Fix issue of missing unique_ptr definition Add tests for SQLMoreResults Implement SQLNativeSql Add diagnostic records * remove `SQLErrors`, as the driver manager is supposed to map `SQLErrors` to `SQLGetDiagRec` Implement tests for SQLNumResultCols SQLCloseCursor Implementation * fix error state to return the correct state 24000. * add tests for SQLCloseCursor and SQLFreeStmt(SQL_CLOSE) Add SQLColumns and SQLColAttribute tests with nullptr * Modify SQLTables test to run on both mock and remote * reorder error msg alphabetically Implement tests for SQLRowCount Fix merge conflicts Adjust searchable return value based on GH-46920 fix Fix `getCTypeForSQLType` return for interval types - Fix `getCTypeForSQLType` function to return the correct `C type` for interval SQL data types. Previously the function was checking for interval `C type` and returning interval `SQL type`, which was the opposite of the intended functionality. SQLGetTypeInfo implementation * Add tests for SQLGetTypeInfo * fix bug of createParams returning empty string instead of null * fix bug of non-concise data type being returned for datetime/interval * fix bug of longvarchar not converted to wlongvarchar when driver has unicode enabled * Address comments from James and add checks * test SQL_ALL_TYPES in ODBC 2.x. * test SQL_TYPE_* in ODBC 2.x, the driver manager reports invalid data type error. * test SQL_* datetime in ODBC 3.x for backwards compatibility. * add check for valid/invalid data type. Fix segfault issue from empty metadata * Use empty map in bug fix * Address comments from James Implement SQLDescribeCol Add SQLError Tests * add test to free null handles. Without handle value initialization, segfault error was seen Move `SQLGetDiagField` and `SQLGetDiagRec` tests to `errors_test.cc` * Address comments from James * Add ODBC Ver 2 tests * update test name to indicate if error handling is from driver manager. * add tests for warnings. * fix lint errors. * remove SQL_ATTR_APP_ROW_DESC that is not applicable to Environment Attribute. Add tests for SQLGetFunctions * Using `SQL_FUNC_EXISTS` macro fixed the issue of `api_exists` not read correctly * Fix SQLGetTypeInfo naming for ODBC ver 2 tests * Add more tests Add unsupported API checks. Add `TestSQLGetFunctionsODBCVer2`. Add `TestSQLGetFunctionsCheckSingleAPI`. * Update reset value * Add SQLDescribeCol function check Add additional SQLDescribeCol test cases Add Descriptor support in SQLAllocHandle and SQLFreeHandle * Descriptor allocation initial impl * Add descriptor handle tests * Add diagnostic error test for descriptor handle * the error is from driver manager as our driver doesn't have descriptor-specific APIs implemented yet * Add SQLGetDescField test from driver manager * Add doxygen doc comment Fix connection issues servers that require data in handshake * revert back to passing empty string in handshake Fix bug of system trust store not loaded properly * Bug: system trust store has default value of "true" regardless of "encryption" value. * But if encryption is set to false, system trust store cannot be used, so system trust store should be saved as false if user does not enable encryption. * It may confuse the user if they see encryption is set to false but system trust store is set to true. In this case, the driver will not do system trust store verification. The DSN window should reflect this accurately. CPack ODBC Windows msi installer Add ARROW_FLIGHT_SQL_ODBC_INSTALLER environment variable to enable creation of Windows installer. * Use CPack + WiX to create a `msi` installer. * run command `cpack` under the build directory to generate the installer. Register ODBC Driver on Windows in installer * In `wxs`, cannot use `package`, need to use `fragment` instead * Use component as feature will automatically be generated * Set Patch version + add installer instructions Use Arrow logo banner * replaces default banner with Arrow logo banner Indicate Driver Company and Version on Driver Manager * use versioninfo.rc.in template * use `1 VERSIONINFO` for it to work properly * Set version variables * Add `.rc` to gitignore * Add `@` variables to rc template Return nameLengthPtr value as length in characters Allow spaces while parsing Table Type mac changes added changes to build in MacOS Enable ODBC build on Windows Glib & Ruby workflows * Fix `boost::get` issue on GLib workflow * Fix GLib build issue with conflicts Compile entry_points.cc before odbc_api.cc due to conflict from sql.h and flight/types.h. Fix build warnings treated as error on Windows SQLDescribeCol expects `bufCharLen` to be of type `SQLSMALLINT`, so build warnings occur when `bufCharLen` is set to `SQLINTEGER`. Add more types for Array Conversion Set column attribute SQL_DESC_TYPE_NAME [Refactor] Remove reference of `Configuration` in odbc_connection.cc Remove reference of `Configuration` in odbc_connection.cc which is in library `odbcabstraction`. The reasoning is that `Configuration` is in `arrow_odbc_spi_impl` and this is a library that `odbcabstraction` depends on. I moved the logic for reading DSN values into `odbc_api.cc` instead, since that is a place that will result in no conflicts. The ODBC driver now does not save the DSN value in the connection string if the DSN value is put after Driver value. * Address comments from James Simply logic to check for first key value pair only Add TODO for non-UTC time zone support * Currently, the driver only supports UTC time zone Replace `spdlogs` with Arrow's Internal Logging * Add logging README * register kernel function conditionally * Resolves errors of kernel function already registered Logging files are not supported since GLOG is not enabled on Windows in Arrow repo. We can work + test on log file support on macOS/Linux phase Remove `RUN_ALL_TESTS` as they are not needed Previously, tests on Windows were not being run due to https://github.com/apache/arrow/issues/47434. So we added `RUN_ALL_TESTS` as a workaround in early May, before https://github.com/apache/arrow/issues/47434 was raised. Now that https://github.com/apache/arrow/issues/47434 is resolved, we can remove `RUN_ALL_TESTS` as they are not needed. Other changes: * Renamed `arrow_odbc_spi_impl_test` to `odbc_spi_impl_test` so the test executable will be `arrow_odbc_spi_impl_test.exe` instead of `arrow_arrow_odbc_spi_impl_test.exe` Fix Function & Variable Naming Fix member variable naming Address code review comments for Arrow#40939 * Reword comment on null check Addresses comment https://github.com/apache/arrow/pull/40939#discussion_r2099120394 * Use `std::memcpy` instead of `memcpy` Addresses comment https://github.com/apache/arrow/pull/40939#discussion_r2099120764 * add include cstring * Remove warpdrive mentions * Fix lint * Address Justin's comments Combine utils and define utils namespace Use arrow::flight::sql::odbc namespace Update namespaces driver::flight_sql and driver::odbcabstraction to arrow::flight::sql::odbc. Fix seg fault from register log PR to keep apache-odbc in sync with arrow main branch. I found that, if the driver registers logs before compute kernel registration, segfault somehow occurs at when register kernel function is being used. I am not sure the root cause, but registering the logs after compute kernel registration seems to resolve the issue. Refactor odbcabstraction and odbc_impl flight_sql is renamed to odbc_impl. The files inside odbcabstraction have been moved to odbc_impl and odbcabstraction is removed. Replace boost::variant with std::variant * Remove `boost-optional` and `boost-variant` dependencies from vcpkg.json * Replace boost::variant with std::variant Fix EXPECT_EQ() to have expected on the left and actual on the right Code style naming convention Remove underscores in test names Sync arrow_flight_testing changes from Arrow::main branch Fix test failure due to typo Fix `FlightSQLODBCRemoteTestBase.TestSQLExecDirectDataQueryDefaultType` test failure due to typo The precision should be 38 in the original code. I think it got changed to 0 by accident somewhere. Disable tests with `DISABLED_` * Enable `TestSQLGetInfoDriverHdesc` We missed this test before, enabling the test now * Disable tests with `DISABLED_` Fix test assertions Define Test Fixtures * Connect/disconnect in SetUp/TearDown * Define Test Fixtures Resolve ODBC CI Build Errors * Fix `ConfigDSNW` not found error * Fix `TestFreeNullHandles` test `ConnectionTest` sets up the handles, so the `this->...` handles will not be null at beginning of test. Define New Base Test Fixtures Skip tests without connecting/disconnecting Address general test code review comments * Use RAII helper for allocating and freeing env/conn handles Avoids duplicated code Move test helper functions to anonymous namespace Add `[[nodiscard]]` for ODBC APIs Addresses community comment: https://github.com/apache/arrow/pull/47763/files#r2450014186 Remove `using List=` in test suite definitions Use static_cast for `SQLWCHAR` in type info test Use static_cast for `SQLWCHAR` in tables test * use mutable arrays for places where characters cannot be const Use static_cast for `SQLWCHAR` in columns test Update comment Use static_cast for `SQLWCHAR` in SQLGetInfo test * Address Justin's comments * Add fix for skipping the allocation of handles ^ Conflicts: ^ .github/workflows/cpp.yml ^ cpp/src/arrow/flight/sql/odbc/odbc_api.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/attribute_utils.h ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/config/configuration.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/config/configuration.h ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/encoding_utils.h ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection_test.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/main.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.h ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.h ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/util.cc ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h ^ cpp/src/arrow/flight/sql/odbc/tests/connection_test.cc ^ cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc Enable ODBC build in MacOS CI ODBC Standalone MSVC Build CI Remove DataSet Fix cache key Use odbc-specific cache key Fix Lint and Add odbc registration step Link appropriate GitHub issues that blocks ODBC tests Disable test phase and add schedule Run everyday 7 am Vancouver time Enable ODBC build on MSVC CI Code Clean up and enable ODBC tests in CI Still need to modify ODBCUtilEnvironment if we decide to use it. Still need to add ODBC V2 support so a different env and conn is used for ODBC 2 tests. Draft enable ODBC global setup/teardown Run ODBC test once outside of test script If ODBC test is run using MinGW Shell, segfault occurs. I am not getting any seg fault on my local MSVC Windows when I run the tests without the bash script. But if Windows CI breaks from running the standalone exe then I will look into this Prepend vcpkg to search vcpkg before `/lib/cmake` Since we are installing dependencies on vcpkg, if `lib/cmake` is searched first, then cmake will look into that directory and use the wrong paths. Convert VCPKG Windows path to MSYS path Set `VCPKG_ROOT` in test phase Fix ODBC dll name Use `arrow_flight_sql_odbc.dll` instead of `libarrow_flight_sql_odbc.dll` which is the naming convention used on MinGW Windows Link ODBC library on all platforms On my local MSVC Windows machine, Visual Studio is used to build without needing to link ` ${ODBCINST}` explicitly, its behavior might be different from Ninja which is what CI uses. `${ODBCINST}` is likely needed by Linux as well, so adding it for all platforms. Attempt to resolve `arrow-compute-grouper-benchmark` build issue I think `if(ARROW_FLIGHT_TEST_LINKAGE STREQUAL "static")` might be more appropriate here, as I am seeing build issues due to both dynamic and static linking occurring. I see in https://github.com/apache/arrow/commit/59903d089ec5b1766e1622d94c1f618b539b205d, this check is used for `arrow-filesystem-s3fs-benchmark` Disable `UNITY_BUILD` for ODBC test due to conflict on `sqlite_sql_info.cc` On some workflows, unity build is set to ON to make the build faster. This is an attempt to resolve the build issue. Remove debug messages Fix lint Add `sql_info_undef.h` to sqlite_sql_info.cc Undefine duplicates in SQLGetInfo Add `#pragma once` to odbc_test_suite.h To avoid redefinition error Attempt to resolve conflict with sql/types.h Attempt to include Arrow headers before ODBC headers to avoid conflict with arrow/flight/sql/types.h [GH-48084] Replace boost::optional with std::optional Addresses comment https://github.com/apache/arrow/pull/40939#discussion_r2099118497 Replace boost::optional with std::optional in ODBC codebase https://github.com/apache/arrow/issues/48084 Fix lint Remove `BOOST_SOURCE=BUNDLED` to use vcpkg's dynamic link to boost Since we need to use link to boost dynamically, we need to remove the bundled boost library flag. Add debug messages. Remove `ARROW_BOOST_USE_SHARED = OFF` Since we have a release build, can enable boost as shared. With `ARROW_BOOST_USE_SHARED = OFF`, I am getting error ``` D:\a\arrow\arrow\build\cpp\vcpkg_installed\x64-windows\include\boost/filesystem/config.hpp(96): fatal error C1189: #error: Must not define both BOOST_FILESYSTEM_DYN_LINK and BOOST_FILESYSTEM_STATIC_LINK ``` Also increased time limit, since 2 hours don't seem to be enough to finish the vcpkg build Change to release build Getting ``` orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in unity_2_cxx.cxx.obj orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in unity_2_cxx.cxx.obj ``` when building with debug and presumably ARROW_ORC Set ARROW_BOOST_USE_SHARED to OFF `ARROW_BOOST_USE_SHARED` is restored to `OFF`, which according to Windows doc should help with the debug build now. Retore value of `ARROW_BUILD_BENCHMARKS`, though noting it is set to `OFF` in GLib MSVC workflow. Add VCPKG set up Borrowed from the Glib workflow Add `/EHsc` flag * Add back `CMAKE_CXX_STANDARD: "17"` Remove `CMAKE_CXX_STANDARD` 17 * reason: gtest can't be used with C++17. Arrow project doesn't support C++ 17 yet. Extend run-time to 2hr windows-mingw also has timeout of 2hr Empty commit to trigger CI Specify `VCPKG_BINARY_SOURCES` and `VCPKG_DEFAULT_TRIPLET` Specify VCPKG_TARGET_TRIPLET as x64 windows Set ARROW_DEPENDENCY_SOURCE to VCPKG To make it easier to manage dependencies Enable static build on MSVC `ARROW_BUILD_BENCHMARKS` prevents Flight from being built Remove `ARROW_BOOST_USE_SHARED` Having static vs. shared issue Enable Flight & Flight SQL for ODBC Enable ODBC build on MSVC CI Enable regular ctest tests Remove undef items Add concurrency and permissions to odbc yml Create cpp_odbc.yml Fix architecture in CI * Add ODBC installer MSVC CI Mac Setup ODBC ini Script * added setup script for mac and platform folders updated read me windows file path updated windows folder related file paths added license Fix Lint issues Add install ODBC script Update install_odbc_ini.sh This script depends on the ODBC install script. Add TODO note and update DSN * Fix typo in README --------- Fix lint Fix lint Update flight_sql_stream_chunk_buffer.cc Update statement_test.cc Move MacOS ODBC workflows to cpp_odbc.yml Fix Lint Fix `ODBC::GetSqlWCharSize()` Add back missing checks for error handle tests SQLError implementation in macOS * SQLError implementation for macOS Disable `SQLGetConnectOption` on macOS. Still need to check if SQLGetConnectOption is needed on macOS Excel. Update type_info_test.cc Update columns_test.cc Remove unneeded component code - Removed unneeded code for `COMPONENT` - Use `install(PROGRAMS ${CMAKE_INSTALL_SYSTEM_RUNTIME_LIBS}` to include MSVC redistributable DLLs instead of picking them up with `unspecified` components. - Use better way to include ODBC dll files: replace DLL filtering with `install(TARGETS` Fix deadlock error from C++ 20 * Fix ODBC deadlock issue `absl::SetMutexDeadlockDetectionMode(absl::OnDeadlockCycle::kIgnore);` fixes ODBC deadlock issue. We turn off the detection as the deadlock originated from upstream projects. * Fix ODBC Extra slow CI issue Attempt to resolve caching issue with ODBC CI Add write permission to store vcpkg cache Check nuget paths in Ruby Glib and ODBC Run vcpkg install after vcpkg is set in GLib Enable vcpkg verbose in GLib * confirmed `vcpkg install` command returns 403 error Add ARROW_HOME Remove `ARROW_SIMD_LEVEL` as it is not required for ODBC. Remove debug msgs Cannot get vcpkg in cpp_build to show debug messages Move location of `packages: write` Attempt to add `ARROW_DEPENDENCY_USE_SHARED - off` Revert "Move location of `packages: write`" This reverts commit b62e04ed6d88296217bd8ed05d67ee2fb0b927c2. Revert "Attempt to add `ARROW_DEPENDENCY_USE_SHARED - off`" This reverts commit 5fdf4251df18a4f2c1b7c697f99fe91ef6e1b9aa. Add GLib MSVC build to C++ Extra TEMP disable ODBC build Disable cmake and enable build Re-enable cmake 4.1.2 and use install_vcpkg.sh to install vcpkg Clean up test changes Push after previous run is complete. Revert "Re-enable cmake 4.1.2 and use install_vcpkg.sh to install vcpkg" This reverts commit dfea3774de18f764ee6bcba8293d347e4e358813. Trigger CI Apply Kou's suggestion for nuget timeout Remove `C++ ODBC` workflow * missed item from rebase Remove changes added from rebase Co-Authored-By: Victor Tsang Co-Authored-By: Alina (Xi) Li Co-Authored-By: vic-tsang ^ Conflicts: ^ cpp/src/arrow/flight/sql/odbc/odbc_impl/config/configuration.h ^ cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt ^ cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.h --- .github/workflows/cpp_extra.yml | 3 +- .gitignore | 1 + ci/scripts/cpp_build.sh | 1 + .../sqlite_tables_schema_batch_reader.cc | 54 ++++++---- cpp/src/arrow/flight/sql/odbc/README | 83 ++++++++++++++ cpp/src/arrow/flight/sql/odbc/entry_points.cc | 12 +++ .../sql/odbc/install/mac/install_odbc.sh | 76 +++++++++++++ .../sql/odbc/install/mac/install_odbc_ini.sh | 82 ++++++++++++++ cpp/src/arrow/flight/sql/odbc/odbc_api.cc | 54 +++++++++- .../arrow/flight/sql/odbc/odbc_api_internal.h | 6 ++ .../accessors/binary_array_accessor.cc | 3 +- .../accessors/binary_array_accessor_test.cc | 7 +- .../accessors/boolean_array_accessor_test.cc | 5 +- .../sql/odbc/odbc_impl/accessors/common.h | 5 +- .../accessors/date_array_accessor_test.cc | 7 +- .../accessors/decimal_array_accessor_test.cc | 5 +- .../primitive_array_accessor_test.cc | 23 ++-- .../accessors/string_array_accessor.cc | 3 +- .../accessors/string_array_accessor_test.cc | 11 +- .../accessors/time_array_accessor_test.cc | 11 +- .../accessors/timestamp_array_accessor.cc | 10 +- .../timestamp_array_accessor_test.cc | 11 +- .../sql/odbc/odbc_impl/accessors/types.h | 2 +- .../flight/sql/odbc/odbc_impl/address_info.cc | 5 +- .../flight/sql/odbc/odbc_impl/address_info.h | 5 +- .../sql/odbc/odbc_impl/attribute_utils.h | 22 ++-- .../sql/odbc/odbc_impl/calendar_utils.cc | 3 + .../odbc/odbc_impl/config/configuration.cc | 2 +- .../sql/odbc/odbc_impl/encoding_utils.h | 26 ++--- .../odbc/odbc_impl/flight_sql_auth_method.cc | 28 +++-- .../odbc/odbc_impl/flight_sql_connection.cc | 13 +-- .../odbc/odbc_impl/flight_sql_connection.h | 2 +- .../odbc_impl/flight_sql_connection_test.cc | 18 ++-- .../odbc_impl/flight_sql_get_tables_reader.cc | 6 +- .../odbc_impl/flight_sql_result_set_column.h | 1 + .../flight_sql_result_set_metadata.cc | 16 ++- .../odbc/odbc_impl/flight_sql_ssl_config.h | 4 +- .../odbc/odbc_impl/flight_sql_statement.cc | 4 +- .../flight_sql_statement_get_columns.cc | 13 +-- .../flight_sql_statement_get_tables.h | 1 - .../flight_sql_stream_chunk_buffer.cc | 14 ++- .../sql/odbc/odbc_impl/get_info_cache.h | 5 +- .../sql/odbc/odbc_impl/json_converter.cc | 2 +- .../sql/odbc/odbc_impl/json_converter.h | 2 +- .../sql/odbc/odbc_impl/json_converter_test.cc | 5 +- .../sql/odbc/odbc_impl/odbc_connection.cc | 23 ++-- .../sql/odbc/odbc_impl/odbc_connection.h | 12 ++- .../sql/odbc/odbc_impl/odbc_descriptor.cc | 14 +-- .../sql/odbc/odbc_impl/odbc_descriptor.h | 6 +- .../flight/sql/odbc/odbc_impl/odbc_handle.h | 5 +- .../sql/odbc/odbc_impl/odbc_statement.cc | 11 +- .../sql/odbc/odbc_impl/odbc_statement.h | 5 +- .../odbc/odbc_impl/parse_table_types_test.cc | 3 +- .../odbc/odbc_impl/record_batch_transformer.h | 4 +- .../record_batch_transformer_test.cc | 3 +- .../odbc/odbc_impl/scalar_function_reporter.h | 2 +- .../sql/odbc/odbc_impl/spi/connection.h | 5 +- .../flight/sql/odbc/odbc_impl/spi/statement.h | 4 +- .../sql/odbc/odbc_impl/ui/custom_window.cc | 1 + .../odbc_impl/ui/dsn_configuration_window.cc | 6 +- .../flight/sql/odbc/odbc_impl/ui/window.cc | 1 - .../arrow/flight/sql/odbc/odbc_impl/util.cc | 19 ++-- .../arrow/flight/sql/odbc/odbc_impl/util.h | 14 +-- .../flight/sql/odbc/odbc_impl/util_test.cc | 11 +- .../flight/sql/odbc/tests/CMakeLists.txt | 8 ++ cpp/src/arrow/flight/sql/odbc/tests/README | 23 ++++ .../flight/sql/odbc/tests/columns_test.cc | 46 +++++++- .../sql/odbc/tests/connection_attr_test.cc | 18 +++- .../sql/odbc/tests/connection_info_test.cc | 10 +- .../sql/odbc/tests/dremio/docker-compose.yml | 35 ++++++ .../tests/dremio/set_up_dremio_instance.sh | 66 ++++++++++++ .../flight/sql/odbc/tests/errors_test.cc | 101 ++++++++++++------ .../sql/odbc/tests/get_functions_test.cc | 9 +- .../flight/sql/odbc/tests/odbc_test_suite.h | 9 ++ .../sql/odbc/tests/statement_attr_test.cc | 42 +++++--- .../flight/sql/odbc/tests/statement_test.cc | 81 ++++++++++++-- .../flight/sql/odbc/tests/type_info_test.cc | 12 +++ cpp/vcpkg.json | 2 - 78 files changed, 993 insertions(+), 290 deletions(-) create mode 100644 cpp/src/arrow/flight/sql/odbc/README create mode 100644 cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh create mode 100644 cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh create mode 100644 cpp/src/arrow/flight/sql/odbc/tests/README create mode 100644 cpp/src/arrow/flight/sql/odbc/tests/dremio/docker-compose.yml create mode 100644 cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh diff --git a/.github/workflows/cpp_extra.yml b/.github/workflows/cpp_extra.yml index b38ccaa27795..ee4fa696ac84 100644 --- a/.github/workflows/cpp_extra.yml +++ b/.github/workflows/cpp_extra.yml @@ -410,7 +410,8 @@ jobs: if: >- needs.check-labels.outputs.force == 'true' || contains(fromJSON(needs.check-labels.outputs.ci-extra-labels || '[]'), 'CI: Extra') || - contains(fromJSON(needs.check-labels.outputs.ci-extra-labels || '[]'), 'CI: Extra: C++') + contains(fromJSON(needs.check-labels.outputs.ci-extra-labels || '[]'), 'CI: Extra: C++') || + contains(join(github.event.pull_request.changed_files, ' '), 'cpp/src/arrow/flight/sql/odbc/') timeout-minutes: 240 permissions: packages: write diff --git a/.gitignore b/.gitignore index 83458ab2057b..c7690ec2637f 100644 --- a/.gitignore +++ b/.gitignore @@ -32,6 +32,7 @@ dependency-reduced-pom.xml MANIFEST compile_commands.json build.ninja +build*/ # Generated Visual Studio files *.vcxproj diff --git a/ci/scripts/cpp_build.sh b/ci/scripts/cpp_build.sh index 79b64dbc2a47..4738d0306149 100755 --- a/ci/scripts/cpp_build.sh +++ b/ci/scripts/cpp_build.sh @@ -272,6 +272,7 @@ else -DgRPC_SOURCE=${gRPC_SOURCE:-} \ -DGTest_SOURCE=${GTest_SOURCE:-} \ -Dlz4_SOURCE=${lz4_SOURCE:-} \ + -DODBC_INCLUDE_DIR="${ODBC_INCLUDE_DIR:-}" \ -Dopentelemetry-cpp_SOURCE=${opentelemetry_cpp_SOURCE:-} \ -DORC_SOURCE=${ORC_SOURCE:-} \ -DPARQUET_BUILD_EXAMPLES=${PARQUET_BUILD_EXAMPLES:-OFF} \ diff --git a/cpp/src/arrow/flight/sql/example/sqlite_tables_schema_batch_reader.cc b/cpp/src/arrow/flight/sql/example/sqlite_tables_schema_batch_reader.cc index 85332e6c4dfc..674377f6b58b 100644 --- a/cpp/src/arrow/flight/sql/example/sqlite_tables_schema_batch_reader.cc +++ b/cpp/src/arrow/flight/sql/example/sqlite_tables_schema_batch_reader.cc @@ -65,33 +65,43 @@ Status SqliteTablesWithSchemaBatchReader::ReadNext(std::shared_ptr* auto* string_array = reinterpret_cast(table_name_array.get()); - std::vector> column_fields; + std::map>> table_columns_map; for (int i = 0; i < table_name_array->length(); i++) { const std::string& table_name = string_array->GetString(i); + table_columns_map[table_name]; + } - while (sqlite3_step(schema_statement->GetSqlite3Stmt()) == SQLITE_ROW) { - std::string sqlite_table_name = std::string(reinterpret_cast( - sqlite3_column_text(schema_statement->GetSqlite3Stmt(), 0))); - if (sqlite_table_name == table_name) { - const char* column_name = reinterpret_cast( - sqlite3_column_text(schema_statement->GetSqlite3Stmt(), 1)); - const char* column_type = reinterpret_cast( - sqlite3_column_text(schema_statement->GetSqlite3Stmt(), 2)); - int nullable = sqlite3_column_int(schema_statement->GetSqlite3Stmt(), 3); - - const ColumnMetadata& column_metadata = GetColumnMetadata( - GetSqlTypeFromTypeName(column_type), sqlite_table_name.c_str()); - std::shared_ptr arrow_type; - auto status = GetArrowType(column_type).Value(&arrow_type); - if (!status.ok()) { - return Status::NotImplemented("Unknown SQLite type '", column_type, - "' for column '", column_name, "' in table '", - table_name, "': ", status); - } - column_fields.push_back(arrow::field(column_name, arrow_type, nullable == 0, - column_metadata.metadata_map())); + while (sqlite3_step(schema_statement->GetSqlite3Stmt()) == SQLITE_ROW) { + std::string table_name = std::string(reinterpret_cast( + sqlite3_column_text(schema_statement->GetSqlite3Stmt(), 0))); + + if (table_columns_map.contains(table_name)) { + const char* column_name = reinterpret_cast( + sqlite3_column_text(schema_statement->GetSqlite3Stmt(), 1)); + const char* column_type = reinterpret_cast( + sqlite3_column_text(schema_statement->GetSqlite3Stmt(), 2)); + int nullable = sqlite3_column_int(schema_statement->GetSqlite3Stmt(), 3); + + const ColumnMetadata& column_metadata = + GetColumnMetadata(GetSqlTypeFromTypeName(column_type), table_name.c_str()); + + std::shared_ptr arrow_type; + auto status = GetArrowType(column_type).Value(&arrow_type); + if (!status.ok()) { + return Status::NotImplemented("Unknown SQLite type '", column_type, + "' for column '", column_name, "' in table '", + table_name, "': ", status); } + table_columns_map[table_name].push_back(arrow::field( + column_name, arrow_type, nullable == 0, column_metadata.metadata_map())); } + } + + std::vector> column_fields; + for (int i = 0; i < table_name_array->length(); i++) { + const std::string& table_name = string_array->GetString(i); + column_fields = table_columns_map[table_name]; + ARROW_ASSIGN_OR_RAISE(std::shared_ptr schema_buffer, ipc::SerializeSchema(*arrow::schema(column_fields))); diff --git a/cpp/src/arrow/flight/sql/odbc/README b/cpp/src/arrow/flight/sql/odbc/README new file mode 100644 index 000000000000..120a92950f9b --- /dev/null +++ b/cpp/src/arrow/flight/sql/odbc/README @@ -0,0 +1,83 @@ + + +## Steps to Register the 64-bit Apache Arrow ODBC driver on Windows + +After the build succeeds, the ODBC DLL will be located in +`build\debug\Debug` for a debug build and `build\release\Release` for a release build. + +1. Open Power Shell as administrator. + +2. Register your ODBC DLL: + Need to replace with actual path to repository in the commands. + + i. `cd to repo.` + ii. `cd ` + iii. Run script to register your ODBC DLL as Apache Arrow Flight SQL ODBC Driver + `.\cpp\src\arrow\flight\sql\odbc\tests\install_odbc.cmd \cpp\build\< release | debug >\< Release | Debug>\arrow_flight_sql_odbc.dll` + Example command for reference: + `.\cpp\src\arrow\flight\sql\odbc\tests\install_odbc.cmd C:\path\to\arrow\cpp\build\release\Release\arrow_flight_sql_odbc.dll` + +If the registration is successful, then Apache Arrow Flight SQL ODBC Driver +should show as an available ODBC driver in the x64 ODBC Driver Manager. + +## Steps to Generate Windows Installer +1. Build with `ARROW_FLIGHT_SQL_ODBC=ON` and `ARROW_FLIGHT_SQL_ODBC_INSTALLER=ON`. +2. `cd` to `build` folder. +3. Run `cpack`. + +If the generation is successful, you will find `Apache Arrow Flight SQL ODBC--win64.msi` generated under the `build` folder. + +## Steps to Register the 64-bit Apache Arrow ODBC driver on macOS + +After the build succeeds, the ODBC DYLIB will be located in +`build\debug` for a debug build and `build\release` for a release build. + +1. Open terminal shell. + +2. Register your ODBC DYLIB: + Need to replace with actual path to repository in the commands. + + i. `cd to repo.` + ii. `cd ` + iii. Give script permission to execute + `chmod +x cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh` + iv. Run script with `sudo` to register your ODBC DYLIB as Apache Arrow Flight SQL ODBC Driver + `sudo cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh /cpp/build/< release | debug >/libarrow_flight_sql_odbc.dylib` + Example command for reference: + `sudo cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh /path/to/arrow/cpp/build/release/libarrow_flight_sql_odbc.dylib` + +If the registration is successful, then Apache Arrow Flight SQL ODBC Driver +should be shown at `~/Library/ODBC/odbcinst.ini` + +## Steps to Enable Logging +Arrow Flight SQL ODBC driver uses Arrow's internal logging framework. By default, the log messages are printed to the terminal. +1. Set environment variable `ARROW_ODBC_LOG_LEVEL` to any of the following valid values to enable logging. If `ARROW_ODBC_LOG_LEVEL` is set to a non-empty string that does not match any of the following values, `DEBUG` level is used by default. + +The characters are case-insensitive. +- TRACE +- DEBUG +- INFO +- WARNING +- ERROR +- FATAL + +The Windows ODBC driver currently does not support writing log files. `ARROW_USE_GLOG` is required to write log files, and `ARROW_USE_GLOG` is disabled on Windows platform since plasma using `glog` is not fully tested on windows. + +Note: GH-47670 running more than 1 tests with logging enabled is not fully supported. diff --git a/cpp/src/arrow/flight/sql/odbc/entry_points.cc b/cpp/src/arrow/flight/sql/odbc/entry_points.cc index 8e7c2e2be713..c03089ffcc6d 100644 --- a/cpp/src/arrow/flight/sql/odbc/entry_points.cc +++ b/cpp/src/arrow/flight/sql/odbc/entry_points.cc @@ -83,6 +83,18 @@ SQLRETURN SQL_API SQLGetDiagRec(SQLSMALLINT handle_type, SQLHANDLE handle, buffer_length, text_length_ptr); } +#if defined(__APPLE__) +// macOS ODBC Driver Manager doesn't map SQLError to SQLGetDiagRec, so we need to +// implement SQLError for macOS. +// on Windows, SQLError mapping implemented by Driver Manager is preferred. +SQLRETURN SQL_API SQLError(SQLHENV env, SQLHDBC conn, SQLHSTMT stmt, SQLWCHAR* sql_state, + SQLINTEGER* native_error_ptr, SQLWCHAR* message_text, + SQLSMALLINT buffer_length, SQLSMALLINT* text_length_ptr) { + return arrow::flight::sql::odbc::SQLError(env, conn, stmt, sql_state, native_error_ptr, + message_text, buffer_length, text_length_ptr); +} +#endif // __APPLE__ + SQLRETURN SQL_API SQLGetEnvAttr(SQLHENV env, SQLINTEGER attr, SQLPOINTER value_ptr, SQLINTEGER buffer_len, SQLINTEGER* str_len_ptr) { return arrow::flight::sql::odbc::SQLGetEnvAttr(env, attr, value_ptr, buffer_len, diff --git a/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh b/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh new file mode 100644 index 000000000000..0916bff0845a --- /dev/null +++ b/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh @@ -0,0 +1,76 @@ +#!/bin/sh +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Used by macOS ODBC installer script and macOS ODBC testing + +ODBC_64BIT="$1" + +if [[ -z "$ODBC_64BIT" ]]; then + echo "error: 64-bit driver is not specified. Call format: install_odbc abs_path_to_64_bit_driver" + exit 1 +fi + +if [ ! -f $ODBC_64BIT ]; then + echo "64-bit driver can not be found: $ODBC_64BIT" + echo "Call format: install_odbc abs_path_to_64_bit_driver" + exit 1 +fi + +USER_ODBCINST_FILE="$HOME/Library/ODBC/odbcinst.ini" +DRIVER_NAME="Apache Arrow Flight SQL ODBC Driver" +DSN_NAME="Apache Arrow Flight SQL ODBC DSN" + +mkdir -p $HOME/Library/ODBC + +touch "$USER_ODBCINST_FILE" + +# Admin privilege is needed to add ODBC driver registration +if [ $EUID -ne 0 ]; then + echo "Please run this script with sudo" + exit 1 +fi + +if grep -q "^\[$DRIVER_NAME\]" "$USER_ODBCINST_FILE"; then + echo "Driver [$DRIVER_NAME] already exists in odbcinst.ini" +else + echo "Adding [$DRIVER_NAME] to odbcinst.ini..." + echo " +[$DRIVER_NAME] +Description=An ODBC Driver for Apache Arrow Flight SQL +Driver=$ODBC_64BIT +" >> "$USER_ODBCINST_FILE" +fi + +# Check if [ODBC Drivers] section exists +if grep -q '^\[ODBC Drivers\]' "$USER_ODBCINST_FILE"; then + # Section exists: check if driver entry exists + if ! grep -q "^${DRIVER_NAME}=" "$USER_ODBCINST_FILE"; then + # Driver entry does not exist, add under [ODBC Drivers] + sed -i '' "/^\[ODBC Drivers\]/a\\ +${DRIVER_NAME}=Installed +" "$USER_ODBCINST_FILE" + fi +else + # Section doesn't exist, append both section and driver entry at end + { + echo "" + echo "[ODBC Drivers]" + echo "${DRIVER_NAME}=Installed" + } >> "$USER_ODBCINST_FILE" +fi diff --git a/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh b/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh new file mode 100644 index 000000000000..cfc9729ed7e4 --- /dev/null +++ b/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh @@ -0,0 +1,82 @@ +#!/bin/sh +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# GH-47876 TODO: create macOS ODBC Installer. +# Script for installing macOS ODBC driver, to be used for macOS installer. + +source_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +odbc_install_script="${source_dir}/install_odbc.sh" + +chmod +x "$odbc_install_script" +. "$odbc_install_script" /Library/Apache/ArrowFlightSQLODBC/lib/libarrow_flight_sql_odbc.dylib + +USER_ODBC_FILE="$HOME/Library/ODBC/odbc.ini" +DRIVER_NAME="Apache Arrow Flight SQL ODBC Driver" +DSN_NAME="Apache Arrow Flight SQL ODBC DSN" + +touch "$USER_ODBC_FILE" + +if [ $EUID -ne 0 ]; then + echo "Please run this script with sudo" + exit 1 +fi + +if grep -q "^\[$DSN_NAME\]" "$USER_ODBC_FILE"; then + echo "DSN [$DSN_NAME] already exists in $USER_ODBC_FILE" +else + echo "Adding [$DSN_NAME] to $USER_ODBC_FILE..." + cat >> "$USER_ODBC_FILE" < "${USER_ODBC_FILE}.tmp" && mv "${USER_ODBC_FILE}.tmp" "$USER_ODBC_FILE" + fi +else + # Section doesn't exist, append section and DSN entry at end + { + echo "" + echo "[ODBC Data Sources]" + echo "${DSN_NAME}=${DRIVER_NAME}" + } >> "$USER_ODBC_FILE" +fi + diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_api.cc b/cpp/src/arrow/flight/sql/odbc/odbc_api.cc index 5676b9b05ed9..d0451a551e56 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_api.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_api.cc @@ -247,6 +247,56 @@ SQLRETURN SQLFreeStmt(SQLHSTMT handle, SQLUSMALLINT option) { return SQL_ERROR; } +#if defined(__APPLE__) +SQLRETURN SQLError(SQLHENV env, SQLHDBC conn, SQLHSTMT stmt, SQLWCHAR* sql_state, + SQLINTEGER* native_error_ptr, SQLWCHAR* message_text, + SQLSMALLINT buffer_length, SQLSMALLINT* text_length_ptr) { + ARROW_LOG(DEBUG) << "SQLError called with env: " << env << ", conn: " << conn + << ", stmt: " << stmt + << ", sql_state: " << static_cast(sql_state) + << ", native_error_ptr: " << static_cast(native_error_ptr) + << ", message_text: " << static_cast(message_text) + << ", buffer_length: " << buffer_length + << ", text_length_ptr: " << static_cast(text_length_ptr); + + SQLSMALLINT handle_type; + SQLHANDLE handle; + + if (env) { + handle_type = SQL_HANDLE_ENV; + handle = static_cast(env); + } else if (conn) { + handle_type = SQL_HANDLE_DBC; + handle = static_cast(conn); + } else if (stmt) { + handle_type = SQL_HANDLE_STMT; + handle = static_cast(stmt); + } else { + return static_cast(SQL_INVALID_HANDLE); + } + + // Use the last record + SQLINTEGER diag_number; + SQLSMALLINT diag_number_length; + + SQLRETURN ret = arrow::flight::sql::odbc::SQLGetDiagField( + handle_type, handle, 0, SQL_DIAG_NUMBER, &diag_number, sizeof(SQLINTEGER), 0); + if (ret != SQL_SUCCESS) { + return ret; + } + + if (diag_number == 0) { + return SQL_NO_DATA; + } + + SQLSMALLINT rec_number = static_cast(diag_number); + + return arrow::flight::sql::odbc::SQLGetDiagRec( + handle_type, handle, rec_number, sql_state, native_error_ptr, message_text, + buffer_length, text_length_ptr); +} +#endif // __APPLE__ + inline bool IsValidStringFieldArgs(SQLPOINTER diag_info_ptr, SQLSMALLINT buffer_length, SQLSMALLINT* string_length_ptr, bool is_unicode) { const SQLSMALLINT char_size = is_unicode ? GetSqlWCharSize() : sizeof(char); @@ -536,7 +586,6 @@ SQLRETURN SQLGetDiagRec(SQLSMALLINT handle_type, SQLHANDLE handle, SQLSMALLINT r << ", message_text: " << static_cast(message_text) << ", buffer_length: " << buffer_length << ", text_length_ptr: " << static_cast(text_length_ptr); - using arrow::flight::sql::odbc::Diagnostics; using ODBC::GetStringAttribute; using ODBC::ODBCConnection; using ODBC::ODBCDescriptor; @@ -736,7 +785,6 @@ SQLRETURN SQLGetConnectAttr(SQLHDBC conn, SQLINTEGER attribute, SQLPOINTER value << ", attribute: " << attribute << ", value_ptr: " << value_ptr << ", buffer_length: " << buffer_length << ", string_length_ptr: " << static_cast(string_length_ptr); - using ODBC::ODBCConnection; return ODBCConnection::ExecuteWithDiagnostics(conn, SQL_ERROR, [=]() { @@ -1384,7 +1432,7 @@ SQLRETURN SQLColAttribute(SQLHSTMT stmt, SQLUSMALLINT record_number, } SQLRETURN SQLGetTypeInfo(SQLHSTMT stmt, SQLSMALLINT data_type) { - // GH-47237 TODO: return SQL_PRED_CHAR and SQL_PRED_BASIC for + // GH-47237 return SQL_PRED_CHAR and SQL_PRED_BASIC for // appropriate data types in `SEARCHABLE` field ARROW_LOG(DEBUG) << "SQLGetTypeInfoW called with stmt: " << stmt << " data_type: " << data_type; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_api_internal.h b/cpp/src/arrow/flight/sql/odbc/odbc_api_internal.h index 4fea8569acb5..f9d8d887cb87 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_api_internal.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_api_internal.h @@ -31,6 +31,12 @@ namespace arrow::flight::sql::odbc { SQLHANDLE* result); [[nodiscard]] SQLRETURN SQLFreeHandle(SQLSMALLINT type, SQLHANDLE handle); [[nodiscard]] SQLRETURN SQLFreeStmt(SQLHSTMT stmt, SQLUSMALLINT option); +#if defined(__APPLE__) +[[nodiscard]] SQLRETURN SQLError(SQLHENV env, SQLHDBC conn, SQLHSTMT stmt, + SQLWCHAR* sql_state, SQLINTEGER* native_error_ptr, + SQLWCHAR* message_text, SQLSMALLINT buffer_length, + SQLSMALLINT* text_length_ptr); +#endif // __APPLE__ [[nodiscard]] SQLRETURN SQLGetDiagField(SQLSMALLINT handle_type, SQLHANDLE handle, SQLSMALLINT rec_number, SQLSMALLINT diag_identifier, diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor.cc index e4625ace370f..939264bedb7d 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor.cc @@ -19,6 +19,7 @@ #include #include +#include #include "arrow/array.h" namespace arrow::flight::sql::odbc { @@ -39,7 +40,7 @@ inline RowStatus MoveSingleCellToBinaryBuffer(ColumnBinding* binding, BinaryArra auto* byte_buffer = static_cast(binding->buffer) + i * binding->buffer_length; - memcpy(byte_buffer, ((char*)value) + value_offset, value_length); + std::memcpy(byte_buffer, ((char*)value) + value_offset, value_length); if (remaining_length > binding->buffer_length) { result = RowStatus_SUCCESS_WITH_INFO; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor_test.cc index 39d03692da6c..423870eb3bed 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor_test.cc @@ -18,11 +18,12 @@ #include "arrow/flight/sql/odbc/odbc_impl/accessors/binary_array_accessor.h" #include "arrow/testing/builder.h" #include "arrow/testing/gtest_util.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { -TEST(BinaryArrayAccessor, Test_CDataType_BINARY_Basic) { +TEST(BinaryArrayAccessor, TestCDataTypeBinaryBasic) { std::vector values = {"foo", "barx", "baz123"}; std::shared_ptr array; ArrayFromVector(values, &array); @@ -53,7 +54,7 @@ TEST(BinaryArrayAccessor, Test_CDataType_BINARY_Basic) { } } -TEST(BinaryArrayAccessor, Test_CDataType_BINARY_Truncation) { +TEST(BinaryArrayAccessor, TestCDataTypeBinaryTruncation) { std::vector values = {"ABCDEFABCDEFABCDEFABCDEFABCDEFABCDEFABCDEF"}; std::shared_ptr array; ArrayFromVector(values, &array); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/boolean_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/boolean_array_accessor_test.cc index 9b17a9045981..b3f402dd7c1f 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/boolean_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/boolean_array_accessor_test.cc @@ -17,11 +17,12 @@ #include "arrow/flight/sql/odbc/odbc_impl/accessors/boolean_array_accessor.h" #include "arrow/testing/builder.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { -TEST(BooleanArrayFlightSqlAccessor, Test_BooleanArray_CDataType_BIT) { +TEST(BooleanArrayFlightSqlAccessor, TestBooleanArrayCDataTypeBit) { const std::vector values = {true, false, true}; std::shared_ptr array; ArrayFromVector(values, &array); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/common.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/common.h index 0a79bc39dfb8..45f88b50fb8c 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/common.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/common.h @@ -19,6 +19,7 @@ #include #include +#include #include "arrow/array.h" #include "arrow/flight/sql/odbc/odbc_impl/accessors/types.h" #include "arrow/flight/sql/odbc/odbc_impl/diagnostics.h" @@ -42,7 +43,7 @@ inline size_t CopyFromArrayValuesToBinding(ARRAY_TYPE* array, ColumnBinding* bin } } } else { - // Duplicate this loop to avoid null checks within the loop. + // Duplicate above for-loop to exit early when null value is found for (int64_t i = starting_row; i < starting_row + cells; ++i) { if (array->IsNull(i)) { throw NullWithoutIndicatorException(); @@ -54,7 +55,7 @@ inline size_t CopyFromArrayValuesToBinding(ARRAY_TYPE* array, ColumnBinding* bin // Note that the array should already have been sliced down to the same number // of elements in the ODBC data array by the point in which this function is called. const auto* values = array->raw_values(); - memcpy(binding->buffer, &values[starting_row], element_size * cells); + std::memcpy(binding->buffer, &values[starting_row], element_size * cells); return cells; } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/date_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/date_array_accessor_test.cc index 03716e2477a4..a482a838101c 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/date_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/date_array_accessor_test.cc @@ -20,11 +20,12 @@ #include "arrow/flight/sql/odbc/odbc_impl/accessors/boolean_array_accessor.h" #include "arrow/flight/sql/odbc/odbc_impl/calendar_utils.h" #include "arrow/testing/builder.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { -TEST(DateArrayAccessor, Test_Date32Array_CDataType_DATE) { +TEST(DateArrayAccessor, TestDate32ArrayCDataTypeDate) { std::vector values = {7589, 12320, 18980, 19095, -1, 0}; std::vector expected = { {1990, 10, 12}, {2003, 9, 25}, {2021, 12, 19}, @@ -57,7 +58,7 @@ TEST(DateArrayAccessor, Test_Date32Array_CDataType_DATE) { } } -TEST(DateArrayAccessor, Test_Date64Array_CDataType_DATE) { +TEST(DateArrayAccessor, TestDate64ArrayCDataTypeDate) { std::vector values = { 86400000, 172800000, 259200000, 1649793238110, 0, 345600000, 432000000, 518400000, -86400000, -17987443200000, -24268068949000}; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/decimal_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/decimal_array_accessor_test.cc index b2eb9450c2fa..6664b2d6e60a 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/decimal_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/decimal_array_accessor_test.cc @@ -19,7 +19,8 @@ #include "arrow/builder.h" #include "arrow/testing/builder.h" #include "arrow/util/decimal.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { namespace { @@ -93,7 +94,7 @@ void AssertNumericOutput(int input_precision, int input_scale, } } -TEST(DecimalArrayFlightSqlAccessor, Test_Decimal128Array_CDataType_NUMERIC_SameScale) { +TEST(DecimalArrayFlightSqlAccessor, TestDecimal128ArrayCDataTypeNumericSameScale) { const std::vector& input_values = {"25.212", "-25.212", "-123456789.123", "123456789.123"}; const std::vector& output_values = diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/primitive_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/primitive_array_accessor_test.cc index 2f04b7324a56..a5ce05fb717c 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/primitive_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/primitive_array_accessor_test.cc @@ -19,7 +19,8 @@ #include "arrow/flight/sql/odbc/odbc_impl/diagnostics.h" #include "arrow/testing/builder.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { @@ -52,43 +53,43 @@ void TestPrimitiveArraySqlAccessor() { } } -TEST(PrimitiveArrayFlightSqlAccessor, Test_Int64Array_CDataType_SBIGINT) { +TEST(PrimitiveArrayFlightSqlAccessor, TestInt64ArrayCDataTypeSbigint) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_Int32Array_CDataType_SLONG) { +TEST(PrimitiveArrayFlightSqlAccessor, TestInt32ArrayCDataTypeSlong) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_Int16Array_CDataType_SSHORT) { +TEST(PrimitiveArrayFlightSqlAccessor, TestInt16ArrayCDataTypeSshort) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_Int8Array_CDataType_STINYINT) { +TEST(PrimitiveArrayFlightSqlAccessor, TestInt8ArrayCDataTypeStinyint) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_UInt64Array_CDataType_UBIGINT) { +TEST(PrimitiveArrayFlightSqlAccessor, TestUInt64ArrayCDataTypeUbigint) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_UInt32Array_CDataType_ULONG) { +TEST(PrimitiveArrayFlightSqlAccessor, TestUInt32ArrayCDataTypeUlong) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_UInt16Array_CDataType_USHORT) { +TEST(PrimitiveArrayFlightSqlAccessor, TestUInt16ArrayCDataTypeUshort) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_UInt8Array_CDataType_UTINYINT) { +TEST(PrimitiveArrayFlightSqlAccessor, TestUInt8ArrayCDataTypeUtinyint) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_FloatArray_CDataType_FLOAT) { +TEST(PrimitiveArrayFlightSqlAccessor, TestFloatArrayCDataTypeFloat) { TestPrimitiveArraySqlAccessor(); } -TEST(PrimitiveArrayFlightSqlAccessor, Test_DoubleArray_CDataType_DOUBLE) { +TEST(PrimitiveArrayFlightSqlAccessor, TestDoubleArrayCDataTypeDouble) { TestPrimitiveArraySqlAccessor(); } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor.cc index 69b3b3049454..441b2a3394e7 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor.cc @@ -18,6 +18,7 @@ #include "arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor.h" #include +#include #include "arrow/array.h" #include "arrow/flight/sql/odbc/odbc_impl/encoding.h" @@ -79,7 +80,7 @@ inline RowStatus MoveSingleCellToCharBuffer( auto* byte_buffer = static_cast(binding->buffer) + i * binding->buffer_length; auto* char_buffer = (CHAR_TYPE*)byte_buffer; - memcpy(char_buffer, ((char*)value) + value_offset, value_length); + std::memcpy(char_buffer, ((char*)value) + value_offset, value_length); // Write a NUL terminator if (binding->buffer_length >= remaining_length + sizeof(CHAR_TYPE)) { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor_test.cc index eb7a9c88b3b3..4d0e13934072 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/string_array_accessor_test.cc @@ -19,11 +19,12 @@ #include "arrow/flight/sql/odbc/odbc_impl/encoding.h" #include "arrow/testing/builder.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { -TEST(StringArrayAccessor, Test_CDataType_CHAR_Basic) { +TEST(StringArrayAccessor, TestCDataTypeCharBasic) { std::vector values = {"foo", "barx", "baz123"}; std::shared_ptr array; ArrayFromVector(values, &array); @@ -49,7 +50,7 @@ TEST(StringArrayAccessor, Test_CDataType_CHAR_Basic) { } } -TEST(StringArrayAccessor, Test_CDataType_CHAR_Truncation) { +TEST(StringArrayAccessor, TestCDataTypeCharTruncation) { std::vector values = {"ABCDEFABCDEFABCDEFABCDEFABCDEFABCDEFABCDEF"}; std::shared_ptr array; ArrayFromVector(values, &array); @@ -82,7 +83,7 @@ TEST(StringArrayAccessor, Test_CDataType_CHAR_Truncation) { ASSERT_EQ(values[0], ss.str()); } -TEST(StringArrayAccessor, Test_CDataType_WCHAR_Basic) { +TEST(StringArrayAccessor, TestCDataTypeWcharBasic) { std::vector values = {"foo", "barx", "baz123"}; std::shared_ptr array; ArrayFromVector(values, &array); @@ -112,7 +113,7 @@ TEST(StringArrayAccessor, Test_CDataType_WCHAR_Basic) { } } -TEST(StringArrayAccessor, Test_CDataType_WCHAR_Truncation) { +TEST(StringArrayAccessor, TestCDataTypeWcharTruncation) { std::vector values = {"ABCDEFA"}; std::shared_ptr array; ArrayFromVector(values, &array); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/time_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/time_array_accessor_test.cc index eb49e4078cd8..41bd0d73ea77 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/time_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/time_array_accessor_test.cc @@ -20,11 +20,12 @@ #include "arrow/flight/sql/odbc/odbc_impl/calendar_utils.h" #include "arrow/flight/sql/odbc/odbc_impl/util.h" #include "arrow/testing/builder.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { -TEST(TEST_TIME32, TIME_WITH_SECONDS) { +TEST(TestTime32, TimeWithSeconds) { auto value_field = field("f0", time32(TimeUnit::SECOND)); std::vector t32_values = {14896, 14897, 14892, 85400, 14893, 14895}; @@ -58,7 +59,7 @@ TEST(TEST_TIME32, TIME_WITH_SECONDS) { } } -TEST(TEST_TIME32, TIME_WITH_MILLI) { +TEST(TestTime32, TimeWithMilli) { auto value_field = field("f0", time32(TimeUnit::MILLI)); std::vector t32_values = {14896000, 14897000, 14892000, 85400000, 14893000, 14895000}; @@ -94,7 +95,7 @@ TEST(TEST_TIME32, TIME_WITH_MILLI) { } } -TEST(TEST_TIME64, TIME_WITH_MICRO) { +TEST(TestTime32, TimeWithMicro) { auto value_field = field("f0", time64(TimeUnit::MICRO)); std::vector t64_values = {14896000, 14897000, 14892000, @@ -131,7 +132,7 @@ TEST(TEST_TIME64, TIME_WITH_MICRO) { } } -TEST(TEST_TIME64, TIME_WITH_NANO) { +TEST(TestTime32, TimeWithNano) { auto value_field = field("f0", time64(TimeUnit::NANO)); std::vector t64_values = {14896000000, 14897000000, 14892000000, 85400000000, 14893000000, 14895000000}; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor.cc index 37f14ebd9c52..93af94655769 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor.cc @@ -19,12 +19,12 @@ #include "arrow/flight/sql/odbc/odbc_impl/calendar_utils.h" -using arrow::TimeUnit; +#include +#include namespace arrow::flight::sql::odbc { namespace { - -int64_t GetConversionToSecondsDivisor(TimeUnit::type unit) { +inline int64_t GetConversionToSecondsDivisor(TimeUnit::type unit) { int64_t divisor = 1; switch (unit) { case TimeUnit::SECOND: @@ -79,6 +79,10 @@ template RowStatus TimestampArrayFlightSqlAccessor::MoveSingleCellImpl( ColumnBinding* binding, int64_t arrow_row, int64_t cell_counter, int64_t& value_offset, bool update_value_offset, Diagnostics& diagnostics) { + // Times less than the minimum integer number of seconds that can be represented + // for each time unit will not convert correctly. This is mostly interesting for + // nanoseconds as timestamps in other units are outside of the accepted range of + // Gregorian dates. auto* buffer = static_cast(binding->buffer); int64_t value = this->GetArray()->Value(arrow_row); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor_test.cc index 393dd98501d9..dd4917b0e378 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/timestamp_array_accessor_test.cc @@ -20,11 +20,12 @@ #include "arrow/flight/sql/odbc/odbc_impl/calendar_utils.h" #include "arrow/flight/sql/odbc/odbc_impl/util.h" #include "arrow/testing/builder.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { -TEST(TEST_TIMESTAMP, TIMESTAMP_WITH_MILLI) { +TEST(TestTimestamp, TimestampWithMilli) { std::vector values = {86400370, 172800000, 259200000, 1649793238110LL, 345600000, 432000000, 518400000, -86399000, 0, @@ -88,7 +89,7 @@ TEST(TEST_TIMESTAMP, TIMESTAMP_WITH_MILLI) { } } -TEST(TEST_TIMESTAMP, TIMESTAMP_WITH_SECONDS) { +TEST(TestTimestamp, TimestampWithSeconds) { std::vector values = {86400, 172800, 259200, 1649793238, 345600, 432000, 518400}; @@ -130,7 +131,7 @@ TEST(TEST_TIMESTAMP, TIMESTAMP_WITH_SECONDS) { } } -TEST(TEST_TIMESTAMP, TIMESTAMP_WITH_MICRO) { +TEST(TestTimestamp, TimestampWithMicro) { std::vector values = {86400000000, 1649793238000000}; std::shared_ptr timestamp_array; @@ -174,7 +175,7 @@ TEST(TEST_TIMESTAMP, TIMESTAMP_WITH_MICRO) { } } -TEST(TEST_TIMESTAMP, TIMESTAMP_WITH_NANO) { +TEST(TestTimestamp, TimestampWithNano) { std::vector values = {86400000010000, 1649793238000000000}; std::shared_ptr timestamp_array; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/types.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/types.h index ca33d872fa77..c0084a5ab155 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/types.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/accessors/types.h @@ -102,7 +102,7 @@ class FlightSqlAccessor : public Accessor { throw NullWithoutIndicatorException(); } } else { - // TODO: Optimize this by creating different versions of MoveSingleCell + // GH-47849 TODO: Optimize this by creating different versions of MoveSingleCell // depending on if str_len_buffer is null. auto row_status = MoveSingleCell(binding, current_arrow_row, i, value_offset, update_value_offset, diagnostics); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.cc index 7bdb4d58cf82..5dfc85a58f77 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.cc @@ -18,7 +18,7 @@ #include "arrow/flight/sql/odbc/odbc_impl/address_info.h" #include -namespace driver { +namespace arrow::flight::sql::odbc { bool AddressInfo::GetAddressInfo(const std::string& host, char* host_name_info, int64_t max_host) { @@ -47,4 +47,5 @@ AddressInfo::~AddressInfo() { } AddressInfo::AddressInfo() : addrinfo_result_(nullptr) {} -} // namespace driver + +} // namespace arrow::flight::sql::odbc diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.h index c127c0f4a29d..7d538f912e0d 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/address_info.h @@ -27,7 +27,7 @@ # include #endif -namespace driver { +namespace arrow::flight::sql::odbc { class AddressInfo { private: @@ -40,4 +40,5 @@ class AddressInfo { bool GetAddressInfo(const std::string& host, char* host_name_info, int64_t max_host); }; -} // namespace driver + +} // namespace arrow::flight::sql::odbc diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/attribute_utils.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/attribute_utils.h index 315d854b60d5..cfda4cf29a74 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/attribute_utils.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/attribute_utils.h @@ -29,6 +29,8 @@ // GH-48083 TODO: replace `namespace ODBC` with `namespace arrow::flight::sql::odbc` namespace ODBC { +using arrow::flight::sql::odbc::Diagnostics; +using arrow::flight::sql::odbc::WcsToUtf8; template inline void GetAttribute(T attribute_value, SQLPOINTER output, O output_size, @@ -66,7 +68,7 @@ inline SQLRETURN GetAttributeUTF8(std::string_view attribute_value, SQLPOINTER o template inline SQLRETURN GetAttributeUTF8(std::string_view attribute_value, SQLPOINTER output, O output_size, O* output_len_ptr, - arrow::flight::sql::odbc::Diagnostics& diagnostics) { + Diagnostics& diagnostics) { SQLRETURN result = GetAttributeUTF8(attribute_value, output, output_size, output_len_ptr); if (SQL_SUCCESS_WITH_INFO == result) { @@ -103,10 +105,10 @@ inline SQLRETURN GetAttributeSQLWCHAR(std::string_view attribute_value, } template -inline SQLRETURN GetAttributeSQLWCHAR( - const std::string& attribute_value, bool is_length_in_bytes, SQLPOINTER output, - O output_size, O* output_len_ptr, - arrow::flight::sql::odbc::Diagnostics& diagnostics) { +inline SQLRETURN GetAttributeSQLWCHAR(const std::string& attribute_value, + bool is_length_in_bytes, SQLPOINTER output, + O output_size, O* output_len_ptr, + Diagnostics& diagnostics) { SQLRETURN result = GetAttributeSQLWCHAR(attribute_value, is_length_in_bytes, output, output_size, output_len_ptr); if (SQL_SUCCESS_WITH_INFO == result) { @@ -119,7 +121,7 @@ template inline SQLRETURN GetStringAttribute(bool is_unicode, std::string_view attribute_value, bool is_length_in_bytes, SQLPOINTER output, O output_size, O* output_len_ptr, - arrow::flight::sql::odbc::Diagnostics& diagnostics) { + Diagnostics& diagnostics) { SQLRETURN result = SQL_SUCCESS; if (is_unicode) { result = GetAttributeSQLWCHAR(attribute_value, is_length_in_bytes, output, @@ -157,11 +159,11 @@ inline void SetAttributeSQLWCHAR(SQLPOINTER new_value, SQLINTEGER input_length_i std::string& attribute_to_write) { thread_local std::vector utf8_str; if (input_length_in_bytes == SQL_NTS) { - arrow::flight::sql::odbc::WcsToUtf8(new_value, &utf8_str); + WcsToUtf8(new_value, &utf8_str); } else { - arrow::flight::sql::odbc::WcsToUtf8( - new_value, input_length_in_bytes / arrow::flight::sql::odbc::GetSqlWCharSize(), - &utf8_str); + WcsToUtf8(new_value, + input_length_in_bytes / arrow::flight::sql::odbc::GetSqlWCharSize(), + &utf8_str); } attribute_to_write.assign((char*)utf8_str.data()); } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/calendar_utils.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/calendar_utils.cc index 1dddae2a7c71..b47b33f2d93f 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/calendar_utils.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/calendar_utils.cc @@ -40,6 +40,9 @@ int64_t GetTodayTimeFromEpoch() { #endif } +// GH-47631: add support for non-UTC time zone data. +// Read the time zone value from Arrow::Timestamp, and use the time zone value to convert +// seconds_since_epoch instead of converting to UTC time zone by default void GetTimeForSecondsSinceEpoch(const int64_t seconds_since_epoch, std::tm& out_tm) { std::memset(&out_tm, 0, sizeof(std::tm)); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/configuration.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/configuration.cc index 866749e7e0f8..75498710d23f 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/configuration.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/configuration.cc @@ -16,6 +16,7 @@ // under the License. #include "arrow/flight/sql/odbc/odbc_impl/config/configuration.h" + #include "arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.h" #include "arrow/flight/sql/odbc/odbc_impl/util.h" #include "arrow/result.h" @@ -195,6 +196,5 @@ std::vector Configuration::GetCustomKeys() const { boost::copy(copy_props | boost::adaptors::map_keys, std::back_inserter(keys)); return keys; } - } // namespace config } // namespace arrow::flight::sql::odbc diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/encoding_utils.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/encoding_utils.h index 7f8a4a7ef856..5e3a4ecbdae3 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/encoding_utils.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/encoding_utils.h @@ -31,13 +31,17 @@ #define _SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING namespace ODBC { +using arrow::flight::sql::odbc::DriverException; +using arrow::flight::sql::odbc::GetSqlWCharSize; +using arrow::flight::sql::odbc::Utf8ToWcs; +using arrow::flight::sql::odbc::WcsToUtf8; // Return the number of bytes required for the conversion. template inline size_t ConvertToSqlWChar(std::string_view str, SQLWCHAR* buffer, SQLLEN buffer_size_in_bytes) { thread_local std::vector wstr; - arrow::flight::sql::odbc::Utf8ToWcs(str.data(), str.size(), &wstr); + Utf8ToWcs(str.data(), str.size(), &wstr); SQLLEN value_length_in_bytes = wstr.size(); if (buffer) { @@ -46,14 +50,11 @@ inline size_t ConvertToSqlWChar(std::string_view str, SQLWCHAR* buffer, // Write a NUL terminator if (buffer_size_in_bytes >= - value_length_in_bytes + - static_cast(arrow::flight::sql::odbc::GetSqlWCharSize())) { - reinterpret_cast( - buffer)[value_length_in_bytes / arrow::flight::sql::odbc::GetSqlWCharSize()] = + value_length_in_bytes + static_cast(GetSqlWCharSize())) { + reinterpret_cast(buffer)[value_length_in_bytes / GetSqlWCharSize()] = '\0'; } else { - SQLLEN num_chars_written = - buffer_size_in_bytes / arrow::flight::sql::odbc::GetSqlWCharSize(); + SQLLEN num_chars_written = buffer_size_in_bytes / GetSqlWCharSize(); // If we failed to even write one char, the buffer is too small to hold a // NUL-terminator. if (num_chars_written > 0) { @@ -66,16 +67,15 @@ inline size_t ConvertToSqlWChar(std::string_view str, SQLWCHAR* buffer, inline size_t ConvertToSqlWChar(std::string_view str, SQLWCHAR* buffer, SQLLEN buffer_size_in_bytes) { - switch (arrow::flight::sql::odbc::GetSqlWCharSize()) { + switch (GetSqlWCharSize()) { case sizeof(char16_t): return ConvertToSqlWChar(str, buffer, buffer_size_in_bytes); case sizeof(char32_t): return ConvertToSqlWChar(str, buffer, buffer_size_in_bytes); default: assert(false); - throw arrow::flight::sql::odbc::DriverException( - "Encoding is unsupported, SQLWCHAR size: " + - std::to_string(arrow::flight::sql::odbc::GetSqlWCharSize())); + throw DriverException("Encoding is unsupported, SQLWCHAR size: " + + std::to_string(GetSqlWCharSize())); } } @@ -91,9 +91,9 @@ inline std::string SqlWcharToString(SQLWCHAR* wchar_msg, SQLINTEGER msg_len = SQ thread_local std::vector utf8_str; if (msg_len == SQL_NTS) { - arrow::flight::sql::odbc::WcsToUtf8((void*)wchar_msg, &utf8_str); + WcsToUtf8((void*)wchar_msg, &utf8_str); } else { - arrow::flight::sql::odbc::WcsToUtf8((void*)wchar_msg, msg_len, &utf8_str); + WcsToUtf8((void*)wchar_msg, msg_len, &utf8_str); } return std::string(utf8_str.begin(), utf8_str.end()); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_auth_method.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_auth_method.cc index 587dbdfb96ba..5da662e7aded 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_auth_method.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_auth_method.cc @@ -37,6 +37,9 @@ class NoOpAuthMethod : public FlightSqlAuthMethod { void Authenticate(FlightSqlConnection& connection, FlightCallOptions& call_options) override { // Do nothing + + // GH-46733 TODO: implement NoOpAuthMethod to validate server address. + // Can use NoOpClientAuthHandler. } }; @@ -66,10 +69,10 @@ class UserPasswordAuthMethod : public FlightSqlAuthMethod { FlightCallOptions auth_call_options; const std::optional& login_timeout = connection.GetAttribute(Connection::LOGIN_TIMEOUT); - if (login_timeout && boost::get(*login_timeout) > 0) { + if (login_timeout && std::get(*login_timeout) > 0) { // ODBC's LOGIN_TIMEOUT attribute and FlightCallOptions.timeout use // seconds as time unit. - double timeout_seconds = static_cast(boost::get(*login_timeout)); + double timeout_seconds = static_cast(std::get(*login_timeout)); if (timeout_seconds > 0) { auth_call_options.timeout = TimeoutDuration{timeout_seconds}; } @@ -94,7 +97,9 @@ class UserPasswordAuthMethod : public FlightSqlAuthMethod { throw DriverException(bearer_result.status().message()); } - call_options.headers.push_back(bearer_result.ValueOrDie()); + // call_options may have already been populated with data from the connection string + // or DSN. Ensure auth-generated headers are placed at the front of the header list. + call_options.headers.insert(call_options.headers.begin(), bearer_result.ValueOrDie()); } std::string GetUser() override { return user_; } @@ -116,10 +121,11 @@ class TokenAuthMethod : public FlightSqlAuthMethod { void Authenticate(FlightSqlConnection& connection, FlightCallOptions& call_options) override { - // add the token to the headers + // add the token to the front of the headers. For consistency auth headers should be + // at the front. const std::pair token_header("authorization", "Bearer " + token_); - call_options.headers.push_back(token_header); + call_options.headers.insert(call_options.headers.begin(), token_header); const Status status = client_.Authenticate( call_options, std::unique_ptr(new NoOpClientAuthHandler())); @@ -143,22 +149,22 @@ std::unique_ptr FlightSqlAuthMethod::FromProperties( const std::unique_ptr& client, const Connection::ConnPropertyMap& properties) { // Check if should use user-password authentication - auto it_user = properties.find(FlightSqlConnection::USER); + auto it_user = properties.find(std::string(FlightSqlConnection::USER)); if (it_user == properties.end()) { // The Microsoft OLE DB to ODBC bridge provider (MSDASQL) will write // "User ID" and "Password" properties instead of mapping // to ODBC compliant UID/PWD keys. - it_user = properties.find(FlightSqlConnection::USER_ID); + it_user = properties.find(std::string(FlightSqlConnection::USER_ID)); } - auto it_password = properties.find(FlightSqlConnection::PASSWORD); - auto it_token = properties.find(FlightSqlConnection::TOKEN); + auto it_password = properties.find(std::string(FlightSqlConnection::PASSWORD)); + auto it_token = properties.find(std::string(FlightSqlConnection::TOKEN)); if (it_user == properties.end() || it_password == properties.end()) { // Accept UID/PWD as aliases for User/Password. These are suggested as // standard properties in the documentation for SQLDriverConnect. - it_user = properties.find(FlightSqlConnection::UID); - it_password = properties.find(FlightSqlConnection::PWD); + it_user = properties.find(std::string(FlightSqlConnection::UID)); + it_password = properties.find(std::string(FlightSqlConnection::PWD)); } if (it_user != properties.end() || it_password != properties.end()) { const std::string& user = it_user != properties.end() ? it_user->second : ""; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.cc index 8b2b564d8db8..abe82b36c454 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.cc @@ -119,7 +119,7 @@ const std::set BUILT_IN_PROPERTIES Connection::ConnPropertyMap::const_iterator TrackMissingRequiredProperty( std::string_view property, const Connection::ConnPropertyMap& properties, std::vector& missing_attr) { - auto prop_iter = properties.find(property); + auto prop_iter = properties.find(std::string(property)); if (properties.end() == prop_iter) { missing_attr.push_back(property); } @@ -138,6 +138,7 @@ std::shared_ptr LoadFlightSslConfigs( AsBool(conn_property_map, FlightSqlConnection::USE_SYSTEM_TRUST_STORE) .value_or(SYSTEM_TRUST_STORE_DEFAULT); + // GH-47630: find co-located TLS certificate if `trusted certs` path is not specified auto trusted_certs_iterator = conn_property_map.find(std::string(FlightSqlConnection::TRUSTED_CERTS)); auto trusted_certs = trusted_certs_iterator != conn_property_map.end() @@ -243,9 +244,9 @@ const FlightCallOptions& FlightSqlConnection::PopulateCallOptions( // is the first request. const std::optional& connection_timeout = closed_ ? GetAttribute(LOGIN_TIMEOUT) : GetAttribute(CONNECTION_TIMEOUT); - if (connection_timeout && boost::get(*connection_timeout) > 0) { + if (connection_timeout && std::get(*connection_timeout) > 0) { call_options_.timeout = - TimeoutDuration{static_cast(boost::get(*connection_timeout))}; + TimeoutDuration{static_cast(std::get(*connection_timeout))}; } for (auto prop : props) { @@ -320,7 +321,7 @@ Location FlightSqlConnection::BuildLocation( Location location; if (ssl_config->UseEncryption()) { - driver::AddressInfo address_info; + AddressInfo address_info; char host_name_info[NI_MAXHOST] = ""; bool operation_result = false; @@ -334,7 +335,7 @@ Location FlightSqlConnection::BuildLocation( ThrowIfNotOK(Location::ForGrpcTls(host_name_info, port).Value(&location)); return location; } - // TODO: We should log that we could not convert an IP to hostname here. + // GH-47852 TODO: We should log that we could not convert an IP to hostname here. } } catch (...) { // This is expected. The Host attribute can be an IP or name, but make_address will @@ -402,7 +403,7 @@ Connection::Info FlightSqlConnection::GetInfo(uint16_t info_type) { if (info_type == SQL_DBMS_NAME || info_type == SQL_SERVER_NAME) { // Update the database component reported in error messages. // We do this lazily for performance reasons. - diagnostics_.SetDataSourceComponent(boost::get(result)); + diagnostics_.SetDataSourceComponent(std::get(result)); } return result; } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.h index 2561ea492f05..d1a194abcbd8 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.h @@ -34,7 +34,7 @@ class FlightSqlSslConfig; /// \brief Create an instance of the FlightSqlSslConfig class, from the properties passed /// into the map. /// \param conn_property_map the map with the Connection properties. -/// \return An instance of the FlightSqlSslConfig. +/// \return An instance of the FlightSqlSslConfig. std::shared_ptr LoadFlightSslConfigs( const Connection::ConnPropertyMap& conn_property_map); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection_test.cc index 87ae526f1582..bc26a113d77e 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection_test.cc @@ -19,7 +19,8 @@ #include "arrow/flight/sql/odbc/odbc_impl/platform.h" #include "arrow/flight/types.h" -#include "gtest/gtest.h" + +#include #include @@ -33,17 +34,16 @@ TEST(AttributeTests, SetAndGetAttribute) { const std::optional first_value = connection.GetAttribute(Connection::CONNECTION_TIMEOUT); - EXPECT_TRUE(first_value); - - EXPECT_EQ(static_cast(200), boost::get(*first_value)); + ASSERT_TRUE(first_value); + ASSERT_EQ(static_cast(200), std::get(*first_value)); connection.SetAttribute(Connection::CONNECTION_TIMEOUT, static_cast(300)); const std::optional change_value = connection.GetAttribute(Connection::CONNECTION_TIMEOUT); - EXPECT_TRUE(change_value); - EXPECT_EQ(static_cast(300), boost::get(*change_value)); + ASSERT_TRUE(change_value); + ASSERT_EQ(static_cast(300), std::get(*change_value)); connection.Close(); } @@ -55,7 +55,7 @@ TEST(AttributeTests, GetAttributeWithoutSetting) { connection.GetAttribute(Connection::CONNECTION_TIMEOUT); connection.SetClosed(false); - EXPECT_EQ(0, boost::get(*optional)); + EXPECT_EQ(0, std::get(*optional)); connection.Close(); } @@ -77,8 +77,8 @@ TEST(MetadataSettingsTest, StringColumnLengthTest) { const std::optional actual_string_column_length = connection.GetStringColumnLength(properties); - EXPECT_TRUE(actual_string_column_length); - EXPECT_EQ(expected_string_column_length, *actual_string_column_length); + ASSERT_TRUE(actual_string_column_length); + ASSERT_EQ(expected_string_column_length, *actual_string_column_length); connection.Close(); } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_get_tables_reader.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_get_tables_reader.cc index ebff8c40f2cb..a668353272b5 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_get_tables_reader.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_get_tables_reader.cc @@ -77,9 +77,9 @@ std::shared_ptr GetTablesReader::GetSchema() { const arrow::Result>& result = arrow::ipc::ReadSchema(&dataset_schema_reader, &in_memo); if (!result.ok()) { - // TODO: Ignoring this error until we fix the problem on Dremio server - // The problem is that complex types columns are being returned without the children - // types. + // GH-46561 TODO: Test and build the driver against a server that returns + // complex types columns with the children + // types and handle the failure properly return nullptr; } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_column.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_column.h index ede53038f1a2..d09e550900ba 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_column.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_column.h @@ -70,4 +70,5 @@ class FlightSqlResultSetColumn { } } }; + } // namespace arrow::flight::sql::odbc diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_metadata.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_metadata.cc index 9e2f45647a52..8278b42e36b1 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_metadata.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_result_set_metadata.cc @@ -158,19 +158,27 @@ size_t FlightSqlResultSetMetadata::GetLength(int column_position) { } std::string FlightSqlResultSetMetadata::GetLiteralPrefix(int column_position) { - // TODO: Flight SQL column metadata does not have this, should we add to the spec? + // GH-47853 TODO: use `ColumnMetadata` to get literal prefix after Flight SQL protocol + // adds support for it + + // Flight SQL column metadata does not have literal prefix, empty string is returned return ""; } std::string FlightSqlResultSetMetadata::GetLiteralSuffix(int column_position) { - // TODO: Flight SQL column metadata does not have this, should we add to the spec? + // GH-47853 TODO: use `ColumnMetadata` to get literal suffix after Flight SQL protocol + // adds support for it + + // Flight SQL column metadata does not have literal suffix, empty string is returned return ""; } std::string FlightSqlResultSetMetadata::GetLocalTypeName(int column_position) { ColumnMetadata metadata = GetMetadata(schema_->field(column_position - 1)); - // TODO: Is local type name the same as type name? + // Local type name is for display purpose only. + // Return type name as local type name as Flight SQL protocol doesn't have support for + // local type name. return metadata.GetTypeName().ValueOrElse([] { return ""; }); } @@ -193,7 +201,7 @@ size_t FlightSqlResultSetMetadata::GetOctetLength(int column_position) { // Workaround to get the precision for Decimal and Numeric types, since server doesn't // return it currently. - // TODO: Use the server precision when its fixed. + // GH-47854 TODO: Use the server precision when its fixed. std::shared_ptr arrow_type = field->type(); if (arrow_type->id() == Type::DECIMAL128) { int32_t precision = util::GetDecimalTypePrecision(arrow_type); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_ssl_config.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_ssl_config.h index 10b121497121..c2d1423e97e6 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_ssl_config.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_ssl_config.h @@ -17,9 +17,9 @@ #pragma once -#include -#include #include +#include "arrow/flight/types.h" +#include "arrow/status.h" namespace arrow::flight::sql::odbc { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement.cc index c35154b44eab..ef652d34fac1 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement.cc @@ -83,9 +83,9 @@ bool FlightSqlStatement::SetAttribute(StatementAttributeId attribute, case MAX_LENGTH: return CheckIfSetToOnlyValidValue(value, static_cast(0)); case QUERY_TIMEOUT: - if (boost::get(value) > 0) { + if (std::get(value) > 0) { call_options_.timeout = - TimeoutDuration{static_cast(boost::get(value))}; + TimeoutDuration{static_cast(std::get(value))}; } else { call_options_.timeout = TimeoutDuration{-1}; // Intentional fall-through. diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_columns.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_columns.cc index 914cd8fa4520..a6200c0b1c15 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_columns.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_columns.cc @@ -26,6 +26,8 @@ namespace arrow::flight::sql::odbc { using arrow::Result; +using arrow::Schema; +using arrow::flight::sql::ColumnMetadata; using util::AppendToBuilder; using std::make_optional; @@ -99,10 +101,9 @@ Result> TransformInner( const auto& table_name = reader.GetTableName(); const std::shared_ptr& schema = reader.GetSchema(); if (schema == nullptr) { - // TODO: Remove this if after fixing TODO on GetTablesReader::GetSchema() - // This is because of a problem on Dremio server, where complex types columns - // are being returned without the children types, so we are simply ignoring - // it by now. + // GH-46561 TODO: Test and build the driver against a server that returns + // complex types columns with the children + // types and handle the failure properly. continue; } for (int i = 0; i < schema->num_fields(); ++i) { @@ -126,8 +127,8 @@ Result> TransformInner( ? data_type_v3 : util::ConvertSqlDataTypeFromV3ToV2(data_type_v3); - // TODO: Use `metadata.GetTypeName()` when ARROW-16064 is merged. - const auto& type_name_result = field->metadata()->Get("ARROW:FLIGHT:SQL:TYPE_NAME"); + const auto& type_name_result = metadata.GetTypeName(); + data.type_name = type_name_result.ok() ? type_name_result.ValueOrDie() : util::GetTypeNameFromSqlDataType(data_type_v3); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_tables.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_tables.h index 0c3ad10f97b5..5687134f1eb7 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_tables.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_tables.h @@ -61,5 +61,4 @@ std::shared_ptr GetTablesForGenericUse( const std::string* catalog_name, const std::string* schema_name, const std::string* table_name, const std::vector& table_types, Diagnostics& diagnostics, const MetadataSettings& metadata_settings); - } // namespace arrow::flight::sql::odbc diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_stream_chunk_buffer.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_stream_chunk_buffer.cc index be788d9d08aa..d4812e30eee4 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_stream_chunk_buffer.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_stream_chunk_buffer.cc @@ -20,8 +20,6 @@ namespace arrow::flight::sql::odbc { -using arrow::Result; - FlightStreamChunkBuffer::FlightStreamChunkBuffer( FlightSqlClient& flight_sql_client, const FlightClientOptions& client_options, const FlightCallOptions& call_options, const std::shared_ptr& flight_info, @@ -58,10 +56,10 @@ FlightStreamChunkBuffer::FlightStreamChunkBuffer( util::ThrowIfNotOK(result.status()); std::shared_ptr stream_reader_ptr(std::move(result.ValueOrDie())); - BlockingQueue, - std::shared_ptr>>::Supplier supplier = [=]() - -> std::optional< - std::pair, std::shared_ptr>> { + BlockingQueue, + std::shared_ptr>>::Supplier supplier = + [=]() -> std::optional, + std::shared_ptr>> { auto result = stream_reader_ptr->Next(); bool is_not_ok = !result.ok(); bool is_not_empty = result.ok() && (result.ValueOrDie().data != nullptr); @@ -82,13 +80,13 @@ FlightStreamChunkBuffer::FlightStreamChunkBuffer( } bool FlightStreamChunkBuffer::GetNext(FlightStreamChunk* chunk) { - std::pair, std::shared_ptr> + std::pair, std::shared_ptr> closeable_endpoint_stream_pair; if (!queue_.Pop(&closeable_endpoint_stream_pair)) { return false; } - Result result = closeable_endpoint_stream_pair.first; + arrow::Result result = closeable_endpoint_stream_pair.first; if (!result.status().ok()) { Close(); throw DriverException(result.status().message()); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/get_info_cache.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/get_info_cache.h index a1452e4b466f..693ee000de52 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/get_info_cache.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/get_info_cache.h @@ -17,13 +17,12 @@ #pragma once -#include "arrow/flight/sql/client.h" -#include "arrow/flight/sql/odbc/odbc_impl/spi/connection.h" - #include #include #include #include +#include "arrow/flight/sql/client.h" +#include "arrow/flight/sql/odbc/odbc_impl/spi/connection.h" namespace arrow::flight::sql::odbc { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.cc index db6170f31027..acd68f0eef3c 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.cc @@ -221,7 +221,7 @@ class ScalarToJson : public ScalarVisitor { } Status Visit(const DurationScalar& scalar) override { - // TODO: Append TimeUnit on conversion + // GH-47857 TODO: Append TimeUnit on conversion return ConvertScalarToStringAndWrite(scalar, writer_); } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.h index 9c0b42748a8c..44321398b6fa 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter.h @@ -17,8 +17,8 @@ #pragma once -#include #include +#include "arrow/type_fwd.h" namespace arrow::flight::sql::odbc { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter_test.cc index a3c3275affd5..d6d7a3ed5062 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/json_converter_test.cc @@ -19,7 +19,8 @@ #include "arrow/scalar.h" #include "arrow/testing/builder.h" #include "arrow/type.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { @@ -163,7 +164,7 @@ TEST(ConvertToJson, MonthInterval) { } TEST(ConvertToJson, Duration) { - // TODO: Append TimeUnit on conversion + // GH-47857 TODO: Append TimeUnit on conversion ASSERT_EQ("\"123\"", ConvertToJson(DurationScalar(123, TimeUnit::SECOND))); ASSERT_EQ("\"123\"", ConvertToJson(DurationScalar(123, TimeUnit::MILLI))); ASSERT_EQ("\"123\"", ConvertToJson(DurationScalar(123, TimeUnit::MICRO))); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.cc index 59cdc2e12d90..a2464aa00e36 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.cc @@ -21,6 +21,7 @@ #include "arrow/util/utf8.h" #include "arrow/flight/sql/odbc/odbc_impl/attribute_utils.h" +#include "arrow/flight/sql/odbc/odbc_impl/config/configuration.h" #include "arrow/flight/sql/odbc/odbc_impl/exceptions.h" #include "arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.h" #include "arrow/flight/sql/odbc/odbc_impl/odbc_environment.h" @@ -233,8 +234,8 @@ SQLRETURN ODBCConnection::GetInfo(SQLUSMALLINT info_type, SQLPOINTER value, case SQL_COLUMN_ALIAS: case SQL_DBMS_NAME: case SQL_DBMS_VER: - case SQL_DRIVER_NAME: // TODO: This should be the driver's filename and shouldn't - // come from the SPI. + case SQL_DRIVER_NAME: // GH-47858 TODO: This should be the driver's filename and + // shouldn't come from the SPI. case SQL_DRIVER_VER: case SQL_SEARCH_PATTERN_ESCAPE: case SQL_SERVER_NAME: @@ -260,7 +261,7 @@ SQLRETURN ODBCConnection::GetInfo(SQLUSMALLINT info_type, SQLPOINTER value, case SQL_SPECIAL_CHARACTERS: case SQL_XOPEN_CLI_YEAR: { const auto& info = spi_connection_->GetInfo(info_type); - const std::string& info_value = boost::get(info); + const std::string& info_value = std::get(info); return GetStringAttribute(is_unicode, info_value, true, value, buffer_length, output_length, GetDiagnostics()); } @@ -350,7 +351,7 @@ SQLRETURN ODBCConnection::GetInfo(SQLUSMALLINT info_type, SQLPOINTER value, case SQL_SQL92_VALUE_EXPRESSIONS: case SQL_STANDARD_CLI_CONFORMANCE: { const auto& info = spi_connection_->GetInfo(info_type); - uint32_t info_value = boost::get(info); + uint32_t info_value = std::get(info); GetAttribute(info_value, value, buffer_length, output_length); return SQL_SUCCESS; } @@ -385,7 +386,7 @@ SQLRETURN ODBCConnection::GetInfo(SQLUSMALLINT info_type, SQLPOINTER value, case SQL_ODBC_SQL_CONFORMANCE: case SQL_ODBC_SAG_CLI_CONFORMANCE: { const auto& info = spi_connection_->GetInfo(info_type); - uint16_t info_value = boost::get(info); + uint16_t info_value = std::get(info); GetAttribute(info_value, value, buffer_length, output_length); return SQL_SUCCESS; } @@ -396,7 +397,7 @@ SQLRETURN ODBCConnection::GetInfo(SQLUSMALLINT info_type, SQLPOINTER value, if (!attr) { throw DriverException("Optional feature not supported.", "HYC00"); } - const std::string& info_value = boost::get(*attr); + const std::string& info_value = std::get(*attr); return GetStringAttribute(is_unicode, info_value, true, value, buffer_length, output_length, GetDiagnostics()); } @@ -414,7 +415,7 @@ void ODBCConnection::SetConnectAttr(SQLINTEGER attribute, SQLPOINTER value, bool successfully_written = false; switch (attribute) { // Internal connection attributes -#ifdef SQL_ATR_ASYNC_DBC_EVENT +#ifdef SQL_ATTR_ASYNC_DBC_EVENT case SQL_ATTR_ASYNC_DBC_EVENT: throw DriverException("Optional feature not supported.", "HYC00"); #endif @@ -422,7 +423,7 @@ void ODBCConnection::SetConnectAttr(SQLINTEGER attribute, SQLPOINTER value, case SQL_ATTR_ASYNC_DBC_FUNCTIONS_ENABLE: throw DriverException("Optional feature not supported.", "HYC00"); #endif -#ifdef SQL_ATTR_ASYNC_PCALLBACK +#ifdef SQL_ATTR_ASYNC_DBC_PCALLBACK case SQL_ATTR_ASYNC_DBC_PCALLBACK: throw DriverException("Optional feature not supported.", "HYC00"); #endif @@ -450,7 +451,7 @@ void ODBCConnection::SetConnectAttr(SQLINTEGER attribute, SQLPOINTER value, throw DriverException("Cannot set read-only attribute", "HY092"); case SQL_ATTR_TRACE: // DM-only throw DriverException("Cannot set read-only attribute", "HY092"); - case SQL_ATTR_TRACEFILE: + case SQL_ATTR_TRACEFILE: // DM-only throw DriverException("Optional feature not supported.", "HYC00"); case SQL_ATTR_TRANSLATE_LIB: throw DriverException("Optional feature not supported.", "HYC00"); @@ -591,7 +592,7 @@ SQLRETURN ODBCConnection::GetConnectAttr(SQLINTEGER attribute, SQLPOINTER value, if (!catalog) { throw DriverException("Optional feature not supported.", "HYC00"); } - const std::string& info_value = boost::get(*catalog); + const std::string& info_value = std::get(*catalog); return GetStringAttribute(is_unicode, info_value, true, value, buffer_length, output_length, GetDiagnostics()); } @@ -620,7 +621,7 @@ SQLRETURN ODBCConnection::GetConnectAttr(SQLINTEGER attribute, SQLPOINTER value, throw DriverException("Invalid attribute", "HY092"); } - GetAttribute(static_cast(boost::get(*spi_attribute)), value, + GetAttribute(static_cast(std::get(*spi_attribute)), value, buffer_length, output_length); return SQL_SUCCESS; } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.h index d50e99d0b619..4b7519f74a4e 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.h @@ -17,8 +17,8 @@ #pragma once -#include #include "arrow/flight/sql/odbc/odbc_impl/odbc_handle.h" +#include "arrow/flight/sql/odbc/odbc_impl/spi/connection.h" #include "arrow/flight/sql/odbc/odbc_impl/type_fwd.h" #include @@ -37,6 +37,9 @@ class ODBCConnection : public ODBCHandle { ODBCConnection(const ODBCConnection&) = delete; ODBCConnection& operator=(const ODBCConnection&) = delete; + /// \brief Constructor for ODBCConnection. + /// \param[in] environment the parent environment. + /// \param[in] spi_connection the underlying spi connection. ODBCConnection(ODBCEnvironment& environment, std::shared_ptr spi_connection); @@ -44,6 +47,11 @@ class ODBCConnection : public ODBCHandle { const std::string& GetDSN() const; bool IsConnected() const; + + /// \brief Connect to Arrow Flight SQL server. + /// \param[in] dsn the dsn name. + /// \param[in] properties the connection property map extracted from connection string. + /// \param[out] missing_properties report the properties that are missing void Connect(std::string dsn, const arrow::flight::sql::odbc::Connection::ConnPropertyMap& properties, std::vector& missing_properties); @@ -51,7 +59,7 @@ class ODBCConnection : public ODBCHandle { SQLRETURN GetInfo(SQLUSMALLINT info_type, SQLPOINTER value, SQLSMALLINT buffer_length, SQLSMALLINT* output_length, bool is_unicode); void SetConnectAttr(SQLINTEGER attribute, SQLPOINTER value, SQLINTEGER string_length, - bool isUnicode); + bool is_unicode); SQLRETURN GetConnectAttr(SQLINTEGER attribute, SQLPOINTER value, SQLINTEGER buffer_length, SQLINTEGER* output_length, bool is_unicode); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.cc index eae90ac2b77a..8c856fdbd6bc 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.cc @@ -62,7 +62,7 @@ ODBCDescriptor::ODBCDescriptor(Diagnostics& base_diagnostics, ODBCConnection* co parent_statement_(stmt), array_status_ptr_(nullptr), bind_offset_ptr_(nullptr), - rows_processed_ptr_(nullptr), + rows_proccessed_ptr_(nullptr), array_size_(1), bind_type_(SQL_BIND_BY_COLUMN), highest_one_based_bound_record_(0), @@ -109,7 +109,7 @@ void ODBCDescriptor::SetHeaderField(SQLSMALLINT field_identifier, SQLPOINTER val has_bindings_changed_ = true; break; case SQL_DESC_ROWS_PROCESSED_PTR: - SetPointerAttribute(value, rows_processed_ptr_); + SetPointerAttribute(value, rows_proccessed_ptr_); has_bindings_changed_ = true; break; case SQL_DESC_COUNT: { @@ -273,7 +273,7 @@ void ODBCDescriptor::GetHeaderField(SQLSMALLINT field_identifier, SQLPOINTER val GetAttribute(bind_type_, value, buffer_length, output_length); break; case SQL_DESC_ROWS_PROCESSED_PTR: - GetAttribute(rows_processed_ptr_, value, buffer_length, output_length); + GetAttribute(rows_proccessed_ptr_, value, buffer_length, output_length); break; case SQL_DESC_COUNT: { // highest_one_based_bound_record_ equals number of records + 1 @@ -507,10 +507,12 @@ void ODBCDescriptor::PopulateFromResultSetMetadata(ResultSetMetadata* rsmd) { rsmd->IsAutoUnique(one_based_index) ? SQL_TRUE : SQL_FALSE; records_[i].case_sensitive = rsmd->IsCaseSensitive(one_based_index) ? SQL_TRUE : SQL_FALSE; - records_[i].datetime_interval_precision; // TODO - update when rsmd adds this + records_[i].datetime_interval_precision; // GH-47869 TODO implement + // `SQL_DESC_DATETIME_INTERVAL_PRECISION` SQLINTEGER num_prec_radix = rsmd->GetNumPrecRadix(one_based_index); records_[i].num_prec_radix = num_prec_radix > 0 ? num_prec_radix : 0; - records_[i].datetime_interval_code; // TODO + records_[i].datetime_interval_code; // GH-47868 TODO implement + // `SQL_DESC_DATETIME_INTERVAL_CODE` records_[i].fixed_prec_scale = rsmd->IsFixedPrecScale(one_based_index) ? SQL_TRUE : SQL_FALSE; records_[i].nullable = rsmd->IsNullable(one_based_index); @@ -579,5 +581,5 @@ void ODBCDescriptor::SetDataPtrOnRecord(SQLPOINTER data_ptr, SQLSMALLINT record_ } void DescriptorRecord::CheckConsistency() { - // TODO + // GH-47870 TODO implement } diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.h index f0259d40098c..7d59db91b6ef 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_descriptor.h @@ -121,8 +121,8 @@ class ODBCDescriptor : public ODBCHandle { inline SQLUSMALLINT* GetArrayStatusPtr() { return array_status_ptr_; } inline void SetRowsProcessed(SQLULEN rows) { - if (rows_processed_ptr_) { - *rows_processed_ptr_ = rows; + if (rows_proccessed_ptr_) { + *rows_proccessed_ptr_ = rows; } } @@ -139,7 +139,7 @@ class ODBCDescriptor : public ODBCHandle { ODBCStatement* parent_statement_; SQLUSMALLINT* array_status_ptr_; SQLULEN* bind_offset_ptr_; - SQLULEN* rows_processed_ptr_; + SQLULEN* rows_proccessed_ptr_; SQLULEN array_size_; SQLINTEGER bind_type_; SQLSMALLINT highest_one_based_bound_record_; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_handle.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_handle.h index 9dd8fe37baf6..4674a4c9ff16 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_handle.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_handle.h @@ -17,13 +17,14 @@ #pragma once -#include -#include +// platform.h includes windows.h, so it needs to be included first +#include "arrow/flight/sql/odbc/odbc_impl/platform.h" #include #include #include #include +#include "arrow/flight/sql/odbc/odbc_impl/diagnostics.h" /** * @brief An abstraction over a generic ODBC handle. diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc index c77e97934ae9..82656d1a38f8 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc @@ -251,7 +251,7 @@ void ODBCStatement::CopyAttributesFromConnection(ODBCConnection& connection) { ODBCStatement& tracking_statement = connection.GetTrackingStatement(); // Get abstraction attributes and copy to this spi_statement_. - // Possible ODBC attributes are below, but many of these are not supported by warpdrive + // Possible ODBC attributes are below, but many of these are not supported by Arrow ODBC // or ODBCAbstaction: // SQL_ATTR_ASYNC_ENABLE: // SQL_ATTR_METADATA_ID: @@ -329,7 +329,7 @@ bool ODBCStatement::Fetch(size_t rows, SQLULEN* row_count_ptr, } if (current_ard_->HaveBindingsChanged()) { - // TODO: Deal handle when offset != buffer_length. + // GH-47871 TODO: handle when offset != buffer_length. // Wipe out all bindings in the ResultSet. // Note that the number of ARD records can both be more or less @@ -537,7 +537,7 @@ void ODBCStatement::GetStmtAttr(SQLINTEGER statement_attribute, SQLPOINTER outpu } if (spi_attribute) { - GetAttribute(static_cast(boost::get(*spi_attribute)), output, + GetAttribute(static_cast(std::get(*spi_attribute)), output, buffer_size, str_len_ptr); return; } @@ -554,7 +554,7 @@ void ODBCStatement::SetStmtAttr(SQLINTEGER statement_attribute, SQLPOINTER value switch (statement_attribute) { case SQL_ATTR_APP_PARAM_DESC: { ODBCDescriptor* desc = static_cast(value); - if (current_apd_ != desc) { + if (desc && current_apd_ != desc) { if (current_apd_ != built_in_apd_.get()) { current_apd_->DetachFromStatement(this, true); } @@ -567,7 +567,7 @@ void ODBCStatement::SetStmtAttr(SQLINTEGER statement_attribute, SQLPOINTER value } case SQL_ATTR_APP_ROW_DESC: { ODBCDescriptor* desc = static_cast(value); - if (current_ard_ != desc) { + if (desc && current_ard_ != desc) { if (current_ard_ != built_in_ard_.get()) { current_ard_->DetachFromStatement(this, false); } @@ -621,6 +621,7 @@ void ODBCStatement::SetStmtAttr(SQLINTEGER statement_attribute, SQLPOINTER value return; case SQL_ATTR_ASYNC_ENABLE: + throw DriverException("Unsupported attribute", "HYC00"); #ifdef SQL_ATTR_ASYNC_STMT_EVENT case SQL_ATTR_ASYNC_STMT_EVENT: throw DriverException("Unsupported attribute", "HYC00"); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.h index 7f974b0a85e6..a223b1867e05 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.h @@ -17,10 +17,12 @@ #pragma once +// platform.h platform.h includes windows.h so it needs to be included first +#include "arrow/flight/sql/odbc/odbc_impl/platform.h" + #include "arrow/flight/sql/odbc/odbc_impl/odbc_handle.h" #include "arrow/flight/sql/odbc/odbc_impl/type_fwd.h" -#include #include #include #include @@ -57,7 +59,6 @@ class ODBCStatement : public ODBCHandle { /// row_count_ptr and row_status_array are optional arguments, they are only needed for /// SQLExtendedFetch bool Fetch(size_t rows, SQLULEN* row_count_ptr = 0, SQLUSMALLINT* row_status_array = 0); - bool IsPrepared() const; void GetStmtAttr(SQLINTEGER statement_attribute, SQLPOINTER output, diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/parse_table_types_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/parse_table_types_test.cc index cf1e5930a827..3749c276d4f3 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/parse_table_types_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/parse_table_types_test.cc @@ -18,7 +18,8 @@ #include "arrow/flight/sql/odbc/odbc_impl/flight_sql_statement_get_tables.h" #include "arrow/flight/sql/odbc/odbc_impl/platform.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer.h index 539583aac29f..15548b52c63d 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer.h @@ -17,9 +17,9 @@ #pragma once -#include -#include #include +#include "arrow/flight/client.h" +#include "arrow/type.h" namespace arrow::flight::sql::odbc { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer_test.cc index 9727167a500c..a5e094317ead 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/record_batch_transformer_test.cc @@ -20,7 +20,8 @@ #include "arrow/flight/sql/odbc/odbc_impl/platform.h" #include "arrow/record_batch.h" #include "arrow/testing/builder.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { namespace { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/scalar_function_reporter.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/scalar_function_reporter.h index f4855812bf93..e9cf18dc55a3 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/scalar_function_reporter.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/scalar_function_reporter.h @@ -17,7 +17,7 @@ #pragma once -#include +#include "arrow/type.h" namespace arrow::flight::sql::odbc { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h index 541848916934..b7f24628a64a 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h @@ -23,6 +23,7 @@ #include #include #include +#include #include #include "arrow/flight/sql/odbc/odbc_impl/diagnostics.h" @@ -62,8 +63,8 @@ class Connection { PACKET_SIZE, // uint32_t - The Packet Size }; - typedef boost::variant Attribute; - typedef boost::variant Info; + typedef std::variant Attribute; + typedef std::variant Info; typedef PropertyMap ConnPropertyMap; /// \brief Establish the connection. diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/statement.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/statement.h index 240c51b67202..2c58371d1080 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/statement.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/statement.h @@ -17,9 +17,9 @@ #pragma once -#include #include #include +#include #include #include "arrow/flight/sql/odbc/odbc_impl/type_fwd.h" @@ -50,7 +50,7 @@ class Statement { QUERY_TIMEOUT, }; - typedef boost::variant Attribute; + typedef std::variant Attribute; /// \brief Set a statement attribute (may be called at any time) /// diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc index 179303b68e38..0d24f0cca82d 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc @@ -25,6 +25,7 @@ #include #include +#include #include #include "arrow/flight/sql/odbc/odbc_impl/exceptions.h" diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.cc index 0432836a16f8..be6b3fe4799b 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.cc @@ -44,9 +44,9 @@ std::string TestConnection(const config::Configuration& config) { // This should have been checked before enabling the Test button. assert(missing_properties.empty()); std::string server_name = - boost::get(flight_sql_conn->GetInfo(SQL_SERVER_NAME)); + std::get(flight_sql_conn->GetInfo(SQL_SERVER_NAME)); std::string server_version = - boost::get(flight_sql_conn->GetInfo(SQL_DBMS_VER)); + std::get(flight_sql_conn->GetInfo(SQL_DBMS_VER)); return "Server Name: " + server_name + "\n" + "Server Version: " + server_version; } } // namespace @@ -565,7 +565,7 @@ bool DsnConfigurationWindow::OnMessage(UINT msg, WPARAM wparam, LPARAM lparam) { open_file_name.lpstrFile = file_name; open_file_name.lpstrFile[0] = '\0'; open_file_name.nMaxFile = FILENAME_MAX; - // TODO: What type should this be? + // GH-47851 TODO: Update `lpstrFilter` to correct value open_file_name.lpstrFilter = L"All\0*.*"; open_file_name.nFilterIndex = 1; open_file_name.lpstrFileTitle = NULL; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/window.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/window.cc index f21329977ba2..ce10ddd3bf9e 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/window.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/window.cc @@ -36,7 +36,6 @@ HINSTANCE GetHInstance() { TCHAR sz_file_name[MAX_PATH]; GetModuleFileName(NULL, sz_file_name, MAX_PATH); - // TODO: This needs to be the module name. HINSTANCE h_instance = GetModuleHandle(sz_file_name); if (h_instance == NULL) { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.cc index fa0a35274a8e..c23933287fb1 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.cc @@ -117,12 +117,13 @@ SqlDataType GetDataTypeFromArrowFieldV3(const std::shared_ptr& field, case Type::TIME64: return SqlDataType_TYPE_TIME; case Type::INTERVAL_MONTHS: - return SqlDataType_INTERVAL_MONTH; // TODO: maybe - // SqlDataType_INTERVAL_YEAR_TO_MONTH + return SqlDataType_INTERVAL_MONTH; // GH-47873 TODO: check and update to + // SqlDataType_INTERVAL_YEAR_TO_MONTH if it is + // more appropriate case Type::INTERVAL_DAY_TIME: return SqlDataType_INTERVAL_DAY; - // TODO: Handle remaining types. + // GH-47873 TODO: Handle remaining types. case Type::INTERVAL_MONTH_DAY_NANO: case Type::LIST: case Type::STRUCT: @@ -670,7 +671,7 @@ optional GetDisplaySize(SqlDataType data_type, case SqlDataType_INTERVAL_HOUR_TO_MINUTE: case SqlDataType_INTERVAL_HOUR_TO_SECOND: case SqlDataType_INTERVAL_MINUTE_TO_SECOND: - return nullopt; // TODO: Implement for INTERVAL types + return nullopt; // GH-47874 TODO: Implement for INTERVAL types case SqlDataType_GUID: return 36; default: @@ -935,9 +936,9 @@ ArrayConvertTask GetConverter(Type::type original_type_id, CDataType target_type auto seconds_from_epoch = GetTodayTimeFromEpoch(); - auto third_converted_array = CheckConversion( - arrow::compute::Add(second_converted_array, - std::make_shared(seconds_from_epoch * 1000))); + auto third_converted_array = CheckConversion(arrow::compute::Add( + second_converted_array, + std::make_shared(seconds_from_epoch * 1000))); arrow::compute::CastOptions cast_options_2; cast_options_2.to_type = arrow::timestamp(TimeUnit::MILLI); @@ -956,7 +957,7 @@ ArrayConvertTask GetConverter(Type::type original_type_id, CDataType target_type auto second_converted_array = CheckConversion(arrow::compute::Add( first_converted_array, - std::make_shared(seconds_from_epoch * 1000000000))); + std::make_shared(seconds_from_epoch * 1000000000))); arrow::compute::CastOptions cast_options_2; cast_options_2.to_type = arrow::timestamp(TimeUnit::NANO); @@ -980,7 +981,7 @@ ArrayConvertTask GetConverter(Type::type original_type_id, CDataType target_type } else if (original_type_id == Type::DECIMAL128 && (target_type == CDataType_CHAR || target_type == CDataType_WCHAR)) { return [=](const std::shared_ptr& original_array) { - StringBuilder builder; + arrow::StringBuilder builder; int64_t length = original_array->length(); ThrowIfNotOK(builder.ReserveData(length)); diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h index f513c8d340dd..e9a6794f104b 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h @@ -17,17 +17,17 @@ #pragma once -#include "arrow/util/utf8.h" +#include +#include +#include +#include +#include #include "arrow/flight/sql/odbc/odbc_impl/exceptions.h" #include "arrow/flight/sql/odbc/odbc_impl/spi/connection.h" #include "arrow/flight/sql/odbc/odbc_impl/types.h" #include "arrow/flight/types.h" - -#include -#include -#include -#include +#include "arrow/util/utf8.h" #define CONVERT_WIDE_STR(wstring_var, utf8_target) \ wstring_var = [&] { \ @@ -57,7 +57,7 @@ inline void ThrowIfNotOK(const Status& status) { template inline bool CheckIfSetToOnlyValidValue(const AttributeTypeT& value, T allowed_value) { - return boost::get(value) == allowed_value; + return std::get(value) == allowed_value; } template diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util_test.cc index bfcec15b4dac..c5f4ac5c2c47 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util_test.cc @@ -23,7 +23,8 @@ #include "arrow/testing/builder.h" #include "arrow/testing/gtest_util.h" #include "arrow/testing/util.h" -#include "gtest/gtest.h" + +#include namespace arrow::flight::sql::odbc { @@ -48,7 +49,7 @@ void AssertConvertedArray(const std::shared_ptr& expected_array, ASSERT_EQ(expected_array->ToString(), converted_array->ToString()); } -std::shared_ptr convertArray(const std::shared_ptr& original_array, +std::shared_ptr ConvertArray(const std::shared_ptr& original_array, CDataType c_type) { auto converter = util::GetConverter(original_array->type_id(), c_type); return converter(original_array); @@ -60,7 +61,7 @@ void TestArrayConversion(const std::vector& input, std::shared_ptr original_array; ArrayFromVector(input, &original_array); - auto converted_array = convertArray(original_array, c_type); + auto converted_array = ConvertArray(original_array, c_type); AssertConvertedArray(expected_array, converted_array, input.size(), arrow_type); } @@ -71,7 +72,7 @@ void TestTime32ArrayConversion(const std::vector& input, std::shared_ptr original_array; ArrayFromVector(time32(TimeUnit::MILLI), input, &original_array); - auto converted_array = convertArray(original_array, c_type); + auto converted_array = ConvertArray(original_array, c_type); AssertConvertedArray(expected_array, converted_array, input.size(), arrow_type); } @@ -82,7 +83,7 @@ void TestTime64ArrayConversion(const std::vector& input, std::shared_ptr original_array; ArrayFromVector(time64(TimeUnit::NANO), input, &original_array); - auto converted_array = convertArray(original_array, c_type); + auto converted_array = ConvertArray(original_array, c_type); AssertConvertedArray(expected_array, converted_array, input.size(), arrow_type); } diff --git a/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt index ef0c7271ec23..1a00f304b118 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt +++ b/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt @@ -15,6 +15,14 @@ # specific language governing permissions and limitations # under the License. +add_custom_target(tests) + +if(WIN32) + include_directories(${ODBC_INCLUDE_DIRS}) +else() + include_directories(${ODBC_INCLUDE_DIR}) +endif() + find_package(SQLite3Alt REQUIRED) set(ARROW_FLIGHT_SQL_MOCK_SERVER_SRCS diff --git a/cpp/src/arrow/flight/sql/odbc/tests/README b/cpp/src/arrow/flight/sql/odbc/tests/README new file mode 100644 index 000000000000..fe74c98b72f1 --- /dev/null +++ b/cpp/src/arrow/flight/sql/odbc/tests/README @@ -0,0 +1,23 @@ + + +Prior to running the tests, set environment variable `ARROW_FLIGHT_SQL_ODBC_CONN` +to a valid connection string. +A valid connection string looks like: +driver={Apache Arrow Flight SQL ODBC Driver};HOST=localhost;port=32010;pwd=myPassword;uid=myName;useEncryption=false; diff --git a/cpp/src/arrow/flight/sql/odbc/tests/columns_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/columns_test.cc index f9936a83467b..0eeff7d4d2cb 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/columns_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/columns_test.cc @@ -24,6 +24,8 @@ #include +// Many tests are disabled for MacOS due to iODBC limitations. + namespace arrow::flight::sql::odbc { template @@ -237,6 +239,7 @@ void CheckSQLColAttribute(SQLHSTMT stmt, SQLUSMALLINT idx, EXPECT_EQ(expected_unsigned_column, unsigned_col); } +#ifndef __APPLE__ void CheckSQLColAttributes(SQLHSTMT stmt, SQLUSMALLINT idx, const std::wstring& expected_column_name, SQLLEN expected_data_type, SQLLEN expected_display_size, @@ -306,6 +309,7 @@ void CheckSQLColAttributes(SQLHSTMT stmt, SQLUSMALLINT idx, EXPECT_EQ(expected_searchable, searchable); EXPECT_EQ(expected_unsigned_column, unsigned_col); } +#endif // __APPLE__ void GetSQLColAttributeString(SQLHSTMT stmt, const std::wstring& wsql, SQLUSMALLINT idx, SQLUSMALLINT field_identifier, std::wstring& value) { @@ -364,6 +368,7 @@ void GetSQLColAttributeNumeric(SQLHSTMT stmt, const std::wstring& wsql, SQLUSMAL SQLColAttribute(stmt, idx, field_identifier, 0, 0, nullptr, value)); } +#ifndef __APPLE__ void GetSQLColAttributesNumeric(SQLHSTMT stmt, const std::wstring& wsql, SQLUSMALLINT idx, SQLUSMALLINT field_identifier, SQLLEN* value) { // Execute query and check SQLColAttribute numeric attribute @@ -377,7 +382,7 @@ void GetSQLColAttributesNumeric(SQLHSTMT stmt, const std::wstring& wsql, SQLUSMA ASSERT_EQ(SQL_SUCCESS, SQLColAttributes(stmt, idx, field_identifier, 0, 0, nullptr, value)); } - +#endif // __APPLE__ } // namespace TYPED_TEST(ColumnsTest, SQLColumnsTestInputData) { @@ -1115,7 +1120,7 @@ TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColumnsAllTypesODBCVer2) { EXPECT_EQ(SQL_NO_DATA, SQLFetch(this->stmt)); } -TEST_F(ColumnsMockTest, TestSQLColumnsColumnPattern) { +TEST_F(ColumnsMockTest, TestSQLColumnsColumnPatternSegFault) { // Checks filtering table with column name pattern. // Only check table and column name @@ -1387,7 +1392,9 @@ TEST_F(ColumnsMockTest, TestSQLColAttributeAllTypes) { SQL_FALSE); // expected_unsigned_column } -TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesAllTypesODBCVer2) { +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. +#ifndef __APPLE__ +TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesAllTypes) { // Tests ODBC 2.0 API SQLColAttributes this->CreateTableAllDataType(); @@ -1446,6 +1453,7 @@ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesAllTypesODBCVer2) { SQL_PRED_NONE, // expected_searchable SQL_FALSE); // expected_unsigned_column } +#endif // __APPLE__ TEST_F(ColumnsRemoteTest, TestSQLColAttributeAllTypes) { // Test assumes there is a table $scratch.ODBCTest in remote server @@ -1612,6 +1620,8 @@ TEST_F(ColumnsRemoteTest, TestSQLColAttributeAllTypes) { SQL_TRUE); // expected_unsigned_column } +// iODBC does not support SQLColAttribute in ODBC 2.0 mode. +#ifndef __APPLE__ TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColAttributeAllTypesODBCVer2) { // Test assumes there is a table $scratch.ODBCTest in remote server std::wstring wsql = L"SELECT * from $scratch.ODBCTest;"; @@ -1776,6 +1786,7 @@ TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColAttributeAllTypesODBCVer2) { SQL_TRUE); // expected_unsigned_column } +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColAttributesAllTypesODBCVer2) { // Tests ODBC 2.0 API SQLColAttributes // Test assumes there is a table $scratch.ODBCTest in remote server @@ -1895,6 +1906,7 @@ TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColAttributesAllTypesODBCVer2) { SQL_SEARCHABLE, // expected_searchable SQL_TRUE); // expected_unsigned_column } +#endif // __APPLE__ TYPED_TEST(ColumnsTest, TestSQLColAttributeCaseSensitive) { // Arrow limitation: returns SQL_FALSE for case sensitive column @@ -1910,6 +1922,8 @@ TYPED_TEST(ColumnsTest, TestSQLColAttributeCaseSensitive) { ASSERT_EQ(SQL_FALSE, value); } +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. +#ifndef __APPLE__ TYPED_TEST(ColumnsOdbcV2Test, TestSQLColAttributesCaseSensitive) { // Arrow limitation: returns SQL_FALSE for case sensitive column // Tests ODBC 2.0 API SQLColAttributes @@ -1924,6 +1938,7 @@ TYPED_TEST(ColumnsOdbcV2Test, TestSQLColAttributesCaseSensitive) { GetSQLColAttributesNumeric(this->stmt, wsql, 28, SQL_COLUMN_CASE_SENSITIVE, &value); ASSERT_EQ(SQL_FALSE, value); } +#endif // __APPLE__ TEST_F(ColumnsMockTest, TestSQLColAttributeUniqueValue) { // Mock server limitation: returns false for auto-increment column @@ -1935,6 +1950,8 @@ TEST_F(ColumnsMockTest, TestSQLColAttributeUniqueValue) { ASSERT_EQ(SQL_FALSE, value); } +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. +#ifndef __APPLE__ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesAutoIncrement) { // Tests ODBC 2.0 API SQLColAttributes // Mock server limitation: returns false for auto-increment column @@ -1945,6 +1962,7 @@ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesAutoIncrement) { GetSQLColAttributeNumeric(this->stmt, wsql, 1, SQL_COLUMN_AUTO_INCREMENT, &value); ASSERT_EQ(SQL_FALSE, value); } +#endif // __APPLE__ TEST_F(ColumnsMockTest, TestSQLColAttributeBaseTableName) { this->CreateTableAllDataType(); @@ -1955,6 +1973,8 @@ TEST_F(ColumnsMockTest, TestSQLColAttributeBaseTableName) { ASSERT_EQ(std::wstring(L"AllTypesTable"), value); } +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. +#ifndef __APPLE__ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesTableName) { // Tests ODBC 2.0 API SQLColAttributes this->CreateTableAllDataType(); @@ -1964,6 +1984,7 @@ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesTableName) { GetSQLColAttributesString(this->stmt, wsql, 1, SQL_COLUMN_TABLE_NAME, value); ASSERT_EQ(std::wstring(L"AllTypesTable"), value); } +#endif // __APPLE__ TEST_F(ColumnsMockTest, TestSQLColAttributeCatalogName) { // Mock server limitattion: mock doesn't return catalog for result metadata, @@ -1985,6 +2006,8 @@ TEST_F(ColumnsRemoteTest, TestSQLColAttributeCatalogName) { ASSERT_EQ(std::wstring(L""), value); } +// iODBC does not support SQLColAttribute in ODBC 2.0 mode. +#ifndef __APPLE__ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesQualifierName) { // Mock server limitattion: mock doesn't return catalog for result metadata, // and the defautl catalog should be 'main' @@ -2005,6 +2028,7 @@ TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColAttributesQualifierName) { GetSQLColAttributeString(this->stmt, wsql, 1, SQL_COLUMN_QUALIFIER_NAME, value); ASSERT_EQ(std::wstring(L""), value); } +#endif // __APPLE__ TYPED_TEST(ColumnsTest, TestSQLColAttributeCount) { std::wstring wsql = this->GetQueryAllDataTypes(); @@ -2050,6 +2074,8 @@ TEST_F(ColumnsRemoteTest, TestSQLColAttributeSchemaName) { ASSERT_EQ(std::wstring(L""), value); } +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. +#ifndef __APPLE__ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesOwnerName) { // Tests ODBC 2.0 API SQLColAttributes this->CreateTableAllDataType(); @@ -2071,6 +2097,7 @@ TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColAttributesOwnerName) { GetSQLColAttributesString(this->stmt, wsql, 1, SQL_COLUMN_OWNER_NAME, value); ASSERT_EQ(std::wstring(L""), value); } +#endif // __APPLE__ TEST_F(ColumnsMockTest, TestSQLColAttributeTableName) { this->CreateTableAllDataType(); @@ -2119,6 +2146,8 @@ TEST_F(ColumnsRemoteTest, TestSQLColAttributeTypeName) { ASSERT_EQ(std::wstring(L"TIMESTAMP"), value); } +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. +#ifndef __APPLE__ TEST_F(ColumnsOdbcV2MockTest, TestSQLColAttributesTypeName) { // Tests ODBC 2.0 API SQLColAttributes this->CreateTableAllDataType(); @@ -2159,6 +2188,7 @@ TEST_F(ColumnsOdbcV2RemoteTest, TestSQLColAttributesTypeName) { GetSQLColAttributesString(this->stmt, L"", 9, SQL_COLUMN_TYPE_NAME, value); ASSERT_EQ(std::wstring(L"TIMESTAMP"), value); } +#endif // __APPLE__ TYPED_TEST(ColumnsTest, TestSQLColAttributeUnnamed) { std::wstring wsql = this->GetQueryAllDataTypes(); @@ -2175,6 +2205,8 @@ TYPED_TEST(ColumnsTest, TestSQLColAttributeUpdatable) { ASSERT_EQ(SQL_ATTR_READWRITE_UNKNOWN, value); } +// iODBC does not support SQLColAttributes for ODBC 3.0 attributes. +#ifndef __APPLE__ TYPED_TEST(ColumnsOdbcV2Test, TestSQLColAttributesUpdatable) { // Tests ODBC 2.0 API SQLColAttributes std::wstring wsql = this->GetQueryAllDataTypes(); @@ -2183,6 +2215,7 @@ TYPED_TEST(ColumnsOdbcV2Test, TestSQLColAttributesUpdatable) { GetSQLColAttributesNumeric(this->stmt, wsql, 1, SQL_COLUMN_UPDATABLE, &value); ASSERT_EQ(SQL_ATTR_READWRITE_UNKNOWN, value); } +#endif // __APPLE__ TEST_F(ColumnsMockTest, SQLDescribeColValidateInput) { this->CreateTestTables(); @@ -2193,7 +2226,7 @@ TEST_F(ColumnsMockTest, SQLDescribeColValidateInput) { SQLUSMALLINT bookmark_column = 0; SQLUSMALLINT out_of_range_column = 4; SQLUSMALLINT negative_column = -1; - SQLWCHAR column_name[1024]; + SQLWCHAR column_name[1024] = {0}; SQLSMALLINT buf_char_len = static_cast(sizeof(column_name) / GetSqlWCharSize()); SQLSMALLINT name_length = 0; @@ -2210,7 +2243,12 @@ TEST_F(ColumnsMockTest, SQLDescribeColValidateInput) { EXPECT_EQ(SQL_ERROR, SQLDescribeCol(this->stmt, bookmark_column, column_name, buf_char_len, &name_length, &data_type, &column_size, &decimal_digits, &nullable)); +#ifdef __APPLE__ + // non-standard odbc error code for invalid column index + VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1002); +#else VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorState07009); +#endif // __APPLE__ // Invalid descriptor index - index out of range EXPECT_EQ(SQL_ERROR, SQLDescribeCol(this->stmt, out_of_range_column, column_name, diff --git a/cpp/src/arrow/flight/sql/odbc/tests/connection_attr_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/connection_attr_test.cc index 413bc43c0ad9..e12a27b2a887 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/connection_attr_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/connection_attr_test.cc @@ -104,12 +104,15 @@ TYPED_TEST(ConnectionAttributeTest, TestSQLSetConnectAttrQuietModeReadOnly) { VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorStateHY092); } +// iODBC needs to be compiled with tracing enabled to handle SQL_ATTR_TRACE +#ifndef __APPLE__ TYPED_TEST(ConnectionAttributeTest, TestSQLSetConnectAttrTraceDMOnly) { // Verify DM-only attribute is settable via Driver Manager ASSERT_EQ(SQL_SUCCESS, SQLSetConnectAttr(this->conn, SQL_ATTR_TRACE, reinterpret_cast(SQL_OPT_TRACE_OFF), 0)); } +#endif // __APPLE__ TYPED_TEST(ConnectionAttributeTest, TestSQLSetConnectAttrTracefileDMOnly) { // Verify DM-only attribute is handled by Driver Manager @@ -120,14 +123,22 @@ TYPED_TEST(ConnectionAttributeTest, TestSQLSetConnectAttrTracefileDMOnly) { std::vector trace_file0(trace_file.begin(), trace_file.end()); ASSERT_EQ(SQL_ERROR, SQLSetConnectAttr(this->conn, SQL_ATTR_TRACEFILE, &trace_file0[0], static_cast(trace_file0.size()))); +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorStateHYC00); +#else VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorStateHY000); +#endif // __APPLE__ } TYPED_TEST(ConnectionAttributeTest, TestSQLSetConnectAttrTranslateLabDMOnly) { // Verify DM-only attribute is handled by Driver Manager ASSERT_EQ(SQL_ERROR, SQLSetConnectAttr(this->conn, SQL_ATTR_TRANSLATE_LIB, 0, 0)); // Checks for invalid argument return error +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorStateHYC00); +#else VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorStateHY024); +#endif // __APPLE__ } TYPED_TEST(ConnectionAttributeTest, TestSQLSetConnectAttrTranslateOptionUnsupported) { @@ -152,6 +163,8 @@ TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrDbcInfoTokenSetOnly) { } #endif +// iODBC does not treat SQL_ATTR_ODBC_CURSORS as DM-only +#ifndef __APPLE__ TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrOdbcCursorsDMOnly) { // Verify that DM-only attribute is handled by driver manager SQLULEN cursor_attr; @@ -160,6 +173,7 @@ TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrOdbcCursorsDMOnly) { EXPECT_EQ(SQL_CUR_USE_DRIVER, cursor_attr); } +// iODBC needs to be compiled with tracing enabled to handle SQL_ATTR_TRACE TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrTraceDMOnly) { // Verify that DM-only attribute is handled by driver manager SQLUINTEGER trace; @@ -168,6 +182,7 @@ TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrTraceDMOnly) { EXPECT_EQ(SQL_OPT_TRACE_OFF, trace); } +// iODBC needs to be compiled with tracing enabled to handle SQL_ATTR_TRACEFILE TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrTraceFileDMOnly) { // Verify that DM-only attribute is handled by driver manager SQLWCHAR out_str[kOdbcBufferSize]; @@ -176,11 +191,12 @@ TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrTraceFileDMOnly) { kOdbcBufferSize, &out_str_len)); // Length is returned in bytes for SQLGetConnectAttr, // we want the number of characters - out_str_len /= arrow::flight::sql::odbc::GetSqlWCharSize(); + out_str_len /= GetSqlWCharSize(); std::string out_connection_string = ODBC::SqlWcharToString(out_str, static_cast(out_str_len)); EXPECT_FALSE(out_connection_string.empty()); } +#endif // __APPLE__ TYPED_TEST(ConnectionAttributeTest, TestSQLGetConnectAttrTranslateLibUnsupported) { SQLWCHAR out_str[kOdbcBufferSize]; diff --git a/cpp/src/arrow/flight/sql/odbc/tests/connection_info_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/connection_info_test.cc index c9ee212224a1..09cffefaa31e 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/connection_info_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/connection_info_test.cc @@ -73,7 +73,7 @@ TYPED_TEST(ConnectionInfoTest, TestSQLGetInfoTruncation) { SQLSMALLINT message_length; ASSERT_EQ(SQL_SUCCESS_WITH_INFO, - SQLGetInfo(this->conn, SQL_KEYWORDS, value, info_len, &message_length)); + SQLGetInfo(this->conn, SQL_INTEGRITY, value, info_len, &message_length)); // Verify string truncation is reported VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorState01004); @@ -319,7 +319,11 @@ TYPED_TEST(ConnectionInfoTest, TestSQLGetInfoOdbcVer) { SQLWCHAR value[kOdbcBufferSize] = L""; GetInfo(this->conn, SQL_ODBC_VER, value); +#ifdef __APPLE__ + EXPECT_STREQ(static_cast(L"03.52.0000"), value); +#else EXPECT_STREQ(static_cast(L"03.80.0000"), value); +#endif // __APPLE__ } TYPED_TEST(ConnectionInfoTest, TestSQLGetInfoParamArrayRowCounts) { @@ -786,8 +790,8 @@ TYPED_TEST(ConnectionInfoTest, TestSQLGetInfoIntegrity) { } TYPED_TEST(ConnectionInfoTest, TestSQLGetInfoKeywords) { - // Keyword strings can require 5000 buffer length - static constexpr int info_len = kOdbcBufferSize * 5; + // Keyword strings can require 10000 buffer length + static constexpr int info_len = kOdbcBufferSize * 10; SQLWCHAR value[info_len] = L""; GetInfo(this->conn, SQL_KEYWORDS, value, info_len); diff --git a/cpp/src/arrow/flight/sql/odbc/tests/dremio/docker-compose.yml b/cpp/src/arrow/flight/sql/odbc/tests/dremio/docker-compose.yml new file mode 100644 index 000000000000..eaab4d02b731 --- /dev/null +++ b/cpp/src/arrow/flight/sql/odbc/tests/dremio/docker-compose.yml @@ -0,0 +1,35 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# GH-48068 TODO: run remote ODBC tests on Linux + +services: + dremio: + platform: linux/x86_64 + image: dremio/dremio-oss:latest + ports: + - 9047:9047 # REST API + - 31010:31010 # JDBC/ODBC + - 32010:32010 + container_name: dremio_container + environment: + - DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dsaffron.default.charset=UTF-8 -Dsaffron.default.nationalcharset=UTF-8 -Dsaffron.default.collation.name=UTF-8$$en_US + healthcheck: + test: curl --fail http://localhost:9047 || exit 1 + interval: 10s + timeout: 5s + retries: 30 diff --git a/cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh b/cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh new file mode 100644 index 000000000000..8d632bb2c3e1 --- /dev/null +++ b/cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh @@ -0,0 +1,66 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# GH-48068 TODO: run remote ODBC tests on Linux + +#!/bin/bash +set -e + +HOST_URL="http://localhost:9047" +NEW_USER_URL="$HOST_URL/apiv2/bootstrap/firstuser" +LOGIN_URL="$HOST_URL/apiv2/login" +SQL_URL="$HOST_URL/api/v3/sql" + +ADMIN_USER="admin" +ADMIN_PASSWORD="admin2025" + +# Wait for Dremio to be available. +until curl -s "$NEW_USER_URL"; do + echo 'Waiting for Dremio to start...' + sleep 5 +done + +echo "" +echo 'Creating admin user...' + +# Create new admin account. +curl -X PUT "$NEW_USER_URL" \ + -H "Content-Type: application/json" \ + -d "{ \"userName\": \"$ADMIN_USER\", \"password\": \"$ADMIN_PASSWORD\" }" + +echo "" +echo "Created admin user." + +# Use admin account to login and acquire a token. +TOKEN=$(curl -s -X POST "$LOGIN_URL" \ + -H "Content-Type: application/json" \ + -d "{ \"userName\": \"$ADMIN_USER\", \"password\": \"$ADMIN_PASSWORD\" }" \ + | grep -oP '(?<="token":")[^"]+') + +SQL_QUERY="Create Table \$scratch.ODBCTest As SELECT CAST(2147483647 AS INTEGER) AS sinteger_max, CAST(9223372036854775807 AS BIGINT) AS sbigint_max, CAST(999999999 AS DECIMAL(38,0)) AS decimal_positive, CAST(3.40282347E38 AS FLOAT) AS float_max, CAST(1.7976931348623157E308 AS DOUBLE) AS double_max, CAST(true AS BOOLEAN) AS bit_true, CAST(DATE '9999-12-31' AS DATE) AS date_max, CAST(TIME '23:59:59' AS TIME) AS time_max, CAST(TIMESTAMP '9999-12-31 23:59:59' AS TIMESTAMP) AS timestamp_max;" +ESCAPED_QUERY=$(printf '%s' "$SQL_QUERY" | sed 's/"/\\"/g') + +echo "Creating \$scratch.ODBCTest table." + +# Create a new table by sending a SQL query. +curl -i -X POST "$SQL_URL" \ + -H "Authorization: _dremio$TOKEN" \ + -H "Content-Type: application/json" \ + -d "{\"sql\": \"$ESCAPED_QUERY\"}" + +echo "" +echo "Finished setting up dremio docker instance." diff --git a/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc index 34a32738455a..53eeac1aba39 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc @@ -47,6 +47,8 @@ using TestTypesHandle = ::testing::Types; TYPED_TEST_SUITE(ErrorsHandleTest, TestTypesHandle); +using ODBC::SqlWcharToString; + TYPED_TEST(ErrorsHandleTest, TestSQLGetDiagFieldWForConnectFailure) { // Invalid connect string std::string connect_str = this->GetInvalidConnectionString(); @@ -90,9 +92,12 @@ TYPED_TEST(ErrorsHandleTest, TestSQLGetDiagFieldWForConnectFailure) { SQLWCHAR message_text[kOdbcBufferSize]; SQLSMALLINT message_text_length; - EXPECT_EQ(SQL_SUCCESS, - SQLGetDiagField(SQL_HANDLE_DBC, this->conn, RECORD_1, SQL_DIAG_MESSAGE_TEXT, - message_text, kOdbcBufferSize, &message_text_length)); + SQLRETURN ret = + SQLGetDiagField(SQL_HANDLE_DBC, this->conn, RECORD_1, SQL_DIAG_MESSAGE_TEXT, + message_text, kOdbcBufferSize, &message_text_length); + + // dependent on the size of the message it could output SQL_SUCCESS_WITH_INFO + EXPECT_TRUE(ret == SQL_SUCCESS || ret == SQL_SUCCESS_WITH_INFO); EXPECT_GT(message_text_length, 100); @@ -114,10 +119,9 @@ TYPED_TEST(ErrorsHandleTest, TestSQLGetDiagFieldWForConnectFailure) { EXPECT_EQ( SQL_SUCCESS, SQLGetDiagField(SQL_HANDLE_DBC, this->conn, RECORD_1, SQL_DIAG_SQLSTATE, sql_state, - sql_state_size * arrow::flight::sql::odbc::GetSqlWCharSize(), - &sql_state_length)); + sql_state_size * GetSqlWCharSize(), &sql_state_length)); - EXPECT_EQ(std::wstring(L"28000"), std::wstring(sql_state)); + EXPECT_EQ(kErrorState28000, SqlWcharToString(sql_state)); } TYPED_TEST(ErrorsHandleTest, DISABLED_TestSQLGetDiagFieldWForConnectFailureNTS) { @@ -156,6 +160,8 @@ TYPED_TEST(ErrorsHandleTest, DISABLED_TestSQLGetDiagFieldWForConnectFailureNTS) EXPECT_GT(message_text_length, 100); } +// iODBC does not support application allocated descriptors. +#ifndef __APPLE__ TYPED_TEST(ErrorsTest, TestSQLGetDiagFieldWForDescriptorFailureFromDriverManager) { SQLHDESC descriptor; @@ -216,7 +222,7 @@ TYPED_TEST(ErrorsTest, TestSQLGetDiagFieldWForDescriptorFailureFromDriverManager SQLGetDiagField(SQL_HANDLE_DESC, descriptor, RECORD_1, SQL_DIAG_SQLSTATE, sql_state, sql_state_size * GetSqlWCharSize(), &sql_state_length)); - EXPECT_EQ(std::wstring(L"IM001"), std::wstring(sql_state)); + EXPECT_EQ(kErrorStateIM001, SqlWcharToString(sql_state)); // Free descriptor handle EXPECT_EQ(SQL_SUCCESS, SQLFreeHandle(SQL_HANDLE_DESC, descriptor)); @@ -245,13 +251,14 @@ TYPED_TEST(ErrorsTest, TestSQLGetDiagRecForDescriptorFailureFromDriverManager) { EXPECT_EQ(0, native_error); // API not implemented error from driver manager - EXPECT_EQ(std::wstring(L"IM001"), std::wstring(sql_state)); + EXPECT_EQ(kErrorStateIM001, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); // Free descriptor handle EXPECT_EQ(SQL_SUCCESS, SQLFreeHandle(SQL_HANDLE_DESC, descriptor)); } +#endif // __APPLE__ TYPED_TEST(ErrorsHandleTest, TestSQLGetDiagRecForConnectFailure) { // Invalid connect string @@ -282,7 +289,7 @@ TYPED_TEST(ErrorsHandleTest, TestSQLGetDiagRecForConnectFailure) { EXPECT_EQ(200, native_error); - EXPECT_EQ(std::wstring(L"28000"), std::wstring(sql_state)); + EXPECT_EQ(kErrorState28000, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } @@ -305,11 +312,17 @@ TYPED_TEST(ErrorsTest, TestSQLGetDiagRecInputData) { nullptr, 0, nullptr)); // Invalid handle +#ifdef __APPLE__ + // MacOS ODBC driver manager requires connection handle + EXPECT_EQ(SQL_INVALID_HANDLE, + SQLGetDiagRec(0, this->conn, 1, nullptr, nullptr, nullptr, 0, nullptr)); +#else EXPECT_EQ(SQL_INVALID_HANDLE, SQLGetDiagRec(0, nullptr, 0, nullptr, nullptr, nullptr, 0, nullptr)); +#endif // __APPLE__ } -TYPED_TEST(ErrorsTest, TestSQLErrorInputData) { +TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorInputData) { // Test ODBC 2.0 API SQLError. Driver manager maps SQLError to SQLGetDiagRec. // SQLError does not post diagnostic records for itself. @@ -320,8 +333,13 @@ TYPED_TEST(ErrorsTest, TestSQLErrorInputData) { EXPECT_EQ(SQL_NO_DATA, SQLError(nullptr, this->conn, nullptr, nullptr, nullptr, nullptr, 0, nullptr)); +#ifdef __APPLE__ + EXPECT_EQ(SQL_NO_DATA, SQLError(SQL_NULL_HENV, this->conn, this->stmt, nullptr, nullptr, + nullptr, 0, nullptr)); +#else EXPECT_EQ(SQL_NO_DATA, SQLError(nullptr, nullptr, this->stmt, nullptr, nullptr, nullptr, 0, nullptr)); +#endif // __APPLE__ // Invalid handle EXPECT_EQ(SQL_INVALID_HANDLE, @@ -345,12 +363,12 @@ TYPED_TEST(ErrorsTest, TestSQLErrorEnvErrorFromDriverManager) { ASSERT_EQ(SQL_SUCCESS, SQLError(this->env, nullptr, nullptr, sql_state, &native_error, message, SQL_MAX_MESSAGE_LENGTH, &message_length)); - EXPECT_GT(message_length, 50); + EXPECT_GT(message_length, 40); EXPECT_EQ(0, native_error); // Function sequence error state from driver manager - EXPECT_EQ(std::wstring(L"HY010"), std::wstring(sql_state)); + EXPECT_EQ(kErrorStateHY010, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } @@ -362,9 +380,8 @@ TYPED_TEST(ErrorsTest, TestSQLErrorConnError) { // DM passes 512 as buffer length to SQLError. // Attempt to set unsupported attribute - SQLRETURN ret = SQLGetConnectAttr(this->conn, SQL_ATTR_TXN_ISOLATION, 0, 0, nullptr); - - ASSERT_EQ(SQL_ERROR, ret); + ASSERT_EQ(SQL_ERROR, + SQLGetConnectAttr(this->conn, SQL_ATTR_TXN_ISOLATION, 0, 0, nullptr)); SQLWCHAR sql_state[6] = {0}; SQLINTEGER native_error = 0; @@ -378,7 +395,7 @@ TYPED_TEST(ErrorsTest, TestSQLErrorConnError) { EXPECT_EQ(100, native_error); // optional feature not supported error state - EXPECT_EQ(std::wstring(L"HYC00"), std::wstring(sql_state)); + EXPECT_EQ(kErrorStateHYC00, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } @@ -399,14 +416,16 @@ TYPED_TEST(ErrorsTest, TestSQLErrorStmtError) { SQLINTEGER native_error = 0; SQLWCHAR message[SQL_MAX_MESSAGE_LENGTH] = {0}; SQLSMALLINT message_length = 0; - ASSERT_EQ(SQL_SUCCESS, SQLError(nullptr, nullptr, this->stmt, sql_state, &native_error, - message, SQL_MAX_MESSAGE_LENGTH, &message_length)); + SQLRETURN ret = SQLError(nullptr, this->conn, this->stmt, sql_state, &native_error, + message, SQL_MAX_MESSAGE_LENGTH, &message_length); + + EXPECT_TRUE(ret == SQL_SUCCESS || ret == SQL_SUCCESS_WITH_INFO); EXPECT_GT(message_length, 70); EXPECT_EQ(100, native_error); - EXPECT_EQ(std::wstring(L"HY000"), std::wstring(sql_state)); + EXPECT_EQ(kErrorStateHY000, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } @@ -434,20 +453,21 @@ TYPED_TEST(ErrorsTest, TestSQLErrorStmtWarning) { SQLINTEGER native_error = 0; SQLWCHAR message[SQL_MAX_MESSAGE_LENGTH] = {0}; SQLSMALLINT message_length = 0; - ASSERT_EQ(SQL_SUCCESS, SQLError(nullptr, nullptr, this->stmt, sql_state, &native_error, - message, SQL_MAX_MESSAGE_LENGTH, &message_length)); + ASSERT_EQ(SQL_SUCCESS, + SQLError(SQL_NULL_HENV, this->conn, this->stmt, sql_state, &native_error, + message, SQL_MAX_MESSAGE_LENGTH, &message_length)); EXPECT_GT(message_length, 50); EXPECT_EQ(1000100, native_error); // Verify string truncation warning is reported - EXPECT_EQ(std::wstring(L"01004"), std::wstring(sql_state)); + EXPECT_EQ(kErrorState01004, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } -TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorEnvErrorODBCVer2FromDriverManager) { +TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorEnvErrorFromDriverManager) { // Test ODBC 2.0 API SQLError with ODBC ver 2. // Known Windows Driver Manager (DM) behavior: // When application passes buffer length greater than SQL_MAX_MESSAGE_LENGTH (512), @@ -464,22 +484,35 @@ TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorEnvErrorODBCVer2FromDriverManager) { ASSERT_EQ(SQL_SUCCESS, SQLError(this->env, nullptr, nullptr, sql_state, &native_error, message, SQL_MAX_MESSAGE_LENGTH, &message_length)); - EXPECT_GT(message_length, 50); + EXPECT_GT(message_length, 40); EXPECT_EQ(0, native_error); // Function sequence error state from driver manager - EXPECT_EQ(std::wstring(L"S1010"), std::wstring(sql_state)); +#ifdef _WIN32 + // Windows Driver Manager returns S1010 + EXPECT_EQ(kErrorStateS1010, SqlWcharToString(sql_state)); +#else + // unix Driver Manager returns HY010 + EXPECT_EQ(kErrorStateHY010, SqlWcharToString(sql_state)); +#endif // _WIN32 EXPECT_FALSE(std::wstring(message).empty()); } -TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorConnErrorODBCVer2) { +// TODO: verify that `SQLGetConnectOption` is not required by Excel. +#ifndef __APPLE__ +TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorConnError) { // Test ODBC 2.0 API SQLError with ODBC ver 2. // Known Windows Driver Manager (DM) behavior: // When application passes buffer length greater than SQL_MAX_MESSAGE_LENGTH (512), // DM passes 512 as buffer length to SQLError. + // Known macOS Driver Manager (DM) behavior: + // Attempts to call SQLGetConnectOption without redirecting the API call to + // SQLGetConnectAttr. SQLGetConnectOption is not implemented as it is not required by + // macOS Excel. + // Attempt to set unsupported attribute ASSERT_EQ(SQL_ERROR, SQLGetConnectAttr(this->conn, SQL_ATTR_TXN_ISOLATION, 0, 0, nullptr)); @@ -496,12 +529,13 @@ TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorConnErrorODBCVer2) { EXPECT_EQ(100, native_error); // optional feature not supported error state. Driver Manager maps state to S1C00 - EXPECT_EQ(std::wstring(L"S1C00"), std::wstring(sql_state)); + EXPECT_EQ(kErrorStateS1C00, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } +#endif // __APPLE__ -TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtErrorODBCVer2) { +TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtError) { // Test ODBC 2.0 API SQLError with ODBC ver 2. // Known Windows Driver Manager (DM) behavior: // When application passes buffer length greater than SQL_MAX_MESSAGE_LENGTH (512), @@ -525,12 +559,12 @@ TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtErrorODBCVer2) { EXPECT_EQ(100, native_error); // Driver Manager maps error state to S1000 - EXPECT_EQ(std::wstring(L"S1000"), std::wstring(sql_state)); + EXPECT_EQ(kErrorStateS1000, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } -TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtWarningODBCVer2) { +TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtWarning) { // Test ODBC 2.0 API SQLError. std::wstring wsql = L"SELECT 'VERY LONG STRING here' AS string_col;"; @@ -553,15 +587,16 @@ TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtWarningODBCVer2) { SQLINTEGER native_error = 0; SQLWCHAR message[SQL_MAX_MESSAGE_LENGTH] = {0}; SQLSMALLINT message_length = 0; - ASSERT_EQ(SQL_SUCCESS, SQLError(nullptr, nullptr, this->stmt, sql_state, &native_error, - message, SQL_MAX_MESSAGE_LENGTH, &message_length)); + ASSERT_EQ(SQL_SUCCESS, + SQLError(SQL_NULL_HENV, this->conn, this->stmt, sql_state, &native_error, + message, SQL_MAX_MESSAGE_LENGTH, &message_length)); EXPECT_GT(message_length, 50); EXPECT_EQ(1000100, native_error); // Verify string truncation warning is reported - EXPECT_EQ(std::wstring(L"01004"), std::wstring(sql_state)); + EXPECT_EQ(kErrorState01004, SqlWcharToString(sql_state)); EXPECT_FALSE(std::wstring(message).empty()); } diff --git a/cpp/src/arrow/flight/sql/odbc/tests/get_functions_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/get_functions_test.cc index 35d7f2f935d6..2181f8a3b408 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/get_functions_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/get_functions_test.cc @@ -40,6 +40,8 @@ using TestTypesOdbcV2 = ::testing::Types; TYPED_TEST_SUITE(GetFunctionsOdbcV2Test, TestTypesOdbcV2); +// MacOS driver manager iODBC does not support SQLGetFunctions for ODBC 3.x or 2.x driver +#ifndef __APPLE__ TYPED_TEST(GetFunctionsTest, TestSQLGetFunctionsAllFunctions) { // Verify driver manager return values for SQLGetFunctions @@ -87,7 +89,7 @@ TYPED_TEST(GetFunctionsTest, TestSQLGetFunctionsAllFunctions) { } } -TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsAllFunctionsODBCVer2) { +TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsAllFunctions) { // Verify driver manager return values for SQLGetFunctions // ODBC 2.0 SQLGetFunctions returns 100 elements according to spec @@ -175,7 +177,7 @@ TYPED_TEST(GetFunctionsTest, TestSQLGetFunctionsUnsupportedSingleAPI) { } } -TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsSupportedSingleAPIODBCVer2) { +TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsSupportedSingleAPI) { const std::vector supported_functions = { SQL_API_SQLCONNECT, SQL_API_SQLGETINFO, SQL_API_SQLDESCRIBECOL, SQL_API_SQLGETTYPEINFO, SQL_API_SQLDISCONNECT, SQL_API_SQLNUMRESULTCOLS, @@ -199,7 +201,7 @@ TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsSupportedSingleAPIODBCVer2 } } -TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsUnsupportedSingleAPIODBCVer2) { +TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsUnsupportedSingleAPI) { const std::vector unsupported_functions = { SQL_API_SQLPUTDATA, SQL_API_SQLPARAMDATA, SQL_API_SQLSETCURSORNAME, SQL_API_SQLGETCURSORNAME, SQL_API_SQLSTATISTICS, SQL_API_SQLSPECIALCOLUMNS, @@ -216,5 +218,6 @@ TYPED_TEST(GetFunctionsOdbcV2Test, TestSQLGetFunctionsUnsupportedSingleAPIODBCVe api_exists = -1; } } +#endif // __APPLE__ } // namespace arrow::flight::sql::odbc diff --git a/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.h b/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.h index 3115cd627547..2de951690b15 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.h +++ b/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.h @@ -206,7 +206,13 @@ class FlightSQLOdbcEnvConnHandleMockTestBase : public FlightSQLODBCMockTestBase }; /** ODBC read buffer size. */ +#ifdef __APPLE__ +// iODBC driver manager may crash with smaller buffer sizes +// so we use a larger buffer on MacOS +static constexpr int kOdbcBufferSize = 2048; +#else static constexpr int kOdbcBufferSize = 1024; +#endif // __APPLE__ /// Compare ConnPropertyMap, key value is case-insensitive bool CompareConnPropertyMap(Connection::ConnPropertyMap map1, @@ -236,10 +242,13 @@ static constexpr std::string_view kErrorStateHY106 = "HY106"; static constexpr std::string_view kErrorStateHY114 = "HY114"; static constexpr std::string_view kErrorStateHY118 = "HY118"; static constexpr std::string_view kErrorStateHYC00 = "HYC00"; +static constexpr std::string_view kErrorStateIM001 = "IM001"; +static constexpr std::string_view kErrorStateS1000 = "S1000"; static constexpr std::string_view kErrorStateS1002 = "S1002"; static constexpr std::string_view kErrorStateS1004 = "S1004"; static constexpr std::string_view kErrorStateS1010 = "S1010"; static constexpr std::string_view kErrorStateS1090 = "S1090"; +static constexpr std::string_view kErrorStateS1C00 = "S1C00"; /// Verify ODBC Error State void VerifyOdbcErrorState(SQLSMALLINT handle_type, SQLHANDLE handle, diff --git a/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc index 0a4e99d33a6f..ac2cbb79864c 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc @@ -104,14 +104,18 @@ void ValidateSetStmtAttr(SQLHSTMT statement, SQLINTEGER attribute, SQLPOINTER va // Validate error return value and code void ValidateSetStmtAttrErrorCode(SQLHSTMT statement, SQLINTEGER attribute, - SQLULEN new_value, std::string_view error_code) { + SQLULEN new_value, SQLRETURN expected_rc, + std::string_view error_code) { SQLINTEGER string_length_ptr = sizeof(SQLULEN); - ASSERT_EQ(SQL_ERROR, + ASSERT_EQ(expected_rc, SQLSetStmtAttr(statement, attribute, reinterpret_cast(new_value), - string_length_ptr)); + string_length_ptr)) + << GetOdbcErrorMessage(SQL_HANDLE_STMT, statement); - VerifyOdbcErrorState(SQL_HANDLE_STMT, statement, error_code); + if (expected_rc == SQL_ERROR) { + VerifyOdbcErrorState(SQL_HANDLE_STMT, statement, error_code); + } } } // namespace @@ -329,8 +333,7 @@ TYPED_TEST(StatementAttributeTest, TestSQLGetStmtAttrRowBindType) { EXPECT_EQ(static_cast(0), value); } -TYPED_TEST(StatementAttributeTest, DISABLED_TestSQLGetStmtAttrRowNumber) { - // GH-47711 TODO: enable test after SQLExecDirect support +TYPED_TEST(StatementAttributeTest, TestSQLGetStmtAttrRowNumber) { std::wstring wsql = L"SELECT 1;"; std::vector sql0(wsql.begin(), wsql.end()); @@ -418,21 +421,21 @@ TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrAppRowDesc) { TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrAsyncEnableUnsupported) { // Optional feature not implemented ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ASYNC_ENABLE, SQL_ASYNC_ENABLE_OFF, - kErrorStateHYC00); + SQL_ERROR, kErrorStateHYC00); } #endif #ifdef SQL_ATTR_ASYNC_STMT_EVENT TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrAsyncStmtEventUnsupported) { // Driver does not support asynchronous notification - ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ASYNC_STMT_EVENT, 0, + ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ASYNC_STMT_EVENT, 0, SQL_ERROR, kErrorStateHY118); } #endif #ifdef SQL_ATTR_ASYNC_STMT_PCALLBACK TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrAsyncStmtPCCallbackUnsupported) { - ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ASYNC_STMT_PCALLBACK, 0, + ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ASYNC_STMT_PCALLBACK, 0, SQL_ERROR, kErrorStateHYC00); } #endif @@ -440,7 +443,7 @@ TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrAsyncStmtPCCallbackUnsuppor #ifdef SQL_ATTR_ASYNC_STMT_PCONTEXT TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrAsyncStmtPCContextUnsupported) { // Optional feature not implemented - ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ASYNC_STMT_PCONTEXT, 0, + ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ASYNC_STMT_PCONTEXT, 0, SQL_ERROR, kErrorStateHYC00); } #endif @@ -477,12 +480,25 @@ TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrFetchBookmarkPointer) { TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrIMPParamDesc) { // Invalid use of an automatically allocated descriptor handle ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_IMP_PARAM_DESC, - static_cast(0), kErrorStateHY017); + static_cast(0), +#ifdef __APPLE__ + // iODBC on MacOS returns SQL_INVALID_HANDLE for this case + SQL_INVALID_HANDLE, +#else + SQL_ERROR, +#endif + kErrorStateHY017); } TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrIMPRowDesc) { // Invalid use of an automatically allocated descriptor handle ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_IMP_ROW_DESC, static_cast(0), +#ifdef __APPLE__ + // iODBC on MacOS returns SQL_INVALID_HANDLE for this case + SQL_INVALID_HANDLE, +#else + SQL_ERROR, +#endif kErrorStateHY017); } @@ -497,7 +513,7 @@ TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrMaxLength) { TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrMaxRows) { // Cannot set read-only attribute ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_MAX_ROWS, static_cast(0), - kErrorStateHY092); + SQL_ERROR, kErrorStateHY092); } TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrMetadataID) { @@ -602,7 +618,7 @@ TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrRowBindType) { TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrRowNumber) { // Cannot set read-only attribute ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_ROW_NUMBER, static_cast(0), - kErrorStateHY092); + SQL_ERROR, kErrorStateHY092); } TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrRowOperationPtr) { diff --git a/cpp/src/arrow/flight/sql/odbc/tests/statement_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/statement_test.cc index 1a38bd0e2429..a4e120a27e3e 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/statement_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/statement_test.cc @@ -54,9 +54,15 @@ TYPED_TEST(StatementTest, TestSQLExecDirectSimpleQuery) { ASSERT_EQ(SQL_NO_DATA, SQLFetch(this->stmt)); +#ifdef __APPLE__ + // With iODBC we expect SQL_SUCCESS and the buffer unchanged in this situation. + ASSERT_EQ(SQL_SUCCESS, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, nullptr)); + EXPECT_EQ(1, val); +#else ASSERT_EQ(SQL_ERROR, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, nullptr)); // Invalid cursor state VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorState24000); +#endif } TYPED_TEST(StatementTest, TestSQLExecDirectInvalidQuery) { @@ -89,9 +95,15 @@ TYPED_TEST(StatementTest, TestSQLExecuteSimpleQuery) { ASSERT_EQ(SQL_NO_DATA, SQLFetch(this->stmt)); +#ifdef __APPLE__ + // With iODBC we expect SQL_SUCCESS and the buffer unchanged in this situation. + ASSERT_EQ(SQL_SUCCESS, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, nullptr)); + EXPECT_EQ(1, val); +#else ASSERT_EQ(SQL_ERROR, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, nullptr)); // Invalid cursor state VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorState24000); +#endif } TYPED_TEST(StatementTest, TestSQLPrepareInvalidQuery) { @@ -105,7 +117,11 @@ TYPED_TEST(StatementTest, TestSQLPrepareInvalidQuery) { ASSERT_EQ(SQL_ERROR, SQLExecute(this->stmt)); // Verify function sequence error state is returned +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1010); +#else VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateHY010); +#endif // __APPLE__ } TYPED_TEST(StatementTest, TestSQLExecDirectDataQuery) { @@ -805,10 +821,15 @@ TYPED_TEST(StatementTest, TestSQLExecDirectRowFetching) { // Verify result set has no more data beyond row 3 ASSERT_EQ(SQL_NO_DATA, SQLFetch(this->stmt)); +#ifdef __APPLE__ + // With iODBC we expect SQL_SUCCESS and the buffer unchanged in this situation. + ASSERT_EQ(SQL_SUCCESS, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, nullptr)); + EXPECT_EQ(3, val); +#else ASSERT_EQ(SQL_ERROR, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, &ind)); - // Invalid cursor state VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorState24000); +#endif } TYPED_TEST(StatementTest, TestSQLFetchScrollRowFetching) { @@ -864,9 +885,15 @@ TYPED_TEST(StatementTest, TestSQLFetchScrollRowFetching) { // Verify result set has no more data beyond row 3 ASSERT_EQ(SQL_NO_DATA, SQLFetchScroll(this->stmt, SQL_FETCH_NEXT, 0)); +#ifdef __APPLE__ + // With iODBC we expect SQL_SUCCESS and the buffer unchanged in this situation. + ASSERT_EQ(SQL_SUCCESS, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, nullptr)); + EXPECT_EQ(3, val); +#else ASSERT_EQ(SQL_ERROR, SQLGetData(this->stmt, 1, SQL_C_LONG, &val, 0, &ind)); // Invalid cursor state VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorState24000); +#endif } TYPED_TEST(StatementTest, TestSQLFetchScrollUnsupportedOrientation) { @@ -901,8 +928,12 @@ TYPED_TEST(StatementTest, TestSQLFetchScrollUnsupportedOrientation) { ASSERT_EQ(SQL_ERROR, SQLFetchScroll(this->stmt, SQL_FETCH_BOOKMARK, fetch_offset)); - // DM returns state HY106 for SQL_FETCH_BOOKMARK +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateHYC00); +#else + // Windows DM returns state HY106 for SQL_FETCH_BOOKMARK VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateHY106); +#endif // __APPLE__ } TYPED_TEST(StatementTest, TestSQLExecDirectVarcharTruncation) { @@ -1170,6 +1201,9 @@ TEST_F(StatementRemoteTest, TestSQLExecDirectNullQueryNullIndicator) { VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorState22002); } +// MacOS Driver Manager iODBC returns SQL_ERROR when invalid buffer length is provided to +// SQLGetData +#ifndef __APPLE__ TYPED_TEST(StatementTest, TestSQLExecDirectIgnoreInvalidBufLen) { // Verify the driver ignores invalid buffer length for fixed data types @@ -1367,6 +1401,7 @@ TYPED_TEST(StatementTest, TestSQLExecDirectIgnoreInvalidBufLen) { EXPECT_EQ(59, timestamp_var.second); EXPECT_EQ(0, timestamp_var.fraction); } +#endif // __APPLE__ TYPED_TEST(StatementTest, TestSQLBindColDataQuery) { // Numeric Types @@ -1503,7 +1538,7 @@ TYPED_TEST(StatementTest, TestSQLBindColDataQuery) { SQLBindCol(this->stmt, 25, SQL_C_CHAR, &char_val, buf_len, &ind)); SQLWCHAR wchar_val[2]; - size_t wchar_size = arrow::flight::sql::odbc::GetSqlWCharSize(); + size_t wchar_size = GetSqlWCharSize(); buf_len = wchar_size * 2; ASSERT_EQ(SQL_SUCCESS, @@ -1533,10 +1568,10 @@ TYPED_TEST(StatementTest, TestSQLBindColDataQuery) { SQL_TIMESTAMP_STRUCT timestamp_val_min{}, timestamp_val_max{}; - EXPECT_EQ(SQL_SUCCESS, SQLBindCol(this->stmt, 31, SQL_C_TYPE_TIMESTAMP, + ASSERT_EQ(SQL_SUCCESS, SQLBindCol(this->stmt, 31, SQL_C_TYPE_TIMESTAMP, ×tamp_val_min, buf_len, &ind)); - EXPECT_EQ(SQL_SUCCESS, SQLBindCol(this->stmt, 32, SQL_C_TYPE_TIMESTAMP, + ASSERT_EQ(SQL_SUCCESS, SQLBindCol(this->stmt, 32, SQL_C_TYPE_TIMESTAMP, ×tamp_val_max, buf_len, &ind)); // Execute query and fetch data once since there is only 1 row. @@ -1990,7 +2025,8 @@ TEST_F(StatementRemoteTest, DISABLED_TestSQLExtendedFetchQueryNullIndicator) { } TYPED_TEST(StatementTest, TestSQLMoreResultsNoData) { - // Verify SQLMoreResults returns SQL_NO_DATA by default. + // Verify SQLMoreResults is stubbed to return SQL_NO_DATA + std::wstring wsql = L"SELECT 1;"; std::vector sql0(wsql.begin(), wsql.end()); @@ -2107,7 +2143,11 @@ TYPED_TEST(StatementTest, TestSQLNativeSqlReturnsErrorOnBadInputs) { ASSERT_EQ(SQL_ERROR, SQLNativeSql(this->conn, input_str, -100, buf, buf_char_len, &output_char_len)); +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorStateS1090); +#else VerifyOdbcErrorState(SQL_HANDLE_DBC, this->conn, kErrorStateHY090); +#endif // __APPLE__ } TYPED_TEST(StatementTest, SQLNumResultColsReturnsColumnsOnSelect) { @@ -2149,9 +2189,20 @@ TYPED_TEST(StatementTest, SQLNumResultColsFunctionSequenceErrorOnNoQuery) { SQLSMALLINT expected_value = 0; ASSERT_EQ(SQL_ERROR, SQLNumResultCols(this->stmt, &column_count)); +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1010); +#else VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateHY010); +#endif // __APPLE__ - EXPECT_EQ(expected_value, column_count); + ASSERT_EQ(SQL_ERROR, SQLNumResultCols(this->stmt, &column_count)); +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1010); +#else + VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateHY010); +#endif // __APPLE__ + + ASSERT_EQ(expected_value, column_count); } TYPED_TEST(StatementTest, SQLRowCountReturnsNegativeOneOnSelect) { @@ -2193,11 +2244,25 @@ TYPED_TEST(StatementTest, SQLRowCountFunctionSequenceErrorOnNoQuery) { SQLLEN expected_value = 0; ASSERT_EQ(SQL_ERROR, SQLRowCount(this->stmt, &row_count)); +#ifdef __APPLE__ + VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1010); +#else VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateHY010); +#endif // __APPLE__ EXPECT_EQ(expected_value, row_count); } +TYPED_TEST(StatementTest, TestSQLFreeStmtSQLClose) { + std::wstring wsql = L"SELECT 1;"; + std::vector sql0(wsql.begin(), wsql.end()); + + ASSERT_EQ(SQL_SUCCESS, + SQLExecDirect(this->stmt, &sql0[0], static_cast(sql0.size()))); + + ASSERT_EQ(SQL_SUCCESS, SQLFreeStmt(this->stmt, SQL_CLOSE)); +} + TYPED_TEST(StatementTest, TestSQLCloseCursor) { std::wstring wsql = L"SELECT 1;"; std::vector sql0(wsql.begin(), wsql.end()); @@ -2209,7 +2274,7 @@ TYPED_TEST(StatementTest, TestSQLCloseCursor) { } TYPED_TEST(StatementTest, TestSQLFreeStmtSQLCloseWithoutCursor) { - // Verify SQLFreeStmt(SQL_CLOSE) does not throw error with invalid cursor + // SQLFreeStmt(SQL_CLOSE) does not throw error with invalid cursor ASSERT_EQ(SQL_SUCCESS, SQLFreeStmt(this->stmt, SQL_CLOSE)); } diff --git a/cpp/src/arrow/flight/sql/odbc/tests/type_info_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/type_info_test.cc index 0adaf51c71f1..24c46d8017e0 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/type_info_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/type_info_test.cc @@ -1656,11 +1656,15 @@ TEST_F(TypeInfoOdbcV2MockTest, TestSQLGetTypeInfoDateODBCVer2) { } TEST_F(TypeInfoOdbcV2MockTest, TestSQLGetTypeInfoSQLTypeDateODBCVer2) { +#ifdef __APPLE__ + ASSERT_EQ(SQL_SUCCESS, SQLGetTypeInfo(this->stmt, SQL_TYPE_DATE)); +#else // Pass ODBC Ver 3 data type ASSERT_EQ(SQL_ERROR, SQLGetTypeInfo(this->stmt, SQL_TYPE_DATE)); // Driver manager returns SQL data type out of range error state VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1004); +#endif // __APPLE__ } TEST_F(TypeInfoMockTest, TestSQLGetTypeInfoSQLTypeTime) { @@ -1764,11 +1768,15 @@ TEST_F(TypeInfoOdbcV2MockTest, TestSQLGetTypeInfoTimeODBCVer2) { } TEST_F(TypeInfoOdbcV2MockTest, TestSQLGetTypeInfoSQLTypeTimeODBCVer2) { +#ifdef __APPLE__ + ASSERT_EQ(SQL_SUCCESS, SQLGetTypeInfo(this->stmt, SQL_TYPE_DATE)); +#else // Pass ODBC Ver 3 data type ASSERT_EQ(SQL_ERROR, SQLGetTypeInfo(this->stmt, SQL_TYPE_TIME)); // Driver manager returns SQL data type out of range error state VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1004); +#endif // __APPLE__ } TEST_F(TypeInfoMockTest, TestSQLGetTypeInfoSQLTypeTimestamp) { @@ -1872,11 +1880,15 @@ TEST_F(TypeInfoOdbcV2MockTest, TestSQLGetTypeInfoSQLTimestampODBCVer2) { } TEST_F(TypeInfoOdbcV2MockTest, TestSQLGetTypeInfoSQLTypeTimestampODBCVer2) { +#ifdef __APPLE__ + ASSERT_EQ(SQL_SUCCESS, SQLGetTypeInfo(this->stmt, SQL_TYPE_TIMESTAMP)); +#else // Pass ODBC Ver 3 data type ASSERT_EQ(SQL_ERROR, SQLGetTypeInfo(this->stmt, SQL_TYPE_TIMESTAMP)); // Driver manager returns SQL data type out of range error state VerifyOdbcErrorState(SQL_HANDLE_STMT, this->stmt, kErrorStateS1004); +#endif // __APPLE__ } TEST_F(TypeInfoMockTest, TestSQLGetTypeInfoInvalidDataType) { diff --git a/cpp/vcpkg.json b/cpp/vcpkg.json index ba3c8e1851b0..5fa97292ff01 100644 --- a/cpp/vcpkg.json +++ b/cpp/vcpkg.json @@ -21,10 +21,8 @@ "boost-filesystem", "boost-locale", "boost-multiprecision", - "boost-optional", "boost-process", "boost-system", - "boost-variant", "boost-xpressive", "brotli", "bzip2", From c09449377f10d8ff6f9204934368d6861448fab7 Mon Sep 17 00:00:00 2001 From: "Alina (Xi) Li" <96995091+alinaliBQ@users.noreply.github.com> Date: Tue, 10 Feb 2026 10:21:53 -0800 Subject: [PATCH 2/7] Build ODBC statically on macOS * Link ODBC library statically on unix Change unit tests to re-use code. Testing on windows is needed. Now seeing error ``` Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.76.dylib ``` on Excel. Attempt to unlink arrow_flight.proto Now I see ODBC.dylib has 100MB, and error from Excel is ``` Library not loaded: /usr/local/opt/libiodbc/lib/libiodbc.2.dylib ``` instead of looking for flightsql. `iodbctest` command outside of Excel works. in-progress link arrow statically for `flightsql-odbc` * both ODBC driver and `arrow_odbc_spi_impl` need to link Arrow statically for tests to work. * getting error `File already exists in database: Flight.proto` since the ODBC layer is linking dynamically while I successfully got `arrow_odbc_spi_impl` to link to Arrow statically Draft for resolving `File already exists in database: FlightSql.proto` error Note: make ODBC available on Excel first, test with `iodbctest`, worry about ODBC test executable later. Add unix build (same as Windows build) * Attempted to link static libraries with `arrow_odbc_spi_impl`, failed as it caused linking errors and rendered the ODBC driver unusable. Undo changes to resolve broken driver Resolves dependency errors Attempt to switch to static build Still getting error ``` [iODBC][Driver Manager]dlopen(/Library/ODBC/arrow-odbc/libarrow_flight_sql_odbc.2300.0.0.dylib, 0x0006): Library not loaded: @rpath/libarrow_flight_sql.2300.dylib ``` In-progress attempt to build ODBC statically on unix systems In-progress changes to enable static odbc_impl library pushing the changes just for saving my code. Currently odbc test still has double registration issue, work on this next. I think it should be solvable by either adding library flags or just using odbc spi shared. Because `odbc_spi_impl_test` works. And the ODBC tests worked just fine before I switched to static odbc spi impl. Fix for gtest missing issue Attempting to fix for issue: IMPORTANT NOTICE - DO NOT IGNORE: This test program did NOT call testing::InitGoogleTest() before calling RUN_ALL_TESTS(). This is INVALID. Soon Google Test will start to enforce the valid usage. Please fix it ASAP, or IT WILL START TO FAIL. Issue fixed with adding `${ARROW_TEST_LINK_LIBS}`. It is needed alongside `arrow_flight_testing_shared`. IN-PROGRESS build static test on macOS * Trying to have 2 builds. One `arrow_odbc_spi_impl_static` for static build and one `arrow_odbc_spi_impl_shared` for a shared lib that links Arrow dynamically --- only committing the approach that worked. Trying to work from `flight_sql_odbc_test` to debug the real issue. Link 3rd party dependencies statically Fix 3rd party headers cannot find issue for: - boost xpressive and beast headers - rapidjson headers Resolve unneeded link for ODBC ---- Moving `set(APPEND CMAKE_FIND_LIBRARY_SUFFIXES ".a")` up didn't seem to have any effect ---- Fix for static gRPC connection issue: export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=/etc/ssl/cert.pem Fix cares `cannot modify alias target` issue Make arrow_odbc_spi_impl static lib In progress for getting gprc linked statically Add debug msgs in add_arrow_lib BuildUtils.cmake Remove commented out code in odbc lib Fix build issue from odbc impl tests Added draft code for Link Arrow libs statically on Unix for unit tests Add commented out code`ARROW_DEPENDENCY_USE_SHARED` in ODBC cpp yml Uncomment when it is verified to work. In-progress Fix ODBC test build and make it run. Build executable from scratch Revert "Build executable from scratch" This reverts commit a51dc395d69c50b42090f50c22cdc9247d3baf9f. Dummy test dummy test without using ODBC passed without double registration issue Make ODBC test run Issue to resolve: [libprotobuf FATAL /path/to/arrow/cpp/debug-build/_deps/protobuf-src/src/google/protobuf/extension_set.cc:100] Multiple extension registrations for type "google.protobuf.MessageOptions", field number 1000. unknown file: Failure C++ exception with description "Multiple extension registrations for type "google.protobuf.MessageOptions", field number 1000." thrown in SetUp(). Fix: only link test executable directly with ODBC shared dylib. Do not link test executable with odbc spi impl dylib. And change ODBC to link odbc spi impl dylib publicly instead of privately in-progress changes to accomodate macos CI Test ODBC CI - to be reverted Can revert later Only install ODBC dependencies in mac As dependencies are static and built from source Uninstall absl as attempt to resolve build on Intel Add C++ standard 20 Remove absl header from intel Use static linking in dependency install step Use bundled boost for static linking In-progress Fix Windows build Fix macOS (after Windows build) Remove unneeded ODBC link Resolves the issue of ODBC on macOS being dynamically linked to ODBC Fix ODBC double proto issue on macOS Need to check if Windows CI passes Clean up PR - Remove dummy tests - Remove unneeded changes Revert "Test ODBC CI - to be reverted" This reverts commit 711b3e0863ebbf5fe2c5cb1401fbaf72eed58aa4. Remove unneeded code Attempt to revert c-ares change Fix `set_target_properties can not be used on an ALIAS target.` error. Example of failed run in forked repo: https://github.com/Bit-Quill/arrow/actions/runs/21459770235/job/61809740997?pr=153#step:7:540 Fix descriptor pointer tests on static macOS Fix mimalloc issue Fixes issue: ``` mimalloc: assertion failed: at "arrow/cpp/debug-build/mimalloc_ep-prefix/src/mimalloc_ep/include/mimalloc/internal.h":658, _mi_ptr_page assertion: "p==NULL || mi_is_in_heap_region(p)" zsh: abort debug/arrow-flight-sql-odbc-test ``` Enable ODBC tests for static build Add iodbc link to macOS build on ODBC layer Add iodbc link to macOS build on ODBC Test Add iodbc link to macOS build on ODBC layer did not resolve issue of unixodbc being picked in CI * Fix segfault at SQLError * work on Justin's comments --- .github/workflows/cpp_extra.yml | 25 ++++++ ci/scripts/cpp_test.sh | 1 - cpp/cmake_modules/ThirdpartyToolchain.cmake | 9 +- cpp/src/arrow/flight/sql/odbc/CMakeLists.txt | 82 +++++++++++++------ .../flight/sql/odbc/odbc_impl/CMakeLists.txt | 29 ++++--- .../config/connection_string_parser.h | 2 +- .../sql/odbc/odbc_impl/odbc_statement.cc | 1 - .../sql/odbc/odbc_impl/spi/connection.h | 1 - .../odbc/odbc_impl/ui/add_property_window.cc | 6 +- .../sql/odbc/odbc_impl/ui/custom_window.cc | 2 +- .../sql/odbc/odbc_impl/ui/custom_window.h | 2 +- .../odbc_impl/ui/dsn_configuration_window.h | 4 +- .../arrow/flight/sql/odbc/odbc_impl/util.h | 1 - .../flight/sql/odbc/tests/CMakeLists.txt | 79 ++++++++++++------ .../flight/sql/odbc/tests/errors_test.cc | 7 +- .../sql/odbc/tests/statement_attr_test.cc | 17 ++-- 16 files changed, 177 insertions(+), 91 deletions(-) diff --git a/.github/workflows/cpp_extra.yml b/.github/workflows/cpp_extra.yml index ee4fa696ac84..cd39a6d65d80 100644 --- a/.github/workflows/cpp_extra.yml +++ b/.github/workflows/cpp_extra.yml @@ -357,6 +357,10 @@ jobs: ARROW_BUILD_TESTS: ON ARROW_FLIGHT_SQL_ODBC: ON ARROW_HOME: /tmp/local + ARROW_DEPENDENCY_USE_SHARED: OFF + ARROW_DEPENDENCY_SOURCE: BUNDLED + ARROW_MIMALLOC: OFF + CMAKE_CXX_STANDARD: "20" steps: - name: Checkout Arrow uses: actions/checkout@v6.0.1 @@ -366,6 +370,22 @@ jobs: - name: Install Dependencies run: | brew bundle --file=cpp/Brewfile + + # We want to use bundled RE2 for static linking. If + # Homebrew's RE2 is installed, its header file may be used. + # We uninstall Homebrew's RE2 to ensure using bundled RE2. + brew uninstall grpc || : # gRPC depends on RE2 + brew uninstall grpc@1.54 || : # gRPC 1.54 may be installed too + brew uninstall re2 + + # We want to use bundled Protobuf for static linking. If + # Homebrew's Protobuf is installed, its library file may be + # used on test We uninstall Homebrew's Protobuf to ensure using + # bundled Protobuf. + brew uninstall protobuf + + # We want to use bundled Boost for static linking. + brew uninstall boost - name: Setup ccache run: | ci/scripts/ccache_setup.sh @@ -395,6 +415,10 @@ jobs: export ARROW_CMAKE_ARGS="-DODBC_INCLUDE_DIR=$ODBC_INCLUDE_DIR" export CXXFLAGS="$CXXFLAGS -I$ODBC_INCLUDE_DIR" ci/scripts/cpp_build.sh $(pwd) $(pwd)/build + - name: Register Flight SQL ODBC Driver + run: | + chmod +x cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh + sudo cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh $(pwd)/build/cpp/debug/libarrow_flight_sql_odbc.dylib - name: Test shell: bash run: | @@ -431,6 +455,7 @@ jobs: CMAKE_INSTALL_PREFIX: /usr VCPKG_BINARY_SOURCES: 'clear;nugettimeout,600;nuget,GitHub,readwrite' VCPKG_DEFAULT_TRIPLET: x64-windows + CMAKE_CXX_STANDARD: "20" steps: - name: Disable Crash Dialogs run: | diff --git a/ci/scripts/cpp_test.sh b/ci/scripts/cpp_test.sh index 88239a0bd1e7..5d6d5e099ab1 100755 --- a/ci/scripts/cpp_test.sh +++ b/ci/scripts/cpp_test.sh @@ -59,7 +59,6 @@ case "$(uname)" in ;; Darwin) n_jobs=$(sysctl -n hw.ncpu) - exclude_tests+=("arrow-flight-sql-odbc-test") # TODO: https://github.com/apache/arrow/issues/40410 exclude_tests+=("arrow-s3fs-test") ;; diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake index e84b2accb8b2..3c8565e99c40 100644 --- a/cpp/cmake_modules/ThirdpartyToolchain.cmake +++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake @@ -1114,6 +1114,11 @@ function(build_boost) else() list(APPEND BOOST_EXCLUDE_LIBRARIES uuid) endif() + if(ARROW_FLIGHT_SQL_ODBC) + list(APPEND BOOST_INCLUDE_LIBRARIES beast xpressive) + else() + list(APPEND BOOST_EXCLUDE_LIBRARIES beast xpressive) + endif() set(BOOST_SKIP_INSTALL_RULES ON) if(NOT ARROW_ENABLE_THREADING) set(BOOST_UUID_LINK_LIBATOMIC OFF) @@ -3023,8 +3028,8 @@ function(build_cares) if(APPLE) # libresolv must be linked from c-ares version 1.16.1 find_library(LIBRESOLV_LIBRARY NAMES resolv libresolv REQUIRED) - set_target_properties(c-ares::cares PROPERTIES INTERFACE_LINK_LIBRARIES - "${LIBRESOLV_LIBRARY}") + set_target_properties(c-ares PROPERTIES INTERFACE_LINK_LIBRARIES + "${LIBRESOLV_LIBRARY}") endif() set(ARROW_BUNDLED_STATIC_LIBS diff --git a/cpp/src/arrow/flight/sql/odbc/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/CMakeLists.txt index 39040c45024d..80d3a67c2931 100644 --- a/cpp/src/arrow/flight/sql/odbc/CMakeLists.txt +++ b/cpp/src/arrow/flight/sql/odbc/CMakeLists.txt @@ -19,6 +19,11 @@ # GH-44792: Arrow will switch to C++ 20 set(CMAKE_CXX_STANDARD 20) +if(APPLE) + # CMAKE_FIND_LIBRARY_SUFFIXES. + set(APPEND CMAKE_FIND_LIBRARY_SUFFIXES ".a") +endif() + if(WIN32) if(MSVC_VERSION GREATER_EQUAL 1900) set(ODBCINST legacy_stdio_definitions odbccp32 shlwapi) @@ -61,33 +66,56 @@ if(WIN32) list(APPEND ARROW_FLIGHT_SQL_ODBC_SRCS odbc.def install/versioninfo.rc) endif() -add_arrow_lib(arrow_flight_sql_odbc - CMAKE_PACKAGE_NAME - ArrowFlightSqlOdbc - PKG_CONFIG_NAME - arrow-flight-sql-odbc - OUTPUTS - ARROW_FLIGHT_SQL_ODBC_LIBRARIES - SOURCES - ${ARROW_FLIGHT_SQL_ODBC_SRCS} - DEPENDENCIES - arrow_flight_sql - DEFINITIONS - UNICODE - SHARED_LINK_FLAGS - ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in cpp/arrow/CMakeLists.txt - SHARED_LINK_LIBS - arrow_flight_sql_shared - SHARED_INSTALL_INTERFACE_LIBS - ArrowFlight::arrow_flight_sql_shared - STATIC_LINK_LIBS - arrow_flight_sql_static - STATIC_INSTALL_INTERFACE_LIBS - ArrowFlight::arrow_flight_sql_static - SHARED_PRIVATE_LINK_LIBS - ODBC::ODBC - ${ODBCINST} - arrow_odbc_spi_impl) +# On Windows, dynmaic build for ODBC is supported. +# On unix systems, static build for ODBC is supported, all libraries are linked statically on unix. +if(WIN32) + add_arrow_lib(arrow_flight_sql_odbc + CMAKE_PACKAGE_NAME + ArrowFlightSqlOdbc + PKG_CONFIG_NAME + arrow-flight-sql-odbc + OUTPUTS + ARROW_FLIGHT_SQL_ODBC_LIBRARIES + SOURCES + ${ARROW_FLIGHT_SQL_ODBC_SRCS} + DEFINITIONS + UNICODE + SHARED_LINK_FLAGS + ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in cpp/arrow/CMakeLists.txt + SHARED_LINK_LIBS + arrow_flight_sql_shared + arrow_odbc_spi_impl + SHARED_INSTALL_INTERFACE_LIBS + ArrowFlight::arrow_flight_sql_shared + STATIC_LINK_LIBS + arrow_flight_sql_static + STATIC_INSTALL_INTERFACE_LIBS + ArrowFlight::arrow_flight_sql_static + SHARED_PRIVATE_LINK_LIBS + ODBC::ODBC + ${ODBCINST}) +else() + # Unix + add_arrow_lib(arrow_flight_sql_odbc + CMAKE_PACKAGE_NAME + ArrowFlightSqlOdbc + PKG_CONFIG_NAME + arrow-flight-sql-odbc + OUTPUTS + ARROW_FLIGHT_SQL_ODBC_LIBRARIES + SOURCES + ${ARROW_FLIGHT_SQL_ODBC_SRCS} + DEFINITIONS + UNICODE + SHARED_LINK_FLAGS + ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in cpp/arrow/CMakeLists.txt + STATIC_LINK_LIBS + iodbc + ODBC::ODBC + ${ODBCINST} + SHARED_LINK_LIBS + arrow_odbc_spi_impl) +endif() foreach(LIB_TARGET ${ARROW_FLIGHT_SQL_ODBC_LIBRARIES}) target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_FLIGHT_SQL_ODBC_EXPORTING) diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt index e58558258df0..0897248f6cea 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt @@ -15,7 +15,7 @@ # specific language governing permissions and limitations # under the License. -add_library(arrow_odbc_spi_impl +add_library(arrow_odbc_spi_impl STATIC accessors/binary_array_accessor.cc accessors/binary_array_accessor.h accessors/boolean_array_accessor.cc @@ -133,8 +133,11 @@ endif() if(APPLE) target_include_directories(arrow_odbc_spi_impl SYSTEM BEFORE PUBLIC ${ODBC_INCLUDE_DIR}) target_link_libraries(arrow_odbc_spi_impl - PUBLIC arrow_flight_sql_shared arrow_compute_shared Boost::locale - iodbc) + PUBLIC arrow_flight_sql_static + arrow_compute_static + Boost::locale + Boost::headers + RapidJSON) else() find_package(ODBC REQUIRED) target_include_directories(arrow_odbc_spi_impl PUBLIC ${ODBC_INCLUDE_DIR}) @@ -143,14 +146,6 @@ else() ${ODBCINST}) endif() -set_target_properties(arrow_odbc_spi_impl - PROPERTIES ARCHIVE_OUTPUT_DIRECTORY - ${CMAKE_BINARY_DIR}/$/lib - LIBRARY_OUTPUT_DIRECTORY - ${CMAKE_BINARY_DIR}/$/lib - RUNTIME_OUTPUT_DIRECTORY - ${CMAKE_BINARY_DIR}/$/lib) - # CLI add_executable(arrow_odbc_spi_impl_cli main.cc) set_target_properties(arrow_odbc_spi_impl_cli @@ -159,6 +154,16 @@ set_target_properties(arrow_odbc_spi_impl_cli target_link_libraries(arrow_odbc_spi_impl_cli arrow_odbc_spi_impl) # Unit tests + +# On Windows, dynamic linking ODBC is supported. +# On unix systems, static linking ODBC is supported, thus the library linking is static. +if(WIN32) + set(ODBC_SPI_IMPL_TEST_LINK_LIBS arrow_flight_testing_shared) +else() + # unix + set(ODBC_SPI_IMPL_TEST_LINK_LIBS arrow_flight_testing_static) +endif() + add_arrow_test(odbc_spi_impl_test SOURCES accessors/boolean_array_accessor_test.cc @@ -177,4 +182,4 @@ add_arrow_test(odbc_spi_impl_test util_test.cc EXTRA_LINK_LIBS arrow_odbc_spi_impl - arrow_flight_testing_shared) + ${ODBC_SPI_IMPL_TEST_LINK_LIBS}) diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/connection_string_parser.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/connection_string_parser.h index dd937410adc1..2e7196e322fa 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/connection_string_parser.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/config/connection_string_parser.h @@ -19,7 +19,7 @@ #include -#include "config/configuration.h" +#include "arrow/flight/sql/odbc/odbc_impl/config/configuration.h" namespace arrow::flight::sql::odbc { namespace config { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc index 82656d1a38f8..c9cedfe3b3f3 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_statement.cc @@ -30,7 +30,6 @@ #include #include #include -#include #include #include diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h index b7f24628a64a..00a7b511ef20 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/spi/connection.h @@ -18,7 +18,6 @@ #pragma once #include -#include #include #include #include diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/add_property_window.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/add_property_window.cc index 634fd77862e9..6518e48459e7 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/add_property_window.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/add_property_window.cc @@ -15,7 +15,7 @@ // specific language governing permissions and limitations // under the License. -#include "ui/add_property_window.h" +#include "arrow/flight/sql/odbc/odbc_impl/ui/add_property_window.h" #include @@ -25,8 +25,8 @@ #include #include "arrow/flight/sql/odbc/odbc_impl/exceptions.h" -#include "ui/custom_window.h" -#include "ui/window.h" +#include "arrow/flight/sql/odbc/odbc_impl/ui/custom_window.h" +#include "arrow/flight/sql/odbc/odbc_impl/ui/window.h" namespace arrow::flight::sql::odbc { namespace config { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc index 0d24f0cca82d..ff8416a43a86 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.cc @@ -29,7 +29,7 @@ #include #include "arrow/flight/sql/odbc/odbc_impl/exceptions.h" -#include "ui/custom_window.h" +#include "arrow/flight/sql/odbc/odbc_impl/ui/custom_window.h" namespace arrow::flight::sql::odbc { namespace config { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.h index a65800114855..9745f58b2e22 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/custom_window.h @@ -17,7 +17,7 @@ #pragma once -#include "ui/window.h" +#include "arrow/flight/sql/odbc/odbc_impl/ui/window.h" namespace arrow::flight::sql::odbc { namespace config { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.h index bfac68df8b98..74a750537362 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.h @@ -17,8 +17,8 @@ #pragma once -#include "config/configuration.h" -#include "ui/custom_window.h" +#include "arrow/flight/sql/odbc/odbc_impl/config/configuration.h" +#include "arrow/flight/sql/odbc/odbc_impl/ui/custom_window.h" namespace arrow::flight::sql::odbc { namespace config { diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h index e9a6794f104b..3bd11d60113b 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/util.h @@ -17,7 +17,6 @@ #pragma once -#include #include #include diff --git a/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt index 1a00f304b118..e74e7768b1c3 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt +++ b/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt @@ -15,8 +15,6 @@ # specific language governing permissions and limitations # under the License. -add_custom_target(tests) - if(WIN32) include_directories(${ODBC_INCLUDE_DIRS}) else() @@ -33,29 +31,60 @@ set(ARROW_FLIGHT_SQL_MOCK_SERVER_SRCS ../../example/sqlite_server.cc ../../example/sqlite_tables_schema_batch_reader.cc) -add_arrow_test(flight_sql_odbc_test - SOURCES - odbc_test_suite.cc - odbc_test_suite.h - columns_test.cc - connection_attr_test.cc - connection_info_test.cc - connection_test.cc - errors_test.cc - get_functions_test.cc - statement_attr_test.cc - statement_test.cc - tables_test.cc - type_info_test.cc - # Enable Protobuf cleanup after test execution - # GH-46889: move protobuf_test_util to a more common location - ../../../../engine/substrait/protobuf_test_util.cc - ${ARROW_FLIGHT_SQL_MOCK_SERVER_SRCS} - EXTRA_LINK_LIBS - ${ODBC_LIBRARIES} - ${ODBCINST} - ${SQLite3_LIBRARIES} - arrow_odbc_spi_impl) +if(ARROW_TEST_LINKAGE STREQUAL "static") + set(ARROW_FLIGHT_SQL_ODBC_TEST_LINK_LIBS arrow_flight_sql_odbc_static + ${ARROW_TEST_STATIC_LINK_LIBS}) +else() + set(ARROW_FLIGHT_SQL_ODBC_TEST_LINK_LIBS arrow_flight_sql_odbc_shared + ${ARROW_TEST_SHARED_LINK_LIBS}) +endif() + +set(ARROW_FLIGHT_SQL_ODBC_TEST_SRCS + odbc_test_suite.cc + odbc_test_suite.h + columns_test.cc + connection_attr_test.cc + connection_info_test.cc + connection_test.cc + errors_test.cc + get_functions_test.cc + statement_attr_test.cc + statement_test.cc + tables_test.cc + type_info_test.cc + # Enable Protobuf cleanup after test execution + # GH-46889: move protobuf_test_util to a more common location + ../../../../engine/substrait/protobuf_test_util.cc) + +# On Windows, dynamic linking ODBC is supported, tests link libraries dynamically. +# On unix systems, static linking ODBC is supported, thus tests link libraries statically. +if(WIN32) + add_arrow_test(flight_sql_odbc_test + SOURCES + ${ARROW_FLIGHT_SQL_ODBC_TEST_SRCS} + ${ARROW_FLIGHT_SQL_MOCK_SERVER_SRCS} + DEFINITIONS + UNICODE + EXTRA_LINK_LIBS + ${ODBC_LIBRARIES} + ${ODBCINST} + ${SQLite3_LIBRARIES} + ${ARROW_FLIGHT_SQL_ODBC_TEST_LINK_LIBS}) +elseif(APPLE) + # macOS + add_arrow_test(flight_sql_odbc_test + SOURCES + ${ARROW_FLIGHT_SQL_ODBC_TEST_SRCS} + ${ARROW_FLIGHT_SQL_MOCK_SERVER_SRCS} + DEFINITIONS + UNICODE + STATIC_LINK_LIBS + iodbc + ${ODBC_LIBRARIES} + ${ODBCINST} + ${SQLite3_LIBRARIES} + ${ARROW_FLIGHT_SQL_ODBC_TEST_LINK_LIBS}) +endif() find_package(ODBC REQUIRED) target_link_libraries(arrow-flight-sql-odbc-test PRIVATE ODBC::ODBC) diff --git a/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc index 53eeac1aba39..47401c5ed565 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc @@ -541,7 +541,7 @@ TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtError) { // When application passes buffer length greater than SQL_MAX_MESSAGE_LENGTH (512), // DM passes 512 as buffer length to SQLError. - std::wstring wsql = L"1"; + std::wstring wsql = L"SELECT * from non_existent_table;"; std::vector sql0(wsql.begin(), wsql.end()); ASSERT_EQ(SQL_ERROR, @@ -549,11 +549,10 @@ TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtError) { SQLWCHAR sql_state[6] = {0}; SQLINTEGER native_error = 0; - SQLWCHAR message[SQL_MAX_MESSAGE_LENGTH] = {0}; SQLSMALLINT message_length = 0; - ASSERT_EQ(SQL_SUCCESS, SQLError(nullptr, nullptr, this->stmt, sql_state, &native_error, + SQLWCHAR message[SQL_MAX_MESSAGE_LENGTH] = {0}; + ASSERT_EQ(SQL_SUCCESS, SQLError(SQL_NULL_HENV, this->conn, this->stmt, sql_state, &native_error, message, SQL_MAX_MESSAGE_LENGTH, &message_length)); - EXPECT_GT(message_length, 70); EXPECT_EQ(100, native_error); diff --git a/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc index ac2cbb79864c..07d309fa9f54 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc @@ -480,26 +480,25 @@ TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrFetchBookmarkPointer) { TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrIMPParamDesc) { // Invalid use of an automatically allocated descriptor handle ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_IMP_PARAM_DESC, - static_cast(0), + static_cast(0), SQL_ERROR, #ifdef __APPLE__ - // iODBC on MacOS returns SQL_INVALID_HANDLE for this case - SQL_INVALID_HANDLE, + // static iODBC on MacOS returns IM001 for this case + kErrorStateIM001); #else - SQL_ERROR, -#endif kErrorStateHY017); +#endif // __APPLE__ } TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrIMPRowDesc) { // Invalid use of an automatically allocated descriptor handle ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_IMP_ROW_DESC, static_cast(0), + SQL_ERROR, #ifdef __APPLE__ - // iODBC on MacOS returns SQL_INVALID_HANDLE for this case - SQL_INVALID_HANDLE, + // static iODBC on MacOS returns IM001 for this case + kErrorStateIM001); #else - SQL_ERROR, -#endif kErrorStateHY017); +#endif // __APPLE__ } TYPED_TEST(StatementAttributeTest, TestSQLSetStmtAttrKeysetSizeUnsupported) { From 555146a4f59771517a27db627d58fde9ac0f068f Mon Sep 17 00:00:00 2001 From: "Alina (Xi) Li" Date: Tue, 10 Feb 2026 10:37:43 -0800 Subject: [PATCH 3/7] Fix rebase conflicts --- .github/workflows/cpp_extra.yml | 3 +-- ci/scripts/cpp_build.sh | 1 - cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc | 5 +++-- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/.github/workflows/cpp_extra.yml b/.github/workflows/cpp_extra.yml index cd39a6d65d80..768f95e48659 100644 --- a/.github/workflows/cpp_extra.yml +++ b/.github/workflows/cpp_extra.yml @@ -434,8 +434,7 @@ jobs: if: >- needs.check-labels.outputs.force == 'true' || contains(fromJSON(needs.check-labels.outputs.ci-extra-labels || '[]'), 'CI: Extra') || - contains(fromJSON(needs.check-labels.outputs.ci-extra-labels || '[]'), 'CI: Extra: C++') || - contains(join(github.event.pull_request.changed_files, ' '), 'cpp/src/arrow/flight/sql/odbc/') + contains(fromJSON(needs.check-labels.outputs.ci-extra-labels || '[]'), 'CI: Extra: C++') timeout-minutes: 240 permissions: packages: write diff --git a/ci/scripts/cpp_build.sh b/ci/scripts/cpp_build.sh index 4738d0306149..79b64dbc2a47 100755 --- a/ci/scripts/cpp_build.sh +++ b/ci/scripts/cpp_build.sh @@ -272,7 +272,6 @@ else -DgRPC_SOURCE=${gRPC_SOURCE:-} \ -DGTest_SOURCE=${GTest_SOURCE:-} \ -Dlz4_SOURCE=${lz4_SOURCE:-} \ - -DODBC_INCLUDE_DIR="${ODBC_INCLUDE_DIR:-}" \ -Dopentelemetry-cpp_SOURCE=${opentelemetry_cpp_SOURCE:-} \ -DORC_SOURCE=${ORC_SOURCE:-} \ -DPARQUET_BUILD_EXAMPLES=${PARQUET_BUILD_EXAMPLES:-OFF} \ diff --git a/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc index 47401c5ed565..60399bd3a71a 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/errors_test.cc @@ -551,8 +551,9 @@ TYPED_TEST(ErrorsOdbcV2Test, TestSQLErrorStmtError) { SQLINTEGER native_error = 0; SQLSMALLINT message_length = 0; SQLWCHAR message[SQL_MAX_MESSAGE_LENGTH] = {0}; - ASSERT_EQ(SQL_SUCCESS, SQLError(SQL_NULL_HENV, this->conn, this->stmt, sql_state, &native_error, - message, SQL_MAX_MESSAGE_LENGTH, &message_length)); + ASSERT_EQ(SQL_SUCCESS, + SQLError(SQL_NULL_HENV, this->conn, this->stmt, sql_state, &native_error, + message, SQL_MAX_MESSAGE_LENGTH, &message_length)); EXPECT_GT(message_length, 70); EXPECT_EQ(100, native_error); From bd8d3baf437326e7bc8934bf53460e2ed0253f0e Mon Sep 17 00:00:00 2001 From: "Alina (Xi) Li" Date: Tue, 10 Feb 2026 15:21:52 -0800 Subject: [PATCH 4/7] Remove code added during conflict resolution --- cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt | 6 ------ 1 file changed, 6 deletions(-) diff --git a/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt index e74e7768b1c3..b44846fb3500 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt +++ b/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt @@ -15,12 +15,6 @@ # specific language governing permissions and limitations # under the License. -if(WIN32) - include_directories(${ODBC_INCLUDE_DIRS}) -else() - include_directories(${ODBC_INCLUDE_DIR}) -endif() - find_package(SQLite3Alt REQUIRED) set(ARROW_FLIGHT_SQL_MOCK_SERVER_SRCS From 18949976126b028e26108aee84b2ea917f0c252b Mon Sep 17 00:00:00 2001 From: justing-bq <62349012+justing-bq@users.noreply.github.com> Date: Wed, 28 Jan 2026 13:45:36 -0800 Subject: [PATCH 5/7] Implement system_dsn for Mac --- ci/scripts/cpp_test.sh | 1 + .../arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt | 6 +++--- .../arrow/flight/sql/odbc/odbc_impl/system_dsn.cc | 14 +++++--------- .../arrow/flight/sql/odbc/odbc_impl/system_dsn.h | 5 ++++- .../flight/sql/odbc/odbc_impl/win_system_dsn.cc | 2 +- .../arrow/flight/sql/odbc/tests/connection_test.cc | 3 --- .../arrow/flight/sql/odbc/tests/odbc_test_suite.cc | 4 ---- 7 files changed, 14 insertions(+), 21 deletions(-) diff --git a/ci/scripts/cpp_test.sh b/ci/scripts/cpp_test.sh index 5d6d5e099ab1..88239a0bd1e7 100755 --- a/ci/scripts/cpp_test.sh +++ b/ci/scripts/cpp_test.sh @@ -59,6 +59,7 @@ case "$(uname)" in ;; Darwin) n_jobs=$(sysctl -n hw.ncpu) + exclude_tests+=("arrow-flight-sql-odbc-test") # TODO: https://github.com/apache/arrow/issues/40410 exclude_tests+=("arrow-s3fs-test") ;; diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt index 0897248f6cea..74c60cd91632 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt @@ -104,6 +104,8 @@ add_library(arrow_odbc_spi_impl STATIC spi/result_set.h spi/result_set_metadata.h spi/statement.h + system_dsn.cc + system_dsn.h system_trust_store.cc system_trust_store.h types.h @@ -125,9 +127,7 @@ if(WIN32) ui/dsn_configuration_window.h ui/window.cc ui/window.h - win_system_dsn.cc - system_dsn.cc - system_dsn.h) + win_system_dsn.cc) endif() if(APPLE) diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.cc index 468f05e4cf4d..902b370ba181 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.cc @@ -17,23 +17,19 @@ #include "arrow/flight/sql/odbc/odbc_impl/system_dsn.h" -#include "arrow/flight/sql/odbc/odbc_impl/config/configuration.h" -#include "arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.h" -#include "arrow/flight/sql/odbc/odbc_impl/ui/dsn_configuration_window.h" -#include "arrow/flight/sql/odbc/odbc_impl/ui/window.h" -#include "arrow/flight/sql/odbc/odbc_impl/util.h" #include "arrow/result.h" #include "arrow/util/utf8.h" -#include #include namespace arrow::flight::sql::odbc { using config::Configuration; -void PostError(DWORD error_code, LPCWSTR error_msg) { +void PostError(DWORD error_code, LPWSTR error_msg) { +#if defined _WIN32 MessageBox(NULL, error_msg, L"Error!", MB_ICONEXCLAMATION | MB_OK); +#endif // _WIN32 SQLPostInstallerError(error_code, error_msg); } @@ -42,7 +38,7 @@ void PostArrowUtilError(arrow::Status error_status) { std::wstring werror_msg = arrow::util::UTF8ToWideString(error_msg).ValueOr( L"Error during utf8 to wide string conversion"); - PostError(ODBC_ERROR_GENERAL_ERR, werror_msg.c_str()); + PostError(ODBC_ERROR_GENERAL_ERR, (LPWSTR)werror_msg.c_str()); } void PostLastInstallerError() { @@ -55,7 +51,7 @@ void PostLastInstallerError() { buf << L"Message: \"" << msg << L"\", Code: " << code; std::wstring error_msg = buf.str(); - PostError(code, error_msg.c_str()); + PostError(code, (LPWSTR)error_msg.c_str()); } /** diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.h b/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.h index f1fee84fbd4e..59c8cc9601f0 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.h +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/system_dsn.h @@ -19,8 +19,11 @@ #include "arrow/flight/sql/odbc/odbc_impl/platform.h" #include "arrow/flight/sql/odbc/odbc_impl/config/configuration.h" +#include "arrow/flight/sql/odbc/odbc_impl/flight_sql_connection.h" #include "arrow/status.h" +#include + namespace arrow::flight::sql::odbc { #if defined _WIN32 @@ -64,7 +67,7 @@ bool RegisterDsn(const config::Configuration& config, LPCWSTR driver); */ bool UnregisterDsn(const std::wstring& dsn); -void PostError(DWORD error_code, LPCWSTR error_msg); +void PostError(DWORD error_code, LPWSTR error_msg); void PostArrowUtilError(arrow::Status error_status); } // namespace arrow::flight::sql::odbc diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/win_system_dsn.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/win_system_dsn.cc index 2ea9a2451c20..3140b9ade426 100644 --- a/cpp/src/arrow/flight/sql/odbc/odbc_impl/win_system_dsn.cc +++ b/cpp/src/arrow/flight/sql/odbc/odbc_impl/win_system_dsn.cc @@ -144,7 +144,7 @@ BOOL INSTAPI ConfigDSNW(HWND hwnd_parent, WORD req, LPCWSTR wdriver, std::wstring werror_msg = arrow::util::UTF8ToWideString(error_msg).ValueOr(L"Error during DSN load"); - PostError(err.GetNativeError(), werror_msg.c_str()); + PostError(err.GetNativeError(), (LPWSTR)werror_msg.c_str()); return FALSE; } diff --git a/cpp/src/arrow/flight/sql/odbc/tests/connection_test.cc b/cpp/src/arrow/flight/sql/odbc/tests/connection_test.cc index 3ca4a50ef769..57f6c9349bff 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/connection_test.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/connection_test.cc @@ -248,7 +248,6 @@ TYPED_TEST(ConnectionHandleTest, TestSQLDriverConnect) { << GetOdbcErrorMessage(SQL_HANDLE_DBC, this->conn); } -#if defined _WIN32 TYPED_TEST(ConnectionHandleTest, TestSQLDriverConnectDsn) { // Connect string std::string connect_str = this->GetConnectionString(); @@ -432,8 +431,6 @@ TEST_F(ConnectionRemoteTest, TestSQLConnectDSNPrecedence) { << GetOdbcErrorMessage(SQL_HANDLE_DBC, conn); } -#endif // _WIN32 - TEST_F(ConnectionRemoteTest, TestSQLDriverConnectInvalidUid) { // Invalid connect string std::string connect_str = GetInvalidConnectionString(); diff --git a/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc b/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc index 470a68b3beb3..d35713581cdb 100644 --- a/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc +++ b/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc @@ -463,9 +463,6 @@ std::string GetOdbcErrorMessage(SQLSMALLINT handle_type, SQLHANDLE handle) { return res; } -// GH-47822 TODO: once RegisterDsn is implemented in Mac and Linux, the following can be -// re-enabled. -#if defined _WIN32 bool WriteDSN(std::string connection_str) { Connection::ConnPropertyMap properties; @@ -490,7 +487,6 @@ bool WriteDSN(Connection::ConnPropertyMap properties) { std::wstring w_driver = arrow::util::UTF8ToWideString(driver).ValueOr(L""); return RegisterDsn(config, w_driver.c_str()); } -#endif std::wstring GetStringColumnW(SQLHSTMT stmt, int col_id) { SQLWCHAR buf[1024]; From 7560a91c1da355540eba215dbcd7c125cca5f785 Mon Sep 17 00:00:00 2001 From: "Alina (Xi) Li" <96995091+alinaliBQ@users.noreply.github.com> Date: Wed, 11 Feb 2026 11:54:51 -0800 Subject: [PATCH 6/7] Address community comments for `install_odbc_ini.sh` --- .../sql/odbc/install/mac/install_odbc_ini.sh | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) mode change 100644 => 100755 cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh diff --git a/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh b/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh old mode 100644 new mode 100755 index cfc9729ed7e4..652034f49460 --- a/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh +++ b/cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc_ini.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/bin/bash # # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file @@ -19,13 +19,21 @@ # GH-47876 TODO: create macOS ODBC Installer. # Script for installing macOS ODBC driver, to be used for macOS installer. +# This script assumes ODBC driver is at +# /Library/ODBC/arrow-odbc/libarrow_flight_sql_odbc.dylib + +set -euo pipefail + +if [ $EUID -ne 0 ]; then + echo "Please run this script with sudo" + exit 1 +fi source_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" odbc_install_script="${source_dir}/install_odbc.sh" -chmod +x "$odbc_install_script" -. "$odbc_install_script" /Library/Apache/ArrowFlightSQLODBC/lib/libarrow_flight_sql_odbc.dylib +"$odbc_install_script" /Library/ODBC/arrow-odbc/libarrow_flight_sql_odbc.dylib USER_ODBC_FILE="$HOME/Library/ODBC/odbc.ini" DRIVER_NAME="Apache Arrow Flight SQL ODBC Driver" @@ -33,11 +41,6 @@ DSN_NAME="Apache Arrow Flight SQL ODBC DSN" touch "$USER_ODBC_FILE" -if [ $EUID -ne 0 ]; then - echo "Please run this script with sudo" - exit 1 -fi - if grep -q "^\[$DSN_NAME\]" "$USER_ODBC_FILE"; then echo "DSN [$DSN_NAME] already exists in $USER_ODBC_FILE" else From 2336a2d475c8a9d01a84a22a3028433b70b1512a Mon Sep 17 00:00:00 2001 From: justing-bq <62349012+justing-bq@users.noreply.github.com> Date: Thu, 12 Feb 2026 12:17:40 -0800 Subject: [PATCH 7/7] Add separate release & debug workflows for MacOS --- .github/workflows/cpp_extra.yml | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/.github/workflows/cpp_extra.yml b/.github/workflows/cpp_extra.yml index 768f95e48659..365af03ae6c6 100644 --- a/.github/workflows/cpp_extra.yml +++ b/.github/workflows/cpp_extra.yml @@ -338,7 +338,7 @@ jobs: odbc-macos: needs: check-labels - name: ODBC ${{ matrix.architecture }} macOS ${{ matrix.macos-version }} + name: ODBC ${{ matrix.build_type }} ${{ matrix.architecture }} macOS ${{ matrix.macos-version }} runs-on: macos-${{ matrix.macos-version }} if: >- needs.check-labels.outputs.force == 'true' || @@ -349,12 +349,13 @@ jobs: fail-fast: false matrix: include: - - architecture: AMD64 - macos-version: "15-intel" - - architecture: ARM64 - macos-version: "14" + - { architecture: AMD64, macos-version: "15-intel", build_type: debug } + - { architecture: AMD64, macos-version: "15-intel", build_type: release } + - { architecture: ARM64, macos-version: "14", build_type: debug } + - { architecture: ARM64, macos-version: "14", build_type: release } env: ARROW_BUILD_TESTS: ON + ARROW_BUILD_TYPE: ${{ matrix.build_type }} ARROW_FLIGHT_SQL_ODBC: ON ARROW_HOME: /tmp/local ARROW_DEPENDENCY_USE_SHARED: OFF @@ -397,8 +398,8 @@ jobs: uses: actions/cache@v5.0.2 with: path: ${{ steps.ccache-info.outputs.cache-dir }} - key: cpp-odbc-ccache-macos-${{ matrix.macos-version }}-${{ hashFiles('cpp/**') }} - restore-keys: cpp-odbc-ccache-macos-${{ matrix.macos-version }}- + key: cpp-odbc-ccache-macos-${{ matrix.macos-version }}-${{ matrix.build_type }}-${{ hashFiles('cpp/**') }} + restore-keys: cpp-odbc-ccache-macos-${{ matrix.macos-version }}-${{ matrix.build_type }}- - name: Build run: | # Homebrew uses /usr/local as prefix. So packages @@ -418,7 +419,7 @@ jobs: - name: Register Flight SQL ODBC Driver run: | chmod +x cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh - sudo cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh $(pwd)/build/cpp/debug/libarrow_flight_sql_odbc.dylib + sudo cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc.sh $(pwd)/build/cpp/${{ matrix.build_type }}/libarrow_flight_sql_odbc.dylib - name: Test shell: bash run: |