Skip to content

Add DateTime64 regression test for text/binary epoch equality (Fixes …#472

Open
ansh0014 wants to merge 1 commit intoClickHouse:masterfrom
ansh0014:fix/issue-398-datetime64-regression
Open

Add DateTime64 regression test for text/binary epoch equality (Fixes …#472
ansh0014 wants to merge 1 commit intoClickHouse:masterfrom
ansh0014:fix/issue-398-datetime64-regression

Conversation

@ansh0014
Copy link
Copy Markdown

@ansh0014 ansh0014 commented Apr 3, 2026

Fix DateTime64 text vs binary epoch mismatch regression test (Issue #398)

Summary

Adds a regression test to verify that DateTime64 values inserted via text and binary paths represent the same microsecond epoch.

Problem

Issue #398 reported inconsistent behavior between:

  • text insert path: INSERT ... VALUES ('2024-10-10 11:00:00')
  • binary insert path: ColumnDateTime64::Append(epoch_microseconds)

For the same logical timestamp, both paths must produce identical epoch values.

Root Cause Context

The mismatch risk exists around DateTime64 interpretation boundaries (precision/timezone handling between insert paths). This PR introduces a guard test to prevent regressions.

Changes

  • Added new unit test: DateTime64.Issue398_TextAndBinarySameEpoch
  • File: ut/roundtrip_tests.cpp

Test behavior:

  1. Creates temporary Memory table with DateTime64(6, 'UTC')
  2. Inserts one row via text path
  3. Inserts one row via binary path using explicit microsecond epoch
  4. Verifies both rows map to identical toUnixTimestamp64Micro(...) values using SQL aggregation (count/min/max)

Why This Approach

  • Avoids fragile row-index assumptions in callback handling.
  • Keeps assertion deterministic and server-side.
  • Minimal and focused change only in UT coverage.

Validation

Executed locally against isolated ClickHouse Docker instance:

build/ut/Debug/clickhouse-cpp-ut.exe --gtest_filter=DateTime64.Issue398_TextAndBinarySameEpoch

Result: Passed.

Scope / Non-goals

  • No production client behavior changes in this PR.
  • No unrelated test infra/auth default changes included.

Checklist

  • Added regression UT
  • Verified test passes locally
  • Kept change scoped to issue
  • No temporary artifacts included in commit (Testing/Temporary/*)

Fixes #398

@ansh0014 ansh0014 requested a review from slabko as a code owner April 3, 2026 13:02
Copilot AI review requested due to automatic review settings April 3, 2026 13:02
@ansh0014 ansh0014 requested a review from mzitnik as a code owner April 3, 2026 13:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds coverage to guard against regressions where DateTime64 values inserted via text vs. binary paths resolve to different underlying epoch microseconds (Issue #398), and adds a corresponding check to the tests/simple runner.

Changes:

  • Added a new gtest regression test DateTime64.Issue398_TextAndBinarySameEpoch validating server-side epoch equality.
  • Added a tests/simple repro/check for Issue #398 and wired it into the simple test runner.
  • Simplified tests/simple/main.cpp connection setup and main().

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
ut/roundtrip_tests.cpp Adds a regression UT comparing text vs binary inserted DateTime64(6, 'UTC') values via toUnixTimestamp64Micro.
tests/simple/main.cpp Adds an Issue #398 check and refactors the simple runner’s main()/connection defaults.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +345 to +347
const char* table = "test_datetime64_issue398";
client.Execute(std::string("DROP TABLE IF EXISTS ") + table);
client.Execute(std::string("CREATE TABLE ") + table + " (dt64 DateTime64(6, 'UTC')) ENGINE = Memory");
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test creates/drops a persistent table with a fixed name. If an assertion fails before the final DROP, the table will be left behind and can make subsequent test runs flaky (and can conflict under parallel test execution). Prefer using a TEMPORARY table (session-scoped) and/or a unique table name (e.g., with a UUID suffix), and ensure cleanup via RAII/SCOPE_EXIT so it runs even on ASSERT failures.

Copilot uses AI. Check for mistakes.
Comment on lines +621 to +622
options.SetUser(EnvOrDefault("CLICKHOUSE_USER", "test"));
options.SetPassword(EnvOrDefault("CLICKHOUSE_PASSWORD", "test"));
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default connection credentials were changed to user/password "test"/"test". This diverges from the rest of the repo’s test/bench setup (typically user "default" with empty password) and will fail against a standard local ClickHouse instance unless users set env vars. Recommend keeping the prior defaults (or at least matching the repo-wide convention) and only overriding via CLICKHOUSE_* env vars.

Suggested change
options.SetUser(EnvOrDefault("CLICKHOUSE_USER", "test"));
options.SetPassword(EnvOrDefault("CLICKHOUSE_PASSWORD", "test"));
options.SetUser(EnvOrDefault("CLICKHOUSE_USER", "default"));
options.SetPassword(EnvOrDefault("CLICKHOUSE_PASSWORD", ""));

Copilot uses AI. Check for mistakes.
Comment on lines +172 to +183
client.Execute("CREATE TEMPORARY TABLE test_datetime64_issue398 (dt64 DateTime64(6))");

// Text insert (server parses this in server timezone for DateTime64 without explicit timezone).
client.Execute("INSERT INTO test_datetime64_issue398 VALUES ('2024-10-10 11:00:00')");

// Ask server for exact epoch micros for the same wall-clock value.
Int64 expected_micros = 0;
client.Select(
"SELECT toUnixTimestamp64Micro(toDateTime64('2024-10-10 11:00:00', 6))",
[&expected_micros](const Block& block) {
expected_micros = block[0]->As<ColumnInt64>()->At(0);
});
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Issue398 check computes expected_micros using toDateTime64(...) without an explicit timezone and inserts into a DateTime64(6) column without timezone. That makes the check depend on server timezone and largely tautological (binary insert uses the same server-derived epoch), so it may not detect the original text-vs-binary timezone mismatch. To make this a meaningful guard, use an explicit timezone (e.g., DateTime64(6, 'UTC') and/or toDateTime64(..., 'UTC')) and compare the two inserted rows’ epochs against that expected value.

Copilot uses AI. Check for mistakes.
Comment on lines +617 to 627
int main() {
clickhouse::ClientOptions options;
options.SetHost(EnvOrDefault("CLICKHOUSE_HOST", "127.0.0.1"));
options.SetPort(static_cast<uint16_t>(std::stoul(EnvOrDefault("CLICKHOUSE_PORT", "9000"))));
options.SetUser(EnvOrDefault("CLICKHOUSE_USER", "test"));
options.SetPassword(EnvOrDefault("CLICKHOUSE_PASSWORD", "test"));
options.SetDefaultDatabase(EnvOrDefault("CLICKHOUSE_DB", "default"));

clickhouse::Client ch_client(options);
RunTests(ch_client);
return 0;
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Port parsing uses std::stoul(...) and then static_cast<uint16_t>(...). If CLICKHOUSE_PORT is out of range, this will silently truncate to an unexpected port. Consider validating the parsed value is within [1, 65535] (and surfacing a clear error) before casting.

Suggested change
int main() {
clickhouse::ClientOptions options;
options.SetHost(EnvOrDefault("CLICKHOUSE_HOST", "127.0.0.1"));
options.SetPort(static_cast<uint16_t>(std::stoul(EnvOrDefault("CLICKHOUSE_PORT", "9000"))));
options.SetUser(EnvOrDefault("CLICKHOUSE_USER", "test"));
options.SetPassword(EnvOrDefault("CLICKHOUSE_PASSWORD", "test"));
options.SetDefaultDatabase(EnvOrDefault("CLICKHOUSE_DB", "default"));
clickhouse::Client ch_client(options);
RunTests(ch_client);
return 0;
static uint16_t ParsePortFromEnvOrDefault(const char* name, const char* fallback) {
const std::string port_str = EnvOrDefault(name, fallback);
size_t parsed = 0;
const unsigned long port = std::stoul(port_str, &parsed);
if (parsed != port_str.size()) {
throw std::invalid_argument(std::string(name) + " must be a valid integer port number");
}
if (port < 1 || port > 65535) {
throw std::out_of_range(std::string(name) + " must be in range [1, 65535]");
}
return static_cast<uint16_t>(port);
}
int main() {
try {
clickhouse::ClientOptions options;
options.SetHost(EnvOrDefault("CLICKHOUSE_HOST", "127.0.0.1"));
options.SetPort(ParsePortFromEnvOrDefault("CLICKHOUSE_PORT", "9000"));
options.SetUser(EnvOrDefault("CLICKHOUSE_USER", "test"));
options.SetPassword(EnvOrDefault("CLICKHOUSE_PASSWORD", "test"));
options.SetDefaultDatabase(EnvOrDefault("CLICKHOUSE_DB", "default"));
clickhouse::Client ch_client(options);
RunTests(ch_client);
return 0;
} catch (const std::exception& ex) {
std::cerr << "Configuration error: " << ex.what() << std::endl;
return 1;
}

Copilot uses AI. Check for mistakes.
Comment on lines 598 to 600
DateTime64Example(client);
DateTime64Issue398Check(client);
DecimalExample(client);
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description states the change is scoped to adding a regression UT in ut/roundtrip_tests.cpp, but this PR also modifies tests/simple/main.cpp (adds a new check and rewrites main()/connection defaults). If those changes are intended, please update the PR description/scope; otherwise consider splitting them into a separate PR to keep the regression-test change focused.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Strange behavior with dateTime

2 participants