Fix: populate PostgresType.oid for array columns to enable binary COPY#432
Merged
staticlibs merged 1 commit intoduckdb:mainfrom Apr 7, 2026
Merged
Conversation
PostgresType.oid was initialized to 0 and never populated for array child types in both code paths: 1. CreateEmptyPostgresType (DDL-created tables): when DuckDB creates a table via DDL (e.g. CREATE TABLE with VARCHAR[]), the child PostgresType.oid was left at 0. Fixed by setting it via ToPostgresOid(), which is correct since DuckDB controls the DDL and the Postgres types will match. 2. TypeToLogicalType (catalog-read tables): when reading existing Postgres tables from the catalog, the child PostgresType.oid was also never set. Here ToPostgresOid() cannot be used because multiple Postgres types map to the same DuckDB LogicalType (e.g. both "text" and "varchar" map to VARCHAR, but have different OIDs 25 vs 1043). Added TypeNameToPostgresOid() to resolve the OID from the actual Postgres type name, preserving the text/varchar distinction. This caused CopyRequiresText to always evaluate: 0 != ToPostgresOid(child_type) → 0 != 1043 → true → FORMAT TEXT With the fix: - varchar[] columns now correctly use binary COPY in both paths - text[] columns correctly fall back to TEXT (TEXTOID != VARCHAROID), which is needed because the binary writer always emits VARCHAROID Fixes duckdb#431
Contributor
Author
Local test results (macOS ARM64, PostgreSQL 17, DuckDB v1.5.1)
No regressions in existing array or binary copy tests. |
Collaborator
|
Thanks! |
Contributor
Author
|
that's cool! thanks @staticlibs |
staticlibs
added a commit
to staticlibs/duckdb
that referenced
this pull request
Apr 8, 2026
This Postgres update includes 2 following fixes that were added recently: - duckdb/duckdb-postgres#432 - duckdb/duckdb-postgres#434
Mytherin
added a commit
to duckdb/duckdb
that referenced
this pull request
Apr 9, 2026
This Postgres scanner update includes 2 following fixes that were added recently: - duckdb/duckdb-postgres#432 - duckdb/duckdb-postgres#434
evertlammerts
pushed a commit
to evertlammerts/duckdb
that referenced
this pull request
Apr 10, 2026
This Postgres update includes 2 following fixes that were added recently: - duckdb/duckdb-postgres#432 - duckdb/duckdb-postgres#434
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #431
PostgresType.oidwas initialized to 0 and never populated for array child types, causingCopyRequiresTextto always fall back toFORMAT TEXTfor any table with array columns. This resulted in ~2x slower bulk write throughput.Root Cause
The check in
CopyRequiresTextfor LIST types:Always evaluated as
0 != 1043 → true → FORMAT TEXTbecause theoidfield was never set in either code path.Changes
1.
CreateEmptyPostgresType(DDL-created tables)When DuckDB creates a table via DDL (e.g.
CREATE TABLE ... VARCHAR[]), the childPostgresType.oidwas left at 0. Fixed by setting it viaToPostgresOid(), which is correct since DuckDB controls the DDL and the Postgres types will match.2.
TypeToLogicalType(catalog-read tables)When reading existing Postgres tables from the catalog, the child
PostgresType.oidwas also never set. HereToPostgresOid()cannot be used because multiple Postgres type names map to the same DuckDBLogicalType(e.g. bothtextandvarcharmap toVARCHAR, but have different OIDs: 25 vs 1043). AddedTypeNameToPostgresOid()to resolve the OID from the actual Postgres type name, preserving the text/varchar distinction.Result
varchar[]columns now correctly use binary COPY in both pathstext[]columns correctly fall back to TEXT copy (sinceTEXTOID != VARCHAROIDand the binary writer always emitsVARCHAROID)int[],boolean[],uuid[], etc.) also benefit from binary COPYTest plan
attach_array_binary_copy.testcovering 4 scenarios:VARCHAR[]andINT[]BOOLEAN[],DATE[],DOUBLE[],UUID[]varchar[]table created viapostgres_executetext[]table created viapostgres_execute(verifies TEXT fallback still works)