Skip to content

Fail with a clear error when scanning a partitioned table#435

Open
simi wants to merge 1 commit intoduckdb:mainfrom
simi:graceful-partition-scan-fail
Open

Fail with a clear error when scanning a partitioned table#435
simi wants to merge 1 commit intoduckdb:mainfrom
simi:graceful-partition-scan-fail

Conversation

@simi
Copy link
Copy Markdown

@simi simi commented Apr 11, 2026

Scanning a partitioned table caused an infinite hang because PostgreSQL sets relpages=-1 for relkind='p' tables. Cast to idx_t (uint64) gives pages_approx = 2^64-1, so the scanner loops over ~18 quintillion pages.

Now throws NotImplementedException at bind time with a message that includes the relation kind and table name.


Initial step to provide support (yes, by gracefully fail for now, better than hang IMHO) for partitioned tables, since those are broken as reported at #220.

I have plan to open set of smaller PRs to improve the situation ending with full support of partitioned tables. This is initial foundation - accepting the broken state, add friendly error and add testing infrastructure.

Example EXPLAIN (SELECT hangs):

lake D EXPLAIN (FORMAT JSON) SELECT * FROM pg.public.sales_revenues;

┌─────────────────────────────┐
│┌───────────────────────────┐│
││       Physical Plan       ││
│└───────────────────────────┘│
└─────────────────────────────┘
[
    {
        "name": "POSTGRES_SCAN",
        "children": [],
        "extra_info": {
            "Table": "sales_revenues",
            "Projections": [
                "id",
                "account_id",
                "sale_id",
                "report_id",
                "report_group_id",
                "organization_id",
                "contract_id",
                "configuration_id",
                "product_id",
                "product_asset_id",
                "asset_id",
                "base",
                "share",
                "royalty",
                "currency",
                "chain",
                "statement_id",
                "unit_amount"
            ],
            "Estimated Cardinality": "18446744073709551568"
        }
    }

pg_class returns

reporting_development=# SELECT relkind, relpages FROM pg_class WHERE relname = 'sales_revenues';
 relkind | relpages 
---------+----------
 p       |       -1

Scanning a partitioned table caused an infinite hang because PostgreSQL
sets relpages=-1 for relkind='p' tables. Cast to idx_t (uint64) gives
pages_approx = 2^64-1, so the scanner loops over ~18 quintillion pages.

Now throws NotImplementedException at bind time with a message that
includes the relation kind and table name.
return result;
}

string PostgresUtils::RelkindToString(const string &relkind) {
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even patch is targeted to partitioned tables, same issue would be now for any relkind. This mapping in here is to provide nice error message only for user with safe fallback. It handles all relkinds loaded at

WHERE attnum > 0 AND relkind IN ('r', 'v', 'm', 'f', 'p') ${CONDITION}
.

info->approx_num_pages = approx_num_pages;
auto relpages = result.IsNull(row, 2) ? 0 : result.GetInt64(row, 2);
auto relkind = result.IsNull(row, 12) ? string() : result.GetString(row, 12);
info->approx_num_pages = GetScanRelpages(relpages, relkind, table_name);
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetScanRelpages is planned to be kept even once partitioned tables are fully supported as a guard for unexpected raw values.

for (idx_t row = 0; row < rows; row++) {
AddColumnOrConstraint(&transaction, &schema, *result, row, *table_info);
}
table_info->approx_num_pages = result->GetInt64(0, 2);
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and following GetInt64 missed the null check. Was it intended? I have added it in my change.

@simi
Copy link
Copy Markdown
Author

simi commented Apr 11, 2026

I'll check on CI failures.

@staticlibs
Copy link
Copy Markdown
Collaborator

@simi

Just FYI, some details on local test runs: #431 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants