Pool: add configuration and introspection function#430
Merged
staticlibs merged 3 commits intoduckdb:mainfrom Apr 6, 2026
Merged
Pool: add configuration and introspection function#430staticlibs merged 3 commits intoduckdb:mainfrom
staticlibs merged 3 commits intoduckdb:mainfrom
Conversation
This PR adds `postgres_configure_pool` table function that can be used to configure the connection pool for the specified attached Postgres DB (all for all attached Postgres DBs). This function return the details about the configured pool. Testing: basic test added, more test coverage pending.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a Postgres database is being attached, a connection pool is created for this database.
This PR adds the ability to configure connection pool details for both newly created pools (newly attached databases) and also for already existing pools (for databases that are already attached).
It adds the following configuration options that only apply to databases attached after the option is set:
pg_pool_max_connections(UBIGINT, default:max(num_cpus, 8)): maximum number of connections that are allowed to be cached in a connection pool for each attached Postgres database. This number can be temporary exceeded when parallel scans are used.pg_pool_wait_timeout_millis(UBIGINT, default:30000): maximum number of milliseconds to wait when acquiring a connection from a pool where all available connections are already taken.pg_pool_enable_thread_local_cache(BOOLEAN, default:true): whether to enable the connection caching in thread-local cache. Such connections are getting pinned to the threads and are not made available to other threads, while still taking the place in the pool.pg_pool_max_lifetime_millis(UBIGINT, default:0- unlimited): maximum number of milliseconds the connection can be kept open. This number is checked when the connection is taken from the pool and returned to the pool. When the connection pool reaper thread is enabled ('pg_pool_enable_reaper_thread' option), then this number is checked in background periodically.pg_pool_idle_timeout_millis(UBIGINT, default:0- unlimited): maximum number of milliseconds the connection can be kept idle in the pool. This number is checked when the connection is taken from the pool. When the connection pool reaper thread is enabled ('pg_pool_enable_reaper_thread' option), then this number is checked in background periodically.pg_pool_enable_reaper_thread(default:FALSE): whether to enable the connection pool reaper thread, that periodically scans the pool to check the 'max_lifetime_millis' and 'idle_timeout_millis' and closes the connection which exceed the specified values. Either 'max_lifetime_millis' or 'idle_timeout_millis' must be set to a non-zero value for this option to be effective.pg_pool_health_check_query(VARCHAR, default:SELECT 1): the query that is used to check that the connection is healthy. Setting this option to an empty string disables the health check.Pool options that existed before, these options apply to both newly created pools and also to all existing pools:
pg_connection_limit(default:max(num_cpus, 8)): the same aspg_pool_max_connectionspg_connection_cache(default:TRUE): the same asSET pg_pool_max_connections = 0orRESET pg_pool_max_connections.To change the configuration of an exiting pool new table function is added:
Function: postgres_configure_pool
Allows to change the configuration of an existing connection pool (in an attached Postgres database). Returns the current state of a single pool or of all pools.
Parameters:
catalog_name(VARCHAR): the name (alias) of the attached Postgres database to which pool the configuration change is applied and details are returned. WhenNULL(default) returns the current state of pools for all attached catalogs without changing their configuration. Must be specified and non-NULL when any other option is specified.max_connections(UBIGINT): maximum number of connections that are allowed to be cached in a connection pool for each attached Postgres database. This number can be temporary exceeded when parallel scans are used.wait_timeout_millis(UBIGINT): maximum number of milliseconds to wait when acquiring a connection from a pool where all available connections are already taken.enable_thread_local_cache(BOOLEAN): whether to enable the connection caching in thread-local cache. Such connections are getting pinned to the threads and are not made available to other threads, while still taking the place in the pool.max_lifetime_millis(UBIGINT): maximum number of milliseconds the connection can be kept open. This number is checked when the connection is taken from the pool and returned to the pool. When the connection pool reaper thread is enabled ('enable_reaper_thread' argument), then this number is checked in background periodically.idle_timeout_millis(UBIGINT): maximum number of milliseconds the connection can be kept idle in the pool. This number is checked when the connection is taken from the pool. When the connection pool reaper thread is enabled ('enable_reaper_thread' option), then this number is checked in background periodically.enable_reaper_thread(BOOLEAN): whether to enable the connection pool reaper thread, that periodically scans the pool to check the 'max_lifetime_millis' and 'idle_timeout_millis' and closes the connection which exceed the specified values. Either 'max_lifetime_millis' or 'idle_timeout_millis' must be set to a non-zero value for this option to be effective.health_check_query(VARCHAR): the query that is used to check that the connection is healthy. Setting this option to an empty string disables the health check.Returns:
A table with 1 row for each pool with the following columns:
catalog_name(VARCHAR): the name (alias) of the attached Postgres databaseavailable_connections(UBIGINT): the number of idle connection that are currently available in the poolmax_connections(UBIGINT): maximum number of connections that are allowed to be cached in the pool. This number can be temporary exceeded when parallel scans are used.wait_timeout_millis(UBIGINT): maximum number of milliseconds to wait when acquiring a connection from a pool where all available connections are already takenthread_local_cache_enabled(BOOLEAN): whether caching the connections caching in thread-local cache is enabledthread_local_cache_hits(UBIGINT): the number of times connections were successfully acquired from a thread-local cache without going to the main poolthread_local_cache_misses(UBIGINT): the number of times connections were not available in a thread-local cache and were taken from the main pool insteadmax_lifetime_millis(UBIGINT): maximum number of milliseconds the connection can be kept openidle_timeout_millis(UBIGINT): maximum number of milliseconds the connection can be kept idle in the poolreaper_thread_running(BOOLEAN): whether the pool reaper thread is running; this thread periodically scans the pool to check the 'max_lifetime_millis' and 'idle_timeout_millis' and closes the connection which exceed the specified valueshealth_check_query(VARCHAR): the query that is used to check that the connection is healthyExamples:
Change 2 options on a pool for the specified attached database:
Return current pool state for all attached databases:
FROM postgres_configure_pool();Testing: new tests added for new configuration options and for
postgres_configure_poolfunction.