Open
Conversation
robertmhaas
force-pushed
the
pg_plan_advice
branch
5 times, most recently
from
3124131 to
30a9774
Compare
cost_tidrangescan() was setting the disabled_nodes value correctly, and then immediately resetting it to zero, due to poor code editing on my part. materialized_finished_plan correctly set matpath.parent to zero, but forgot to also set matpath.parallel_workers = 0, causing an access to uninitialized memory in cost_material. (This shouldn't result in any real problem, but it makes valgrind unhappy.) reparameterize_path was dereferencing a variable before verifying that it was not NULL. Reported-by: Tom Lane <tgl@sss.pgh.pa.us> (issue #1) Reported-by: Michael Paquier <michael@paquier.xyz> (issue #1) Diagnosed-by: Lukas Fittl <lukas@fittl.com> (issue #1) Reported-by: Zsolt Parragi <zsolt.parragi@percona.com> (issue #2) Reported-by: Richard Guo <guofenglinux@gmail.com> (issue #3) Discussion: http://postgr.es/m/CAN4CZFPvwjNJEZ_JT9Y67yR7C=KMNa=LNefOB8ZY7TKDcmAXOA@mail.gmail.com Discussion: http://postgr.es/m/aXrnPgrq6Gggb5TG@paquier.xyz
Use the proper constant InvalidXLogRecPtr instead of literal 0 when assigning XLogRecPtr variables and struct fields. This improves code clarity by making it explicit that these are invalid LSN values rather than ambiguous zero literals. Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Discussion: https://postgr.es/m/aRtd2dw8FO1nNX7k@ip-10-97-1-34.eu-west-3.compute.internal
The leaks were hard to reach in practice and the impact was low. The callers provide a buffer the same number of bytes as the source string (plus one for NUL terminator) as a starting size, and libc never increases the number of characters. But, if the byte length of one of the converted characters is larger, then it might need a larger destination buffer. Previously, in that case, the working buffers would be leaked. Even in that case, the call typically happens within a context that will soon be reset. Regardless, it's worth fixing to avoid such assumptions, and the fix is simple so it's worth backporting. Discussion: https://postgr.es/m/e2b7a0a88aaadded7e2d19f42d5ab03c9e182ad8.camel@j-davis.com Backpatch-through: 18
robertmhaas
force-pushed
the
pg_plan_advice
branch
from
30a9774 to
c930d75
Compare
Commit 6ceef94 was still one brick shy of a load, because it caused any usage at all of PGIOAlignedBlock or PGAlignedXLogBlock to fail under older g++. Notably, this broke "headerscheck --cplusplus". We can permit references to these structs as abstract structs though; only actual declaration of such a variable needs to be forbidden. Discussion: https://www.postgresql.org/message-id/3119480.1769189606@sss.pgh.pa.us
In fcb9c97 I included an assertion in BufferLockConditional() to detect if a conditional lock acquisition is done on a buffer that we already have locked. The assertion was added in the course of adding other assertions. Unfortunately I failed to realize that some of our code relies on such lock acquisitions to silently fail. E.g. spgist and nbtree may try to conditionally lock an already locked buffer when acquiring a empty buffer. LWLockAcquireConditional(), which was previously used to implement ConditionalLockBuffer(), does not have such an assert. Instead of just removing the assert, and relying on the lock acquisition to fail due to the buffer already locked, this commit changes the behaviour of conditional content lock acquisition to fail if the current backend has any pre-existing lock on the buffer, even if the lock modes would not conflict. The reason for that is that we currently do not have space to track multiple lock acquisitions on a single buffer. Allowing multiple locks on the same buffer by a backend also seems likely to lead to bugs. There is only one non-self-exclusive conditional content lock acquisition, in GetVictimBuffer(), but it only is used if the target buffer is not pinned and thus can't already be locked by the current backend. Reported-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/90bd2cbb-49ce-4092-9f61-5ac2ab782c94@gmail.com
Previously, the CheckXidAlive check was performed within the table_scan*next* functions. This caused the check to be executed for every fetched tuple, an unnecessary overhead. To fix, move the check to table_beginscan* so it is performed once per scan rather than once per row. Note: table_tuple_fetch_row_version() does not use a scan descriptor; therefore, the CheckXidAlive check is retained in that function. The overhead is unlikely to be relevant for the existing callers. Reported-by: Andres Freund <andres@anarazel.de> Author: Dilip Kumar <dilipbalaut@gmail.com> Suggested-by: Andres Freund <andres@anarazel.de> Suggested-by: Amit Kapila <akapila@postgresql.org> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://www.postgresql.org/message-id/tlpltqm5jjwj7mp66dtebwwhppe4ri36vdypux2zoczrc2i3mp%40dhv4v4nikyfg
Author: Yugo Nagata <nagata@sraoss.co.jp> Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com> Discussion: https://postgr.es/m/20260128120056.b2a3e8184712ab5a537879eb@sraoss.co.jp
This makes the arrays somewhat easier to read. Author: Álvaro Herrera <alvherre@kurilemu.de> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl> Discussion: https://postgr.es/m/202601281204.sdxbr5qvpunk@alvherre.pgsql
These changes should have been done by 2f96613, but were overlooked. I noticed while reviewing the code for commit b8926a5. Author: Álvaro Herrera <alvherre@kurilemu.de> Discussion: https://postgr.es/m/18984-0f4778a6599ac3ae@postgresql.org
For readability. It was a slight modularity violation to have fields in PGShmemHeader that were only used by the allocator code in shmem.c. And it was inconsistent that ShmemLock was nevertheless not stored there. Moving all the allocator-related fields to a separate struct makes it more consistent and modular, and removes the need to allocate and pass ShmemLock separately via BackendParameters. Merge InitShmemAccess() and InitShmemAllocation() into a single function that initializes the struct when called from postmaster, and when called from backends in EXEC_BACKEND mode, re-establishes the global variables. That's similar to all the *ShmemInit() functions that we have. Co-authored-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Discussion: https://www.postgresql.org/message-id/CAExHW5uNRB9oT4pdo54qAo025MXFX4MfYrD9K15OCqe-ExnNvg@mail.gmail.com
BackgroundPsql needs to wait for all the output from an interactive psql command to come back. To make sure that's happened, it issues the command, then issues \echo and \warn psql commands that echo a "banner" string (which we assume won't appear in the command's output), then waits for the banner strings to appear. The hazard in this approach is that the banner will also appear in the echoed psql commands themselves, so we need to distinguish those echoes from the desired output. Commit 8b886a4 tried to do that by positing that the desired output would be directly preceded and followed by newlines, but it turns out that that assumption is timing-sensitive. In particular, it tends to fail in builds made --without-readline, wherein the command echoes will be made by the pty driver and may be interspersed with prompts issued by psql proper. It does seem safe to assume that the banner output we want will be followed by a newline, since that should be the last output before things quiesce. Therefore, we can improve matters by putting quotes around the banner strings in the \echo and \warn psql commands, so that their echoes cannot include banner directly followed by newline, and then checking for just banner-and-newline in the match pattern. While at it, spruce up the pump() call in sub query() to look like the neater version in wait_connect(), and don't die on timeout until after printing whatever we got. Reported-by: Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru> Diagnosed-by: Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Soumya S Murali <soumyamurali.work@gmail.com> Discussion: https://postgr.es/m/db6fdb35a8665ad3c18be01181d44b31@postgrespro.ru Backpatch-through: 14
Similarly to the preceding commit, 030_pager.pl was assuming that patterns it looks for in interactive psql output would appear by themselves on a line, but that assumption tends to fall over in builds made --without-readline: the output we get might have a psql prompt immediately followed by the expected line of output. For several of these tests, just checking for the pattern followed by newline seems sufficient, because we could not get a false match against the command echo, nor against the unreplaced command output if the pager fails to be invoked when expected. However, that's fairly scary for the test that was relying on information_schema.referential_constraints: "\d+" could easily appear at the end of a line in that view. Let's get rid of that hazard by making a custom test view instead of using information_schema.referential_constraints. This test script is new in v19, so no need for back-patch. Reported-by: Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru> Author: Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Soumya S Murali <soumyamurali.work@gmail.com> Discussion: https://postgr.es/m/db6fdb35a8665ad3c18be01181d44b31@postgrespro.ru
The build generates four files based on the wait event contents stored in wait_event_names.txt: - wait_event_types.h - pgstat_wait_event.c - wait_event_funcs_data.c - wait_event_types.sgml The SGML file is generated as part of a documentation build, with its data stored in doc/src/sgml/ for meson and configure. The three others are handled differently for meson and configure: - In configure, all the files are created in src/backend/utils/activity/. A link to wait_event_types.h is created in src/include/utils/. - In meson, all the files are created in src/include/utils/. The two C files, pgstat_wait_event.c and wait_event_funcs_data.c, are then included in respectively wait_event.c and wait_event_funcs.c, without the "utils/" path. For configure, this does not present a problem. For meson, this has to be combined with a trick in src/backend/utils/activity/meson.build, where include_directories needs to point to include/utils/ to make the inclusion of the C files work properly, causing builds to pull in PostgreSQL headers rather than system headers in some build paths, as src/include/utils/ would take priority. In order to fix this issue, this commit reworks the way the C/H files are generated, becoming consistent with guc_tables.inc.c: - For meson, basically nothing changes. The files are still generated in src/include/utils/. The trick with include_directories is removed. - For configure, the files are now generated in src/backend/utils/, with links in src/include/utils/ pointing to the ones in src/backend/. This requires extra rules in src/backend/utils/activity/Makefile so as a make command in this sub-directory is able to work. - The three files now fall under header-stamp, which is actually simpler as guc_tables.inc.c does the same. - wait_event_funcs_data.c and pgstat_wait_event.c are now included with "utils/" in their path. This problem has not been an issue in the buildfarm; it has been noted with AIX and a conflict with float.h. This issue could, however, create conflicts in the buildfarm depending on the environment with unexpected headers pulled in, so this fix is backpatched down to where the generation of the wait-event files has been introduced. While on it, this commit simplifies wait_event_names.txt regarding the paths of the files generated, to mention just the names of the files generated. The paths where the files are generated became incorrect. The path of the SGML path was wrong. This change has been tested in the CI, down to v17. Locally, I have run tests with configure (with and without VPATH), as well as meson, on the three branches. Combo oversight in fa88928 and 1e68e43. Reported-by: Aditya Kamath <aditya.kamath1@ibm.com> Discussion: https://postgr.es/m/LV8PR15MB64888765A43D229EA5D1CFE6D691A@LV8PR15MB6488.namprd15.prod.outlook.com Backpatch-through: 17
A failing unlink() was reporting an incorrect error message, referring to stat(). Author: Man Zeng <zengman@halodbtech.com> Reviewed-by: Junwang Zhao <zhjwpku@gmail.com> Discussion: https://postgr.es/m/tencent_3BBE865C5F49D452360FF190@qq.com Backpath-through: 17
Up to now we've used GNU-style local labels for branch targets in s_lock.h's assembly blocks. But there's an alternative style, which I for one didn't know about till recently: use regular assembler labels, and insert a per-asm-block number in them using %= to ensure they are distinct across multiple TAS calls within one source file. gcc has had %= since gcc 2.0, and I've verified that clang knows it too. While the immediate motivation for changing this is that AIX's assembler doesn't do local labels, it seems to me that this is a superior solution anyway. There is nothing mnemonic about "1:", while a regular label can convey something useful, and at least to me it feels less error-prone. Therefore let's standardize on this approach, also converting the one other usage in s_lock.h. Discussion: https://postgr.es/m/399291.1769998688@sss.pgh.pa.us
Separate att_align_nominal() into two macros, similarly to what was already done with att_align_datum() and att_align_pointer(). The inner macro att_nominal_alignby() is really just TYPEALIGN(), while att_align_nominal() retains its previous API by mapping TYPALIGN_xxx values to numbers of bytes to align to and then calling att_nominal_alignby(). In support of this, split out tupdesc.c's logic to do that mapping into a publicly visible function typalign_to_alignby(). Having done that, we can replace performance-critical uses of att_align_nominal() with att_nominal_alignby(), where the typalign_to_alignby() mapping is done just once outside the loop. In most places I settled for doing typalign_to_alignby() once per function. We could in many places pass the alignby value in from the caller if we wanted to change function APIs for this purpose; but I'm a bit loath to do that, especially for exported APIs that extensions might call. Replacing a char typalign argument by a uint8 typalignby argument would be an API change that compilers would fail to warn about, thus silently breaking code in hard-to-debug ways. I did revise the APIs of array_iter_setup and array_iter_next, moving the element type attribute arguments to the former; if any external code uses those, the argument-count change will cause visible compile failures. Performance testing shows that ExecEvalScalarArrayOp is sped up by about 10% by this change, when using a simple per-element function such as int8eq. I did not check any of the other loops optimized here, but it's reasonable to expect similar gains. Although the motivation for creating this patch was to avoid a performance loss if we add some more typalign values, it evidently is worth doing whether that patch lands or not. Discussion: https://postgr.es/m/1127261.1769649624@sss.pgh.pa.us
Oversight in commit 5373bc2. Author: Michael Banck <mbanck@gmx.net> Discussion: https://postgr.es/m/20260202173156.GB17962%40p46.dedyn.io%3Blightning.p46.dedyn.io
…porary table. The test relies on VACUUM being able to mark a page all-visible, but this can fail when autovacuum in other sessions prevents the visibility horizon from advancing. Making the test table temporary isolates its horizon from other sessions, including catalog table vacuums, ensuring reliable test behavior. Reported-by: Alexander Lakhin <exclusion@gmail.com> Author: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Discussion: https://postgr.es/m/2b09fba6-6b71-497a-96ef-a6947fcc39f6%40gmail.com
This commit introduces a new prompt escape %i for psql, which shows whether the connected server is operating in hot standby mode. It expands to standby if the server reports in_hot_standby = on, and primary otherwise. This is useful for distinguishing standby servers from primary ones at a glance, especially when working with multiple connections in replicated environments where libpq's multi-host connection strings are used. Author: Jim Jones <jim.jones@uni-muenster.de> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com> Reviewed-by: Srinath Reddy Sadipiralla <srinath2133@gmail.com> Reviewed-by: Nathan Bossart <nathandbossart@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Andreas Karlsson <andreas@proxel.se> Discussion: https://www.postgresql.org/message-id/flat/016f6738-f9a9-4e98-bb5a-e1e4b9591d46@uni-muenster.de
…changes. Previously, when synchronous_standby_names was changed (for example, by reducing the number of required synchronous standbys or modifying the standby list), backends waiting for synchronous replication were not released immediately, even if the new configuration no longer required them to wait. They could remain blocked until additional messages arrived from standbys and triggered their release. This commit improves walsender so that backends waiting for synchronous replication are released as soon as the updated configuration takes effect and the new settings no longer require them to wait, by calling SyncRepReleaseWaiters() when configuration changes are processed. As part of this change, the duplicated code that handles configuration changes in walsender has been refactored into a new helper function, which is now used at the three existing call places. Since this is an improvement rather than a bug fix, it is applied only to the master branch. Author: Shinya Kato <shinya11.kato@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Discussion: https://postgr.es/m/CAOzEurSRii0tEYhu5cePmRcvS=ZrxTLEvxm3Kj0d7_uKGdM23g@mail.gmail.com
This routine has an option to bypass an error if a WAL summary file is opened for read but is missing (missing_ok=true). However, the code incorrectly checked for EEXIST, that matters when using O_CREAT and O_EXCL, rather than ENOENT, for this case. There are currently only two callers of OpenWalSummaryFile() in the tree, and both use missing_ok=false, meaning that the check based on the errno is currently dead code. This issue could matter for out-of-core code or future backpatches that would like to use missing_ok set to true. Issue spotted while monitoring this area of the code, after a9afa02. Author: Michael Paquier <michael@paquier.xyz> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/aYAf8qDHbpBZ3Rml@paquier.xyz Backpatch-through: 17
Two wait events are added to the COPY FROM/TO code: * COPY_FROM_READ: reading data from a copy_file. * COPY_TO_WRITE: writing data to a copy_file. In the COPY code, copy_file can be set when processing a command through the pipe mode (for the non-DestRemote case), the program mode or the file mode, when processing fread() or fwrite() on it. Author: Nikolay Samokhvalov <nik@postgres.ai> Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Discussion: https://postgr.es/m/CAM527d_iDzz0Kqyi7HOfqa-Xzuq29jkR6AGXqfXLqA5PR5qsng@mail.gmail.com
This keeps run-time assertions and static assertions clearly separate. Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/2273bc2a-045d-4a75-8584-7cd9396e5534%40eisentraut.org
The documentation for the INCLUDING COMMENTS option of the LIKE clause in CREATE TABLE was inaccurate and incomplete. It stated that comments for copied columns, constraints, and indexes are copied, but regarding comments on constraints in reality only comments on CHECK and NOT NULL constraints are copied; comments on other constraints (such as primary keys) are not. In addition, comments on extended statistics are copied, but this was not documented. The CREATE FOREIGN TABLE documentation had a similar omission: comments on extended statistics are also copied, but this was not mentioned. This commit updates the documentation to clarify the actual behavior. The CREATE TABLE reference now specifies that comments on copied columns, CHECK constraints, NOT NULL constraints, indexes, and extended statistics are copied. The CREATE FOREIGN TABLE reference now notes that comments on extended statistics are copied as well. Backpatch to all supported versions. Documentation updates related to CREATE FOREIGN TABLE LIKE and NOT NULL constraint comment copying are not applied to v17 and earlier, since those features were introduced in v18. Author: Fujii Masao <masao.fujii@gmail.com> Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com> Discussion: https://postgr.es/m/CAHGQGwHSOSGcaYDvHF8EYCUCfGPjbRwGFsJ23cx5KbJ1X6JouQ@mail.gmail.com Backpatch-through: 14
Settings that ran the new test euc_kr.sql to completion would fail these older src/pl tests. Use alternative expected outputs, for which psql \gset and \if have reduced the maintenance burden. This fixes "LANG=ko_KR.euckr LC_MESSAGES=C make check-world". (LC_MESSAGES=C fixes IO::Pty usage in tests 010_tab_completion and 001_password.) That file is new in commit c67bef3. Back-patch to v14, like that commit. Discussion: https://postgr.es/m/20260217184758.da.noahmisch@microsoft.com Backpatch-through: 14
When the constraint is printed without the column, we were not printing the NO INHERIT flag. Author: Jian He <jian.universality@gmail.com> Backpatch-through: 18 Discussion: https://postgr.es/m/CACJufxEDEOO09G+OQFr=HmFr9ZDLZbRoV7+pj58h3_WeJ_K5UQ@mail.gmail.com
The lack of fallout here is somewhat surprising. Author: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/aY-UE-4t7FiYgH3t@alap3.anarazel.de
pg_dumpall can now produce output in custom, directory, or tar formats in addition to plain text SQL scripts. When using non-text formats, pg_dumpall creates a directory containing: - toc.glo: global data (roles and tablespaces) in custom format - map.dat: mapping between database OIDs and names - databases/: subdirectory with per-database archives named by OID pg_restore is extended to handle these pg_dumpall archives, restoring globals and then each database. The --globals-only option can be used to restore only the global objects. This enables parallel restore of pg_dumpall output and selective restoration of individual databases from a cluster-wide backup. Author: Mahendra Singh Thalor <mahi6run@gmail.com> Co-Author: Andrew Dunstan <andrew@dunslane.net> Reviewed-By: Tushar Ahuja <tushar.ahuja@enterprisedb.com> Reviewed-By: Jian He <jian.universality@gmail.com> Reviewed-By: Vaibhav Dalvi <vaibhav.dalvi@enterprisedb.com> Reviewed-By: Srinath Reddy <srinath2133@gmail.com> Discussion: https://postgr.es/m/cb103623-8ee6-4ba5-a2c9-f32e3a4933fa@dunslane.net
This was incorrectly named "LT" for "larger than" in e5a5e0a, but that is against existing conventions, where "LT" means "less than". Clarify by using "GT" for "greater than" in macro name, and add a missing comment at the top of instr_time.h to note the macro's existence. Reported by: Peter Smith <smithpb2250@gmail.com> Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CAHut%2BPut94CTpjQsqOJHdHkgJ2ZXq%2BqVSfMEcmDKLiWLW-hPfA%40mail.gmail.com
This macro had exactly one user in InstrStartNode, and the caller can instead use INSTR_TIME_IS_ZERO / INSTR_TIME_SET_CURRENT directly. This supports a future change that intends to modify the time source being used in the InstrStartNode case. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CAP53Pkx1bK1FB71_nBqYmzvSSXnp_MbE0ZDnU+baPJF6Ud2WDA@mail.gmail.com
This reduces the inclusion footprint of latch.h a bit. Per suggestion from Andres Freund. Discussion: https://postgr.es/m/pap7mzhcxvuwlfdebjkh646ntyk4brtwm4dbocfpllwdccta5t@w3d7wz6mjpwv
The LVRelState fields that track newly all-visible/all-frozen pages were previously named vm_new_visible_pages, vm_new_frozen_pages, and vm_new_visible_frozen_pages. The correct terminology is all-visible and all-frozen; omitting “all” was open to misinterpretation, as the page isn't visible or invisible, rather all the tuples on the page are visible to all running and future transactions. Rename the members accordingly. Author: Melanie Plageman <melanieplageman@gmail.com> Suggested-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/bqc4kh5midfn44gnjiqez3bjqv4zogydguvdn446riw45jcf3y%404ez66il7ebvk
The backport prior to 18 requires minor modification due to code refactoring. Discussion: https://postgr.es/m/e2b7a0a88aaadded7e2d19f42d5ab03c9e182ad8.camel@j-davis.com Backpatch-through: 16
Commit 84d5efa missed some multibyte issues caused by short-circuit logic in the callers. The callers assumed that if the predicate string is longer than the label string, then it couldn't possibly be a match, but it can be when using case-insensitive matching (LVAR_INCASE) if casefolding changes the byte length. Fix by refactoring to get rid of the short-circuit logic as well as the function pointer, and consolidate the logic in a replacement function ltree_label_match(). Discussion: https://postgr.es/m/02c6ef6cf56a5013ede61ad03c7a26affd27d449.camel@j-davis.com Backpatch-through: 14
heap_page_would_be_all_visible() does not need to distinguish between HEAPTUPLE_RECENTLY_DEAD and HEAPTUPLE_DEAD tuples: any tuple in a state other than HEAPTUPLE_LIVE means the page is not all-visible and heap_page_would_be_all_visible() returns false. Given that, calling HeapTupleSatisfiesVacuum() is unnecessary, since it performs extra work to distinguish between dead and recently dead tuples using OldestXmin. Replace it with the more minimal HeapTupleSatisfiesVacuumHorizon(). Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/CALdSSPjvhGXihT_9f-GJabYU%3D_PjrFDUxYaURuTbfLyQM6TErg%40mail.gmail.com
heapam_scan_analyze_next_tuple() doesn't distinguish between dead and recently dead tuples when counting them, so it doesn't need OldestXmin. GetOldestNonRemovableTransactionId() isn't free, so removing it is a win. Looking at other table AMs implementing table_scan_analyze_next_tuple(), we couldn't find one using OldestXmin either, so remove it from the callback. Author: Melanie Plageman <melanieplageman@gmail.com> Suggested-by: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/CALdSSPjvhGXihT_9f-GJabYU%3D_PjrFDUxYaURuTbfLyQM6TErg%40mail.gmail.com
robertmhaas
force-pushed
the
pg_plan_advice
branch
from
e16f735 to
264c072
Compare
This commit also rephrases this comment to improve readability. Oversight in commit 6136e94. Reported-by: Etsuro Fujita <etsuro.fujita@gmail.com> Author: Andreas Karlsson <andreas@proxel.se> Co-authored-by: Etsuro Fujita <etsuro.fujita@gmail.com> Discussion: https://postgr.es/m/CAPmGK16pDnM_wU3kmquPj-M9MYqG3y0BdntRZ0eytqbCaFY3WQ%40mail.gmail.com Backpatch-through: 14
This prevents proliferation of proc.h to tons of other places; shm_mq.h is widely included. Discussion: https://postgr.es/m/202602261733.s2rkxezwuif6@alvherre.pgsql
This commit updates the frontend tools (src/bin/, contrib/ and src/test/) to use the memory allocation variants based on pg_malloc_object() and pg_malloc_array() in various code paths. This does not cover all the allocations, but a good chunk of them. Like all the changes of this kind (31d3847, etc.), this should encourage any future code to use this new style. Author: Andreas Karlsson <andreas@proxel.se> Discussion: https://postgr.es/m/cfb645da-6b3a-4f22-9bcc-5bc46b0e9c61@proxel.se
Oversight in commit 763aaa0. When nodes are going out of scope, we should stop the underlying postmasters rather than waiting for the script to end. Per gripe from Tom Lane Discussion: https://postgr.es/m/740033.1772142754@sss.pgh.pa.us
We now maintain an array of booleans that indicate which features were detected at runtime. When code wants to check for a given feature, the array is automatically checked if it has been initialized and if not, a single function checks all features at once. Move all x86 feature detection to pg_cpu_x86.c, and move the CRC function choosing logic to the file where the hardware-specific functions are defined, consistent with more recent hardware-specific files in src/port. Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Discussion: https://postgr.es/m/CANWCAZbgEUFw7LuYSVeJ=Tj98R5HoOB1Ffeqk3aLvbw5rU5NTw@mail.gmail.com
The RTE's groupexprs list is used for deparsing views, and for that usage it must contain the original alias Vars; else we can get incorrect SQL output. But since commit 247dea8, parseCheckAggregates put the GROUP BY expressions through flatten_join_alias_vars before building the RTE_GROUP RTE. Changing the order of operations there is enough to fix it. This patch unfortunately can do nothing for already-created views: if they use a coding pattern that is subject to the bug, they will deparse incorrectly and hence present a dump/reload hazard in the future. The only fix is to recreate the view from the original SQL. But the trouble cases seem to be quite narrow. AFAICT the output was only wrong for "SELECT ... t1 LEFT JOIN t2 USING (x) GROUP BY x" where t1.x and t2.x were not of identical data types and t1.x was the side that required an implicit coercion. If there was no hidden coercion, or if the join was plain, RIGHT, or FULL, the deparsed output was uglier than intended but not functionally wrong. Reported-by: Swirl Smog Dowry <swirl-smog-dowry@duck.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Richard Guo <guofenglinux@gmail.com> Discussion: https://postgr.es/m/CA+-gibjCg_vjcq3hWTM0sLs3_TUZ6Q9rkv8+pe2yJrdh4o4uoQ@mail.gmail.com Backpatch-through: 18
Point out that Postgres automatically optimizes away the target list of an EXISTS' subquery, except in weird cases such as target lists containing set-returning functions. Thus, both common conventions EXISTS(SELECT * FROM ...) and EXISTS(SELECT 1 FROM ...) are overhead-free and there's little reason to prefer one over the other. In the code comments, mention that the SQL spec says that EXISTS(SELECT * FROM ...) should be interpreted as EXISTS(SELECT some-literal FROM ...), but we don't choose to do it exactly that way. Author: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/9b301c70-3909-4f0f-98ca-9e3c4d142f3e@eisentraut.org
This fixes some warnings from -Wcast-qual that are easy to fix, without using unconstify or the like. Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Discussion: https://www.postgresql.org/message-id/990c9117-b013-4026-aaf5-261fe2832c3d%40eisentraut.org
Up until now, the only way for a loadable module to disable the use of a particular index was to use get_relation_info_hook to remove it from the index list. While that works, it has some disadvantages. First, the index becomes invisible for all purposes, and can no longer be used for optimizations such as self-join elimination or left join removal, which can severely degrade the resulting plan. Second, if the module attempts to compel the use of a certain index by removing all other indexes from the index list and disabling other scan types, but the planner is unable to use the chosen index for some reason, it will fall back to a sequential scan, because that is only disabled, whereas the other indexes are, from the planner's point of view, completely gone. While this situation ideally shouldn't occur, it's hard for a loadable module to be completely sure whether the planner will view a certain index as usable for a certain query. If it isn't, it may be better to fall back to a scan using a disabled index rather than falling back to an also-disabled sequential scan. Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
For a long time, PostgreSQL has had a get_relation_info_hook which plugins can use to editorialize on the information that get_relation_info obtains from the catalogs. However, this hook is only called for baserels of type RTE_RELATION, and there is potential utility in a similar call back for other types of RTEs. This might have had utility even before commit 4020b37 added pgs_mask to RelOptInfo, but it certainly has utility now. So, move the callback up one level, deleting get_relation_info_hook and adding build_simple_rel_hook instead. The new callback is called just slightly later than before and with slightly different arguments, but it should be fairly straightforward to adjust existing code that currentyy uses get_relation_info_hook: the values previously available as relationObjectId and inhparent are now available via rte->relid and rte->inh, and calls where rte->rtekind != RTE_RELATION can be ignored if desired. Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Provide a facility that (1) can be used to stabilize certain plan choices so that the planner cannot reverse course without authorization and (2) can be used by knowledgeable users to insist on plan choices contrary to what the planner believes best. In both cases, terrible outcomes are possible: users should think twice and perhaps three times before constraining the planner's ability to do as it thinks best; nevertheless, there are problems that are much more easily solved with these facilities than without them. This patch takes the approach of analyzing a finished plan to produce textual output, which we call "plan advice", that describes key decisions made during plan; if that plan advice is provided during future planning cycles, it will force those key decisions to be made in the same way. Not all planner decisions can be controlled using advice; for example, decisions about how to perform aggregation are currently out of scope, as is choice of sort order. Plan advice can also be edited by the user, or even written from scratch in simple cases, making it possible to generate outcomes that the planner would not have produced. Partial advice can be provided to control some planner outcomes but not others. Currently, plan advice is focused only on specific outcomes, such as the choice to use a sequential scan for a particular relation, and not on estimates that might contribute to those outcomes, such as a possibly-incorrect selectivity estimate. While it would be useful to users to be able to provide plan advice that affects selectivity estimates or other aspects of costing, that is out of scope for this commit. Reviewed-by: Lukas Fittl <lukas@fittl.com> Reviewed-by: Jakub Wartak <jakub.wartak@enterprisedb.com> Reviewed-by: Greg Burd <greg@burd.me> Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com> Reviewed-by: Haibo Yan <tristan.yim@gmail.com> Reviewed-by: Dian Fay <di@nmfay.com> Reviewed-by: Ajay Pal <ajay.pal.k@gmail.com> Reviewed-by: John Naylor <johncnaylorls@gmail.com> Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com> Discussion: http://postgr.es/m/CA+TgmoZ-Jh1T6QyWoCODMVQdhTUPYkaZjWztzP1En4=ZHoKPzw@mail.gmail.com
This module allows automated, large-scale collection of queries and the associated plan advice strings using either backend-local memory or dynamic shared memory. In either case, memory usage can be limited by restriction the maximum number of queries and advice strings stored. Care should be taken with these values, and with the use of this module in general, because it's easy to chew up an unreasonably large amount of memory. Unlike pg_stat_statements, this module does not provide for query normalization or even deduplication; it simply makes a record for every query planned. It can be useful to enable query ID computaton before using the module, but it's not required. If not done, all queries will simply show a query ID of zero. Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com> (earlier version)
Previously, the coments stated that there was no purpose to considering startup cost for partial paths, but this is not the case: it's perfectly reasonable to want a fast-start path for a plan that involves a LIMIT (perhaps over an aggregate, so that there is enough data being processed to justify parallel query but yet we don't want all the result rows). Accordingly, rewrite add_partial_path and add_partial_path_precheck to consider startup costs. This also fixes an independent bug in add_partial_path_precheck: commit e222534 failed to update it to do anything with the new disabled_nodes field. That bug fix is formally separate from the rest of this patch and could be committed separately, but I think it makes more sense to fix both issues together, because then we can (as this commit does) just make add_partial_path_precheck do the cost comparisons in the same way as compare_path_costs_fuzzily, which hopefully reduces the chances of ending up with something that's still incorrect. This patch is based on earlier work on this topic by Tomas Vondra, but I have rewritten a great deal of it. Co-authored-by: Robert Haas <rhaas@postgresql.org> Co-authored-by: Tomas Vondra <tomas@vondra.me>
The TAP test included in this new module runs the regression tests with pg_plan_advice loaded. It arranges for each query to be planned twice. The first time, we generate plan advice. The second time, we replan the query using the resulting advice string. If the tests fail, that means that using pg_plan_advice to tell the planner to do what it was going to do anyway breaks something, which indicates a problem either with pg_plan_advice or with the planner. The test also enables pg_plan_advice.feedback_warnings, so that if the plan advice fails to apply cleanly when the query is replanned, a failure will occur.
This module allows plan advice strings to be provided automatically from an in-memory advice stash. Advice stashes are stored in dynamic shared memory and must be recreated and repopulated after a server restart. If pg_stash.advice_stash is set to the name of an advice stash, and if query identifiers are enabled, the query identifier for each query will be looked up in the advice stash and the associated advice string, if any, will be used each time that query is planned.
robertmhaas
force-pushed
the
pg_plan_advice
branch
from
264c072 to
86cf7ca
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mandatory description