Commit Graph

393 Commits

Author SHA1 Message Date
Peter Eisentraut c8e4030d1b Make finding openssl program a configure or meson option
Various test suites use the "openssl" program as part of their setup.
There isn't a way to override which openssl program is to be used,
other than by fiddling with the path, perhaps.  This has gotten
increasingly problematic because different versions of openssl have
different capabilities and do different things by default.

This patch checks for an openssl binary in configure and meson setup,
with appropriate ways to override it.  This is similar to how "lz4"
and "zstd" are handled, for example.  The meson build system actually
already did this, but the result was only used in some places.  This
is now applied more uniformly.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/dc638b75-a16a-007d-9e1c-d16ed6cf0ad2%40enterprisedb.com
2022-10-20 21:05:42 +02:00
Michael Paquier a19e5cee63 Rename SetSingleFuncCall() to InitMaterializedSRF()
Per discussion, the existing routine name able to initialize a SRF
function with materialize mode is unpopular, so rename it.  Equally, the
flags of this function are renamed, as of:
- SRF_SINGLE_USE_EXPECTED -> MAT_SRF_USE_EXPECTED_DESC
- SRF_SINGLE_BLESS -> MAT_SRF_BLESS
The previous function and flags introduced in 9e98583 are kept around
for compatibility purposes, so as any extension code already compiled
with v15 continues to work as-is.  The declarations introduced here for
compatibility will be removed from HEAD in a follow-up commit.

The new names have been suggested by Andres Freund and Melanie
Plageman.

Discussion: https://postgr.es/m/20221013194820.ciktb2sbbpw7cljm@awork3.anarazel.de
Backpatch-through: 15
2022-10-18 10:22:35 +09:00
Alvaro Herrera db1b931a4e
libpq: Reset singlerow flag correctly in pipeline mode
When a query whose results were requested in single-row mode is the last
in the queue by the time those results are being read, the single-row
flag was not being reset, because we were returning early from
pqPipelineProcessQueue.  Move that stanza up so that the flag is always
reset at the end of sending that query's results.

Add a test for the situation.

Backpatch to 14.

Author: Denis Laxalde <denis.laxalde@dalibo.com>
Discussion: https://postgr.es/m/01af18c5-dacc-a8c8-07ee-aecc7650c3e8@dalibo.com
2022-10-14 19:06:26 +02:00
Alvaro Herrera 8a927e3cfc
Fix outdated code reference
ExecCreatePartitionPruneState was renamed by commit 297daa9d43, but
this test file didn't get the memo.  Repair.

Author: Amit Langote
Discussion: https://postgr.es/m/CA+HiwqFLw=oLX0tP9kcKBmoOExNjDaoAe99dRcxo-GdB9abP9A@mail.gmail.com
2022-10-12 09:54:02 +02:00
Michael Paquier 94fd253d56 Fix compilation warning in test_copy_callbacks
A passed-in parameter value was incorrect, for a warning coming from
MSVC.

Oversight in 9fcdf2c.

Reported-by: Andres Freund
Discussion: https://postgr.es/m/20221011224221.dvg5q7e7vhjdtcvv@awork3.anarazel.de
2022-10-12 08:45:01 +09:00
Michael Paquier 9fcdf2c787 Add support for COPY TO callback functions
This is useful as a way for extensions to process COPY TO rows in the
way they see fit (say auditing, analytics, backend, etc.) without the
need to invoke an external process running as the OS user running the
backend through PROGRAM that requires superuser rights.  COPY FROM
already provides a similar callback for logical replication.  For COPY
TO, the callback is triggered when we are ready to send a row in
CopySendEndOfRow(), which is the same code path as when sending a row
to a frontend or a pipe/file.

A small test module, test_copy_callbacks, is added to provide some
coverage for this facility.

Author: Bilva Sanaba, Nathan Bossart
Discussion: https://postgr.es/m/253C21D1-FCEB-41D9-A2AF-E6517015B7D7@amazon.com
2022-10-11 11:45:52 +09:00
David Rowley cd4e8caaa0 Fix final warnings produced by -Wshadow=compatible-local
I thought I had these in d8df67bb1, but per report from Andres Freund, I
missed some.

Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/20221005214052.c4tkudawyp5wxt3c@awork3.anarazel.de
2022-10-07 13:13:27 +13:00
Andres Freund 902ab2fcef meson: Add windows resource files
The generated resource files aren't exactly the same ones as the old
buildsystems generate. Previously "InternalName" and "OriginalFileName" were
mostly wrong / not set (despite being required), but that was hard to fix in
at least the make build. Additionally, the meson build falls back to a
"auto-generated" description when not set, and doesn't set it in a few cases -
unlikely that anybody looks at these descriptions in detail.

Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-10-05 09:56:05 -07:00
Alvaro Herrera 0032a54567
Remove PQsendQuery support in pipeline mode
The extended query protocol implementation I added in commit
acb7e4eb6b has bugs when used in pipeline mode.  Rather than spend
more time trying to fix it, remove that code and make the function rely
on simple query protocol only, meaning it can no longer be used in
pipeline mode.

Users can easily change their applications to use PQsendQueryParams
instead.  We leave PQsendQuery in place for Postgres 14, just in case
somebody is using it and has not hit the mentioned bugs; but we should
recommend that it not be used.

Backpatch to 15.

Per bug report from Gabriele Varrazzo.
Discussion: https://postgr.es/m/CA+mi_8ZGSQNmW6-mk_iSR4JZB_LJ4ww3suOF+1vGNs3MrLsv4g@mail.gmail.com
2022-09-23 18:21:22 +02:00
Alvaro Herrera d11a41a4ce
Stop using PQsendQuery in libpq_pipeline
The "emulation" I wrote for PQsendQuery in pipeline mode to use extended
query protocol, in commit acb7e4eb6b, is problematic.  Due to numerous
bugs we'll soon remove it.  As a first step and for all branches back to
14, stop using PQsendQuery in libpq_pipeline.  Also remove a few test
lines that will no longer be relevant.

Backpatch to 14.

Discussion: https://postgr.es/m/CA+mi_8ZGSQNmW6-mk_iSR4JZB_LJ4ww3suOF+1vGNs3MrLsv4g@mail.gmail.com
2022-09-23 18:11:48 +02:00
Andres Freund e6927270cd meson: Add initial version of meson based build system
Autoconf is showing its age, fewer and fewer contributors know how to wrangle
it. Recursive make has a lot of hard to resolve dependency issues and slow
incremental rebuilds. Our home-grown MSVC build system is hard to maintain for
developers not using Windows and runs tests serially. While these and other
issues could individually be addressed with incremental improvements, together
they seem best addressed by moving to a more modern build system.

After evaluating different build system choices, we chose to use meson, to a
good degree based on the adoption by other open source projects.

We decided that it's more realistic to commit a relatively early version of
the new build system and mature it in tree.

This commit adds an initial version of a meson based build system. It supports
building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD,
Solaris and Windows (however only gcc is supported on aix, solaris). For
Windows/MSVC postgres can now be built with ninja (faster, particularly for
incremental builds) and msbuild (supporting the visual studio GUI, but
building slower).

Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM
bitcode generation, documentation adjustments) are done in subsequent commits
requiring further review. Other aspects (e.g. not installing test-only
extensions) are not yet addressed.

When building on Windows with msbuild, builds are slower when using a visual
studio version older than 2019, because those versions do not support
MultiToolTask, required by meson for intra-target parallelism.

The plan is to remove the MSVC specific build system in src/tools/msvc soon
after reaching feature parity. However, we're not planning to remove the
autoconf/make build system in the near future. Likely we're going to keep at
least the parts required for PGXS to keep working around until all supported
versions build with meson.

Some initial help for postgres developers is at
https://wiki.postgresql.org/wiki/Meson

With contributions from Thomas Munro, John Naylor, Stone Tickle and others.

Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Author: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de
2022-09-21 22:37:17 -07:00
Peter Geoghegan bc2187ed63 Consistently use named parameters in regex code.
Make regex code consistently use named parameters in function
declarations.  Also make sure that parameter names from each function's
declaration match corresponding definition parameter names.

This makes Henry Spencer's regex code follow Postgres coding standards.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAH2-WznJt9CMM9KJTMjJh_zbL5hD9oX44qdJ4aqZtjFi-zA3Tg@mail.gmail.com
2022-09-19 15:10:24 -07:00
Tom Lane cf2c7a736e Detect format-string mistakes in the libpq_pipeline test module.
I happened to notice that libpq_pipeline's private implementation
of pg_fatal lacked any pg_attribute_printf decoration.  Indeed,
adding that turned up a mistake!  We'd likely never have noticed
because the error exits in this code are unlikely to get hit,
but still, it's a bug.

We're so used to having the compiler check this stuff for us that
a printf-like function without pg_attribute_printf is a land mine.
I wonder if there is a way to detect such omissions.

Back-patch to v14 where this code came in.
2022-09-15 17:17:53 -04:00
Tom Lane f40346ff0b Doc: add some doco about using the libpq_pipeline test module.
The README file here was barely a stub.  Try to make it useful.

Jelte Fennema, with some further work by me

Discussion: https://postgr.es/m/AM5PR83MB0178D3B31CA1B6EC4A8ECC42F7529@AM5PR83MB0178.EURPRD83.prod.outlook.com
2022-09-14 16:43:37 -04:00
John Naylor e813e0e168 Add optimized functions for linear search within byte arrays
In similar vein to b6ef167564, add pg_lfind8() and pg_lfind8_le()
to search for bytes equal or less-than-or-equal to a given byte,
respectively. To abstract away platform details, add helper functions
and typedefs to simd.h.

John Naylor and Nathan Bossart, per suggestion from Andres Freund

Discussion: https://www.postgresql.org/message-id/CAFBsxsGzaaGLF%3DNuq61iRXTyspbO9rOjhSqFN%3DV6ozzmta5mXg%40mail.gmail.com
2022-08-26 14:03:39 +07:00
Tom Lane efd0c16bec Avoid using list_length() to test for empty list.
The standard way to check for list emptiness is to compare the
List pointer to NIL; our list code goes out of its way to ensure
that that is the only representation of an empty list.  (An
acceptable alternative is a plain boolean test for non-null
pointer, but explicit mention of NIL is usually preferable.)

Various places didn't get that memo and expressed the condition
with list_length(), which might not be so bad except that there
were such a variety of ways to check it exactly: equal to zero,
less than or equal to zero, less than one, yadda yadda.  In the
name of code readability, let's standardize all those spellings
as "list == NIL" or "list != NIL".  (There's probably some
microscopic efficiency gain too, though few of these look to be
at all performance-critical.)

A very small number of cases were left as-is because they seemed
more consistent with other adjacent list_length tests that way.

Peter Smith, with bikeshedding from a number of us

Discussion: https://postgr.es/m/CAHut+PtQYe+ENX5KrONMfugf0q6NHg4hR5dAhqEXEc2eefFeig@mail.gmail.com
2022-08-17 11:12:35 -04:00
Thomas Munro 7e50b4e3c5 Remove configure probe for sys/select.h.
<sys/select.h> is in SUSv3 and every targeted Unix system has it.
Provide an empty header in src/include/port/win32 so that we can
include it unguarded even on Windows.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKG%2BL_3brvh%3D8e0BW_VfX9h7MtwgN%3DnFHP5o7X2oZucY9dg%40mail.gmail.com
2022-08-14 00:09:47 +12:00
Daniel Gustafsson 92dc33a3a2 Fix typo in test_oat_hooks README
Discussion: https://postgr.es/m/3F066AFE-19F9-4DF5-A498-B09643857A39@yesql.se
2022-08-10 13:49:48 +02:00
John Naylor b6ef167564 Introduce optimized routine for linear searches of arrays
Use SSE2 intrinsics to speed up the search, where available.  Otherwise,
use a simple 'for' loop.  The motivation to add this now is to speed up
XidInMVCCSnapshot(), which is the reason only unsigned 32-bit integer
arrays are optimized. Other types are left for future work, as is the
extension of this technique to non-x86 platforms.

Nathan Bossart

Reviewed by: Andres Freund, Bharath Rupireddy, Masahiko Sawada
Discussion: https://postgr.es/m/20220713170950.GA3116318%40nathanxps13
2022-08-10 10:48:29 +07:00
Tom Lane 71cac850d0 Stabilize output of new regression test.
Per buildfarm, the output order of \dx+ isn't consistent across
locales.  Apply NO_LOCALE to force C locale.  There might be a
more localized way, but I'm not seeing it offhand, and anyway
there is nothing in this test module that particularly cares
about locales.

Security: CVE-2022-2625
2022-08-08 12:16:01 -04:00
Tom Lane b9b21acc76 In extensions, don't replace objects not belonging to the extension.
Previously, if an extension script did CREATE OR REPLACE and there was
an existing object not belonging to the extension, it would overwrite
the object and adopt it into the extension.  This is problematic, first
because the overwrite is probably unintentional, and second because we
didn't change the object's ownership.  Thus a hostile user could create
an object in advance of an expected CREATE EXTENSION command, and would
then have ownership rights on an extension object, which could be
modified for trojan-horse-type attacks.

Hence, forbid CREATE OR REPLACE of an existing object unless it already
belongs to the extension.  (Note that we've always forbidden replacing
an object that belongs to some other extension; only the behavior for
previously-free-standing objects changes here.)

For the same reason, also fail CREATE IF NOT EXISTS when there is
an existing object that doesn't belong to the extension.

Our thanks to Sven Klemm for reporting this problem.

Security: CVE-2022-2625
2022-08-08 11:12:31 -04:00
Alvaro Herrera afe58c8b74
Remove unportable use of timezone in recent test
Per buildfarm member snapper

Discussion: https://postgr.es/m/129951.1659812518@sss.pgh.pa.us
2022-08-07 10:19:40 +02:00
Alvaro Herrera e44dae07f9
BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking
That bit is unlogged and therefore it's wrong to consider it in WAL page
comparison.

Add a test that tickles the case, as branch testing technology allows.

This has been a problem ever since wal consistency checking was
introduced (commit a507b86900 for pg10), so backpatch to all supported
branches.

Author: 王海洋 (Haiyang Wang) <wanghaiyang.001@bytedance.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/CACciXAD2UvLMOhc4jX9VvOKt7DtYLr3OYRBhvOZ-jRxtzc_7Jg@mail.gmail.com
Discussion: https://postgr.es/m/CACciXADOfErX9Bx0nzE_SkdfXr6Bbpo5R=v_B6MUTEYW4ya+cg@mail.gmail.com
2022-08-05 18:00:17 +02:00
Tom Lane 1da0850f0e Reduce test runtime of src/test/modules/snapshot_too_old.
The sto_using_cursor and sto_using_select tests were coded to exercise
every permutation of their test steps, but AFAICS there is no value in
exercising more than one.  This matters because each permutation costs
about six seconds, thanks to the "pg_sleep(6)".  Perhaps we could
reduce that, but the useless permutations seem worth getting rid of
in any case.  (Note that sto_using_hash_index got it right already.)

While here, clean up some other sloppiness such as an unused table.

This doesn't make too much difference in interactive testing, since the
wasted time is typically masked by parallelization with other tests.
However, the buildfarm runs this as a serial step, which means we can
expect to shave ~40 seconds from every buildfarm run.  That makes it
worth back-patching.

Discussion: https://postgr.es/m/2515192.1659454702@sss.pgh.pa.us
2022-08-03 11:14:55 -04:00
Tom Lane 3451a57f77 Remove test_oat_hooks.c's nodetag_to_string().
In the short time this function has existed, it's already proven to be
a nontrivial maintenance burden, since it has to be updated whenever a
node tag is added or removed.  Although in principle we could now
automate that, I see little justification for having such functionality
here at all.  The function is only being applied to utility statements,
for which we already have infrastructure for obtaining string names.
Moreover, that infrastructure produces already-familiar-to-users names,
unlike nodetag_to_string().

So, remove this function and use the existing infrastructure instead.
That saves over a thousand lines of largely-unreachable code.

Back-patch to v15 where this code came in.  Although it seems unlikely
that v15's nodetag list will change anymore, we might as well keep the
two branches looking and acting alike; otherwise back-patching any
test-results changes in this area will be painful.

Discussion: https://postgr.es/m/843818.1659218928@sss.pgh.pa.us
2022-07-31 16:58:25 -04:00
Michael Paquier 43231423da Feed ObjectAddress to event triggers for ALTER TABLE ATTACH/DETACH
These flavors of ALTER TABLE were already shaped to report the
ObjectAddress of the partition attached or detached, but this data was
not added to what is collected for event triggers.  The tests of
test_ddl_deparse are updated to show the modification in the data
reported.

Author: Hou Zhijie
Reviewed-by: Álvaro Herrera, Amit Kapila, Hayato Kuroda, Michael Paquier
Discussion: https://postgr.es/m/OS0PR01MB571626984BD099DADF53F38394899@OS0PR01MB5716.jpnprd01.prod.outlook.com
2022-07-31 13:04:43 +09:00
Michael Paquier 07ff701dbd Expand tests of test_ddl_deparse/ for ALTER TABLE
This module is expanded to track the description of the objects changed
in the subcommands of ALTER TABLE by reworking the function
get_altertable_subcmdtypes() (now named get_altertable_subcmdinfo) used
in the event trigger of the test.  It now returns a set of rows made of
(subcommand type, object description) instead of a text array with only
the information about the subcommand type.

The tests have been lacking a lot of the subcommands added to
AlterTableType over the years.  All the missing subcommands are added,
and the code is now structured so as the addition of a new subcommand
is detected by removing the default clause used in the switch for the
subcommand types.

The coverage of the module is increased from roughly 30% to 50%.  More
could be done but this is already a nice improvement.

Author: Michael Paquier, Hou Zhijie
Reviewed-by: Álvaro Herrera, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/OS0PR01MB571626984BD099DADF53F38394899@OS0PR01MB5716.jpnprd01.prod.outlook.com
2022-07-31 11:48:14 +09:00
Tom Lane 13d8388151 Fix missed corner cases for grantable permissions on GUCs.
We allow users to set the values of not-yet-loaded extension GUCs,
remembering those values in "placeholder" GUC entries.  When/if
the extension is loaded later in the session, we need to verify that
the user had permissions to set the GUC.  That was done correctly
before commit a0ffa885e, but as of that commit, we'd check the
permissions of the active role when the LOAD happens, not the role
that had set the value.  (This'd be a security bug if it had made it
into a released version.)

In principle this is simple enough to fix: we just need to remember
the exact role OID that set each GUC value, and use that not
GetUserID() when verifying permissions.  Maintaining that data in
the guc.c data structures is slightly tedious, but fortunately it's
all basically just copy-n-paste of the logic for tracking the
GucSource of each setting, as we were already doing.

Another oversight is that validate_option_array_item() hadn't
been taught to check for granted GUC privileges.  This appears
to manifest only in that ALTER ROLE/DATABASE RESET ALL will
fail to reset settings that the user should be allowed to reset.

Patch by myself and Nathan Bossart, per report from Nathan Bossart.
Back-patch to v15 where the faulty code came in.

Discussion: https://postgr.es/m/20220706224727.GA2158260@nathanxps13
2022-07-19 17:21:55 -04:00
Andres Freund 8cf64d35ea Mark all symbols exported from extension libraries PGDLLEXPORT.
This is in preparation for defaulting to -fvisibility=hidden in extensions,
instead of relying on all symbols in extensions to be exported.

This should have been committed before 089480c077, but something in my commit
scripts went wrong.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
2022-07-17 18:50:14 -07:00
Andres Freund fd4bad1655 Remove now superfluous declarations of dlsym()ed symbols.
The prior commit declared them centrally.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
2022-07-17 17:29:32 -07:00
Peter Eisentraut 9fd45870c1 Replace many MemSet calls with struct initialization
This replaces all MemSet() calls with struct initialization where that
is easily and obviously possible.  (For example, some cases have to
worry about padding bits, so I left those.)

(The same could be done with appropriate memset() calls, but this
patch is part of an effort to phase out MemSet(), so it doesn't touch
memset() calls.)

Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/9847b13c-b785-f4e2-75c3-12ec77a3b05c@enterprisedb.com
2022-07-16 08:50:49 +02:00
Alvaro Herrera f16180216d
Fix flag tests in src/test/modules/test_oat_hooks
In what must have been a copy'n paste mistake, all the flag tests use
the same flag rather than a different flag each.  The bug is not
suprising, considering that it's dead code; add a minimal, testimonial
line to cover it.

This is all pretty inconsequential, because this is just example code,
but it had better be correct.

Discussion: https://postgr.es/m/20220712152059.fwli2majwgzdmh4r@alvherre.pgsql
2022-07-13 12:58:56 +02:00
Alvaro Herrera 5ca0fe5c8a
Add copy/equal support for XID lists
Commit f10a025cfe added support for List to store Xids, but didn't
handle the new type in all cases.  Add some obviously necessary pieces.
As far as I am aware, this is all dead code as far as core code is
concerned, but it seems unacceptable not to have it in case third-party
code wants to rely on this type of list.  (Some parts of the List API
remain unimplemented, but that can be fixed as and when needed -- see
lack of list_intersection_oid, list_deduplicate_int as precedents.)

Discussion: https://postgr.es/m/20220708164534.nbejhgt4ajz35p65@alvherre.pgsql
2022-07-12 16:11:04 +02:00
Michael Paquier a6434b9515 Fix two portability issues with the tests of test_oat_hooks
This addresses two issues in the tests of test_oat_hooks:
- The role regress_test_user was being left behind, preventing the test
to succeed on repeated runs.  It makes sense to leave some objects
behind to have more coverage for pg_upgrade (as does test_pg_dump), but
the role dropped here does not own any objects so there is no reason to
keep it.
- GRANT SET ON PARAMETER is issued, creating an entry in
pg_parameter_acl without cleaning up the entry created.  This causes
an overlap with unsafe_tests as both use work_mem, making the latter
fail.  This commit adds an extra REVOKE SET ON PARAMETER to clean the
contents of pg_parameter_acl, switching to maintenance_work_mem rather
than work_mem to avoid an overlap between both tests.

The tests of test_oat_hooks cannot use installcheck yet as these are
proving to be unstable with caching and the namespace search hooks, so
the issues fixed here cannot be reached yet, but they would be once the
hook issue is addressed and installcheck is allowed again in
test_oat_hooks.

Discussion: https://postgr.es/m/YrpVkADAY0knF6vM@paquier.xyz
Backpatch-through: 15
2022-07-11 12:47:52 +09:00
Tom Lane 8c73c11a0d Mark Scan as an abstract node type, too.
On further review, this one is never instantiated either.
2022-07-09 13:58:06 -04:00
Tom Lane b4f79d278f Mark PlanState as an abstract node type.
In the same vein as commit 251154beb, make it clear that we never
instantiate PlanState.

Also mark MemoryContextData as abstract.  This has no effect right now,
since memnodes.h isn't one of the files fed to gen_node_support.pl.
But it seems like good documentation and future-proofing.
2022-07-09 13:35:37 -04:00
Alexander Korotkov e57519a463 Add missing inequality searches to rbtree
PostgreSQL contains the implementation of the red-black tree.  The red-black
tree is the ordered data structure, and one of its advantages is the ability
to do inequality searches.  This commit adds rbt_find_less() and
rbt_find_great() functions implementing these searches.  While these searches
aren't yet used in the core code, they might be useful for extensions.

Discussion: https://postgr.es/m/CAGRrpzYE8-7GCoaPjOiL9T_HY605MRax-2jgTtLq236uksZ1Sw%40mail.gmail.com
Author: Steve Chavez, Alexander Korotkov
Reviewed-by: Alexander Korotkov
2022-07-08 22:00:03 +03:00
Peter Eisentraut 251154bebe Remove T_Join and T_Plan
These are abstract node types that don't need to have a node tag
defined.

Discussion: https://www.postgresql.org/message-id/2592455.1657140387%40sss.pgh.pa.us
2022-07-08 10:40:44 +02:00
Alvaro Herrera 054325c5ee
libpq: Improve idle state handling in pipeline mode
We were going into IDLE state too soon when executing queries via
PQsendQuery in pipeline mode, causing several scenarios to misbehave in
different ways -- most notably, as reported by Daniele Varrazzo, that a
warning message is produced by libpq:
  message type 0x33 arrived from server while idle
But it is also possible, if queries are sent and results consumed not in
lockstep, for the expected mediating NULL result values from PQgetResult
to be lost (a problem which has not been reported, but which is more
serious).

Fix this by introducing two new concepts: one is a command queue element
PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server
response to the Close message that is sent by PQsendQuery.  Because the
application is not expecting any PGresult from this, the mechanism to
consume it is a bit hackish.

The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE
state for libpq's state machine to differentiate "really idle" from
merely "the idle state that occurs in between reading results from the
server for elements in the pipeline".  This makes libpq not go fully
IDLE when the libpq command queue contains entries; in normal cases, we
only go IDLE once at the end of the pipeline, when the server response
to the final SYNC message is received.  (However, there are corner cases
it doesn't fix, such as terminating the query sequence by
PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is
what requires PGQUERY_CLOSE bit above.)

This last bit helps make the libpq state machine clearer; in particular
we can get rid of an ugly hack in pqParseInput3 to avoid considering
IDLE as such when the command queue contains entries.

A new test mode is added to libpq_pipeline.c to tickle some related
problematic cases.

Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com>
Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
Peter Eisentraut d746021de1 Add construct_array_builtin, deconstruct_array_builtin
There were many calls to construct_array() and deconstruct_array() for
built-in types, for example, when dealing with system catalog columns.
These all hardcoded the type attributes necessary to pass to these
functions.

To simplify this a bit, add construct_array_builtin(),
deconstruct_array_builtin() as wrappers that centralize this hardcoded
knowledge.  This simplifies many call sites and reduces the amount of
hardcoded stuff that is spread around.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/2914356f-9e5f-8c59-2995-5997fc48bcba%40enterprisedb.com
2022-07-01 11:23:15 +02:00
Tomas Vondra e3fcca0d0d Revert changes in HOT handling of BRIN indexes
This reverts commits 5753d4ee32 and fe60b67250 that modified HOT to
ignore BRIN indexes. The commit message for 5753d4ee32 claims that:

    When determining whether an index update may be skipped by using
    HOT, we can ignore attributes indexed only by BRIN indexes. There
    are no index pointers to individual tuples in BRIN, and the page
    range summary will be updated anyway as it relies on visibility
    info.

This is partially incorrect - it's true BRIN indexes don't point to
individual tuples, so HOT chains are not an issue, but the visibitlity
info is not sufficient to keep the index up to date. This can easily
result in corrupted indexes, as demonstrated in the hackers thread.

This does not mean relaxing the HOT restrictions for BRIN is a lost
cause, but it needs to handle the two aspects (allowing HOT chains and
updating the page range summaries) as separate. But that requires a
major changes, and it's too late for that in the current dev cycle.

Reported-by: Tomas Vondra
Discussion: https://postgr.es/m/05ebcb44-f383-86e3-4f31-0a97a55634cf@enterprisedb.com
2022-06-16 15:02:49 +02:00
Tom Lane 5e5fa32335 Remove portability hazard in unsafe_tests/sql/guc_privs.sql.
This new-in-v15 test case assumed it could set max_stack_depth as high
as 2MB.  You might think that'd be true on any modern platform but
you'd be wrong, as I found out while experimenting with NetBSD/hppa.

This test is about privileges not platform capabilities, so there seems
no need to use any value greater than the 100kB setting already used
in a couple of places in the core regression tests.  There's certainly
no call to expect people to raise their platform's default ulimit just
to run this test.
2022-05-20 13:42:02 -04:00
Alvaro Herrera 6029861916
Fix DDL deparse of CREATE OPERATOR CLASS
When an implicit operator family is created, it wasn't getting reported.
Make it do so.

This has always been missing.  Backpatch to 10.

Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reported-by: Leslie LEMAIRE <leslie.lemaire@developpement-durable.gouv.fr>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Michael Paquiër <michael@paquier.xyz>
Discussion: https://postgr.es/m/f74d69e151b22171e8829551b1159e77@developpement-durable.gouv.fr
2022-05-20 18:52:55 +02:00
Tom Lane 23e7b38bfe Pre-beta mechanical code beautification.
Run pgindent, pgperltidy, and reformat-dat-files.
I manually fixed a couple of comments that pgindent uglified.
2022-05-12 15:17:30 -04:00
Robert Haas ab02d702ef Remove non-functional code for unloading loadable modules.
The code for unloading a library has been commented-out for over 12
years, ever since commit 602a9ef5a7, and we're
no closer to supporting it now than we were back then.

Nathan Bossart, reviewed by Michael Paquier and by me.

Discussion: http://postgr.es/m/Ynsc9bRL1caUSBSE@paquier.xyz
2022-05-11 15:30:30 -04:00
Alvaro Herrera 24d2b2680a
Remove extraneous blank lines before block-closing braces
These are useless and distracting.  We wouldn't have written the code
with them to begin with, so there's no reason to keep them.

Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20220411020336.GB26620@telsasoft.com
Discussion: https://postgr.es/m/attachment/133167/0016-Extraneous-blank-lines.patch
2022-04-13 19:16:02 +02:00
Tom Lane 2f4d0d6799 Extend plsample example to include a trigger handler.
Mark Wong and Konstantina Skovola, reviewed by Chapman Flack

Discussion: https://postgr.es/m/Yd8Cz22eHi80XS30@workstation-mark-wong
2022-04-07 18:26:20 -04:00
Andres Freund 5891c7a8ed pgstat: store statistics in shared memory.
Previously the statistics collector received statistics updates via UDP and
shared statistics data by writing them out to temporary files regularly. These
files can reach tens of megabytes and are written out up to twice a
second. This has repeatedly prevented us from adding additional useful
statistics.

Now statistics are stored in shared memory. Statistics for variable-numbered
objects are stored in a dshash hashtable (backed by dynamic shared
memory). Fixed-numbered stats are stored in plain shared memory.

The header for pgstat.c contains an overview of the architecture.

The stats collector is not needed anymore, remove it.

By utilizing the transactional statistics drop infrastructure introduced in a
prior commit statistics entries cannot "leak" anymore. Previously leaked
statistics were dropped by pgstat_vacuum_stat(), called from [auto-]vacuum. On
systems with many small relations pgstat_vacuum_stat() could be quite
expensive.

Now that replicas drop statistics entries for dropped objects, it is not
necessary anymore to reset stats when starting from a cleanly shut down
replica.

Subsequent commits will perform some further code cleanup, adapt docs and add
tests.

Bumps PGSTAT_FILE_FORMAT_ID.

Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Thomas Munro <thomas.munro@gmail.com>
Reviewed-By: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-By: "David G. Johnston" <david.g.johnston@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@2ndquadrant.com> (in a much earlier version)
Reviewed-By: Arthur Zakirov <a.zakirov@postgrespro.ru> (in a much earlier version)
Reviewed-By: Antonin Houska <ah@cybertec.at> (in a much earlier version)
Discussion: https://postgr.es/m/20220303021600.hs34ghqcw6zcokdh@alap3.anarazel.de
Discussion: https://postgr.es/m/20220308205351.2xcn6k4x5yivcxyd@alap3.anarazel.de
Discussion: https://postgr.es/m/20210319235115.y3wz7hpnnrshdyv6@alap3.anarazel.de
2022-04-06 21:29:46 -07:00
Tom Lane a0ffa885e4 Allow granting SET and ALTER SYSTEM privileges on GUC parameters.
This patch allows "PGC_SUSET" parameters to be set by non-superusers
if they have been explicitly granted the privilege to do so.
The privilege to perform ALTER SYSTEM SET/RESET on a specific parameter
can also be granted.
Such privileges are cluster-wide, not per database.  They are tracked
in a new shared catalog, pg_parameter_acl.

Granting and revoking these new privileges works as one would expect.
One caveat is that PGC_USERSET GUCs are unaffected by the SET privilege
--- one could wish that those were handled by a revocable grant to
PUBLIC, but they are not, because we couldn't make it robust enough
for GUCs defined by extensions.

Mark Dilger, reviewed at various times by Andrew Dunstan, Robert Haas,
Joshua Brindle, and myself

Discussion: https://postgr.es/m/3D691E20-C1D5-4B80-8BA5-6BEB63AF3029@enterprisedb.com
2022-04-06 13:24:33 -04:00
Joe Conway 6198420ad8 Use has_privs_for_roles for predefined role checks
Generally if a role is granted membership to another role with NOINHERIT
they must use SET ROLE to access the privileges of that role, however
with predefined roles the membership and privilege is conflated. Fix that
by replacing is_member_of_role with has_privs_for_role for predefined
roles. Patch does not remove is_member_of_role from acl.h, but it does
add a warning not to use that function for privilege checking. Not
backpatched based on hackers list discussion.

Author: Joshua Brindle
Reviewed-by: Stephen Frost, Nathan Bossart, Joe Conway
Discussion: https://postgr.es/m/flat/CAGB+Vh4Zv_TvKt2tv3QNS6tUM_F_9icmuj0zjywwcgVi4PAhFA@mail.gmail.com
2022-03-28 15:10:04 -04:00