Commit Graph

1791 Commits

Author SHA1 Message Date
Andres Freund e6927270cd meson: Add initial version of meson based build system
Autoconf is showing its age, fewer and fewer contributors know how to wrangle
it. Recursive make has a lot of hard to resolve dependency issues and slow
incremental rebuilds. Our home-grown MSVC build system is hard to maintain for
developers not using Windows and runs tests serially. While these and other
issues could individually be addressed with incremental improvements, together
they seem best addressed by moving to a more modern build system.

After evaluating different build system choices, we chose to use meson, to a
good degree based on the adoption by other open source projects.

We decided that it's more realistic to commit a relatively early version of
the new build system and mature it in tree.

This commit adds an initial version of a meson based build system. It supports
building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD,
Solaris and Windows (however only gcc is supported on aix, solaris). For
Windows/MSVC postgres can now be built with ninja (faster, particularly for
incremental builds) and msbuild (supporting the visual studio GUI, but
building slower).

Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM
bitcode generation, documentation adjustments) are done in subsequent commits
requiring further review. Other aspects (e.g. not installing test-only
extensions) are not yet addressed.

When building on Windows with msbuild, builds are slower when using a visual
studio version older than 2019, because those versions do not support
MultiToolTask, required by meson for intra-target parallelism.

The plan is to remove the MSVC specific build system in src/tools/msvc soon
after reaching feature parity. However, we're not planning to remove the
autoconf/make build system in the near future. Likely we're going to keep at
least the parts required for PGXS to keep working around until all supported
versions build with meson.

Some initial help for postgres developers is at
https://wiki.postgresql.org/wiki/Meson

With contributions from Thomas Munro, John Naylor, Stone Tickle and others.

Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Author: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de
2022-09-21 22:37:17 -07:00
Peter Geoghegan a601366a46 Harmonize more parameter names in bulk.
Make sure that function declarations use names that exactly match the
corresponding names from function definitions in optimizer, parser,
utility, libpq, and "commands" code, as well as in remaining library
code.  Do the same for all code related to frontend programs (with the
exception of pg_dump/pg_dumpall related code).

Like other recent commits that cleaned up function parameter names, this
commit was written with help from clang-tidy.  Later commits will handle
ecpg and pg_dump/pg_dumpall.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAH2-WznJt9CMM9KJTMjJh_zbL5hD9oX44qdJ4aqZtjFi-zA3Tg@mail.gmail.com
2022-09-20 13:09:30 -07:00
Tom Lane bfd6b3bc4e Improve plpgsql's ability to handle arguments declared as RECORD.
Treat arguments declared as RECORD as if that were a polymorphic type
(which it is, sort of), in that we substitute the actual argument type
while forming the function cache lookup key.  This allows the specific
composite type to be known in some cases where it was not before,
at the cost of making a separate function cache entry for each named
composite type that's passed to the function during a session.  The
particular symptom discussed in bug #17610 could be solved in other
more-efficient ways, but only at the cost of considerable development
work, and there are other cases where we'd still fail without this.

Per bug #17610 from Martin Jurča.  Back-patch to v11 where we first
allowed plpgsql functions to be declared as taking type RECORD.

Discussion: https://postgr.es/m/17610-fb1eef75bf6c2364@postgresql.org
2022-09-16 13:23:01 -04:00
John Naylor 4c1532763a Bump minimum Perl version to 5.14
The oldest vendor-shipped Perl in the buildfarm is 5.14.2, which is
the last version that Debian Wheezy shipped. That OS is EOL, but we
keep it running because there is no other convenient way to test certain
non-mainstream 32-bit platforms. There is no bugfix in the 5.14.2 release
that is required, and yet it's also not the latest minor release --
that would be 5.14.4. To clarify the situation, we have thus arranged the
buildfarm to test 5.14.0. That allows configure scripts and documentation
to state 5.14 without fine print.

The MSVC build didn't check the version, since our previous minimum 5.8.3
was considered too old to check for on Windows. We will need a check for
Windows sometime during the v16 cycle, but that could be rendered moot
by the impending Meson conversion, so it seems safe to just document
the requirement for now.

Reviewed by Tom Lane
Discussion: https://www.postgresql.org/message-id/20220902181553.ev4pgzhubhdkguuv@awork3.anarazel.de
2022-09-14 12:37:04 +07:00
John Naylor b086a47a27 Bump minimum version of Bison to 2.3
Since the retirement of some older buildfarm members, the oldest Bison
that gets regular testing is 2.3. MacOS ships that version, and will
continue doing so for the forseeable future because of Apple's policy
regarding GPLv3. While Mac users could use a package manager to install
a newer version, there is no compelling reason to force them do so at
this time.

Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/1097762.1662145681@sss.pgh.pa.us
2022-09-09 12:31:41 +07:00
Peter Eisentraut 6bcda4a721 Fix incorrect uses of Datum conversion macros
Since these macros just cast whatever you give them to the designated
output type, and many normal uses also cast the output type further, a
number of incorrect uses go undiscovered.  The fixes in this patch
have been discovered by changing these macros to inline functions,
which is the subject of a future patch.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/8528fb7e-0aa2-6b54-85fb-0c0886dbd6ed%40enterprisedb.com
2022-09-05 13:30:44 +02:00
Peter Eisentraut 1c3aa54502 Fix PL/Perl build on Cygwin
This was broken by b4e936859d.  The
reason why this fixes it are not entirely clear, but it seemed the
best way to get it working again.

Discussion: https://www.postgresql.org/message-id/flat/8c4fcb72-2574-ff7c-4c25-1f032d4a2a57%40enterprisedb.com
2022-09-02 17:56:14 +02:00
Peter Eisentraut 7d5852ca83 plpython: Don't create pgxsdir subdirectory in installdir target
As of db23464715, we don't install
anything there anymore from plpython, so we don't need to create the
installation directory anymore.
2022-08-31 07:42:01 +02:00
John Naylor 4eec2e03c3 Be more careful to avoid including system headers after perl.h
Commit 121d2d3d70 included simd.h into pg_wchar.h. This caused a problem
on Windows, since Perl has "#define free" (referring to globals), which
breaks the Windows' header. To fix, move the static inline function
definitions from plperl_helpers.h, into plperl.h, where we already
document the necessary inclusion order. Since those functions were the
only reason for the existence of plperl_helpers.h, remove it.

First reported by Justin Pryzby

Diagnosis and review by Andres Freund, patch by myself per suggestion
from Tom Lane

Discussion: https://www.postgresql.org/message-id/20220826115546.GE2342%40telsasoft.com
2022-08-27 14:45:18 +07:00
Alvaro Herrera 1caf915ff3
Add PGDLLEXPORTS to some plpgsql function declarations
After -fvisibility=hidden was added by 089480c077, plpgsql_check no
longer works; this quick hack fixes it.  It would be better to
restructure the plpgsql.h header so that this doesn't look as random,
but we can leave that for another day.

Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRAFxc3-SHMD3URU09JZXEKY3W-RwXKp8xPEnEq8rrka7w@mail.gmail.com
2022-07-20 10:24:50 +02:00
Tom Lane 13d8388151 Fix missed corner cases for grantable permissions on GUCs.
We allow users to set the values of not-yet-loaded extension GUCs,
remembering those values in "placeholder" GUC entries.  When/if
the extension is loaded later in the session, we need to verify that
the user had permissions to set the GUC.  That was done correctly
before commit a0ffa885e, but as of that commit, we'd check the
permissions of the active role when the LOAD happens, not the role
that had set the value.  (This'd be a security bug if it had made it
into a released version.)

In principle this is simple enough to fix: we just need to remember
the exact role OID that set each GUC value, and use that not
GetUserID() when verifying permissions.  Maintaining that data in
the guc.c data structures is slightly tedious, but fortunately it's
all basically just copy-n-paste of the logic for tracking the
GucSource of each setting, as we were already doing.

Another oversight is that validate_option_array_item() hadn't
been taught to check for granted GUC privileges.  This appears
to manifest only in that ALTER ROLE/DATABASE RESET ALL will
fail to reset settings that the user should be allowed to reset.

Patch by myself and Nathan Bossart, per report from Nathan Bossart.
Back-patch to v15 where the faulty code came in.

Discussion: https://postgr.es/m/20220706224727.GA2158260@nathanxps13
2022-07-19 17:21:55 -04:00
Andres Freund 8cf64d35ea Mark all symbols exported from extension libraries PGDLLEXPORT.
This is in preparation for defaulting to -fvisibility=hidden in extensions,
instead of relying on all symbols in extensions to be exported.

This should have been committed before 089480c077, but something in my commit
scripts went wrong.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
2022-07-17 18:50:14 -07:00
Andres Freund fd4bad1655 Remove now superfluous declarations of dlsym()ed symbols.
The prior commit declared them centrally.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
2022-07-17 17:29:32 -07:00
Tom Lane 54ba2f0623 Revert "Use wildcards instead of manually-maintained file lists in */nls.mk."
This reverts commit 617d691412.
While I still think the basic idea is attractive, we need to sort
out what happens with built .c files, and there also seem to be
VPATH issues.
2022-07-13 14:29:10 -04:00
Tom Lane 617d691412 Use wildcards instead of manually-maintained file lists in */nls.mk.
The backend already used a mechanically-generated list of *.c files,
but everywhere else we had a manually-written-out list of files in
which to seek translatable messages.  Commit b0a55e432 contains the
latest in a long line of failures to update those lists.  Rather than
manually fix its oversight, let's change to using "$(wildcard *.c)"
in all these nls.mk files.

Many of these files also have manual references to some *.c files
in other directories, most often src/common/.  Perhaps we should try
to improve that situation too; but it's a bit less clear how, so for
now just fix the local file references.

Kyotaro Horiguchi and Tom Lane

Discussion: https://postgr.es/m/20220713.160853.453362706160476128.horikyota.ntt@gmail.com
2022-07-13 12:56:42 -04:00
Peter Eisentraut 503e3833ef Remove useless assertions
We don't need Assert(IsA(foo, String)) right before running
strVal(foo), since strVal() already does the assertion internally (via
castNode()).
2022-07-13 11:43:40 +02:00
Peter Eisentraut 88dad06b47 NLS: Put list of available languages into LINGUAS files
This moves the list of available languages from nls.mk into a separate
file called po/LINGUAS.  Advantages:

- It keeps the parts notionally managed by programmers (nls.mk)
  separate from the parts notionally managed by translators (LINGUAS).

- It's the standard practice recommended by the Gettext manual
  nowadays.

- The Meson build system also supports this layout (and of course
  doesn't know anything about our custom nls.mk), so this would enable
  sharing the list of languages between the two build systems.

(The MSVC build system currently finds all po files by globbing, so it
is not affected by this change.)

Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/557a9f5c-e871-edc7-2f58-a4140fb65b7b@enterprisedb.com
2022-07-13 08:19:17 +02:00
Thomas Munro 9db300ce6e Remove HP-UX port.
HP-UX hardware is no longer produced, build farm coverage recently
ended, and there are no known active maintainers targeting this OS.
Since there is a major rewrite of the build system in the pipeline for
PostgreSQL 16, and that requires development, testing and maintainance
for each OS and tool chain, it seems like a good time to drop support
for:

 * HP-UX, the operating system.
 * HP aCC, the HP-UX native compiler.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 14:05:05 +12:00
Peter Eisentraut d28e26bbe2 PL/Python: Update guide to alternative expected files
plpython_unicode_3.out was already removed a long time ago, so it
being listed here was very out of date.

plpython_types_3.out was removed with the Python 2 removal.
2022-06-27 12:53:05 +02:00
Peter Eisentraut 41a2b56ae8 PL/Tcl: Don't link with -lc explicitly
It has been reported that PL/Tcl built on macOS with GCC >=11 crashes.
The reason is that there is a hash_search() function in the operating
system's libraries, and that ends up being called instead of the one
in postgres.  This has something to do with how the linker resolves
references between the various possibilities it has been given, and
somehow something changed that it is now picking that one in this
configuration.

We found that removing the -lc from the link command line fixes this
problem.  The -lc was introduced a long time ago in commit
e3909672f1, and we think the reasons
might be obsolete, so we decided that we'll try to just remove it and
see if any problems arise.

Discussion: https://www.postgresql.org/message-id/flat/a78c847a-4f79-9286-be99-e819e9e4139e%40enterprisedb.com
2022-06-23 09:36:12 +02:00
Tom Lane eb39610f82 Fix pl/perl test case so it will still work under Perl 5.36.
Perl 5.36 has reclassified the warning condition that this test
case used, so that the expected error fails to appear.  Tweak
the test so it instead exercises a case that's handled the same
way in all Perl versions of interest.

This appears to meet our standards for back-patching into
out-of-support branches: it changes no user-visible behavior
but enables testing of old branches with newer tools.
Hence, back-patch as far as 9.2.

Dagfinn Ilmari Mannsåker, per report from Jitka Plesníková.

Discussion: https://postgr.es/m/564579.1654093326@sss.pgh.pa.us
2022-06-01 16:15:47 -04:00
Peter Eisentraut 6a8a7b1ccb Translation updates
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: dde45df385dab9032155c1f867b677d55695310c
2022-05-16 11:12:42 +02:00
Peter Eisentraut 30ed71e423 Indent C code in flex and bison files
In the style of pgindent, done semi-manually.

Discussion: https://www.postgresql.org/message-id/flat/7d062ecc-7444-23ec-a159-acd8adf9b586%40enterprisedb.com
2022-05-13 07:17:29 +02:00
Robert Haas ab02d702ef Remove non-functional code for unloading loadable modules.
The code for unloading a library has been commented-out for over 12
years, ever since commit 602a9ef5a7, and we're
no closer to supporting it now than we were back then.

Nathan Bossart, reviewed by Michael Paquier and by me.

Discussion: http://postgr.es/m/Ynsc9bRL1caUSBSE@paquier.xyz
2022-05-11 15:30:30 -04:00
Tom Lane ccd10a9bfa Tighten enforcement of variable CONSTANT markings in plpgsql.
I noticed that plpgsql would allow assignment of a new value to a
variable even when that variable is marked CONSTANT, if the variable
is used as an output parameter in CALL or is a refcursor variable
that OPEN assigns a new value to.  Fix these oversights.

In the CALL case, the check has to be done at runtime because we
cannot know at parse time which parameters are OUT parameters.
For OPEN, it seems best to likewise enforce at runtime because
then we needn't throw error if the variable has a nonnull value
(since OPEN will only try to overwrite a null value).

Although this is surely a bug fix, no back-patch: it seems unlikely
that anyone would thank us for breaking formerly-working code in
minor releases.

Discussion: https://postgr.es/m/214453.1651182729@sss.pgh.pa.us
2022-04-30 11:54:28 -04:00
Alvaro Herrera 24d2b2680a
Remove extraneous blank lines before block-closing braces
These are useless and distracting.  We wouldn't have written the code
with them to begin with, so there's no reason to keep them.

Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20220411020336.GB26620@telsasoft.com
Discussion: https://postgr.es/m/attachment/133167/0016-Extraneous-blank-lines.patch
2022-04-13 19:16:02 +02:00
Tom Lane fb691bbb4c Keep plpgsql.h C++-clean.
I forgot that "typeid" is a C++ keyword.  Per buildfarm.
2022-03-31 18:29:10 -04:00
Tom Lane 53ef6c40f1 Expose a few more PL/pgSQL functions to debugger plugins.
Add exec_assign_value, exec_eval_datum, and exec_cast_value
to the set of functions a PL/pgSQL debugger plugin can
conveniently call.  This allows more convenient manipulation
of the values of PL/pgSQL function variables.

Pavel Stehule, reviewed by Aleksander Alekseev and myself

Discussion: https://postgr.es/m/CAFj8pRD+dBPU0T-KrkP7ef6QNPDEsjYCejEsBe07NDq8TybOkA@mail.gmail.com
2022-03-31 17:05:47 -04:00
Alvaro Herrera 7103ebb7aa
Add support for MERGE SQL command
MERGE performs actions that modify rows in the target table using a
source table or query. MERGE provides a single SQL statement that can
conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise
require multiple PL statements.  For example,

MERGE INTO target AS t
USING source AS s
ON t.tid = s.sid
WHEN MATCHED AND t.balance > s.delta THEN
  UPDATE SET balance = t.balance - s.delta
WHEN MATCHED THEN
  DELETE
WHEN NOT MATCHED AND s.delta > 0 THEN
  INSERT VALUES (s.sid, s.delta)
WHEN NOT MATCHED THEN
  DO NOTHING;

MERGE works with regular tables, partitioned tables and inheritance
hierarchies, including column and row security enforcement, as well as
support for row and statement triggers and transition tables therein.

MERGE is optimized for OLTP and is parameterizable, though also useful
for large scale ETL/ELT. MERGE is not intended to be used in preference
to existing single SQL commands for INSERT, UPDATE or DELETE since there
is some overhead.  MERGE can be used from PL/pgSQL.

MERGE does not support targetting updatable views or foreign tables, and
RETURNING clauses are not allowed either.  These limitations are likely
fixable with sufficient effort.  Rewrite rules are also not supported,
but it's not clear that we'd want to support them.

Author: Pavan Deolasee <pavan.deolasee@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Amit Langote <amitlangote09@gmail.com>
Author: Simon Riggs <simon.riggs@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions)
Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions)
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Reviewed-by: Japin Li <japinli@hotmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com
Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:47:48 +02:00
Andres Freund 43e7787dd3 plpython: Restore alternative output for plpython_error test.
In db23464715 I removed the alternative output for plpython_error. Wrongly
so, because the output changed in Python 3.5, not Python 3.
2022-03-08 10:34:06 -08:00
Andres Freund 54c72eb5e5 plpython: add missing plpython.h include to plpy_plpymodule.h
The include was missing before 9b7e24a2cb, but starting with that commit the
missing include causes cpluspluscheck to fail because the use of
PyMODINIT_FUNC isn't incidentally protected by an ifdef anymore.

Discussion: https://postgr.es/m/20220308045916.7baapelbgftoqeop@alap3.anarazel.de
2022-03-08 09:47:34 -08:00
Andres Freund 9b7e24a2cb plpython: Code cleanup related to removal of Python 2 support.
Since 19252e8ec9 we reject Python 2 during build configuration. Now that the
dust on the buildfarm has settled, remove Python 2 specific code, including
the "Python 2/3 porting layer".

The code to detect conflicts between plpython using Python 2 and 3 is not
removed, in case somebody creates an out-of-tree version adding back support
for Python 2.

Reviewed-By: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211031184548.g4sxfe47n2kyi55r@alap3.anarazel.de
2022-03-07 18:30:28 -08:00
Andres Freund db23464715 plpython: Remove regression test infrastructure for Python 2.
Since 19252e8ec9 we reject Python 2 during build configuration. Now that the
dust on the buildfarm has settled, remove regression testing infrastructure
dealing with differing output between Python 2 / 3.

Reviewed-By: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211031184548.g4sxfe47n2kyi55r@alap3.anarazel.de
2022-03-07 18:20:51 -08:00
Andres Freund 76a29adee7 plpython: Remove plpythonu, plpython2u and associated transform extensions.
Since 19252e8ec9 we reject Python 2 during build configuration. Now that the
dust on the buildfarm has settled, remove extension variants specific to
Python 2.

Reviewed-By: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211031184548.g4sxfe47n2kyi55r@alap3.anarazel.de
2022-03-07 18:20:20 -08:00
Tom Lane 2e517818f4 Fix SPI's handling of errors during transaction commit.
SPI_commit previously left it up to the caller to recover from any error
occurring during commit.  Since that's complicated and requires use of
low-level xact.c facilities, it's not too surprising that no caller got
it right.  Let's move the responsibility for cleanup into spi.c.  Doing
that requires redefining SPI_commit as starting a new transaction, so
that it becomes equivalent to SPI_commit_and_chain except that you get
default transaction characteristics instead of preserving the prior
transaction's characteristics.  We can make this pretty transparent
API-wise by redefining SPI_start_transaction() as a no-op.  Callers
that expect to do something in between might be surprised, but
available evidence is that no callers do so.

Having made that API redefinition, we can fix this mess by having
SPI_commit[_and_chain] trap errors and start a new, clean transaction
before re-throwing the error.  Likewise for SPI_rollback[_and_chain].
Some cleanup is also needed in AtEOXact_SPI, which was nowhere near
smart enough to deal with SPI contexts nested inside a committing
context.

While plperl and pltcl need no changes beyond removing their now-useless
SPI_start_transaction() calls, plpython needs some more work because it
hadn't gotten the memo about catching commit/rollback errors in the
first place.  Such an error resulted in longjmp'ing out of the Python
interpreter, which leaks Python stack entries at present and is reported
to crash Python 3.11 altogether.  Add the missing logic to catch such
errors and convert them into Python exceptions.

We are probably going to have to back-patch this once Python 3.11 ships,
but it's a sufficiently basic change that I'm a bit nervous about doing
so immediately.  Let's let it bake awhile in HEAD first.

Peter Eisentraut and Tom Lane

Discussion: https://postgr.es/m/3375ffd8-d71c-2565-e348-a597d6e739e3@enterprisedb.com
Discussion: https://postgr.es/m/17416-ed8fe5d7213d6c25@postgresql.org
2022-02-28 12:45:36 -05:00
Tom Lane 638300fef5 Disallow execution of SPI functions during plperl function compilation.
Perl can be convinced to execute user-defined code during compilation
of a plperl function (or at least a plperlu function).  That's not
such a big problem as long as the activity is confined within the
Perl interpreter, and it's not clear we could do anything about that
anyway.  However, if such code tries to use plperl's SPI functions,
we have a bigger problem.  In the first place, those functions are
likely to crash because current_call_data->prodesc isn't set up yet.
In the second place, because it isn't set up, we lack critical info
such as whether the function is supposed to be read-only.  And in
the third place, this path allows code execution during function
validation, which is strongly discouraged because of the potential
for security exploits.  Hence, reject execution of the SPI functions
until compilation is finished.

While here, add check_spi_usage_allowed() calls to various functions
that hadn't gotten the memo about checking that.  I think that perhaps
plperl_sv_to_literal may have been intentionally omitted on the grounds
that it was safe at the time; but if so, the addition of transforms
functionality changed that.  The others are more recently added and
seem to be flat-out oversights.

Per report from Mark Murawski.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/9acdf918-7fff-4f40-f750-2ffa84f083d2@intellasoft.net
2022-02-25 17:40:43 -05:00
Michael Paquier e77216fcb0 Simplify more checks related to set-returning functions
This makes more consistent the SRF-related checks in the area of
PL/pgSQL, PL/Perl, PL/Tcl, pageinspect and some of the JSON worker
functions, making it easier to grep for the same error patterns through
the code, reducing a bit the translation work.

It is worth noting that each_worker_jsonb()/each_worker() in jsonfuncs.c
and pageinspect's brin_page_items() were doing a check on expectedDesc
that is not required as they fetch their tuple descriptor directly from
get_call_result_type().  This looks like a set of copy-paste errors that
have spread over the years.

This commit is a continuation of the changes begun in 07daca5, for any
remaining code paths on sight.  Like fcc2817, this makes the code more
consistent, easing the integration of a larger patch that will refactor
the way tuplestores are created and checked in a good portion of the
set-returning functions present in core.

I have worked my way through the changes of this patch by myself, and
Ranier has proposed the same changes in a different thread in parallel,
though there were some inconsistencies related in expectedDesc in what
was proposed by him.

Author: Michael Paquier, Ranier Vilela
Discussion: https://postgr.es/m/CAAKRu_azyd1Z3W_r7Ou4sorTjRCs+PxeHw1CWJeXKofkE6TuZg@mail.gmail.com
Discussion: https://postgr.es/m/CAEudQApm=AFuJjEHLBjBcJbxcw4pBMwg2sHwXyCXYcbBOj3hpg@mail.gmail.com
2022-02-24 16:54:59 +09:00
Tom Lane 88103567cb Disallow setting bogus GUCs within an extension's reserved namespace.
Commit 75d22069e tried to throw a warning for setting a custom GUC whose
prefix belongs to a previously-loaded extension, if there is no such GUC
defined by the extension.  But that caused unstable behavior with
parallel workers, because workers don't necessarily load extensions and
GUCs in the same order their leader did.  To make that work safely, we
have to completely disallow the case.  We now actually remove any such
GUCs at the time of initial extension load, and then throw an error not
just a warning if you try to add one later.  While this might create a
compatibility issue for a few people, the improvement in error-detection
capability seems worth it; it's hard to believe that there's any good
use-case for choosing such GUC names.

This also un-reverts 5609cc01c (Rename EmitWarningsOnPlaceholders() to
MarkGUCPrefixReserved()), since that function's old name is now even
more of a misnomer.

Florin Irion and Tom Lane

Discussion: https://postgr.es/m/1902182.1640711215@sss.pgh.pa.us
2022-02-21 14:10:43 -05:00
Andrew Dunstan ea09a75e1e
Use gendef instead of pexports for building windows .def files
Modern msys systems lack pexports but have gendef instead, so use that.

Discussion: https://postgr.es/m/3ccde7a9-e4f9-e194-30e0-0936e6ad68ba@dunslane.net

Backpatch to release 9.4 to enable building with perl on older branches.
Before that pexports is not used for plperl.
2022-02-10 13:44:05 -05:00
Tom Lane d5c2a91e54 Remove ppport.h's broken re-implementation of eval_pv().
Recent versions of Devel::PPPort try to redefine eval_pv() to
dodge a bug in pre-5.31 Perl versions.  Unfortunately the redefinition
fails on compilers that don't support statements nested within
expressions.  However, we aren't actually interested in this bug fix,
since we always call eval_pv() with croak_on_error = FALSE.
So, until there's an upstream fix for this breakage, just comment
out the macro to revert to the older behavior.

Per report from Wei Sun, as well as previous buildfarm failure
on pademelon (which I'd unfortunately not looked at carefully
enough to understand the cause).  Back-patch to all supported
versions, since we're using the same ppport.h in all.

Discussion: https://postgr.es/m/tencent_2EFCC8BA0107B6EC0F97179E019A8A43C806@qq.com
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pademelon&dt=2022-02-02%2001%3A22%3A58
2022-02-08 19:25:56 -05:00
Andres Freund c89f409749 plperl: windows: Use Perl_setlocale on 5.28+, fixing compile failure.
For older versions we need our own copy of perl's setlocale(), because it was
not exposed (why we need the setlocale in the first place is explained in
plperl_init_interp) . The copy stopped working in 5.28, as some of the used
macros are not public anymore.  But Perl_setlocale is available in 5.28, so
use that.

Author: Victor Wagner <vitus@wagner.pp.ru>
Reviewed-By: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/20200501134711.08750c5f@antares.wagner.home
Backpatch: all versions
2022-01-30 16:42:14 -08:00
Michael Paquier 410aa248e5 Fix various typos, grammar and code style in comments and docs
This fixes a set of issues that have accumulated over the past months
(or years) in various code areas.  Most fixes are related to some recent
additions, as of the development of v15.

Author: Justin Pryzby
Discussion: https://postgr.es/m/20220124030001.GQ23027@telsasoft.com
2022-01-25 09:40:04 +09:00
Bruce Momjian 27b77ecf9f Update copyright for 2022
Backpatch-through: 10
2022-01-07 19:04:57 -05:00
Tom Lane cab5b9ab2c Revert changes about warnings/errors for placeholders.
Revert commits 5609cc01c, 2ed8a8cc5, and 75d22069e until we have
a less broken idea of how this should work in parallel workers.
Per buildfarm.

Discussion: https://postgr.es/m/1640909.1640638123@sss.pgh.pa.us
2021-12-27 16:01:10 -05:00
Tom Lane 5609cc01c6 Rename EmitWarningsOnPlaceholders() to MarkGUCPrefixReserved().
This seems like a clearer name for what it does now.

Provide a compatibility macro so that extensions don't have to convert
to the new name right away.

Discussion: https://postgr.es/m/116024.1640111629@sss.pgh.pa.us
2021-12-27 14:39:08 -05:00
Tom Lane 1fada5d81e Add missing EmitWarningsOnPlaceholders() calls.
Extensions that define any custom GUCs should call
EmitWarningsOnPlaceholders after doing so, to help catch misspellings.
Many of our contrib modules hadn't gotten the memo on that, though.

Also add such calls to src/test/modules extensions that have GUCs.
While these aren't really user-facing, they should illustrate good
practice not faulty practice.

Shinya Kato

Discussion: https://postgr.es/m/524fa2c0a34f34b68fbfa90d0760d515@oss.nttdata.com
2021-12-21 12:12:24 -05:00
Tom Lane dc9c3b0ff2 Remove dynamic translation of regression test scripts, step 2.
"git mv" all the input/*.source and output/*.source files into
the corresponding sql/ and expected/ directories.  Then remove
the pg_regress and Makefile infrastructure associated with
dynamic translation.

Discussion: https://postgr.es/m/1655733.1639871614@sss.pgh.pa.us
2021-12-20 14:15:52 -05:00
Tom Lane d1029bb5a2 Remove dynamic translation of regression test scripts, step 1.
pg_regress has long had provisions for dynamically substituting path
names into regression test scripts and result files, but use of that
feature has always been a serious pain in the neck, mainly because
updating the result files requires tedious manual editing.  Let's
get rid of that in favor of passing down the paths in environment
variables.

In addition to being easier to maintain, this way is capable of
dealing with path names that require escaping at runtime, for example
paths containing single-quote marks.  (There are other stumbling
blocks in the way of actually building in a path that looks like
that, but removing this one seems like a good thing to do.)  The key
coding rule that makes that possible is to concatenate pieces of a
dynamically-variable string using psql's \set command, and then use
the :'variable' notation to quote and escape the string for the next
level of interpretation.

In hopes of making this change more transparent to "git blame",
I've split it into two steps.  This commit adds the necessary
pg_regress.c support and changes all the *.source files in-place
so that they no longer require any dynamic translation.  The next
commit will just "git mv" them into the regular sql/ and expected/
directories.

Discussion: https://postgr.es/m/1655733.1639871614@sss.pgh.pa.us
2021-12-20 14:06:15 -05:00
Peter Eisentraut 99e4d24a9d Remove unneeded Python includes
Inluding <compile.h> and <eval.h> has not been necessary since Python
2.4, since they are included via <Python.h>.  Morever, <eval.h> is
being removed in Python 3.11.  So remove these includes.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/84884.1637723223%40sss.pgh.pa.us
2021-11-25 14:19:22 +01:00
Tom Lane acb2d7d5d2 plpgsql: report proper line number for errors in variable initialization.
Previously, we pointed at the surrounding block's BEGIN keyword.
If there are multiple variables being initialized in a DECLARE section,
this isn't good enough: it can be quite confusing and unhelpful.
We do know where the variable's declaration started, so it just takes
a tiny bit more error-reporting infrastructure to use that.

Discussion: https://postgr.es/m/713975.1635530414@sss.pgh.pa.us
2021-10-31 12:43:47 -04:00