Commit Graph

902 Commits

Author SHA1 Message Date
Daniel Gustafsson a5cf808be5 Read include/exclude commands for dump/restore from file
When there is a need to filter multiple tables with include and/or exclude
options it's quite possible to run into the limitations of the commandline.
This adds a --filter=FILENAME feature to pg_dump, pg_dumpall and pg_restore
which is used to supply a file containing object exclude/include commands
which work just like their commandline counterparts. The format of the file
is one command per row like:

    <command> <object> <objectpattern>

<command> can be "include" or "exclude", <object> can be table_data, index
table_data_and_children, database, extension, foreign_data, function, table
schema, table_and_children or trigger.

This patch has gone through many revisions and design changes over a long
period of time, the list of reviewers reflect reviewers of some version of
the patch, not necessarily the final version.

Patch by Pavel Stehule with some additional hacking by me.

Author: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Erik Rijkers <er@xs4all.nl>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
2023-11-29 14:56:24 +01:00
Thomas Munro 15c9ac3629 Optimize pg_readv/pg_pwritev single vector case.
For the trivial case of iovcnt == 1, kernels are measurably slower at
dealing with the more complex arguments of preadv/pwritev than the
equivalent plain old pread/pwrite.  The overheads are worth it for
iovcnt > 1, but for 1 let's just redirect to the cheaper calls.  While
we could leave it to callers to worry about that, we already have to
have our own pg_ wrappers for portability reasons so it seems
reasonable to centralize this knowledge there (thanks to Heikki for this
suggestion).  Try to avoid function call overheads by making them
inlinable, which might also allow the compiler to avoid the branch in
some cases.  For systems that don't have preadv and pwritev (currently:
Windows and [closed] Solaris), we might as well pull the replacement
functions up into the static inline functions too.

Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CA+hUKGJkOiOCa+mag4BF+zHo7qo=o9CFheB8=g6uT5TUm2gkvA@mail.gmail.com
2023-11-29 17:19:25 +13:00
Tom Lane c82207a548 Use BIO_{get,set}_app_data instead of BIO_{get,set}_data.
We should have done it this way all along, but we accidentally got
away with using the wrong BIO field up until OpenSSL 3.2.  There,
the library's BIO routines that we rely on use the "data" field
for their own purposes, and our conflicting use causes assorted
weird behaviors up to and including core dumps when SSL connections
are attempted.  Switch to using the approved field for the purpose,
i.e. app_data.

While at it, remove our configure probes for BIO_get_data as well
as the fallback implementation.  BIO_{get,set}_app_data have been
there since long before any OpenSSL version that we still support,
even in the back branches.

Also, update src/test/ssl/t/001_ssltests.pl to allow for a minor
change in an error message spelling that evidently came in with 3.2.

Tristan Partin and Bo Andreson.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/CAN55FZ1eDDYsYaL7mv+oSLUij2h_u6hvD4Qmv-7PK7jkji0uyQ@mail.gmail.com
2023-11-28 12:34:03 -05:00
Peter Eisentraut 3849fe7c2b Replace Gen_dummy_probes.sed with Gen_dummy_probes.pl
To generate a dummy probes.h file when dtrace is not available, we had
two different scripts: A sed version, which is the original version,
and a Perl version, which was generated by s2p.  This split was
necessary because Perl was not a mandatory build dependency on Unix,
but sed was not guaranteed to be available on Windows.

(The Meson build system used the sed version even on Windows, which
was probably incorrect and probably would have had to be fixed before
elevating that build system from experimental status.)

As of 721856ff24, Perl is a required build dependency, so this split
is no longer necessary.  We can just use the Perl script in all build
environments and remove a whole bunch of infrastructure to keep the
two variants in sync.

The new Gen_dummy_probes.pl is not the version generated by s2p but a
new implementation written by hand by adapting the sed version to Perl
syntax.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/3fd0f1bc-4483-4ba9-8aa0-64765b052039@eisentraut.org
2023-11-14 10:27:10 +01:00
David Rowley 151ffcf6d8 Try again to fix the MSVC build
My last attempt in 39c959ef2 mistakenly conditionally added the missing
file based on some unrelated condition.

Reported-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGLovvAXim9Fytn=jxks9s=JhP5=8Oyy0cbxGG-ggALJtg@mail.gmail.com
2023-11-04 15:41:16 +13:00
David Rowley 39c959ef25 Add missing unicode_category.c to MSVC build scripts
Fixes MSVC build failure introduced by a02b37fc0
2023-11-03 20:12:36 +13:00
David Rowley 673a17e312 Add missing include dir and references to libpq for MSVC build
66d6086cb adjusted pg_regress to require this but forgot to adjust the
Visual Studio build script.
2023-10-25 15:37:37 +13:00
Nathan Bossart 83223f5f71 Fix MSVC build for changes to binaryheap.
After 5af0263afd, binaryheap.c needs to be listed in Mkvcbuild.pm.
Per buildfarm.
2023-09-18 12:46:57 -07:00
Nathan Bossart f4b54e1ed9 Introduce macros for protocol characters.
This commit introduces descriptively-named macros for the
identifiers used in wire protocol messages.  These new macros are
placed in a new header file so that they can be easily used by
third-party code.

Author: Dave Cramer
Reviewed-by: Alvaro Herrera, Tatsuo Ishii, Peter Smith, Robert Haas, Tom Lane, Peter Eisentraut, Michael Paquier
Discussion: https://postgr.es/m/CADK3HHKbBmK-PKf1bPNFoMC%2BoBt%2BpD9PH8h5nvmBQskEHm-Ehw%40mail.gmail.com
2023-08-22 19:16:12 -07:00
Michael Paquier 1e68e43d3f Add system view pg_wait_events
This new view, wrapped around a SRF, shows some information known about
wait events, as of:
- Name.
- Type (Activity, I/O, Extension, etc.).
- Description.

All the information retrieved comes from wait_event_names.txt, and the
description is the same as the documentation with filters applied to
remove any XML markups.  This view is useful when joined with
pg_stat_activity to get the description of a wait event reported.

Custom wait events for extensions are included in the view.

Original idea by Yves Colin.

Author: Bertrand Drouvot
Reviewed-by: Kyotaro Horiguchi, Masahiro Ikeda, Tom Lane, Michael
Paquier
Discussion: https://postgr.es/m/0e2ae164-dc89-03c3-cf7f-de86378053ac@gmail.com
2023-08-20 15:35:02 +09:00
John Naylor f25b18500a Update Solution.pm for new LoongArch CRC symbol
Oversight in 4d14ccd6a, per report from Amit Kapila
and Michael Paquier.

Discussion: https://postgr.es/m/CAA4eK1LsV3KuyUt8tzZDjPcUds1XfVVeW3Wpeju_59DtRV0%3DxQ%40mail.gmail.com
2023-08-10 18:37:46 +07:00
Thomas Munro 68a4b58eca Remove --disable-thread-safety and related code.
All supported computers have either POSIX or Windows threads, and we no
longer have any automated testing of --disable-thread-safety.  We define
a vestigial ENABLE_THREAD_SAFETY macro to 1 in ecpg_config.h in case it
is useful, but we no longer test it anywhere in PostgreSQL code, and
associated dead code paths are removed.

The Meson and perl-based Windows build scripts never had an equivalent
build option.

Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
2023-07-12 08:20:43 +12:00
Thomas Munro 8d9a9f034e All supported systems have locale_t.
locale_t is defined by POSIX.1-2008 and SUSv4, and available on all
targeted systems.  For Windows, win32_port.h redirects to a partial
implementation called _locale_t.  We can now remove a lot of
compile-time tests for HAVE_LOCALE_T, and associated comments and dead
code branches that were needed for older computers.

Since configure + MinGW builds didn't detect locale_t but now we assume
that all systems have it, further inconsistencies among the 3 Windows build
systems were revealed.  With this commit, we no longer define
HAVE_WCSTOMBS_L and HAVE_MBSTOWCS_L on any Windows build system, but
we have logic to deal with that so that replacements are available where
appropriate.

Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Tristan Partin <tristan@neon.tech>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CA%2BhUKGLg7_T2GKwZFAkEf0V7vbnur-NfCjZPKZb%3DZfAXSV1ORw%40mail.gmail.com
2023-07-09 11:55:18 +12:00
Michael Paquier fa88928470 Generate automatically code and documentation related to wait events
The documentation and the code is generated automatically from a new
file called wait_event_names.txt, formatted in sections dedicated to
each wait event class (Timeout, Lock, IO, etc.) with three tab-separated
fields:
- C symbol in enums
- Format in the system views
- Description in the docs

Using this approach has several advantages, as we have proved to be
rather bad in maintaining this area of the tree across the years:
- The order of each item in the documentation and the code, which should
be alphabetical, has become incorrect multiple times, and the script
generating the code and documentation has a few rules to enforce that,
making the maintenance a no-brainer.
- Some wait events were added to the code, but not documented, so this
cannot be missed now.
- The order of the tables for each wait event class is enforced in the
documentation (the input .txt file does so as well for clarity, though
this is not mandatory).
- Less code, shaving 1.2k lines from the tree, with 1/3 of the savings
coming from the code, the rest from the documentation.

The wait event types "Lock" and "LWLock" still have their own code path
for their code, hence only the documentation is created for them.  These
classes are listed with a special marker called WAIT_EVENT_DOCONLY in
the input file.

Adding a new wait event now requires only an update of
wait_event_names.txt, with "Lock" and "LWLock" treated as exceptions.

This commit has been tested with configure/Makefile, the CI and VPATH
build.  clean, distclean and maintainer-clean were working fine.

Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/77a86b3a-c4a8-5f5d-69b9-d70bbf2e9b98@gmail.com
2023-07-05 10:53:11 +09:00
Michael Paquier 8e278b6576 Remove support for OpenSSL 1.0.1
Here are some notes about this change:
- As X509_get_signature_nid() should always exist (OpenSSL and
LibreSSL), hence HAVE_X509_GET_SIGNATURE_NID is now gone.
- OPENSSL_API_COMPAT is bumped to 0x10002000L.
- One comment related to 1.0.1e introduced by 74242c2 is removed.

Upstream OpenSSL still provides long-term support for 1.0.2 in a closed
fashion, so removing it is out of scope for a few years, at least.

Reviewed-by: Jacob Champion, Daniel Gustafsson
Discussion: https://postgr.es/m/ZG3JNursG69dz1lr@paquier.xyz
2023-07-03 13:20:27 +09:00
Noah Misch 0c52437020 src/tools/msvc/clean.bat: Reconcile with PostgreSQL 16 work. 2023-06-12 07:40:39 -07:00
Noah Misch 6e723f6d7b src/tools/msvc: Move all.sym temporary file back to Debug/postgres.
Commit 70df2df1cc moved it to the
top_srcdir, where it caused "git status" noise.
2023-06-12 07:40:39 -07:00
Tom Lane 0245f8db36 Pre-beta mechanical code beautification.
Run pgindent, pgperltidy, and reformat-dat-files.

This set of diffs is a bit larger than typical.  We've updated to
pg_bsd_indent 2.1.2, which properly indents variable declarations that
have multi-line initialization expressions (the continuation lines are
now indented one tab stop).  We've also updated to perltidy version
20230309 and changed some of its settings, which reduces its desire to
add whitespace to lines to make assignments etc. line up.  Going
forward, that should make for fewer random-seeming changes to existing
code.

Discussion: https://postgr.es/m/20230428092545.qfb3y5wcu4cm75ur@alvherre.pgsql
2023-05-19 17:24:48 -04:00
Tom Lane 38358aab9a Update Solution.pm for HAVE_GSSAPI[_GSSAPI]_EXT_H, too.
Commit d48ac0070 was still a brick shy of a load.
Per buildfarm (via Andrew Dunstan).

Discussion: https://postgr.es/m/08dd6053-deee-52e9-954b-eb69232905c9@dunslane.net
2023-04-17 16:00:39 -04:00
Michael Paquier a923e21631 Fix detection of unseekable files for fseek() and ftello() with MSVC
Calling fseek() or ftello() on a handle to a non-seeking device such as
a pipe or a communications device is not supported.  Unfortunately,
MSVC's flavor of these routines, _fseeki64() and _ftelli64(), do not
return an error when given a pipe as handle.  Some of the logic of
pg_dump and restore relies on these routines to check if a handle is
seekable, causing failures when passing the contents of pg_dump to
pg_restore through a pipe, for example.

This commit introduces wrappers for fseeko() and ftello() on MSVC so as
any callers are able to properly detect the cases of non-seekable
handles.  This relies mainly on GetFileType(), sharing a bit of code
with the MSVC port for fstat().  The code in charge of getting a file
type is refactored into a new file called win32common.c, shared by
win32stat.c and the new win32fseek.c.  It includes the MSVC ports for
fseeko() and ftello().

Like 765f5df, this is backpatched down to 14, where the fstat()
implementation for MSVC is able to understand about files larger than
4GB in size.  Using a TAP test for that is proving to be tricky as
IPC::Run handles the pipes by itself, still I have been able to check
the fix manually.

Reported-by: Daniel Watzinger
Author: Juan José Santamaría Flecha, Michael Paquier
Discussion: https://postgr.es/m/CAC+AXB26a4EmxM2suXxPpJaGrqAdxracd7hskLg-zxtPB50h7A@mail.gmail.com
Backpatch-through: 14
2023-04-12 09:09:38 +09:00
Andrew Dunstan bbec50de16 Try to unbreak MSVC builds for pg_waldump
remedy an omission in commit 7d8219a444
2023-04-08 11:21:53 -04:00
Andrew Dunstan 0e9b271890 Try to unbreak MSVC builds for fuzzystrmatch
Commit a290378a37 neglrected to add a recipe for MSVC to build the
daitch_motokoff.h file.

Per buildfarm animal bowerbird.
2023-04-08 08:28:15 -04:00
Michael Paquier 36f40ce2dc libpq: Add sslcertmode option to control client certificates
The sslcertmode option controls whether the server is allowed and/or
required to request a certificate from the client.  There are three
modes:
- "allow" is the default and follows the current behavior, where a
configured client certificate is sent if the server requests one
(via one of its default locations or sslcert).  With the current
implementation, will happen whenever TLS is negotiated.
- "disable" causes the client to refuse to send a client certificate
even if sslcert is configured or if a client certificate is available in
one of its default locations.
- "require" causes the client to fail if a client certificate is never
sent and the server opens a connection anyway.  This doesn't add any
additional security, since there is no guarantee that the server is
validating the certificate correctly, but it may helpful to troubleshoot
more complicated TLS setups.

sslcertmode=require requires SSL_CTX_set_cert_cb(), available since
OpenSSL 1.0.2.  Note that LibreSSL does not include it.

Using a connection parameter different than require_auth has come up as
the simplest design because certificate authentication does not rely
directly on any of the AUTH_REQ_* codes, and one may want to require a
certificate to be sent in combination of a given authentication method,
like SCRAM-SHA-256.

TAP tests are added in src/test/ssl/, some of them relying on sslinfo to
check if a certificate has been set.  These are compatible across all
the versions of OpenSSL supported on HEAD (currently down to 1.0.1).

Author: Jacob Champion
Reviewed-by: Aleksander Alekseev, Peter Eisentraut, David G. Johnston,
Michael Paquier
Discussion: https://postgr.es/m/9e5a8ccddb8355ea9fa4b75a1e3a9edc88a70cd3.camel@vmware.com
2023-03-24 13:34:26 +09:00
Michael Paquier 98ae2c84a4 libpq: Remove code for SCM credential authentication
Support for SCM credential authentication has been removed in the
backend in 9.1, and libpq has kept some code to handle it for
compatibility.

Commit be4585b, that did the cleanup of the backend code, has done
so because the code was not really portable originally.  And, as there
are likely little chances that this is used these days, this removes the
remaining code from libpq.  An error will now be raised by libpq if
attempting to connect to a server that returns AUTH_REQ_SCM_CREDS,
instead.

References to SCM credential authentication are removed from the
protocol documentation.  This removes some meson and configure checks.

Author: Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/ZBLH8a4otfqgd6Kn@paquier.xyz
2023-03-17 10:52:26 +09:00
Andrew Dunstan dccb4d1c03 MSVC: Don't build ldap_password_func if not building with ldap
Blind attempt to fix issue with 419a8dd814 found on drongo.
2023-03-15 18:30:22 -04:00
Andrew Dunstan 18f2f8b8b0 Don't force SQL_ASCII/no-locale for installcheck in vcregress.pl
It's been this way for a very long time, but it appears to have been
masking an issue that only manifests with different settings. Therefore,
run the tests in the installation's default encoding/locale.

Backpatch to all live branches.
2023-02-26 06:48:41 -05:00
Thomas Munro d2ea2d310d Remove obsolete platforms from ps_status.c.
Time to remove various code, comments and configure/meson probes
relating to ancient BSD, SunOS, GNU/Hurd, IRIX, NeXT and Unixware.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKGJMNGUAqf27WbckYFrM-Mavy0RKJvocfJU%3DJ2XcAZyv%2Bw%40mail.gmail.com
2023-02-17 15:18:18 +13:00
Michael Paquier 9244c11afe Fix handling of SCRAM-SHA-256's channel binding with RSA-PSS certificates
OpenSSL 1.1.1 and newer versions have added support for RSA-PSS
certificates, which requires the use of a specific routine in OpenSSL to
determine which hash function to use when compiling it when using
channel binding in SCRAM-SHA-256.  X509_get_signature_nid(), that is the
original routine the channel binding code has relied on, is not able to
determine which hash algorithm to use for such certificates.  However,
X509_get_signature_info(), new to OpenSSL 1.1.1, is able to do it.  This
commit switches the channel binding logic to rely on
X509_get_signature_info() over X509_get_signature_nid(), which would be
the choice when building with 1.1.1 or newer.

The error could have been triggered on the client or the server, hence
libpq and the backend need to have their related code paths patched.
Note that attempting to load an RSA-PSS certificate with OpenSSL 1.1.0
or older leads to a failure due to an unsupported algorithm.

The discovery of relying on X509_get_signature_info() comes from Jacob,
the tests have been written by Heikki (with few tweaks from me), while I
have bundled the whole together while adding the bits needed for MSVC
and meson.

This issue exists since channel binding exists, so backpatch all the way
down.  Some tests are added in 15~, triggered if compiling with OpenSSL
1.1.1 or newer, where the certificate and key files can easily be
generated for RSA-PSS.

Reported-by: Gunnar "Nick" Bluth
Author: Jacob Champion, Heikki Linnakangas
Discussion: https://postgr.es/m/17760-b6c61e752ec07060@postgresql.org
Backpatch-through: 11
2023-02-15 10:12:16 +09:00
Michael Paquier 2f6e15ac93 Revert refactoring of restore command code to shell_restore.c
This reverts commits 24c35ec and 57169ad.  PreRestoreCommand() and
PostRestoreCommand() need to be put closer to the system() call calling
a restore_command, as they enable in_restore_command for the startup
process which would in turn trigger an immediate proc_exit() in the
SIGTERM handler.  Perhaps we could get rid of this behavior entirely,
but 24c35ec has made the window where the flag is enabled much larger
than it was, and any Postgres-like actions (palloc, etc.) taken by code
paths while the flag is enabled could lead to more severe issues in the
shutdown processing.

Note that curculio has showed that there are much more problems in this
area, unrelated to this change, actually, hence the issues related to
that had better be addressed first.  Keeping the code of HEAD in line
with the stable branches should make that a bit easier.

Per discussion with Andres Freund and Nathan Bossart.

Discussion: https://postgr.es/m/Y979NR3U5VnWrTwB@paquier.xyz
2023-02-06 08:28:42 +09:00
Michael Paquier 14bdb3f13d Refactor code for restoring files via shell commands
Presently, restore_command uses a different code path than
archive_cleanup_command and recovery_end_command.  These code paths
are similar and can be easily combined, as long as it is possible to
identify if a command should:
- Issue a FATAL on signal.
- Exit immediately on SIGTERM.

While on it, this removes src/common/archive.c and its associated
header.  Since the introduction of c96de2c, BuildRestoreCommand() has
become a simple wrapper of replace_percent_placeholders() able to call
make_native_path().  This simplifies shell_restore.c as long as
RestoreArchivedFile() includes a call to make_native_path().

Author: Nathan Bossart
Reviewed-by: Andres Freund, Michael Paquier
Discussion: https://postgr.es/m/20221227192449.GA3672473@nathanxps13
2023-01-18 11:15:48 +09:00
Peter Eisentraut c971a5b27a Fix MSVC build
for commit c96de2ce17
2023-01-11 14:41:52 +01:00
Peter Eisentraut bf03cfd162 Windows support in pg_import_system_collations
Windows can enumerate the locales that are either installed or
supported by calling EnumSystemLocalesEx(), similar to what is already
done in the READ_LOCALE_A_OUTPUT switch.  We can refactor some of the
logic already used in that switch into a new function
create_collation_from_locale().

The enumerated locales have BCP 47 shape, that is with a hyphen
between language and territory, instead of POSIX's underscore.  The
created collations will retain the BCP 47 shape, but we will also
create a POSIX alias, so xx-YY will have an xx_YY alias.

A new test collate.windows.win1252 is added that is like
collate.linux.utf8.

Author: Juan Jose Santamaria Flecha <juanjo.santamaria@gmail.com>
Reviewed-by: Dmitry Koval <d.koval@postgrespro.ru>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/0050ec23-34d9-2765-9015-98c04f0e18ac@postgrespro.ru
2023-01-03 14:21:56 +01:00
Bruce Momjian c8e1ba736b Update copyright for 2023
Backpatch-through: 11
2023-01-02 15:00:37 -05:00
Peter Eisentraut 24b55cd949 Reorder some object files in makefiles
This restores some once-intended alphabetical orders and makes the
lists consistent between the different build systems.
2022-12-28 15:10:27 +01:00
Andrew Dunstan ccc59a83cd Fix binary mismatch for MSVC plperl vs gcc built perl libs
When loading plperl built against Strawberry perl or the msys2 ucrt perl
that have been built with gcc, a binary mismatch has been encountered
which looks like this:

loadable library and perl binaries are mismatched (got handshake key 0000000012800080, needed 0000000012900080)

To cure this we bring the handshake keys into sync by adding
NO_THREAD_SAFE_LOCALE to the defines used to build plperl.

Discussion: https://postgr.es/m/20211005004334.tgjmro4kuachwiuc@alap3.anarazel.de
Discussion: https://postgr.es/m/c2da86a0-2906-744c-923d-16da6047875e@dunslane.net

Backpatch to all live branches.
2022-11-27 09:03:22 -05:00
Peter Eisentraut c8e4030d1b Make finding openssl program a configure or meson option
Various test suites use the "openssl" program as part of their setup.
There isn't a way to override which openssl program is to be used,
other than by fiddling with the path, perhaps.  This has gotten
increasingly problematic because different versions of openssl have
different capabilities and do different things by default.

This patch checks for an openssl binary in configure and meson setup,
with appropriate ways to override it.  This is similar to how "lz4"
and "zstd" are handled, for example.  The meson build system actually
already did this, but the result was only used in some places.  This
is now applied more uniformly.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/dc638b75-a16a-007d-9e1c-d16ed6cf0ad2%40enterprisedb.com
2022-10-20 21:05:42 +02:00
Andres Freund 4289263cf2 windows: Adjust FD_SETSIZE via commandline define
When using precompiled headers, we cannot pre-define macros for the system
headers from within .c files, as headers are already processed before
the #define in the C file is reached. But we can pre-define using
-DFD_SETSIZE, as long as that's also used when building the precompiled header.

A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
am hesitant to change FD_SETSIZE globally on windows, due to
src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
-DFD_SETSIZE=1024 when building the specific targets needing it.

We likely should move away from using select() in those places, but that's a
larger change.

Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20221005190829.lda7ttalh4mzrvf4@awork3.anarazel.de
Discussion: https://postgr.es/m/CA+hUKG+50eOUbN++ocDc0Qnp9Pvmou23DSXu=ZA6fepOcftKqA@mail.gmail.com
Discussion: https://postgr.es/m/20190826054000.GE7005%40paquier.xyz
2022-10-06 13:09:57 -07:00
Andres Freund 31d2c4716e windows: remove date from version number in win32ver.rc
This may have served a purpose at some point, but these days it just
contributes to a non-reproducible build.

Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/c5736f70-bb6d-8d25-e35c-e3d886e4e905@enterprisedb.com
Discussion: https://postgr.es/m/1cef5b48-32bd-5cbf-fb62-fb648860f5ef@enterprisedb.com
2022-09-26 11:38:02 -07:00
Andres Freund eef63941c1 De-special-case pgevent's rc file handling
There's really no need to build win32ver.rc as part of building
pgmsgevent.rc. This will make it sligthly easier to add rc file generation to
the meson build.
2022-09-24 12:04:56 -07:00
Andres Freund c47885bd8b Split TESTDIR into TESTLOGDIR and TESTDATADIR
The motivation for this is twofold. For one the meson patchset would like to
have more control over the logfiles. For another, the log file location for
tap tests (tmp_check/log) is not symmetric to the log location for
pg_regress/isolation tests (log/).

This commit does not change the default location for log files for tap tests,
as that'd break the buildfarm log collection, it just provides the
infrastructure for doing so.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/1131990.1660661896@sss.pgh.pa.us
Discussion: https://postgr.es/m/20220828170806.GN2342@telsasoft.com
2022-09-19 18:03:17 -07:00
Andres Freund bb54bf2290 Don't hardcode tmp_check/ as test directory for tap tests
This is motivated by the meson patchset, which wants to put the log / data for
tests in a different place than the autoconf build. Right now log files for
tap tests have to be inside $TESTDIR/tmp_check, whereas log files for
pg_regress/isolationtester are outside of tmp_check. This change doesn't fix
the latter, but is a prerequisite.

The only test that needs adjustment is 010_tab_completion.pl, as it hardcoded
the tmp_check/ directory. Instead create a dedicated directory for the test
files.  It's also a bit cleaner independently, because it doesn't intermingle
the test files with more important things like the log/ directory.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/1131990.1660661896@sss.pgh.pa.us
Discussion: https://postgr.es/m/d861493c-ed20-c251-7a89-7924f5197341@enterprisedb.com
2022-09-19 18:00:50 -07:00
Andres Freund 70df2df1cc Extend gendef.pl in preparation for meson
The main issue with using gendef.pl as-is for meson is that with meson the
filenames are a bit longer, exceeding the max commandline length when calling
dumpbin with all objects. As it's easier to pass in a library anyway, do so.

The .def file location, input and temporary file location need to be tunable
as well.

This also fixes a bug in gendef.pl: The logic when to regenerate was broken
and never avoid regenerating.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20220809071055.rgikv3qn74ypnnbb@awork3.anarazel.de
Discussion: https://postgr.es/m/7dae5979-c6c0-cec5-7a36-76a85aa8053d@enterprisedb.com
2022-09-19 15:39:35 -07:00
John Naylor 4c1532763a Bump minimum Perl version to 5.14
The oldest vendor-shipped Perl in the buildfarm is 5.14.2, which is
the last version that Debian Wheezy shipped. That OS is EOL, but we
keep it running because there is no other convenient way to test certain
non-mainstream 32-bit platforms. There is no bugfix in the 5.14.2 release
that is required, and yet it's also not the latest minor release --
that would be 5.14.4. To clarify the situation, we have thus arranged the
buildfarm to test 5.14.0. That allows configure scripts and documentation
to state 5.14 without fine print.

The MSVC build didn't check the version, since our previous minimum 5.8.3
was considered too old to check for on Windows. We will need a check for
Windows sometime during the v16 cycle, but that could be rendered moot
by the impending Meson conversion, so it seems safe to just document
the requirement for now.

Reviewed by Tom Lane
Discussion: https://www.postgresql.org/message-id/20220902181553.ev4pgzhubhdkguuv@awork3.anarazel.de
2022-09-14 12:37:04 +07:00
John Naylor ecaf7c5df5 Move gramparse.h to src/backend/parser
This header is semi-private, being used only in files related to
raw parsing, so move to the backend directory where those files
live. This allows removal of Makefile rules that symlink gram.h to
src/include/parser, since gramparse.h can now include gram.h from
within the same directory. This has the side-effect of no longer
installing gram.h and gramparse.h, but there doesn't seem to be a
good reason to continue doing so.

Per suggestion from Andres Freund and Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/20220904181759.px6uosll6zbxcum5%40awork3.anarazel.de
2022-09-14 10:57:13 +07:00
John Naylor 8b878bffa8 Bump minimum version of Flex to 2.5.35
Since the retirement of some older buildfarm members, the oldest Flex
that gets regular testing is 2.5.35.

Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/1097762.1662145681@sss.pgh.pa.us
2022-09-09 12:55:23 +07:00
John Naylor b086a47a27 Bump minimum version of Bison to 2.3
Since the retirement of some older buildfarm members, the oldest Bison
that gets regular testing is 2.3. MacOS ships that version, and will
continue doing so for the forseeable future because of Apple's policy
regarding GPLv3. While Mac users could use a package manager to install
a newer version, there is no compelling reason to force them do so at
this time.

Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/1097762.1662145681@sss.pgh.pa.us
2022-09-09 12:31:41 +07:00
John Naylor 92e7b7722d Fix MSVC linker error for specparse.obj
Per buildfarm animals drongo
2022-09-04 18:01:04 +07:00
Michael Paquier 36389a060c Enable RandomizedBaseAddress (ASLR) on Windows with MSVC builds
This has as effect to add /DYNAMICBASE to the .dll and .exe files
generated by the builds, undoing 7f3e17b.  Note that ASLR was already
enabled in MinGW as we have never added --disable-dynamicbase there.

This change will ease a bit the integration of arm64 with MSVC, as ASLR
support is mandatory in this case.  So, thanks to this commit, we have
no need to make ASLR conditional depending on the architecture used for
the build.

Andres Freund has done a lot of testing with this option while working
on meson, without seeing /DYNAMICBASE as being a problem in the Windows
builds of the CI.  Personally, not supporting anything older than
Windows 10 on HEAD makes me feel safer about this change, as we have
seen ASLR with being a problem in process invocation particularly with
Windows 8 and server 2012 back in 2014, even if Windows 10 was not
really a thing back then.  45e004f is also something that can help in
making the process invocation more stable.  We are very early in the
development of Postgres 16, giving a lot of room to detect stability
issues if any.

Discussion: https://postgr.es/m/20220826012907.gjw3jdqdgsts5y65@awork3.anarazel.de
2022-08-28 16:04:58 +09:00
Thomas Munro bcc8b14ef6 Remove configure probe for sockaddr_in6 and require AF_INET6.
SUSv3 <netinet/in.h> defines struct sockaddr_in6, and all targeted Unix
systems have it.  Windows has it in <ws2ipdef.h>.  Remove the configure
probe, the macro and a small amount of dead code.

Also remove a mention of IPv6-less builds from the documentation, since
there aren't any.

This is similar to commits f5580882 and 077bf2f2 for Unix sockets.  Even
though AF_INET6 is an "optional" component of SUSv3, there are no known
modern operating system without it, and it seems even less likely to be
omitted from future systems than AF_UNIX.

Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGKErNfhmvb_H0UprEmp4LPzGN06yR2_0tYikjzB-2ECMw@mail.gmail.com
2022-08-26 10:18:30 +12:00
Andres Freund 1d77afefbd Don't define FRONTEND for libpq
Not needed anymore after 7143b3e821.

Discussion: https://postgr.es/m/20220820194550.725755r6fj2ro3rx@awork3.anarazel.de
2022-08-22 20:39:30 -07:00