Commit Graph

16921 Commits

Author SHA1 Message Date
Alexander Korotkov 40bbc8cf0a Improve documentation for COPY ... ON_ERROR ...
Discussion: https://postgr.es/m/20240126112829.d420b28859fbe84379fdb7ad%40sraoss.co.jp
Author: Yugo Nagata
Reviewed-by: Masahiko Sawada, David G. Johnston, Atsushi Torikoshi
2024-02-03 01:49:51 +02:00
Bruce Momjian 21912e3c02 doc: improve role option documentation
Role option management was changed in Postgres 16.  This patch improves
the docs around these changes, including CREATE ROLE's INHERIT option,
inheritance handling, and grant's ability to change role options.

Discussion: https://postgr.es/m/Zab9GiV63EENDcWG@momjian.us

Co-authored-by: David G. Johnston

Backpatch-through: 16
2024-02-01 06:11:53 -05:00
Daniel Gustafsson 16a80e8912 doc: remove incorrect grammar for ALTER EVENT TRIGGER
The Parameters subsection had an extra TRIGGER in the grammar
for DISABLE/ENABLE which is incorrect.  Backpatch down to all
supported versions since it's been like this all along.

Discussion: https://postgr.es/m/0AFB171E-7E78-4A90-A140-46AB270212CA@yesql.se
Backpatch-through: v12
2024-02-01 10:45:37 +01:00
Daniel Gustafsson 0293f8579c doc: Fix incorrect openssl option
The openssl command for displaying the DN of a client certificate was
using --subject and not the single-dash option -subject. While recent
versions of openssl handles double dash options,  earlier does not so
fix by using just -subject  (which is per the openssl documentation).

Backpatch to v14 where this was introduced.

Reported-by: konkove@gmail.com
Discussion: https://postgr.es/m/170672168899.666.10442618407194498217@wrigleys.postgresql.org
Backpatch-through: v14
2024-02-01 09:36:34 +01:00
Peter Eisentraut d0283387d3 doc: Document more that relations share a namespace
This was already documented in the CREATE INDEX reference, but not in
the introductory "Data Definition" chapter.  Also, document that the
index that implements a constraint has the same name as the
constraint.

Author: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: jian he <jian.universality@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxFG682tYcP9aH_F-jrqq5End8MHZR77zcp1%3DDUrEsSu1Q%40mail.gmail.com
2024-01-31 12:05:41 +01:00
Amit Kapila 776621a5e4 Add a failover option to subscriptions.
This commit introduces a new subscription option named 'failover', which
provides users with the ability to set the failover property of the
replication slot on the publisher when creating or altering a
subscription.

This uses the replication commands introduced by commit 7329240437 to
enable the failover option for a logical replication slot.

If the failover option is set to true, the associated replication slots
(i.e. the main slot and the table sync slots) in the upstream database are
enabled to be synchronized to the standbys. Note that the capability to
sync the replication slots will be added in subsequent commits.

Thanks to Masahiko Sawada for the design inputs.

Author: Shveta Malik, Hou Zhijie, Ajin Cherian
Reviewed-by: Peter Smith, Bertrand Drouvot, Dilip Kumar, Masahiko Sawada, Nisha Moond, Kuroda Hayato, Amit Kapila
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
2024-01-30 16:49:28 +05:30
David Rowley c85977d8fe Doc: mention foreign keys can reference unique indexes
We seem to have only documented a foreign key can reference the columns of
a primary key or unique constraint.  Here we adjust the documentation
to mention columns in a non-partial unique index can be mentioned too.

The header comment for transformFkeyCheckAttrs() also didn't mention
unique indexes, so fix that too.  In passing make that header comment
reflect reality in the various other aspects where it deviated from it.

Bug: 18295
Reported-by: Gilles PARC
Author: Laurenz Albe, David Rowley
Discussion: https://www.postgresql.org/message-id/18295-0ed0fac5c9f7b17b%40postgresql.org
Backpatch-through: 12
2024-01-30 10:15:17 +13:00
Alvaro Herrera 5de890e361
Add EXPLAIN (MEMORY) to report planner memory consumption
This adds a new "Memory:" line under the "Planning:" group (which
currently only has "Buffers:") when the MEMORY option is specified.

In order to make the reporting reasonably accurate, we create a separate
memory context for planner activities, to be used only when this option
is given.  The total amount of memory allocated by that context is
reported as "allocated"; we subtract memory in the context's freelists
from that and report that result as "used".  We use
MemoryContextStatsInternal() to obtain the quantities.

The code structure to show buffer usage during planning was not in
amazing shape, so I (Álvaro) modified the patch a bit to clean that up
in passing.

Author: Ashutosh Bapat
Reviewed-by: David Rowley, Andrey Lepikhov, Jian He, Andy Fan
Discussion: https://www.postgresql.org/message-id/CAExHW5sZA=5LJ_ZPpRO-w09ck8z9p7eaYAqq3Ks9GDfhrxeWBw@mail.gmail.com
2024-01-29 17:53:03 +01:00
Amit Kapila cf765ff824 Doc: Fix incorrect reference to conflicting column in pg_replication_slots.
Commit 007693f2a3 changes the existing 'conflicting' field to
'conflict_reason' in pg_replication_slots but missed updating one of its
existing references.

Author: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB571690299199ACA80F602D97947E2@OS0PR01MB5716.jpnprd01.prod.outlook.com
2024-01-29 12:13:39 +05:30
Peter Eisentraut 54fac0e505 Remove make function vpathsearch
This function served to support having prebuilt files in the source
tree for vpath builds.  This is no longer possible (since
721856ff24); all built files are now always in the build tree.  The
invocations of this function are no longer required.
2024-01-29 07:24:59 +01:00
Amit Kapila 7329240437 Allow setting failover property in the replication command.
This commit implements a new replication command called
ALTER_REPLICATION_SLOT and a corresponding walreceiver API function named
walrcv_alter_slot. Additionally, the CREATE_REPLICATION_SLOT command has
been extended to support the failover option.

These new additions allow the modification of the failover property of a
replication slot on the publisher. A subsequent commit will make use of
these commands in subscription commands and will add the tests as well to
cover the functionality added/changed by this commit.

Author: Hou Zhijie, Shveta Malik
Reviewed-by: Peter Smith, Bertrand Drouvot, Dilip Kumar, Masahiko Sawada, Nisha Moond, Kuroda, Hayato, Amit Kapila
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
2024-01-29 09:37:23 +05:30
Alvaro Herrera 8c9da1441d
Make spelling of cancelled/cancellation consistent
This fixes places where words derived from cancel were not using their
common en-US ugly^H^H^H^Hspelling.

Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA+hUKG+Lrq+ty6yWXF5572qNQ8KwxGwG5n4fsEcCUap685nWvQ@mail.gmail.com
2024-01-26 12:38:15 +01:00
Tom Lane 8ba6fdf905 Support TZ and OF format codes in to_timestamp().
Formerly, these were only supported in to_char(), but there seems
little reason for that restriction.  We should at least have enough
support to permit round-tripping the output of to_char().

In that spirit, TZ accepts either zone abbreviations or numeric
(HH or HH:MM) offsets, which are the cases that to_char() can output.
In an ideal world we'd make it take full zone names too, but
that seems like it'd introduce an unreasonable amount of ambiguity,
since the rules for POSIX-spec zone names are so lax.

OF is a subset of this, accepting only HH or HH:MM.

One small benefit of this improvement is that we can simplify
jsonpath's executeDateTimeMethod function, which no longer needs
to consider the HH and HH:MM cases separately.  Moreover, letting
it accept zone abbreviations means it will accept "Z" to mean UTC,
which is emitted by JSON.stringify() for example.

Patch by me, reviewed by Aleksander Alekseev and Daniel Gustafsson

Discussion: https://postgr.es/m/1681086.1686673242@sss.pgh.pa.us
2024-01-25 17:47:08 -05:00
Tom Lane 7014c9a4bb Doc: improve documentation for jsonpath behavior.
Clarify the behavior of jsonpath operators and functions by
describing their two different modes of operation explicitly.
In addition to the SQL-spec behavior, where a path returns
a list of matching items, we have a "predicate check" form
that always returns a single boolean result.  That was mentioned
in only one place, but it seems better to annotate each operator
and function as to which form(s) it takes.  Also improve the
examples by converting them into actual executable SQL with
results, and do a bunch of incidental wordsmithing.

David Wheeler, reviewed by Erik Wienhold, Jian He, and myself

Discussion: https://postgr.es/m/7262A188-59CA-4A8A-AAD7-83D4FF0B9758@justatheory.com
2024-01-25 11:32:18 -05:00
Andrew Dunstan 66ea94e8e6 Implement various jsonpath methods
This commit implements ithe jsonpath .bigint(), .boolean(),
.date(), .decimal([precision [, scale]]), .integer(), .number(),
.string(), .time(), .time_tz(), .timestamp(), and .timestamp_tz()
methods.

.bigint() converts the given JSON string or a numeric value to
the bigint type representation.

.boolean() converts the given JSON string, numeric, or boolean
value to the boolean type representation.  In the numeric case, only
integers are allowed. We use the parse_bool() backend function
to convert a string to a bool.

.decimal([precision [, scale]]) converts the given JSON string
or a numeric value to the numeric type representation.  If precision
and scale are provided for .decimal(), then it is converted to the
equivalent numeric typmod and applied to the numeric number.

.integer() and .number() convert the given JSON string or a
numeric value to the int4 and numeric type representation.

.string() uses the datatype's output function to convert numeric
and various date/time types to the string representation.

The JSON string representing a valid date/time is converted to the
specific date or time type representation using jsonpath .date(),
.time(), .time_tz(), .timestamp(), .timestamp_tz() methods.  The
changes use the infrastructure of the .datetime() method and perform
the datatype conversion as appropriate.  Unlike the .datetime()
method, none of these methods accept a format template and use ISO
DateTime format instead.  However, except for .date(), the
date/time related methods take an optional precision to adjust the
fractional seconds.

Jeevan Chalke, reviewed by Peter Eisentraut and Andrew Dunstan.
2024-01-25 10:15:43 -05:00
Amit Kapila c393308b69 Allow to enable failover property for replication slots via SQL API.
This commit adds the failover property to the replication slot. The
failover property indicates whether the slot will be synced to the standby
servers, enabling the resumption of corresponding logical replication
after failover. But note that this commit does not yet include the
capability to sync the replication slot; the subsequent commits will add
that capability.

A new optional parameter 'failover' is added to the
pg_create_logical_replication_slot() function. We will also enable to set
'failover' option for slots via the subscription commands in the
subsequent commits.

The value of the 'failover' flag is displayed as part of
pg_replication_slots view.

Author: Hou Zhijie, Shveta Malik, Ajin Cherian
Reviewed-by: Peter Smith, Bertrand Drouvot, Dilip Kumar, Masahiko Sawada, Nisha Moond, Kuroda, Hayato, Amit Kapila
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
2024-01-25 12:15:46 +05:30
Thomas Munro 820b5af73d jit: Require at least LLVM 10.
Remove support for older LLVM versions.  The default on common software
distributions will be at least LLVM 10 when PostgreSQL 17 ships.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com
2024-01-25 15:42:34 +13:00
Masahiko Sawada 729439607a Add progress reporting of skipped tuples during COPY FROM.
9e2d870119 enabled the COPY command to skip malformed data, however
there was no visibility into how many tuples were actually skipped
during the COPY FROM.

This commit adds a new "tuples_skipped" column to
pg_stat_progress_copy view to report the number of tuples that were
skipped because they contain malformed data.

Bump catalog version.

Author: Atsushi Torikoshi
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/d12fd8c99adcae2744212cb23feff6ed%40oss.nttdata.com
2024-01-25 10:57:41 +09:00
Peter Eisentraut 46a0cd4cef Add temporal PRIMARY KEY and UNIQUE constraints
Add WITHOUT OVERLAPS clause to PRIMARY KEY and UNIQUE constraints.
These are backed by GiST indexes instead of B-tree indexes, since they
are essentially exclusion constraints with = for the scalar parts of
the key and && for the temporal part.

Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
2024-01-24 16:34:37 +01:00
Michael Paquier 94edfe250c pgbench: Add \syncpipeline
This change adds a new meta-command called \syncpipeline to pgbench,
able to send a sync message without flushing using the new libpq
function PQsendPipelineSync().

This meta-command is available within a block made of \startpipeline and
\endpipeline.

Author: Anthonin Bonnefoy
Discussion: https://postgr.es/m/CAO6_XqpcNhW6LZHLF-2NpPzdTbyMm4-RVkr3+AP5cOKSm9hrWA@mail.gmail.com
2024-01-24 16:55:19 +09:00
Amit Langote 1edb3b491b Adjust populate_record_field() to handle errors softly
This adds a Node *escontext parameter to it and a bunch of functions
downstream to it, replacing any ereport()s in that path by either
errsave() or ereturn() as appropriate.  This also adds code to those
functions where necessary to return early upon encountering a soft
error.

The changes here are mainly intended to suppress errors in the
functions of jsonfuncs.c.  Functions in any external modules, such as
arrayfuncs.c, that those functions may in turn call are not changed
here based on the assumption that the various checks in jsonfuncs.c
functions should ensure that only values that are structurally valid
get passed to the functions in those external modules.  An exception
is made for domain_check() to allow handling domain constraint
violation errors softly.

For testing, this adds a function jsonb_populate_record_valid(),
which returns true if jsonb_populate_record() would finish without
causing an error for the provided JSON object, false otherwise.  Note
that jsonb_populate_record() internally calls populate_record(),
which in turn uses populate_record_field().

Extracted from a much larger patch to add SQL/JSON query functions.

Author: Nikita Glukhov <n.gluhov@postgrespro.ru>
Author: Teodor Sigaev <teodor@sigaev.ru>
Author: Oleg Bartunov <obartunov@gmail.com>
Author: Alexander Korotkov <aekorotkov@gmail.com>
Author: Andrew Dunstan <andrew@dunslane.net>
Author: Amit Langote <amitlangote09@gmail.com>

Reviewers have included (in no particular order) Andres Freund,
Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers,
Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby,
Álvaro Herrera, Jian He, Peter Eisentraut

Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de
Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org
Discussion: https://postgr.es/m/CA+HiwqHROpf9e644D8BRqYvaAPmgBZVup-xKMDPk-nd4EpgzHw@mail.gmail.com
Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
2024-01-24 15:04:33 +09:00
Daniel Gustafsson 68387f9bdc doc: Add acronym and glossary term for Access Method
AM was used throughout the documentation to denote Access Method, but
the acronym was not described.  This adds an acronym entry as well as
a glossary term which the acronym links to.  Each page which describe
AMs have the first occurrence with <acronym> markup.

Reported-by: alaa.attya91@gmail.com
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/169974408805.398198.6927340566912872957@wrigleys.postgresql.org
2024-01-23 13:20:15 +01:00
Peter Eisentraut b0f0a9432d Make documentation builds reproducible
Currently, the documentation builds are not fully reproducible (in the
sense of https://reproducible-builds.org/).  A fix is available
upstream (https://github.com/docbook/xslt10-stylesheets/issues/54) but
not released.  This commit patches the upstream fix into our
customization layer.

This patch addresses both the HTML and the FO output.  The man output
is already reproducible.

Discussion: https://www.postgresql.org/message-id/flat/9077b779-a9f8-09c8-6e85-da1ebfba15af@eisentraut.org
2024-01-22 11:01:06 +01:00
Michael Paquier d86d20f0ba Add backend support for injection points
Injection points are a new facility that makes possible for developers
to run custom code in pre-defined code paths.  Its goal is to provide
ways to design and run advanced tests, for cases like:
- Race conditions, where processes need to do actions in a controlled
ordered manner.
- Forcing a state, like an ERROR, FATAL or even PANIC for OOM, to force
recovery, etc.
- Arbitrary sleeps.

This implements some basics, and there are plans to extend it more in
the future depending on what's required.  Hence, this commit adds a set
of routines in the backend that allows developers to attach, detach and
run injection points:
- A code path calling an injection point can be declared with the macro
INJECTION_POINT(name).
- InjectionPointAttach() and InjectionPointDetach() to respectively
attach and detach a callback to/from an injection point.  An injection
point name is registered in a shmem hash table with a library name and a
function name, which will be used to load the callback attached to an
injection point when its code path is run.

Injection point names are just strings, so as an injection point can be
declared and run by out-of-core extensions and modules, with callbacks
defined in external libraries.

This facility is hidden behind a dedicated switch for ./configure and
meson, disabled by default.

Note that backends use a local cache to store callbacks already loaded,
cleaning up their cache if a callback has found to be removed on a
best-effort basis.  This could be refined further but any tests but what
we have here was fine with the tests I've written while implementing
these backend APIs.

Author: Michael Paquier, with doc suggestions from Ashutosh Bapat.
Reviewed-by: Ashutosh Bapat, Nathan Bossart, Álvaro Herrera, Dilip
Kumar, Amul Sul, Nazir Bilal Yavuz
Discussion: https://postgr.es/m/ZTiV8tn_MIb_H2rE@paquier.xyz
2024-01-22 10:15:50 +09:00
Nathan Bossart 8b2bcf3f28 Introduce the dynamic shared memory registry.
Presently, the most straightforward way for a shared library to use
shared memory is to request it at server startup via a
shmem_request_hook, which requires specifying the library in
shared_preload_libraries.  Alternatively, the library can create a
dynamic shared memory (DSM) segment, but absent a shared location
to store the segment's handle, other backends cannot use it.  This
commit introduces a registry for DSM segments so that these other
backends can look up existing segments with a library-specified
string.  This allows libraries to easily use shared memory without
needing to request it at server startup.

The registry is accessed via the new GetNamedDSMSegment() function.
This function handles allocating the segment and initializing it
via a provided callback.  If another backend already created and
initialized the segment, it simply attaches the segment.
GetNamedDSMSegment() locks the registry appropriately to ensure
that only one backend initializes the segment and that all other
backends just attach it.

The registry itself is comprised of a dshash table that stores the
DSM segment handles keyed by a library-specified string.

Reviewed-by: Michael Paquier, Andrei Lepikhov, Nikita Malakhov, Robert Haas, Bharath Rupireddy, Zhang Mingli, Amul Sul
Discussion: https://postgr.es/m/20231205034647.GA2705267%40nathanxps13
2024-01-19 14:24:36 -06:00
Nathan Bossart 964152c476 doc: Reorganize section for shared memory and LWLocks.
Presently, this section meanders through a few different features,
and the text itself is terse.  This commit attempts to improve
matters by splitting the section into smaller sections and by
expanding the text for clarity.  This is preparatory work for a
follow-up commit that will introduce a way for libraries to use
shared memory without needing to request it at startup time.

Reviewed-by: Aleksander Alekseev, Bharath Rupireddy, Abhijit Menon-Sen
Discussion: https://postgr.es/m/20240112041430.GA3557928%40nathanxps13
Discussion: https://postgr.es/m/20231205034647.GA2705267%40nathanxps13
2024-01-19 11:18:32 -06:00
Daniel Gustafsson b2a296b0cf doc: Update compiler for FreeBSD specific instructions
FreeBSD has since v13 switched to Clang from GCC, so update our
instructions to cc instead of gcc to reflect that.

Reported-by: digitaldog4@gmail.com
Discussion: https://postgr.es/m/CAJR80-xzR49jEkAC98Yo-wuSdFDV2=2H4j9Z2AukSVU=-UpynA@mail.gmail.com
2024-01-19 16:27:27 +01:00
Peter Eisentraut 6db4598fcb Add stratnum GiST support function
This is support function 12 for the GiST AM and translates
"well-known" RT*StrategyNumber values into whatever strategy number is
used by the opclass (since no particular numbers are actually
required).  We will use this to support temporal PRIMARY
KEY/UNIQUE/FOREIGN KEY/FOR PORTION OF functionality.

This commit adds two implementations, one for internal GiST opclasses
(just an identity function) and another for btree_gist opclasses.  It
updates btree_gist from 1.7 to 1.8, adding the support function for
all its opclasses.

Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
2024-01-19 15:42:13 +01:00
Alexander Korotkov b725b7eec4 Rename COPY option from SAVE_ERROR_TO to ON_ERROR
The option names now are "stop" (default) and "ignore".  The future options
could be "file 'filename.log'" and "table 'tablename'".

Discussion: https://postgr.es/m/20240117.164859.2242646601795501168.horikyota.ntt%40gmail.com
Author: Jian He
Reviewed-by: Atsushi Torikoshi
2024-01-19 15:15:51 +02:00
Nathan Bossart d891dcc065 Improve some documentation about the bootstrap superuser.
This commit adds some notes about the inability to remove superuser
privileges from the bootstrap superuser.  This has been blocked
since commit e530be2c5c, but it wasn't intended be a supported
feature before that, either.

In passing, change "bootstrap user" to "bootstrap superuser" in a
couple places.

Author: Yurii Rashkovskii
Reviewed-by: Vignesh C, David G. Johnston
Discussion: https://postgr.es/m/CA%2BRLCQzSx_eTC2Fch0EzeNHD3zFUcPvBYOoB%2BpPScFLch1DEQw%40mail.gmail.com
2024-01-18 21:39:51 -06:00
Robert Haas 0a157a4d44 Clarify language in incremental backup glossary entry.
These changes were intended to be included in commit
7b1dbf0a8d, but I goofed.
2024-01-18 10:09:38 -05:00
Robert Haas 7b1dbf0a8d More documentation updates for incremental backup.
Add new terms to glossary. Add a reference to walsummarizer
to monitoring.sgml.

Matthias van de Meent and Robert Haas

Discussion: http://postgr.es/m/CAEze2WjhdVCqEe_qqEok3NA6DwUdOGSBjAxzmYdAqiaaH1uRcg@mail.gmail.com
2024-01-18 09:25:28 -05:00
Alvaro Herrera 27d04ed531
Rework text in replication slots documentation
This is cosmetic, so no backpatch.

Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/20230413111838.e7yxke2dtwrxw3qy@alvherre.pgsql
Discussion: https://postgr.es/m/202401151537.ryhysqfncs6d@alvherre.pgsql
2024-01-18 11:30:53 +01:00
Alexander Korotkov 9e2d870119 Add new COPY option SAVE_ERROR_TO
Currently, when source data contains unexpected data regarding data type or
range, the entire COPY fails. However, in some cases, such data can be ignored
and just copying normal data is preferable.

This commit adds a new option SAVE_ERROR_TO, which specifies where to save the
error information. When this option is specified, COPY skips soft errors and
continues copying.

Currently, SAVE_ERROR_TO only supports "none". This indicates error information
is not saved and COPY just skips the unexpected data and continues running.

Later works are expected to add more choices, such as 'log' and 'table'.

Author: Damir Belyalov, Atsushi Torikoshi, Alex Shulgin, Jian He
Discussion: https://postgr.es/m/87k31ftoe0.fsf_-_%40commandprompt.com
Reviewed-by: Pavel Stehule, Andres Freund, Tom Lane, Daniel Gustafsson,
Reviewed-by: Alena Rybakina, Andy Fan, Andrei Lepikhov, Masahiko Sawada
Reviewed-by: Vignesh C, Atsushi Torikoshi
2024-01-16 23:08:53 +02:00
Peter Eisentraut 6995863157 Support identity columns in partitioned tables
Previously, identity columns were disallowed on partitioned tables.
(The reason was mainly that no one had gotten around to working
through all the details to make it work.)  This makes it work now.

Some details on the behavior:

* A newly created partition inherits identity property

  The partitions of a partitioned table are integral part of the
  partitioned table.  A partition inherits identity columns from the
  partitioned table.  An identity column of a partition shares the
  identity space with the corresponding column of the partitioned
  table.  In other words, the same identity column across all
  partitions of a partitioned table share the same identity space.
  This is effected by sharing the same underlying sequence.

  When INSERTing directly into a partition, the sequence associated
  with the topmost partitioned table is used to calculate the value of
  the corresponding identity column.

  In regular inheritance, identity columns and their properties in a
  child table are independent of those in its parent tables.  A child
  table does not inherit identity columns or their properties
  automatically from the parent.  (This is unchanged.)

* Attached partition inherits identity column

  A table being attached as a partition inherits the identity property
  from the partitioned table.  This should be fine since we expect
  that the partition table's column has the same type as the
  partitioned table's corresponding column.  If the table being
  attached is a partitioned table, the identity properties are
  propagated down its partition hierarchy.

  An identity column in the partitioned table is also marked as NOT
  NULL.  The corresponding column in the partition needs to be marked
  as NOT NULL for the attach to succeed.

* Drop identity property when detaching partition

  A partition's identity column shares the identity space
  (i.e. underlying sequence) as the corresponding column of the
  partitioned table.  If a partition is detached it can longer share
  the identity space as before.  Hence the identity columns of the
  partition being detached loose their identity property.

  When identity of a column of a regular table is dropped it retains
  the NOT NULL constraint that came with the identity property.
  Similarly the columns of the partition being detached retain the NOT
  NULL constraints that came with identity property, even though the
  identity property itself is lost.

  The sequence associated with the identity property is linked to the
  partitioned table (and not the partition being detached).  That
  sequence is not dropped as part of detach operation.

* Partitions with their own identity columns are not allowed.

* The usual ALTER operations (add identity column, add identity
  property to existing column, alter properties of an indentity
  column, drop identity property) are supported for partitioned
  tables.  Changing a column only in a partitioned table or a
  partition is not allowed; the change needs to be applied to the
  whole partition hierarchy.

Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/CAExHW5uOykuTC+C6R1yDSp=o8Q83jr8xJdZxgPkxfZ1Ue5RRGg@mail.gmail.com
2024-01-16 17:24:52 +01:00
Daniel Gustafsson 7bcf57a0ca Decorate WITH with literal markup tags
One instance of "WITH clause" was not using <literal> tags around
WITH, while others were, so add markup to the last one to ensure
consistency.  Backpatch to v15 where MERGE was added.

Reported-by: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxGJKY9ZCPV2WDM6xFsXq9C8r7r3vU6AkScN+p9k6CEpMw@mail.gmail.com
Backpatch-through: v15
2024-01-16 13:51:15 +01:00
Peter Eisentraut a37bb7c139 doc: Add Identity Column section under Data Definition chapter
This seems to be missing since identity column support was added.  Add
the same.  This gives a place to document various pieces of
information in one place that are currently distributed over several
command reference pages or just not documented (e.g., inheritance
behavior).

Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Author: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/CAExHW5uOykuTC+C6R1yDSp=o8Q83jr8xJdZxgPkxfZ1Ue5RRGg@mail.gmail.com
2024-01-16 09:42:40 +01:00
Peter Eisentraut 58b20e6d75 doc: Decorate PostgreSQL with productname tag
... in the section about Generated Columns.

Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAExHW5uOykuTC+C6R1yDSp=o8Q83jr8xJdZxgPkxfZ1Ue5RRGg@mail.gmail.com
2024-01-16 08:56:00 +01:00
Michael Paquier 4794c2d317 libpq: Add PQsendPipelineSync()
This new function is equivalent to PQpipelineSync(), except that it does
not flush anything to the server except if the size threshold of the
output buffer is reached; the user must subsequently call PQflush()
instead.

Its purpose is to reduce the system call overhead of pipeline mode, by
giving to applications more control over the timing of the flushes when
manipulating commands in pipeline mode.

Author: Anton Kirilov
Reviewed-by: Jelte Fennema-Nio, Robert Haas, Álvaro Herrera, Denis
Laxalde, Michael Paquier
Discussion: https://postgr.es/m/CACV6eE5arHFZEA717=iKEa_OewpVFfWJOmsOdGrqqsr8CJVfWQ@mail.gmail.com
2024-01-16 10:13:42 +09:00
Michael Paquier 83eb244e41 Fix a typo and some doc indentation related to libpq pipeline functions
Noticed while reviewing the area for a different patch.  This is
cosmetic, so no backpatch is done.
2024-01-16 09:17:51 +09:00
Robert Haas 237d01139b Fix typos.
Alexander Lakhin

Discussion: http://postgr.es/m/212b0987-83e5-e2ae-c5e8-b8170fdaf3a0@gmail.com
2024-01-15 11:50:44 -05:00
Peter Eisentraut 556799660f doc: Update catalog documentation for attstattarget nullable
Fixup for 4f622503d6.
2024-01-14 07:46:01 +01:00
Peter Eisentraut 4f622503d6 Make attstattarget nullable
This changes the pg_attribute field attstattarget into a nullable
field in the variable-length part of the row.  If no value is set by
the user for attstattarget, it is now null instead of previously -1.
This saves space in pg_attribute and tuple descriptors for most
practical scenarios.  (ATTRIBUTE_FIXED_PART_SIZE is reduced from 108
to 104.)  Also, null is the semantically more correct value.

The ANALYZE code internally continues to represent the default
statistics target by -1, so that that code can avoid having to deal
with null values.  But that is now contained to the ANALYZE code.
Only the DDL code deals with attstattarget possibly null.

For system columns, the field is now always null.  The ANALYZE code
skips system columns anyway.

To set a column's statistics target to the default value, the new
command form ALTER TABLE ... SET STATISTICS DEFAULT can be used.  (SET
STATISTICS -1 still works.)

Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/4da8d211-d54d-44b9-9847-f2a9f1184c76@eisentraut.org
2024-01-13 18:14:53 +01:00
Jeff Davis 5c31669058 Re-validate connection string in libpqrcv_connect().
A superuser may create a subscription with password_required=true, but
which uses a connection string without a password.

Previously, if the owner of such a subscription was changed to a
non-superuser, the non-superuser was able to utilize a password from
another source (like a password file or the PGPASSWORD environment
variable), which should not have been allowed.

This commit adds a step to re-validate the connection string before
connecting.

Reported-by: Jeff Davis
Author: Vignesh C
Reviewed-by: Peter Smith, Robert Haas, Amit Kapila
Discussion: https://www.postgresql.org/message-id/flat/e5892973ae2a80a1a3e0266806640dae3c428100.camel%40j-davis.com
Backpatch-through: 16
2024-01-12 13:41:36 -08:00
Robert Haas 882887e9b4 Fix typo.
Noriyoshi Shinoda

Discussion: http://postgr.es/m/DM4PR84MB17347D9466419413698105D8EE6F2@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
2024-01-12 09:46:46 -05:00
Alvaro Herrera 6780b79d5c
Added literal tag for RETURNING
This is an old mistake (92e38182d7); backpatch all the way back.

Author: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/c0aa00b60a16c0ea2a4c5123b013acb9@oss.nttdata.com
2024-01-12 12:44:20 +01:00
Robert Haas ee1bfd1683 Add new pg_walsummary tool.
This can dump the contents of the WAL summary files found in
pg_wal/summaries. Normally, this shouldn't really be something anyone
needs to do, but it may be needed for debugging problems with
incremental backup, or could possibly be useful to external tools.

Discussion: http://postgr.es/m/CA+Tgmobvqqj-DW9F7uUzT-cQqs6wcVb-Xhs=w=hzJnXSE-kRGw@mail.gmail.com
2024-01-11 12:48:27 -05:00
Robert Haas d9ef650fca Add new function pg_get_wal_summarizer_state().
This makes it possible to access information about the progress
of WAL summarization from SQL. The previously-added functions
pg_available_wal_summaries() and pg_wal_summary_contents() only
examine on-disk state, but this function exposes information from
the server's shared memory.

Discussion: http://postgr.es/m/CA+Tgmobvqqj-DW9F7uUzT-cQqs6wcVb-Xhs=w=hzJnXSE-kRGw@mail.gmail.com
2024-01-11 12:41:18 -05:00
Magnus Hagander ba2d2784f3 Fix omission in partitioning limitation documentation
UNIQUE and PRIMARY KEY constraints can be created on ONLY the
partitioned table.  We already had an example demonstrating that,
but forgot to mention it in the documentation of the limits of
partitioning.

Author: Laurenz Albe
Reviewed-By: shihao zhong, Shubham Khanna, Ashutosh Bapat
Backpatch-through: 12
Discussion: https://postgr.es/m/167299368731.659.16130012959616771853@wrigleys.postgresql.org
2024-01-11 14:27:10 +01:00
John Naylor f7f694b214 Update documentation of default fdw_tuple_cost
Follow up to cac169d68

Umair Shahid

Reviewed by Richard Guo and Chris Travers

Discussion: https://postgr.es/m/CALVUYo9RECc5gwD%2B4SRM5Es%2Bbg9cpNfhd3_qUjf7kVTGyLpFJg%40mail.gmail.com
2024-01-11 08:58:18 +07:00