Commit Graph

42297 Commits

Author SHA1 Message Date
Bruce Momjian 218747d2cf doc: add missing closing 'sect3' tag for ICU patch 2017-03-23 16:59:24 -04:00
Peter Eisentraut 2e0c17dc78 Add ICU_FLAGS to one more place
Reported-by: Thomas Munro <thomas.munro@enterprisedb.com>
2017-03-23 16:53:10 -04:00
Peter Eisentraut 524e0f7ac8 Fix crash in ICU patch
This only happened with single-byte encodings.
2017-03-23 16:31:39 -04:00
Robert Haas c23b186ae9 Fix enum definition.
Commit 249cf070e3 assigned to one of
the labels in the middle the value that should have been assigned
to the first member of the enum.  Rushabh's patch didn't have that
defect as submitted, but I managed to mess it up while editing.
Repair.
2017-03-23 16:10:43 -04:00
Peter Eisentraut eccfef81e1 ICU support
Add a column collprovider to pg_collation that determines which library
provides the collation data.  The existing choices are default and libc,
and this adds an icu choice, which uses the ICU4C library.

The pg_locale_t type is changed to a union that contains the
provider-specific locale handles.  Users of locale information are
changed to look into that struct for the appropriate handle to use.

Also add a collversion column that records the version of the collation
when it is created, and check at run time whether it is still the same.
This detects potentially incompatible library upgrades that can corrupt
indexes and other structures.  This is currently only supported by
ICU-provided collations.

initdb initializes the default collation set as before from the `locale
-a` output but also adds all available ICU locales with a "-x-icu"
appended.

Currently, ICU-provided collations can only be explicitly named
collations.  The global database locales are still always libc-provided.

ICU support is enabled by configure --with-icu.

Reviewed-by: Thomas Munro <thomas.munro@enterprisedb.com>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
2017-03-23 15:28:48 -04:00
Robert Haas ea42cc18c3 Track the oldest XID that can be safely looked up in CLOG.
This provides infrastructure for looking up arbitrary, user-supplied
XIDs without a risk of scary-looking failures from within the clog
module.  Normally, the oldest XID that can be safely looked up in CLOG
is the same as the oldest XID that can reused without causing
wraparound, and the latter is already tracked.  However, while
truncation is in progress, the values are different, so we must
keep track of them separately.

Craig Ringer, reviewed by Simon Riggs and by me.

Discussion: http://postgr.es/m/CAMsr+YHQiWNEi0daCTboS40T+V5s_+dst3PYv_8v2wNVH+Xx4g@mail.gmail.com
2017-03-23 14:26:31 -04:00
Peter Eisentraut 50c956add8 Remove createlang and droplang
They have been deprecated since PostgreSQL 9.1.

Reviewed-by: Magnus Hagander <magnus@hagander.net>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2017-03-23 14:16:45 -04:00
Robert Haas 691b8d5928 Allow for parallel execution whenever ExecutorRun() is done only once.
Previously, it was unsafe to execute a plan in parallel if
ExecutorRun() might be called with a non-zero row count.  However,
it's quite easy to fix things up so that we can support that case,
provided that it is known that we will never call ExecutorRun() a
second time for the same QueryDesc.  Add infrastructure to signal
this, and cross-checks to make sure that a caller who claims this is
true doesn't later reneg.

While that pattern never happens with queries received directly from a
client -- there's no way to know whether multiple Execute messages
will be sent unless the first one requests all the rows -- it's pretty
common for queries originating from procedural languages, which often
limit the result to a single tuple or to a user-specified number of
tuples.

This commit doesn't actually enable parallelism in any additional
cases, because currently none of the places that would be able to
benefit from this infrastructure pass CURSOR_OPT_PARALLEL_OK in the
first place, but it makes it much more palatable to pass
CURSOR_OPT_PARALLEL_OK in places where we currently don't, because it
eliminates some cases where we'd end up having to run the parallel
plan serially.

Patch by me, based on some ideas from Rafia Sabih and corrected by
Rafia Sabih based on feedback from Dilip Kumar and myself.

Discussion: http://postgr.es/m/CA+TgmobXEhvHbJtWDuPZM9bVSLiTj-kShxQJ2uM5GPDze9fRYA@mail.gmail.com
2017-03-23 13:14:36 -04:00
Teodor Sigaev 218f51584d Reduce page locking in GIN vacuum
GIN vacuum during cleaning posting tree can lock this whole tree for a long
time with by holding LockBufferForCleanup() on root. Patch changes it with
two ways: first, cleanup lock will be taken only if there is an empty page
(which should be deleted) and, second, it tries to lock only subtree, not the
whole posting tree.

Author: Andrey Borodin with minor editorization by me
Reviewed-by: Jeff Davis, me

https://commitfest.postgresql.org/13/896/
2017-03-23 19:38:47 +03:00
Peter Eisentraut 73561013e5 Remove trailing comma from enum definition
Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-03-23 11:58:11 -04:00
Peter Eisentraut 128e6ee01d Assorted compilation and test fixes
related to 7c4f52409a, per build farm

Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-03-23 11:44:43 -04:00
Simon Riggs 232c532213 Minor spelling correction in comment
Jon Nelson
2017-03-23 15:29:42 +00:00
Simon Riggs 6912acc04f Replication lag tracking for walsenders
Adds write_lag, flush_lag and replay_lag cols to pg_stat_replication.

Implements a lag tracker module that reports the lag times based upon
measurements of the time taken for recent WAL to be written, flushed and
replayed and for the sender to hear about it. These times
represent the commit lag that was (or would have been) introduced by each
synchronous commit level, if the remote server was configured as a
synchronous standby.  For an asynchronous standby, the replay_lag column
approximates the delay before recent transactions became visible to queries.
If the standby server has entirely caught up with the sending server and
there is no more WAL activity, the most recently measured lag times will
continue to be displayed for a short time and then show NULL.

Physical replication lag tracking is automatic. Logical replication tracking
is possible but is the responsibility of the logical decoding plugin.
Tracking is a private module operating within each walsender individually,
with values reported to shared memory. Module not used outside of walsender.

Design and code is good enough now to commit - kudos to the author.
In many ways a difficult topic, with important and subtle behaviour so this
shoudl be expected to generate discussion and multiple open items: Test now!

Author: Thomas Munro, following designs by Fujii Masao and Simon Riggs
Review: Simon Riggs, Ian Barwick and Craig Ringer
2017-03-23 14:05:28 +00:00
Peter Eisentraut 7c4f52409a Logical replication support for initial data copy
Add functionality for a new subscription to copy the initial data in the
tables and then sync with the ongoing apply process.

For the copying, add a new internal COPY option to have the COPY source
data provided by a callback function.  The initial data copy works on
the subscriber by receiving COPY data from the publisher and then
providing it locally into a COPY that writes to the destination table.

A WAL receiver can now execute full SQL commands.  This is used here to
obtain information about tables and publications.

Several new options were added to CREATE and ALTER SUBSCRIPTION to
control whether and when initial table syncing happens.

Change pg_dump option --no-create-subscription-slots to
--no-subscription-connect and use the new CREATE SUBSCRIPTION
... NOCONNECT option for that.

Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Tested-by: Erik Rijkers <er@xs4all.nl>
2017-03-23 08:55:37 -04:00
Magnus Hagander 707576b571 Fix grammar in comment
Author: Emil Iggland
2017-03-23 10:14:42 +01:00
Stephen Frost 017e4f2588 Expose waitforarchive option through pg_stop_backup()
Internally, we have supported the option to either wait for all of the
WAL associated with a backup to be archived, or to return immediately.
This option is useful to users of pg_stop_backup() as well, when they
are reading the stop backup record position and checking that the WAL
they need has been archived independently.

This patch adds an additional, optional, argument to pg_stop_backup()
which allows the user to indicate if they wish to wait for the WAL to be
archived or not.  The default matches current behavior, which is to
wait.

Author: David Steele, with some minor changes, doc updates by me.
Reviewed by: Takayuki Tsunakawa, Fujii Masao
Discussion: https://postgr.es/m/758e3fd1-45b4-5e28-75cd-e9e7f93a4c02@pgmasters.net
2017-03-22 23:44:58 -04:00
Peter Eisentraut cbf7ed51b8 doc: Improve CREATE PUBLICATION examples 2017-03-22 15:27:40 -04:00
Peter Eisentraut b941891190 doc: Fix logical replication setup instructions
The pg_hba.conf rules were changed in
8df9bd0b44.
2017-03-22 15:27:40 -04:00
Peter Eisentraut a2760915b3 doc: Markup and formatting improvements 2017-03-22 15:27:40 -04:00
Robert Haas dc02c7bca4 Fix wrong costing of Sort under Gather Merge.
There's no mechanism for such a sort to become a top-N sort, so we
should pass -1 rather than limit_tuples to cost_sort().

Rushabh Lathia, per a report from Mithun Cy

Discussion: http://postgr.es/m/CAGPqQf1akRcSgC9=6iwx=sEPap9UvPpHJLzg8_N+OuHdb6fL+g@mail.gmail.com
2017-03-22 14:45:14 -04:00
Magnus Hagander 6b76f1bb58 Support multiple RADIUS servers
This changes all the RADIUS related parameters (radiusserver,
radiussecret, radiusport, radiusidentifier) to be plural and to accept a
comma separated list of servers, which will be tried in order.

Reviewed by Adam Brightwell
2017-03-22 18:11:08 +01:00
Simon Riggs c137c68ea6 Correct erroneous comment in GetOldestXmin()
Craig Ringer
2017-03-22 16:58:12 +00:00
Simon Riggs af4b1a0869 Refactor GetOldestXmin() to use flags
Replace ignoreVacuum parameter with more flexible flags.

Author: Eiji Seki
Review: Haribabu Kommi
2017-03-22 16:51:01 +00:00
Simon Riggs 49bff5300d Assign AccessExclusiveLocks against subxacts in Hot Standby
Previously AELs were registered against the top-level xid, which could
cause locks to be held much longer than necessary in some cases during
Hot Standby replay. We now record locks directly against their appropriate
xids. Requires few code changes because original code allowed for this
situation but didn’t fully implement it.

Discussion: CAKJS1f9vJ841HY=wonnLVbfkTWGYWdPN72VMxnArcGCjF3SywA@mail.gmail.com

Author: Simon Riggs and David Rowley
2017-03-22 16:37:28 +00:00
Peter Eisentraut 8df9bd0b44 Change logical replication pg_hba.conf use
Logical replication no longer uses the "replication" keyword.  It just
matches database entries in the normal way.  The "replication" keyword
now only applies to physical replication.

Reviewed-by: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-03-22 11:19:30 -04:00
Peter Eisentraut 4cfc9484d4 Refine rules for altering publication owner
Previously, the new owner had to be a superuser.  The new rules are more
refined similar to other objects.

Reviewed-by: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-03-22 11:19:30 -04:00
Andrew Dunstan 96a7128b7b Sync pg_dump and pg_dumpall output
Before exiting any files are fsync'ed. A --no-sync option is also
provided for a faster exit if desired.

Michael Paquier.

Reviewed by Albe Laurenz

Discussion: https://postgr.es/m/CAB7nPqS1uZ=Ov+UruW6jr3vB-S_DLVMPc0dQpV-fTDjmm0ZQMg@mail.gmail.com
2017-03-22 10:20:13 -04:00
Simon Riggs 9b013dc238 Improve performance of replay of AccessExclusiveLocks
A hot standby replica keeps a list of Access Exclusive locks for a top
level transaction. These locks are released when the top level transaction
ends. Searching of this list is O(N^2), and each transaction had to pay the
price of searching this list for locks, even if it didn't take any AE
locks itself.

This patch optimizes this case by having the master server track which
transactions took AE locks, and passes that along to the standby server in
the commit/abort record. This allows the standby to only try to release
locks for transactions which actually took any, avoiding the majority of
the performance issue.

Refactor MyXactAccessedTempRel into MyXactFlags to allow minimal additional
cruft with this.

Analysis and initial patch by David Rowley
Author: David Rowley and Simon Riggs
2017-03-22 13:09:36 +00:00
Simon Riggs 1148e22a82 Teach xlogreader to follow timeline switches
Uses page-based mechanism to ensure we’re using the correct timeline.

Tests are included to exercise the functionality using a cold disk-level copy
of the master that's started up as a replica with slots intact, but the
intended use of the functionality is with later features.

Craig Ringer, reviewed by Simon Riggs and Andres Freund
2017-03-22 07:05:12 +00:00
Peter Eisentraut 9ca2dd578d Avoid Perl warning
Perl versions before 5.12 would warn "Use of implicit split to @_ is
deprecated".

Author: Jeff Janes <jeff.janes@gmail.com>
2017-03-22 00:18:49 -04:00
Peter Eisentraut 05227e0c34 pg_dump: Only dump publications when dumping everything
Don't dump publications with pg_dump -t or similar cases that select
specific groups of objects.

Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-03-21 23:34:40 -04:00
Peter Eisentraut 9212810c4f doc: Update tool sets documentation for modern FreeBSD
For several operating systems, we give handy package manager one-liners
to install all the requirements for building our documentation.  All
current production FreeBSD releases have a friendly new package manager
a bit like apt/yum, so give a one line command here.  Also, add a brief
note about gmake vs make in the doc subdirectory.

Author: Thomas Munro <thomas.munro@enterprisedb.com>
2017-03-21 22:11:49 -04:00
Andrew Dunstan 80275561ff Document btree_gin support for enums 2017-03-21 11:11:42 -04:00
Andrew Dunstan 4ad0f88c44 Add btree_gin support for enum types
Reviewed by Tom Lane and Anastasia Lubennikova

Discussion:  http://postgr.es/m/56EA8A71.8060107@dunslane.net
2017-03-21 11:04:17 -04:00
Andrew Dunstan f7946a92b6 Add btree_gist support for enum types.
This will allow enums to be used in exclusion constraints.

The code uses the new CallerFInfoFunctionCall infrastructure in fmgr,
and the support for it added to btree_gist in commit 393bb504d7.

Reviewed by Tom Lane and Anastasia Lubennikova

Discussion:  http://postgr.es/m/56EA8A71.8060107@dunslane.net
2017-03-21 10:43:27 -04:00
Andrew Dunstan 65a9138b9b Use CallerFInfoFunctionCall with btree_gist for varlena types
Follow up to commit 393bb504d7 which did this for numeric types.
2017-03-21 10:43:27 -04:00
Andrew Dunstan 4b1c68d63e Use CallerFInfoFunctionCall with btree_gist for numeric types
None of the existing types actually need to use this mechanism, but this
will allow support for enum types which will need it. A separate patch
will adjust the varlena types support for consistency.

Reviewed by Tom Lane and Anastasia Lubennikova

Discussion:  http://postgr.es/m/27220.1478360811@sss.pgh.pa.us
2017-03-21 10:43:27 -04:00
Simon Riggs eb2a6131be Add a pg_recvlogical wrapper to PostgresNode
Allows testing of logical decoding using SQL interface and/or pg_recvlogical
Most logical decoding tests are in contrib/test_decoding. This module
is for work that doesn't fit well there, like where server restarts
are required.

Craig Ringer
2017-03-21 14:04:49 +00:00
Robert Haas d3cc37f1d8 Don't scan partitioned tables.
Partitioned tables do not contain any data; only their unpartitioned
descendents need to be scanned.  However, the partitioned tables still
need to be locked, even though they're not scanned.  To make that
work, Append and MergeAppend relations now need to carry a list of
(unscanned) partitioned relations that must be locked, and InitPlan
must lock all partitioned result relations.

Aside from the obvious advantage of avoiding some work at execution
time, this has two other advantages.  First, it may improve the
planner's decision-making in some cases since the empty relation
might throw things off.  Second, it paves the way to getting rid of
the storage for partitioned tables altogether.

Amit Langote, reviewed by me.

Discussion: http://postgr.es/m/6837c359-45c4-8044-34d1-736756335a15@lab.ntt.co.jp
2017-03-21 09:48:04 -04:00
Teodor Sigaev d5286aa905 Fix support for some operators (&<, &>, $<|, |&>) in box operator class
of SP-GiST.

Bug exists since initial commit of box opclass for SP-GiST,
so backpath to 9.6

Author: Nikita Glukhov with minor editorization of tests by me
Reviewed-by: Kyotaro Horiguchi, Anastasia Lubennikova

https://commitfest.postgresql.org/13/981/
2017-03-21 16:23:10 +03:00
Andrew Dunstan 29bf501683 Add a direct function call mechanism using the caller's context.
The current DirectFunctionCall functions use NULL as the flinfo in
initializing the FunctionCallInfoData for the call. That means the
called function has no fn_mcxt or fn_extra to work with, and attempting
to do so will result in an access violation. These functions instead use
the provided flinfo, which will usually be the caller's own flinfo. The
caller needs to ensure that it doesn't use the fn_extra in way that is
incompatible with the way the called function will use it. The called
function should not rely on anything else in the provided context, as it
will be relevant to the caller, not the callee.

Original code from Tom Lane.

Discussion: https://postgr.es/m/db2b70a4-78d7-294a-a315-8e7f506c5978@2ndQuadrant.com
2017-03-21 08:57:46 -04:00
Peter Eisentraut 010505546a doc: Improve markup 2017-03-21 08:33:46 -04:00
Bruce Momjian 17fa3e834f doc: clarify that function "ownership" that controls permission
It used to say the creation user.

Reported-by: Nathan Wagner
2017-03-20 22:33:26 -04:00
Bruce Momjian 692ed0567d doc: adjust 'Infinity' example to include minus
This clarifies that quoted infinity values must have the negative signs
inside single quotes.

Reported-by: Don Morrison
2017-03-20 22:22:18 -04:00
Andrew Dunstan 8bc40533d6 Add missing support for new node fields
Commit b6fb534f added two new node fields but neglected to add copy and
comparison support for them, Mea culpa, should have checked for that.

per buildfarm animals with -DCOPY_PARSE_PLAN_TREES
2017-03-20 21:57:33 -04:00
Bruce Momjian c709b113e8 doc: improve createdb example
The previous example could error out due to encoding mismatches;  use
-T/template instead.

Reported-by: Jason O'Donnell
2017-03-20 21:23:56 -04:00
Andrew Dunstan aa740b59a6 Force the regression databases to have bytea_output set to hex
Even if the installation defaults to escape output, this makes
pg_regress make the setting hex, so that installcheck tests can pass in
such a setting.

Jeff Janes.

Discussion: http://postgr.es/m/CAMkU=1xivkTNeyCfzhwdHJ+VH5qpU+4gsipRNuEUbyQf+KN3Kw@mail.gmail.com
2017-03-20 18:12:24 -04:00
Andrew Dunstan b6fb534f10 Add IF NOT EXISTS for CREATE SERVER and CREATE USER MAPPING
There is still some inconsistency with the error messages surrounding
foreign servers. Some use the word "foreign" and some don't. My
inclination is to remove all such uses of "foreign" on the basis that
the  CREATE/ALTER/DROP SERVER commands don't use the word. However, that
is left for another day. In this patch I have kept to the existing usage
in the affected commands, which omits "foreign".

Anastasia Lubennikova, reviewed by Arthur Zakirov and Ashtosh Bapat.

Discussion: http://postgr.es/m/7c2ab9b8-388a-1ce0-23a3-7acf2a0ed3c6@postgrespro.ru
2017-03-20 16:40:45 -04:00
Andrew Dunstan 839cb0649a Use a consistent error message style for user mappings.
User mappings are essentially anonymous, so messages referring to "user
mapping foo on server bar" are wrong, and inconsistent with other error
messages referring to user mappings. To be consistent with existing use,
use "user mapping for foo on server bar" instead.

I dropped the noise word "user" from the original suggestion to be
consistent with other uses.

Discussion: http://postgr.es/m/56c6f8ab-b2d6-f1fa-deb0-1d18cf67f7b9@2ndQuadrant.com
2017-03-20 16:01:45 -04:00
Robert Haas 9abbf4727d Another fix for single-page hash index vacuum.
The WAL consistency checking code needed to be updated for the new
page status bit, but that didn't get done previously.

Ashutosh Sharma, reviewed by Amit Kapila

Discussion: http://postgr.es/m/CAA4eK1LP_oz4EfMen14OjJuzN5CqPdfRkFFuA-MfkcfeE8zGyg@mail.gmail.com
2017-03-20 15:55:27 -04:00