Commit Graph

13739 Commits

Author SHA1 Message Date
Tom Lane
c515ff8d0a Doc: add example of type resolution in nested UNIONs.
Section 10.5 didn't say explicitly that multiple UNIONs are resolved
pairwise.  Since the resolution algorithm is described as taking any
number of inputs, readers might well think that a query like
"select x union select y union select z" would be resolved by
considering x, y, and z in one resolution step.  But that's not what
happens (and I think that behavior is per SQL spec).  Add an example
clarifying this point.

Per bug #15129 from Philippe Beaudoin.

Discussion: https://postgr.es/m/152196085023.32649.9916472370480121694@wrigleys.postgresql.org
2018-03-25 16:15:15 -04:00
Peter Eisentraut
bf4a8676c3 pg_resetwal: Allow users to change the WAL segment size
This adds a new option --wal-segsize (analogous to initdb) that changes
the WAL segment size in pg_control.

Author: Nathan Bossart <bossartn@amazon.com>
2018-03-25 14:58:49 -04:00
Peter Eisentraut
8ad8d916f9 initdb: Further polishing of --wal-segsize option
Extend documentation.  Improve option parsing in case no argument was
specified.
2018-03-25 14:58:21 -04:00
Tom Lane
ee4a2c4a03 Doc: remove extra comma in syntax summary for array_fill().
Noted by Scott Ure.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/152199346794.4544.1888397173908716912@wrigleys.postgresql.org
2018-03-25 12:38:21 -04:00
Peter Eisentraut
e22b27f0cb Add long options to pg_resetwal and pg_controldata
We were running out of good single-letter options for some upcoming
pg_resetwal functionality, so add long options to create more
possibilities.  Add to pg_controldata as well for symmetry.

based on patch by Bossart, Nathan <bossartn@amazon.com>
2018-03-24 21:49:53 -04:00
Peter Eisentraut
4644a1170f Improve pg_resetwal documentation
Clarify that the -l option takes a file name, not an "address", and that
that might be different from the LSN if nondefault WAL segment sizes are
used.
2018-03-24 15:38:57 -04:00
Noah Misch
c92f7c6223 Don't qualify type pg_catalog.text in extend-extensions-example.
Extension scripts begin execution with pg_catalog at the front of the
search path, so type names reliably refer to pg_catalog.  Remove these
superfluous qualifications.  Earlier <programlisting> of this <sect1>
already omitted them.  Back-patch to 9.3 (all supported versions).
2018-03-23 20:31:03 -07:00
Teodor Sigaev
8694cc96b5 Exclude unlogged tables from base backups
Exclude unlogged tables from base backup entirely except init fork which marks
created unlogged table. The next question is do not backup temp table but
it's a story for separate patch.

Author: David Steele
Review by: Adam Brightwell, Masahiko Sawada
Discussion: https://www.postgresql.org/message-id/flat/04791bab-cb04-ba43-e9c0-664a4c1ffb2c@pgmasters.net
2018-03-23 19:14:12 +03:00
Alvaro Herrera
86f575948c Allow FOR EACH ROW triggers on partitioned tables
Previously, FOR EACH ROW triggers were not allowed in partitioned
tables.  Now we allow AFTER triggers on them, and on trigger creation we
cascade to create an identical trigger in each partition.  We also clone
the triggers to each partition that is created or attached later.

This means that deferred unique keys are allowed on partitioned tables,
too.

Author: Álvaro Herrera
Reviewed-by: Peter Eisentraut, Simon Riggs, Amit Langote, Robert Haas,
	Thomas Munro
Discussion: https://postgr.es/m/20171229225319.ajltgss2ojkfd3kp@alvherre.pgsql
2018-03-23 10:48:22 -03:00
Tom Lane
7c91a0364f Sync up our various ways of estimating pg_class.reltuples.
VACUUM thought that reltuples represents the total number of tuples in
the relation, while ANALYZE counted only live tuples.  This can cause
"flapping" in the value when background vacuums and analyzes happen
separately.  The planner's use of reltuples essentially assumes that
it's the count of live (visible) tuples, so let's standardize on having
it mean live tuples.

Another issue is that the definition of "live tuple" isn't totally clear;
what should be done with INSERT_IN_PROGRESS or DELETE_IN_PROGRESS tuples?
ANALYZE's choices in this regard are made on the assumption that if the
originating transaction commits at all, it will happen after ANALYZE
finishes, so we should ignore the effects of the in-progress transaction
--- unless it is our own transaction, and then we should count it.
Let's propagate this definition into VACUUM, too.

Likewise propagate this definition into CREATE INDEX, and into
contrib/pgstattuple's pgstattuple_approx() function.

Tomas Vondra, reviewed by Haribabu Kommi, some corrections by me

Discussion: https://postgr.es/m/16db4468-edfa-830a-f921-39a50498e77e@2ndquadrant.com
2018-03-22 15:47:41 -04:00
Robert Haas
f644c3b386 doc: Update parallel join documentation for Parallel Shared Hash.
Thomas Munro

Discussion: http://postgr.es/m/CAEepm=3XdL=+bn3=WQVCCT5wwfAEv-4onKpk+XQZdwDXv6etzA@mail.gmail.com
2018-03-22 13:26:12 -04:00
Robert Haas
e2f1eb0ee3 Implement partition-wise grouping/aggregation.
If the partition keys of input relation are part of the GROUP BY
clause, all the rows belonging to a given group come from a single
partition.  This allows aggregation/grouping over a partitioned
relation to be broken down * into aggregation/grouping on each
partition.  This should be no worse, and often better, than the normal
approach.

If the GROUP BY clause does not contain all the partition keys, we can
still perform partial aggregation for each partition and then finalize
aggregation after appending the partial results.  This is less certain
to be a win, but it's still useful.

Jeevan Chalke, Ashutosh Bapat, Robert Haas.  The larger patch series
of which this patch is a part was also reviewed and tested by Antonin
Houska, Rajkumar Raghuwanshi, David Rowley, Dilip Kumar, Konstantin
Knizhnik, Pascal Legrand, and Rafia Sabih.

Discussion: http://postgr.es/m/CAM2+6=V64_xhstVHie0Rz=KPEQnLJMZt_e314P0jaT_oJ9MR8A@mail.gmail.com
2018-03-22 12:49:48 -04:00
Teodor Sigaev
f67b113ac6 Add \if support to pgbench
Patch adds \if to pgbench as it done for psql. Implementation shares condition
stack code with psql, so, this code is moved to fe_utils directory.

Author: Fabien COELHO with minor editorization by me
Review by: Vik Fearing, Fedor Sigaev
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.20.1711252200190.28523@lancre
2018-03-22 17:42:03 +03:00
Teodor Sigaev
e51a04840a Add general purpose hasing functions to pgbench.
Hashing function is useful for simulating real-world workload in test like
WEB workload, as an example - YCSB benchmarks.

Author: Ildar Musin with minor editorization by me
Reviewed by: Fabien Coelho, me
Discussion: https://www.postgresql.org/message-id/flat/0e8bd39e-dfcd-2879-f88f-272799ad7ef2@postgrespro.ru
2018-03-21 18:01:23 +03:00
Tatsuo Ishii
8bb3c7d347 Fix typo.
Patch by me.
2018-03-21 23:19:46 +09:00
Peter Eisentraut
325f2ec555 Handle heap rewrites even better in logical decoding
Logical decoding should not publish anything about tables created as
part of a heap rewrite during DDL.  Those tables don't exist externally,
so consumers of logical decoding cannot do anything sensible with that
information.  In ab28feae2b, we worked
around this for built-in logical replication, but that was hack.

This is a more proper fix: We mark such transient heaps using the new
field pg_class.relwrite, linking to the original relation OID.  By
default, we ignore them in logical decoding before they get to the
output plugin.  Optionally, a plugin can register their interest in
getting such changes, if they handle DDL specially, in which case the
new field will help them get information about the actual table.

Reviewed-by: Craig Ringer <craig@2ndquadrant.com>
2018-03-21 09:15:04 -04:00
Teodor Sigaev
be8a7a6866 Add strict_word_similarity to pg_trgm module
strict_word_similarity is similar to existing word_similarity function but
it takes into account word boundaries to compute similarity.

Author: Alexander Korotkov
Review by: David Steele, Liudmila Mantrova, me
Discussion: https://www.postgresql.org/message-id/flat/CY4PR17MB13207ED8310F847CF117EED0D85A0@CY4PR17MB1320.namprd17.prod.outlook.com
2018-03-21 14:57:42 +03:00
Teodor Sigaev
aea7c17e86 Rework word_similarity documentation, make it close to actual algorithm.
word_similarity before claimed as returning similarity of closest word in
string, but, actually it returns similarity of substring. Also fix mistyped
comments.

Author: Alexander Korotkov
Review by: David Steele, Liudmila Mantrova
Discussionis:
https://www.postgresql.org/message-id/flat/CY4PR17MB13207ED8310F847CF117EED0D85A0@CY4PR17MB1320.namprd17.prod.outlook.com
https://www.postgresql.org/message-id/flat/f43b242d-000c-f4c8-cb8b-d37e9752cd93%40postgrespro.ru
2018-03-21 14:35:56 +03:00
Peter Eisentraut
d652e3525b doc: Small wording improvement 2018-03-21 07:27:26 -04:00
Andres Freund
6869b4f258 Add C++ support to configure.
This is an optional dependency. It'll be used for the upcoming LLVM
based just in time compilation support, which needs to wrap a few LLVM
C++ APIs so they're accessible from C..

For now test for C++ compilers unconditionally, without failing if not
present, to ensure wide buildfarm coverage. If we're bothered by the
additional test times (which are quite short) or verbosity, we can
later make the tests conditional on --with-llvm.

Author: Andres Freund
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-20 15:48:48 -07:00
Tom Lane
b6cbe9ea1a Doc: typo fix, "PG_" should be "TG_" here.
Too much PG on the brain in commit 769159fd3, evidently.
Noted by marcelhuberfoo@gmail.com.

Discussion: https://postgr.es/m/152154834496.11957.17112112802418832865@wrigleys.postgresql.org
2018-03-20 11:34:25 -04:00
Tom Lane
a467832047 Doc: note that statement-level view triggers require an INSTEAD OF trigger.
If a view lacks an INSTEAD OF trigger, DML on it can only work by rewriting
the command into a command on the underlying base table(s).  Then we will
fire triggers attached to those table(s), not those for the view.  This
seems appropriate from a consistency standpoint, but nowhere was the
behavior explicitly documented, so let's do that.

There was some discussion of throwing an error or warning if a statement
trigger is created on a view without creating a row INSTEAD OF trigger.
But a simple implementation of that would result in dump/restore ordering
hazards.  Given that it's been like this all along, and we hadn't heard
a complaint till now, a documentation improvement seems sufficient.

Per bug #15106 from Pu Qun.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/152083391168.1215.16892140713507052796@wrigleys.postgresql.org
2018-03-18 15:10:28 -04:00
Peter Eisentraut
e3bdb2d926 Set libpq sslcompression to off by default
Since SSL compression is no longer recommended, turn the default in
libpq from on to off.

OpenSSL 1.1.0 and many distribution packages already turn compression
off by default, so such a server won't accept compression anyway.  So
this will mainly affect users of older OpenSSL installations.

Also update the documentation to make clear that this setting is no
longer recommended.

Discussion: https://www.postgresql.org/message-id/flat/595cf3b1-4ffe-7f05-6f72-f72b7afa7993%402ndquadrant.com
2018-03-17 09:17:33 -04:00
Peter Eisentraut
8a3d942529 Add ssl_passphrase_command setting
This allows specifying an external command for prompting for or
otherwise obtaining passphrases for SSL key files.  This is useful
because in many cases there is no TTY easily available during service
startup.

Also add a setting ssl_passphrase_command_supports_reload, which allows
supporting SSL configuration reload even if SSL files need passphrases.

Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2018-03-17 08:28:51 -04:00
Tom Lane
013c0baadd Doc: explicitly point out that enum values can't be dropped.
This was not stated in so many words anywhere.  Document it to make
clear that it's a design limitation and not just an oversight or
documentation omission.

Discussion: https://postgr.es/m/152089733343.1222.6927268289645380498@wrigleys.postgresql.org
2018-03-16 13:44:43 -04:00
Peter Eisentraut
f66e8bf875 Remove pg_class.relhaspkey
It is not used for anything internally, and it cannot be relied on for
external uses, so it can just be removed.  To correct recommended way to
check for a primary key is in pg_index.

Discussion: https://www.postgresql.org/message-id/flat/b1a24c6c-6913-f89c-674e-0704f0ed69db@2ndquadrant.com
2018-03-14 15:31:34 -04:00
Peter Eisentraut
33803f67f1 Support INOUT arguments in procedures
In a top-level CALL, the values of INOUT arguments will be returned as a
result row.  In PL/pgSQL, the values are assigned back to the input
arguments.  In other languages, the same convention as for return a
record from a function is used.  That does not require any code changes
in the PL implementations.

Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
2018-03-14 12:07:28 -04:00
Alvaro Herrera
484a4a08ab Log when a BRIN autosummarization request fails
Autovacuum's 'workitem' request queue is of limited size, so requests
can fail if they arrive more quickly than autovacuum can process them.
Emit a log message when this happens, to provide better visibility of
this.

Backpatch to 10.  While this represents an API change for
AutoVacuumRequestWork, that function is not yet prepared to deal with
external modules calling it, so there doesn't seem to be any risk (other
than log spam, that is.)

Author: Masahiko Sawada
Reviewed-by: Fabrízio Mello, Ildar Musin, Álvaro Herrera
Discussion: https://postgr.es/m/CAD21AoB1HrQhp6_4rTyHN5kWEJCEsG8YzsjZNt-ctoXSn5Uisw@mail.gmail.com
2018-03-14 11:59:40 -03:00
Alvaro Herrera
63cbee6a78 doc: Reword restriction on partition keys in unique indexes
New wording from David G. Johnston, who noticed the unreadable original
also.  Include his suggested test case as well.

Fix a typo I noticed elsewhere while doing this.

Discussion: https://postgr.es/m/CAKFQuwY4Ld7ecxL_KAmaxwt0FUu5VcPPN2L4dh+3BeYbrdBa5g@mail.gmail.com
2018-03-12 13:32:28 -03:00
Alvaro Herrera
3beb46ae81 docs: Fix typo: a -> an
David Rowley
2018-03-12 12:58:35 -03:00
Alvaro Herrera
c4af6960e3 Remove doc sentence no longer applicable
Amit Langote
2018-03-12 11:38:40 -03:00
Tom Lane
4e0c743c18 Fix cross-checking of ReservedBackends/max_wal_senders/MaxConnections.
We were independently checking ReservedBackends < MaxConnections and
max_wal_senders < MaxConnections, but because walsenders aren't allowed
to use superuser-reserved connections, that's really the wrong thing.
Correct behavior is to insist on ReservedBackends + max_wal_senders being
less than MaxConnections.  Fix the code and associated documentation.

This has been wrong for a long time, but since the situation probably
hardly ever arises in the field (especially pre-v10, when the default
for max_wal_senders was zero), no back-patch.

Discussion: https://postgr.es/m/28271.1520195491@sss.pgh.pa.us
2018-03-08 11:25:26 -05:00
Peter Eisentraut
d40c7cd004 doc: Add more substructure to SSL documentation
The SSL documentation text has gotten a bit long, so add some
subsections and reorder for better flow.
2018-03-07 11:32:51 -05:00
Peter Eisentraut
a3739e376f doc: Improve calculation of vm.nr_hugepages
The previous method worked off the full virtual address space, not just
the shared memory usage.

Author: Tsunakawa, Takayuki <tsunakawa.takay@jp.fujitsu.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Vasundhar Boddapati <bvasundhar@gmail.com>
2018-03-06 21:45:28 -05:00
Peter Eisentraut
0c2c81b403 doc: Add replication parameter to libpq documentation
Author: Michael Paquier <michael@paquier.xyz>
Reported-by: Şahap Aşçı <sahapasci@gmail.com>
Reviewed-by: Vik Fearing <vik.fearing@2ndquadrant.com>
2018-03-06 21:00:10 -05:00
Peter Eisentraut
4c831aeaa7 Tests for Kerberos/GSSAPI authentication
Like the LDAP and SSL tests, these are not run by default but can be
selected via PG_TEST_EXTRA.

Reviewed-by: Thomas Munro <thomas.munro@enterprisedb.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
2018-03-06 10:57:36 -05:00
Andres Freund
854dd8cff5 Add parenthesized options syntax for ANALYZE.
This is analogous to the syntax allowed for VACUUM. This allows us to
avoid making new options reserved keywords and makes it easier to
allow arbitrary argument order. Oh, and it's consistent with the other
commands, too.

Author: Nathan Bossart
Reviewed-By: Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/D3FC73E2-9B1A-4DB4-8180-55F57D116B4E@amazon.com
2018-03-05 16:21:05 -08:00
Alvaro Herrera
5564c11815 Clone extended stats in CREATE TABLE (LIKE INCLUDING ALL)
The LIKE INCLUDING ALL clause to CREATE TABLE intuitively indicates
cloning of extended statistics on the source table, but it failed to do
so.  Patch it up so that it does.  Also include an INCLUDING STATISTICS
option to the LIKE clause, so that the behavior can be requested
individually, or excluded individually.

While at it, reorder the INCLUDING options, both in code and in docs, in
alphabetical order which makes more sense than feature-implementation
order that was previously used.

Backpatch this to Postgres 10, where extended statistics were
introduced, because this is seen as an oversight in a fresh feature
which is better to get consistent from the get-go instead of changing
only in pg11.

In pg11, comments on statistics objects are cloned too.  In pg10 they
are not, because I (Álvaro) was too coward to change the parse node as
required to support it.  Also, in pg10 I chose not to renumber the
parser symbols for the various INCLUDING options in LIKE, for the same
reason.  Any corresponding user-visible changes (docs) are backpatched,
though.

Reported-by: Stephen Froehlich
Author: David Rowley
Reviewed-by: Álvaro Herrera, Tomas Vondra
Discussion: https://postgr.es/m/CY1PR0601MB1927315B45667A1B679D0FD5E5EF0@CY1PR0601MB1927.namprd06.prod.outlook.com
2018-03-05 19:37:19 -03:00
Peter Eisentraut
dd9ed0bf70 doc: Tiny whitespace fix 2018-03-05 11:27:08 -05:00
Peter Eisentraut
7726147f53 doc: Small wording improvement
Replace "checkpoint segment" with "WAL segment".

Reported-by: Maksim Milyutin <milyutinma@gmail.com>
2018-03-03 14:23:13 -05:00
Peter Eisentraut
e568eed2fc doc: Fix links to pg_stat_replication
In PostgreSQL 9.5, the documentation for pg_stat_replication was moved,
so some of the links pointed to an appropriate location.

Author: Maksim Milyutin <milyutinma@gmail.com>
2018-03-03 14:16:39 -05:00
Peter Eisentraut
506652bcae doc: Improve wording 2018-03-03 09:56:17 -05:00
Peter Eisentraut
fdb34824e0 Add PG_TEST_EXTRA to control optional test suites
The SSL and LDAP test suites are not run by default, as they are not
secure for multi-user environments.  This commit adds an extra make
variable to optionally enable them, for example:

make check-world PG_TEST_EXTRA='ldap ssl'

Author: Michael Paquier <michael@paquier.xyz>
2018-03-03 01:40:48 -05:00
Peter Eisentraut
fd1a421fe6 Add prokind column, replacing proisagg and proiswindow
The new column distinguishes normal functions, procedures, aggregates,
and window functions.  This replaces the existing columns proisagg and
proiswindow, and replaces the convention that procedures are indicated
by prorettype == 0.  Also change prorettype to be VOIDOID for procedures.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
2018-03-02 13:48:33 -05:00
Andres Freund
9c4968469a doc: mention PROVE_TESTS in section of TAP tests.
Author: Michael Paquier
Discussion: https://postgr.es/m/20180217140305.GB31338@paquier.xyz
2018-03-01 01:50:27 -08:00
Andres Freund
a88609089a doc: Add WaitForBackgroundWorkerShutdown() to bgw docs.
Commit 924bcf4f16 added WaitForBackgroundWorkerShutdown, but didn't
add it to the documentation. Fix that and two small spelling errors in
the WaitForBackgroundWorkerStartup paragraph.

Author: Daniel Gustafsson
Discussion: https://postgr.es/m/C8738949-0350-4999-A1DA-26E209FF248D@yesql.se
2018-03-01 01:46:04 -08:00
Andres Freund
8c438fcc9f doc: Add random_zipfian to list of random functions with argument.
Author: Ildar Musin
Reviewed-By: Fabian Coelho
Discussion: https://postgr.es/m/6376ed81-3ce8-14f4-4758-099872f4ce7d@postgrespro.ru
2018-03-01 01:40:00 -08:00
Tom Lane
d3b851e9a3 Doc: remove duplicate poly_ops row from SP-GiST opclass table.
Commit ff963b393 added two identical copies of this row.

Dagfinn Ilmari Mannsåker

Discussion: https://postgr.es/m/d8j8tdevb7x.fsf@dalvik.ping.uio.no
2018-02-28 18:54:57 -05:00
Robert Haas
73797b7884 Document LWTRANCHE_PARALLEL_HASH_JOIN.
Thomas Munro

Discussion: http://postgr.es/m/CAEepm=3g1hhbFzYkR_QT9RmBvsGX4UaeCtX-4Js8OOEMmFeaSQ@mail.gmail.com
2018-02-28 11:46:26 -05:00
Peter Eisentraut
6d933da306 doc: Improve man build speed
Turn off man.endnotes.are.numbered parameter, which we don't need, but
which increases performance vastly if off.  Also turn on
man.output.quietly, which also makes things a bit faster, but which is
also less useful now as a progress indicator because the build is so
fast now.
2018-02-28 09:26:36 -05:00
Peter Eisentraut
d21ddc220f Fix warnings in man page build
The changes in the CREATE POLICY man page from commit
87c2a17fee triggered a stylesheet bug that
created some warning messages and incorrect output.  This installs a
workaround.

Also improve the whitespace a bit so it looks better.
2018-02-28 08:22:51 -05:00
Robert Haas
6614aaa699 doc: Fix grammar.
Michael Paquier

Discussion: http://postgr.es/m/20180209135327.GC29003@paquier.xyz
2018-02-27 14:41:10 -05:00
Tom Lane
8af3855699 Last-minute updates for release notes.
Security: CVE-2018-1058
2018-02-26 12:14:27 -05:00
Noah Misch
5770172cb0 Document security implications of search_path and the public schema.
The ability to create like-named objects in different schemas opens up
the potential for users to change the behavior of other users' queries,
maliciously or accidentally.  When you connect to a PostgreSQL server,
you should remove from your search_path any schema for which a user
other than yourself or superusers holds the CREATE privilege.  If you do
not, other users holding CREATE privilege can redefine the behavior of
your commands, causing them to perform arbitrary SQL statements under
your identity.  "SET search_path = ..." and "SELECT
pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one
can use either as the first command of a session.  As special
exceptions, the following client applications behave as documented
regardless of search_path settings and schema privileges: clusterdb
createdb createlang createuser dropdb droplang dropuser ecpg (not
programs it generates) initdb oid2name pg_archivecleanup pg_basebackup
pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready
pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby
pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb
vacuumlo.  Not included are core client programs that run user-specified
SQL commands, namely psql and pgbench.  PostgreSQL encourages non-core
client applications to do likewise.

Document this in the context of libpq connections, psql connections,
dblink connections, ECPG connections, extension packaging, and schema
usage patterns.  The principal defense for applications is "SELECT
pg_catalog.set_config('search_path', '', false)", and the principal
defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC".
Either one is sufficient to prevent attack.  After a REVOKE, consider
auditing the public schema for objects named like pg_catalog objects.

Authors of SECURITY DEFINER functions use some of the same defenses, and
the CREATE FUNCTION reference page already covered them thoroughly.
This is a good opportunity to audit SECURITY DEFINER functions for
robust security practice.

Back-patch to 9.3 (all supported versions).

Reviewed by Michael Paquier and Jonathan S. Katz.  Reported by Arseniy
Sharoglazov.

Security: CVE-2018-1058
2018-02-26 07:39:44 -08:00
Tom Lane
1316417bba Release notes for 10.3, 9.6.8, 9.5.12, 9.4.17, 9.3.22. 2018-02-25 14:52:51 -05:00
Tom Lane
eec1a8cb6c First-draft release notes for 10.3. 2018-02-23 17:20:26 -05:00
Noah Misch
fe35cea7cf Synchronize doc/ copies of src/test/examples/.
This is mostly cosmetic, but it might fix build failures, on some
platform, when copying from the documentation.

Back-patch to 9.3 (all supported versions).
2018-02-23 11:24:04 -08:00
Peter Eisentraut
0db2fc98cd Update gratuitous use of MD5 in documentation
It seems some people are bothered by the outdated MD5 appearing in
example code.  So replace it with more modern alternatives or by
a different example function.

Reported-by: Jon Wolski <jonwolski@gmail.com>
2018-02-22 11:34:54 -05:00
Peter Eisentraut
10cfce34c0 Add user-callable SHA-2 functions
Add the user-callable functions sha224, sha256, sha384, sha512.  We
already had these in the C code to support SCRAM, but there was no test
coverage outside of the SCRAM tests.  Adding these as user-callable
functions allows writing some tests.  Also, we have a user-callable md5
function but no more modern alternative, which led to wide use of md5 as
a general-purpose hash function, which leads to occasional complaints
about using md5.

Also mark the existing md5 functions as leak-proof.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
2018-02-22 11:34:53 -05:00
Alvaro Herrera
9a89f6d854 Adjust ALTER TABLE docs on partitioned constraints
Move the "additional restrictions" comment to ALTER TABLE ADD
CONSTRAINT instead of ADD CONSTRAINT USING INDEX; and in the latter
instead indicate that partitioned tables are unsupported

Noted by David G. Johnston
Discussion: https://postgr.es/m/CAKFQuwY4Ld7ecxL_KAmaxwt0FUu5VcPPN2L4dh+3BeYbrdBa5g@mail.gmail.com
2018-02-20 12:08:55 -03:00
Alvaro Herrera
eb7ed3f306 Allow UNIQUE indexes on partitioned tables
If we restrict unique constraints on partitioned tables so that they
must always include the partition key, then our standard approach to
unique indexes already works --- each unique key is forced to exist
within a single partition, so enforcing the unique restriction in each
index individually is enough to have it enforced globally.  Therefore we
can implement unique indexes on partitions by simply removing a few
restrictions (and adding others.)

Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql
Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql
Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime
	Casanova, Amit Langote
2018-02-19 17:40:00 -03:00
Peter Eisentraut
2fb1abaeb0 Rename enable_partition_wise_join to enable_partitionwise_join
Discussion: https://www.postgresql.org/message-id/flat/ad24e4f4-6481-066e-e3fb-6ef4a3121882%402ndquadrant.com
2018-02-16 10:33:59 -05:00
Tom Lane
439c7bc1a0 Doc: fix minor bug in CREATE TABLE example.
One example in create_table.sgml claimed to be showing table constraint
syntax, but it was really column constraint syntax due to the omission
of a comma.  This is both wrong and confusing, so fix it in all
supported branches.

Per report from neil@postgrescompare.com.

Discussion: https://postgr.es/m/151871659877.1393.2431103178451978795@wrigleys.postgresql.org
2018-02-15 13:56:38 -05:00
Tom Lane
4b93f57999 Make plpgsql use its DTYPE_REC code paths for composite-type variables.
Formerly, DTYPE_REC was used only for variables declared as "record";
variables of named composite types used DTYPE_ROW, which is faster for
some purposes but much less flexible.  In particular, the ROW code paths
are entirely incapable of dealing with DDL-caused changes to the number
or data types of the columns of a row variable, once a particular plpgsql
function has been parsed for the first time in a session.  And, since the
stored representation of a ROW isn't a tuple, there wasn't any easy way
to deal with variables of domain-over-composite types, since the domain
constraint checking code would expect the value to be checked to be a
tuple.  A lesser, but still real, annoyance is that ROW format cannot
represent a true NULL composite value, only a row of per-field NULL
values, which is not exactly the same thing.

Hence, switch to using DTYPE_REC for all composite-typed variables,
whether "record", named composite type, or domain over named composite
type.  DTYPE_ROW remains but is used only for its native purpose, to
represent a fixed-at-compile-time list of variables, for instance the
targets of an INTO clause.

To accomplish this without taking significant performance losses, introduce
infrastructure that allows storing composite-type variables as "expanded
objects", similar to the "expanded array" infrastructure introduced in
commit 1dc5ebc90.  A composite variable's value is thereby kept (most of
the time) in the form of separate Datums, so that field accesses and
updates are not much more expensive than they were in the ROW format.
This holds the line, more or less, on performance of variables of named
composite types in field-access-intensive microbenchmarks, and makes
variables declared "record" perform much better than before in similar
tests.  In addition, the logic involved with enforcing composite-domain
constraints against updates of individual fields is in the expanded
record infrastructure not plpgsql proper, so that it might be reusable
for other purposes.

In further support of this, introduce a typcache feature for assigning a
unique-within-process identifier to each distinct tuple descriptor of
interest; in particular, DDL alterations on composite types result in a new
identifier for that type.  This allows very cheap detection of the need to
refresh tupdesc-dependent data.  This improves on the "tupDescSeqNo" idea
I had in commit 687f096ea: that assigned identifying sequence numbers to
successive versions of individual composite types, but the numbers were not
unique across different types, nor was there support for assigning numbers
to registered record types.

In passing, allow plpgsql functions to accept as well as return type
"record".  There was no good reason for the old restriction, and it
was out of step with most of the other PLs.

Tom Lane, reviewed by Pavel Stehule

Discussion: https://postgr.es/m/8962.1514399547@sss.pgh.pa.us
2018-02-13 18:52:21 -05:00
Peter Eisentraut
2ac3e6acc2 doc: pg_function_is_visible also applies to aggregates and procedures 2018-02-13 15:13:44 -05:00
Peter Eisentraut
7a32ac8a66 Add procedure support to pg_get_functiondef
This also makes procedures work in psql's \ef and \sf commands.

Reported-by: Pavel Stehule <pavel.stehule@gmail.com>
2018-02-13 15:13:44 -05:00
Alvaro Herrera
80f021ef13 Add missing article
Noticed while reviewing nearby text
2018-02-12 11:39:25 -03:00
Alvaro Herrera
fad15f4a54 Mention partitioned indexes in "Data Definition" chapter
We can now create indexes more easily than before, so update this
chapter to use the simpler instructions.

After an idea of Amit Langote.  I (Álvaro) opted to do more invasive
surgery and remove the previous suggestion to create per-partition
indexes, which his patch left in place.

Discussion: https://postgr.es/m/eafaaeb1-f0fd-d010-dd45-07db0300f645@lab.ntt.co.jp
Author: Amit Langote, Álvaro Herrera
2018-02-10 10:06:01 -03:00
Peter Eisentraut
32ff269117 Add more information_schema columns
- table_constraints.enforced
- triggers.action_order
- triggers.action_reference_old_table
- triggers.action_reference_new_table

Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2018-02-07 10:08:02 -05:00
Magnus Hagander
9e03901550 Change default git repo URL to https
Since we now support the server side handler for git over https (so
we're no longer using the "dumb protocol"), make https the primary
choice for cloning the repository, and the git protocol the secondary
choice.

In passing, also change the links to git-scm.com from http to https.

Reviewed by Stefan Kaltenbrunner and David G. Johnston
2018-02-07 11:00:26 +01:00
Tom Lane
0a459cec96 Support all SQL:2011 options for window frame clauses.
This patch adds the ability to use "RANGE offset PRECEDING/FOLLOWING"
frame boundaries in window functions.  We'd punted on that back in the
original patch to add window functions, because it was not clear how to
do it in a reasonably data-type-extensible fashion.  That problem is
resolved here by adding the ability for btree operator classes to provide
an "in_range" support function that defines how to add or subtract the
RANGE offset value.  Factoring it this way also allows the operator class
to avoid overflow problems near the ends of the datatype's range, if it
wishes to expend effort on that.  (In the committed patch, the integer
opclasses handle that issue, but it did not seem worth the trouble to
avoid overflow failures for datetime types.)

The patch includes in_range support for the integer_ops opfamily
(int2/int4/int8) as well as the standard datetime types.  Support for
other numeric types has been requested, but that seems like suitable
material for a follow-on patch.

In addition, the patch adds GROUPS mode which counts the offset in
ORDER-BY peer groups rather than rows, and it adds the frame_exclusion
options specified by SQL:2011.  As far as I can see, we are now fully
up to spec on window framing options.

Existing behaviors remain unchanged, except that I changed the errcode
for a couple of existing error reports to meet the SQL spec's expectation
that negative "offset" values should be reported as SQLSTATE 22013.

Internally and in relevant parts of the documentation, we now consistently
use the terminology "offset PRECEDING/FOLLOWING" rather than "value
PRECEDING/FOLLOWING", since the term "value" is confusingly vague.

Oliver Ford, reviewed and whacked around some by me

Discussion: https://postgr.es/m/CAGMVOdu9sivPAxbNN0X+q19Sfv9edEPv=HibOJhB14TJv_RCQg@mail.gmail.com
2018-02-07 00:06:56 -05:00
Tom Lane
3785f7eee3 Doc: move info for btree opclass implementors into main documentation.
Up to now, useful info for writing a new btree opclass has been buried
in the backend's nbtree/README file.  Let's move it into the SGML docs,
in preparation for extending it with info about "in_range" functions
in the upcoming window RANGE patch.

To do this, I chose to create a new chapter for btree indexes in Part VII
(Internals), parallel to the chapters that exist for the newer index AMs.
This is a pretty short chapter as-is.  At some point somebody might care
to flesh it out with more detail about btree internals, but that is
beyond the scope of my ambition for today.

Discussion: https://postgr.es/m/23141.1517874668@sss.pgh.pa.us
2018-02-06 13:52:27 -05:00
Tom Lane
1eb5d43bee Last-minute updates for release notes.
Security: CVE-2018-1052, CVE-2018-1053
2018-02-05 14:43:40 -05:00
Peter Eisentraut
ad14919ac9 doc: Update mentions of MD5 in the documentation
Reported-by: Shay Rojansky <roji@roji.org>
2018-02-04 16:38:08 -05:00
Tom Lane
cf1cba3110 Release notes for 10.2, 9.6.7, 9.5.11, 9.4.16, 9.3.21. 2018-02-04 15:13:44 -05:00
Tom Lane
64fb645914 Doc: minor clarifications in xindex.sgml.
I noticed some slightly confusing or out-of-date verbiage here
while working on the window RANGE patch.  Seems worth committing
separately.
2018-02-04 11:46:28 -05:00
Peter Eisentraut
4ac583f36a doc: Fix name in release notes
Author: Alexander Lakhin <exclusion@gmail.com>
2018-02-03 11:09:24 -05:00
Peter Eisentraut
1d81c093db doc: Clarify psql --list documentation a bit more 2018-02-03 10:19:57 -05:00
Tom Lane
794eb3a8f0 Minor copy-editing for 10.2 release notes.
Second pass after taking a break ...
2018-02-02 22:33:47 -05:00
Peter Eisentraut
bc38bdba04 doc: Fix index link
The index entry was pointing to a slightly wrong location.
2018-02-02 21:10:59 -05:00
Tom Lane
bf641d3376 First-draft release notes for 10.2.
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
2018-02-02 17:10:46 -05:00
Robert Haas
9da0cc3528 Support parallel btree index builds.
To make this work, tuplesort.c and logtape.c must also support
parallelism, so this patch adds that infrastructure and then applies
it to the particular case of parallel btree index builds.  Testing
to date shows that this can often be 2-3x faster than a serial
index build.

The model for deciding how many workers to use is fairly primitive
at present, but it's better than not having the feature.  We can
refine it as we get more experience.

Peter Geoghegan with some help from Rushabh Lathia.  While Heikki
Linnakangas is not an author of this patch, he wrote other patches
without which this feature would not have been possible, and
therefore the release notes should possibly credit him as an author
of this feature.  Reviewed by Claudio Freire, Heikki Linnakangas,
Thomas Munro, Tels, Amit Kapila, me.

Discussion: http://postgr.es/m/CAM3SWZQKM=Pzc=CAHzRixKjp2eO5Q0Jg1SoFQqeXFQ647JiwqQ@mail.gmail.com
Discussion: http://postgr.es/m/CAH2-Wz=AxWqDoVvGU7dq856S4r6sJAj6DBn7VMtigkB33N5eyg@mail.gmail.com
2018-02-02 13:32:44 -05:00
Stephen Frost
a2a2205761 Improve ALTER TABLE synopsis
Add into the ALTER TABLE synopsis the definition of
partition_bound_spec, column_constraint, index_parameters and
exclude_element.

Initial patch by Lætitia Avrot, with further improvements by Amit
Langote and Thomas Munro.

Discussion: https://postgr.es/m/flat/27ec4df3-d1ab-3411-f87f-647f944897e1%40lab.ntt.co.jp
2018-02-02 05:30:04 -05:00
Bruce Momjian
eab30cc6b5 doc: fix trigger inheritance wording
Fix wording from commit 1cf1112990

Reported-by: Robert Haas

Backpatch-through: 10
2018-01-31 17:52:47 -05:00
Bruce Momjian
59ad246350 doc: clarify major/minor pg_upgrade versions with examples
The previous docs added in PG 10 were not clear enough for someone who
didn't understand the PG 10 version change, so give more specific
examples.

Reported-by: jim@room118solutions.com

Discussion: https://postgr.es/m/20171218213041.25744.8414@wrigleys.postgresql.org

Backpatch-through: 10
2018-01-31 17:09:59 -05:00
Bruce Momjian
1cf1112990 doc: clearify trigger behavior for inheritance
The previous wording added in PG 10 wasn't specific enough about the
behavior of statement and row triggers when using inheritance.

Reported-by: ian@thepathcentral.com

Discussion: https://postgr.es/m/20171129193934.27108.30796@wrigleys.postgresql.org

Backpatch-through: 10
2018-01-31 17:00:17 -05:00
Bruce Momjian
3b15255912 doc: in contrib-spi, mention and link to the meaning of SPI
Also remove outdated comment about SPI subtransactions.

Reported-by: gregory@arenius.com

Discussion: https://postgr.es/m/151726276676.1240.10501743959198501067@wrigleys.postgresql.org

Backpatch-through: 9.3
2018-01-31 16:54:33 -05:00
Bruce Momjian
de71541460 doc: Improve pg_upgrade rsync examples to use clusterdir
Commit 9521ce4a7a from Sep 13, 2017 and
backpatched through 9.5 used rsync examples with datadir.  The reporter
has pointed out, and testing has verified, that clusterdir must be used,
so update the docs accordingly.

Reported-by: Don Seiler

Discussion: https://postgr.es/m/CAHJZqBD0u9dCERpYzK6BkRv=663AmH==DFJpVC=M4Xg_rq2=CQ@mail.gmail.com

Backpatch-through: 9.5
2018-01-31 16:43:40 -05:00
Robert Haas
d40d97d6c7 pgcrypto's encrypt() supports AES-128, AES-192, and AES-256
Previously, only 128 was mentioned, but the others are also supported.

Thomas Munro, reviewed by Michael Paquier and extended a bit by me.

Discussion: http://postgr.es/m/CAEepm=1XbBHXYJKofGjnM2Qfz-ZBVqhGU4AqvtgR+Hegy4fdKg@mail.gmail.com
2018-01-31 16:33:11 -05:00
Bruce Momjian
e5dede9097 doc: mention datadir locations are actually config locations
Technically, pg_upgrade's --old-datadir and --new-datadir are
configuration directories, not necessarily data directories.  This is
reflected in the 'postgres' manual page, so do the same for pg_upgrade.

Reported-by: Yves Goergen

Bug: 14898

Discussion: https://postgr.es/m/20171110220912.31513.13322@wrigleys.postgresql.org

Backpatch-through: 10
2018-01-31 16:25:21 -05:00
Peter Eisentraut
38d485fdaa Fix up references to scram-sha-256
pg_hba_file_rules erroneously reported this as scram-sha256.  Fix that.

To avoid future errors and confusion, also adjust documentation links
and internal symbols to have a separator between "sha" and "256".

Reported-by: Christophe Courtois <christophe.courtois@dalibo.com>
Author: Michael Paquier <michael.paquier@gmail.com>
2018-01-30 16:50:30 -05:00
Peter Eisentraut
1e1e599d66 doc: Clarify pg_upgrade documentation
Clarify that the restriction against reg* types only applies to table
columns using these types, not to the type appearing in any other way,
for example as a function argument.
2018-01-29 14:26:17 -05:00
Magnus Hagander
ba8c2dfffd Add missing semicolons in documentation examples
Author: Daniel Gustafsson <daniel@yesql.se>
2018-01-27 13:13:52 +01:00
Tom Lane
fb8697b31a Avoid unnecessary use of pg_strcasecmp for already-downcased identifiers.
We have a lot of code in which option names, which from the user's
viewpoint are logically keywords, are passed through the grammar as plain
identifiers, and then matched to string literals during command execution.
This approach avoids making words into lexer keywords unnecessarily.  Some
places matched these strings using plain strcmp, some using pg_strcasecmp.
But the latter should be unnecessary since identifiers would have been
downcased on their way through the parser.  Aside from any efficiency
concerns (probably not a big factor), the lack of consistency in this area
creates a hazard of subtle bugs due to different places coming to different
conclusions about whether two option names are the same or different.
Hence, standardize on using strcmp() to match any option names that are
expected to have been fed through the parser.

This does create a user-visible behavioral change, which is that while
formerly all of these would work:
	alter table foo set (fillfactor = 50);
	alter table foo set (FillFactor = 50);
	alter table foo set ("fillfactor" = 50);
	alter table foo set ("FillFactor" = 50);
now the last case will fail because that double-quoted identifier is
different from the others.  However, none of our documentation says that
you can use a quoted identifier in such contexts at all, and we should
discourage doing so since it would break if we ever decide to parse such
constructs as true lexer keywords rather than poor man's substitutes.
So this shouldn't create a significant compatibility issue for users.

Daniel Gustafsson, reviewed by Michael Paquier, small changes by me

Discussion: https://postgr.es/m/29405B24-564E-476B-98C0-677A29805B84@yesql.se
2018-01-26 18:25:14 -05:00
Tom Lane
4971d2a322 Remove the obsolete WITH clause of CREATE FUNCTION.
This clause was superseded by SQL-standard syntax back in 7.3.
We've kept it around for backwards-compatibility purposes ever since;
but 15 years seems like long enough for that, especially seeing that
there are undocumented weirdnesses in how it interacts with the
SQL-standard syntax for specifying the same options.

Michael Paquier, per an observation by Daniel Gustafsson;
some small cosmetic adjustments to nearby code by me.

Discussion: https://postgr.es/m/20180115022748.GB1724@paquier.xyz
2018-01-26 12:25:44 -05:00
Tom Lane
1368e92e16 Support --no-comments in pg_dump, pg_dumpall, pg_restore.
We have switches already to suppress other subsidiary object properties,
such as ACLs, security labels, ownership, and tablespaces, so just on
the grounds of symmetry we should allow suppressing comments as well.
Also, commit 0d4e6ed30 added a positive reason to have this feature,
i.e. to allow obtaining the old behavior of selective pg_restore should
anyone desire that.

Recent commits have removed the cases where pg_dump emitted comments on
built-in objects that the restoring user might not have privileges to
comment on, so the original primary motivation for this feature is gone,
but it still seems at least somewhat useful in its own right.

Robins Tharakan, reviewed by Fabrízio Mello

Discussion: https://postgr.es/m/CAEP4nAx22Z4ch74oJGzr5RyyjcyUSbpiFLyeYXX8pehfou92ug@mail.gmail.com
2018-01-25 15:27:24 -05:00
Tom Lane
0d4e6ed308 Clean up some aspects of pg_dump/pg_restore item-selection logic.
Ensure that CREATE DATABASE and related commands are issued when, and
only when, --create is specified.  Previously there were scenarios
where using selective-dump switches would prevent --create from having
any effect.  For example, it would fail to do anything in pg_restore
if the archive file had been made by a selective dump, because there
would be no TOC entry for the database.

Since we don't issue \connect either if we don't issue CREATE DATABASE,
this could result in unexpectedly restoring objects into the wrong
database.

Also fix pg_restore's selective restore logic so that when an object is
selected to be restored, we also restore its ACL, comment, and security
label if any.  Previously there was no way to get the latter properties
except through tedious mucking about with a -L file.  If, for some
reason, you don't want these properties, you can match the old behavior
by adding --no-acl etc.

While at it, try to make _tocEntryRequired() a little better organized
and better documented.

Discussion: https://postgr.es/m/32668.1516848577@sss.pgh.pa.us
2018-01-25 14:26:15 -05:00
Peter Eisentraut
2a5ecb56d2 Update documentation to mention huge pages on other OSes
Previously, the docs implied that only Linux and Windows could use huge
pages.  That's not quite true: it's just that we only know how to
request them explicitly on those OSes.  Be more explicit about what
huge_pages really does and mention that some OSes may use huge pages
automatically.

Author: Thomas Munro and Catalin Iacob
Reviewed-By: Justin Pryzby, Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm=3qzR-hfjepymohuC4XO5phxoSoipOjm6BEhnJHjNR+jg@mail.gmail.com
2018-01-25 11:15:03 -05:00
Bruce Momjian
d6ab720360 doc: properly indent CREATE TRIGGER paragraph
This was done to match the surrounding indentation.  Text added in PG
10.

Backpatch-through: 10
2018-01-24 15:13:04 -05:00
Bruce Momjian
5b2a8cf96f doc: clarify use of RegisterDynamicBackgroundWorker
Document likely use of RegisterDynamicBackgroundWorker by another
background worker.

Reported-by: Chapman Flack

Discussion: https://postgr.es/m/CAB7nPqTdi=J9HH8PPPiEOohebdd+xkgbbhdY7=VbGnZ3CkZXxA@mail.gmail.com

Author: Chapman Flack
2018-01-24 13:20:37 -05:00