Commit Graph

453 Commits

Author SHA1 Message Date
Bruce Momjian a0efc71453 pg_upgrade: call 'postgres' binary to get data directory location
This matches the binary 'pg_ctl' calls.  Previously we called the
'postmaster'.

Report by Christoph Berg
2015-04-01 18:25:45 -04:00
Andrew Dunstan fa1e5afa8a Run pg_upgrade and pg_resetxlog with restricted token on Windows
As with initdb these programs need to run with a restricted token, and
if they don't pg_upgrade will fail when run as a user with Adminstrator
privileges.

Backpatch to all live branches. On the development branch the code is
reorganized so that the restricted token code is now in a single
location. On the stable bramches a less invasive change is made by
simply copying the relevant code to pg_upgrade.c and pg_resetxlog.c.

Patches and bug report from Muhammad Asif Naeem, reviewed by Michael
Paquier, slightly edited by me.
2015-03-30 17:07:52 -04:00
Tom Lane 785941cdc3 Tweak __attribute__-wrapping macros for better pgindent results.
This improves on commit bbfd7edae5 by
making two simple changes:

* pg_attribute_noreturn now takes parentheses, ie pg_attribute_noreturn().
Likewise pg_attribute_unused(), pg_attribute_packed().  This reduces
pgindent's tendency to misformat declarations involving them.

* attributes are now always attached to function declarations, not
definitions.  Previously some places were taking creative shortcuts,
which were not merely candidates for bad misformatting by pgindent
but often were outright wrong anyway.  (It does little good to put a
noreturn annotation where callers can't see it.)  In any case, if
we would like to believe that these macros can be used with non-gcc
compilers, we should avoid gratuitous variance in usage patterns.

I also went through and manually improved the formatting of a lot of
declarations, and got rid of excessively repetitive (and now obsolete
anyway) comments informing the reader what pg_attribute_printf is for.
2015-03-26 14:03:25 -04:00
Andres Freund bbfd7edae5 Add macros wrapping all usage of gcc's __attribute__.
Until now __attribute__() was defined to be empty for all compilers but
gcc. That's problematic because it prevents using it in other compilers;
which is necessary e.g. for atomics portability.  It's also just
generally dubious to do so in a header as widely included as c.h.

Instead add pg_attribute_format_arg, pg_attribute_printf,
pg_attribute_noreturn macros which are implemented in the compilers that
understand them. Also add pg_attribute_noreturn and pg_attribute_packed,
but don't provide fallbacks, since they can affect functionality.

This means that external code that, possibly unwittingly, relied on
__attribute__ defined to be empty on !gcc compilers may now run into
warnings or errors on those compilers. But there shouldn't be many
occurances of that and it's hard to work around...

Discussion: 54B58BA3.8040302@ohmu.fi
Author: Oskari Saarenmaa, with some minor changes by me.
2015-03-11 14:30:01 +01:00
Bruce Momjian dc01efa5cc pg_upgrade: improve checksum mismatch error message
Patch by Greg Sabino Mullane, slight adjustments by me
2015-02-11 22:22:26 -05:00
Bruce Momjian 056764b102 pg_upgrade: quote directory names in delete_old_cluster script
This allows the delete script to properly function when special
characters appear in directory paths, e.g. spaces.

Backpatch through 9.0
2015-02-11 22:06:04 -05:00
Bruce Momjian ac7009abd2 pg_upgrade: fix one-byte per empty db memory leak
Report by Tatsuo Ishii, Coverity
2015-01-09 12:12:30 -05:00
Bruce Momjian 4baaf863ec Update copyright for 2015
Backpatch certain files through 9.0
2015-01-06 11:43:47 -05:00
Noah Misch f6dc6dd5ba Lock down regression testing temporary clusters on Windows.
Use SSPI authentication to allow connections exclusively from the OS
user that launched the test suite.  This closes on Windows the
vulnerability that commit be76a6d39e
closed on other platforms.  Users of "make installcheck" or custom test
harnesses can run "pg_regress --config-auth=DATADIR" to activate the
same authentication configuration that "make check" would use.
Back-patch to 9.0 (all supported versions).

Security: CVE-2014-0067
2014-12-17 22:48:40 -05:00
Magnus Hagander cef0ae498c Update .gitignore for pg_upgrade
Add Windows versions of generated scripts, and make sure we only
ignore the scripts int he root directory.

Michael Paquier
2014-12-17 11:55:22 +01:00
Alvaro Herrera 73c986adde Keep track of transaction commit timestamps
Transactions can now set their commit timestamp directly as they commit,
or an external transaction commit timestamp can be fed from an outside
system using the new function TransactionTreeSetCommitTsData().  This
data is crash-safe, and truncated at Xid freeze point, same as pg_clog.

This module is disabled by default because it causes a performance hit,
but can be enabled in postgresql.conf requiring only a server restart.

A new test in src/test/modules is included.

Catalog version bumped due to the new subdirectory within PGDATA and a
couple of new SQL functions.

Authors: Álvaro Herrera and Petr Jelínek

Reviewed to varying degrees by Michael Paquier, Andres Freund, Robert
Haas, Amit Kapila, Fujii Masao, Jaime Casanova, Simon Riggs, Steven
Singer, Peter Eisentraut
2014-12-03 11:53:02 -03:00
Bruce Momjian dc9c612767 pg_upgrade: prefix Unix shell script name output with "./"
This more clearly suggests the current directory.  While this also works
on Windows, it might be confusing.

Report by Christoph Berg
2014-10-11 18:38:41 -04:00
Heikki Linnakangas 733be2a5cd Remove unnecessary initialization of local variables.
Oops, forgot these in the prveious commit.
2014-10-10 13:00:53 +03:00
Heikki Linnakangas 33755e8edf Change the way encoding and locale checks are done in pg_upgrade.
Lc_collate and lc_ctype have been per-database settings since server version
8.4, but pg_upgrade was still treating them as cluster-wide options. It
fetched the values for the template0 databases in old and new cluster, and
compared them. That's backwards; the encoding and locale of the template0
database doesn't matter, as template0 is guaranteed to contain only ASCII
characters. But if there are any other databases that exist on both clusters
(in particular template1 and postgres databases), their encodings and
locales must be compatible.

Also, make the locale comparison more lenient. If the locale names are not
equal, try to canonicalize both of them by passing them to setlocale(). We
used to do that only when upgrading from 9.1 or below, but it seems like a
good idea even with newer versions. If we change the canonical form of a
locale, this allows pg_upgrade to still work. I'm about to do just that to
fix bug #11431, by mapping a locale name that contains non-ASCII characters
to a pure-ASCII alias of the same locale.

No backpatching, because earlier versions of pg_upgrade still support
upgrading from 8.3 servers. That would be more complicated, so it doesn't
seem worth it, given that we haven't received any complaints about this
from users.
2014-10-10 10:39:32 +03:00
Bruce Momjian 35419aeb83 pg_upgrade: have pg_upgrade fail for old 9.4 JSONB format
Backpatch through 9.4
2014-09-29 20:19:59 -04:00
Bruce Momjian c3c75fcd7a pg_upgrade: adjust C comments 2014-09-11 18:44:00 -04:00
Bruce Momjian acc8e41681 pg_upgrade: compare control version, not catalog version
Also modify test for the possibility the large object value might not
exist in the old cluster.

Fix for commit e1598a15f4
2014-09-10 20:22:10 -04:00
Bruce Momjian e1598a15f4 pg_upgrade: check for large object size compatibility 2014-09-10 19:23:36 -04:00
Bruce Momjian a74a4aa23b pg_upgrade: preserve the timestamp epoch
This is useful for replication tools like Slony and Skytools.

Report by Sergey Konoplev
2014-09-05 19:19:41 -04:00
Peter Eisentraut 2d759341d9 Fix whitespace 2014-08-26 17:26:45 -04:00
Bruce Momjian 5d16332e96 pg_upgrade: use CTE query rather than temp table
Now that 8.3 is not supported, we can use a CTE and not temp tables.
This allows for auto-oid assignment protection in a future patch.
2014-08-25 21:22:08 -04:00
Bruce Momjian 6cb74a67e2 revert "Throw error for ALTER TABLE RESET of an invalid option"
Reverts commits 73d78e11a0 and
b0488e5c4f.  Also reverts pg_upgrade
changes.
2014-08-25 20:07:37 -04:00
Bruce Momjian 2209b3923a pg_upgrade: remove support for 8.3 old clusters
This trims down the code, and is in preparation for hardening pg_upgrade
against auto-oid assignment.
2014-08-25 20:05:07 -04:00
Bruce Momjian b0488e5c4f pg_upgrade: add SET log_min_error_statement = warning
Add to commit 73d78e11a0
2014-08-25 17:24:33 -04:00
Bruce Momjian 73d78e11a0 Throw error for ALTER TABLE RESET of an invalid option
Also adjust pg_upgrade to not use this method for optional TOAST table
creation.

Patch by Fabrízio de Royes Mello
2014-08-25 17:06:40 -04:00
Bruce Momjian ebe30ad59b pg_ctl, pg_upgrade: allow multiple -o/-O options, append them
Report by Pavel Raiskup
2014-08-25 16:30:26 -04:00
Bruce Momjian 7567d94910 pg_upgrade: adjust logging to use QUERY_ALLOC lengths
Allows the logging to print the entire text of failed queries, rather
than a truncated version.
2014-08-20 13:04:45 -04:00
Peter Eisentraut 9ee9ca545b Fix whitespace 2014-08-15 14:52:06 -04:00
Bruce Momjian ea9b8abd6f pg_upgrade: fix define adjusted during testing
from commit e4c3c99ac3
2014-08-15 12:28:18 -04:00
Bruce Momjian e4c3c99ac3 pg_upgrade: error if run from top of new PGDATA on Windows
Print a clear error message in this case, rather than wait for initdb
--sync-only to generate a "Permission denied" error.
2014-08-15 12:04:03 -04:00
Bruce Momjian 4c6780fd17 pg_upgrade: prevent oid conflicts with new-cluster TOAST tables
Previously, TOAST tables only required in the new cluster could cause
oid conflicts if they were auto-numbered and a later conflicting oid had
to be assigned.

Backpatch through 9.3
2014-08-07 14:56:13 -04:00
Bruce Momjian ee646df59a pg_upgrade: assume user is install user
The user specified to the upgrade was effectively the install user, but
that was not clearly stated in the comments, documentation, or error
messages.
2014-08-04 11:56:16 -04:00
Bruce Momjian b4bd6f6b05 pg_upgrade: remove reference to autovacuum_multixact_freeze_max_age
autovacuum_multixact_freeze_max_age was added as a pg_ctl start
parameter in 9.3.X to prevent autovacuum from running.  However, only
some 9.3.X releases have autovacuum_multixact_freeze_max_age as it was
added in a minor PG 9.3 release.  It also isn't needed because -b turns
off autovacuum in 9.1+.

Without this fix, trying to upgrade from an early 9.3 release to 9.4
would fail.

Report by EDB

Backpatch through 9.3
2014-08-04 11:46:05 -04:00
Bruce Momjian d826d8ecce pg_upgrade: improve C comment wording 2014-07-29 22:09:03 -04:00
Peter Eisentraut e0da6ec618 pg_upgrade: Fix spacing in help output 2014-07-15 15:08:16 -04:00
Bruce Momjian 3088cc3704 pg_upgrade: allow upgrades for new-only TOAST tables
Previously, when calculations on the need for toast tables changed,
pg_upgrade could not handle cases where the new cluster needed a TOAST
table and the old cluster did not.  (It already handled the opposite
case.)  This fixes the "OID mismatch" error typically generated in this
case.

Backpatch through 9.2
2014-07-07 13:24:08 -04:00
Noah Misch 333b7db8b3 Consistently pass an "unsigned char" to ctype.h functions.
The isxdigit() calls relied on undefined behavior.  The isascii() call
was well-defined, but our prevailing style is to include the cast.
Back-patch to 9.4, where the isxdigit() calls were introduced.
2014-07-06 00:29:51 -04:00
Bruce Momjian a61daa14d5 pg_upgrade: preserve database and relation minmxid values
Also set these values for pre-9.3 old clusters that don't have values to
preserve.

Analysis by Alvaro

Backpatch through 9.3
2014-07-02 15:29:38 -04:00
Bruce Momjian 89cf2d5203 pg_upgrade: no need to remove "members" files for pre-9.3 upgrades
Per analysis by Alvaro

Backpatch through 9.3
2014-07-02 13:11:05 -04:00
Tom Lane fbb1d7d73f Allow CREATE/ALTER DATABASE to manipulate datistemplate and datallowconn.
Historically these database properties could be manipulated only by
manually updating pg_database, which is error-prone and only possible for
superusers.  But there seems no good reason not to allow database owners to
set them for their databases, so invent CREATE/ALTER DATABASE options to do
that.  Adjust a couple of places that were doing it the hard way to use the
commands instead.

Vik Fearing, reviewed by Pavel Stehule
2014-07-01 20:10:38 -04:00
Bruce Momjian 55863274d9 pg_upgrade: update C comments about pg_dumpall
There were some C comments that hadn't been updated from the switch of
using only pg_dumpall to using pg_dump and pg_dumpall, so update them.
Also, don't bother using --schema-only for pg_dumpall --globals-only.

Backpatch through 9.4
2014-06-30 19:56:23 -04:00
Bruce Momjian 0f7482733a pg_upgrade: remove pg_multixact files left by initdb
This fixes a bug that caused vacuum to fail when the '0000' files left
by initdb were accessed as part of vacuum's cleanup of old pg_multixact
files.

Backpatch through 9.3
2014-06-24 16:11:06 -04:00
Noah Misch 686f362bee Fix contrib/pg_upgrade/test.sh for $PWD containing spaces.
Most of the necessary quoting was in place; this catches the exceptions.
2014-06-19 21:47:08 -04:00
Noah Misch be76a6d39e Secure Unix-domain sockets of "make check" temporary clusters.
Any OS user able to access the socket can connect as the bootstrap
superuser and proceed to execute arbitrary code as the OS user running
the test.  Protect against that by placing the socket in a temporary,
mode-0700 subdirectory of /tmp.  The pg_regress-based test suites and
the pg_upgrade test suite were vulnerable; the $(prove_check)-based test
suites were already secure.  Back-patch to 8.4 (all supported versions).
The hazard remains wherever the temporary cluster accepts TCP
connections, notably on Windows.

As a convenient side effect, this lets testing proceed smoothly in
builds that override DEFAULT_PGSOCKET_DIR.  Popular non-default values
like /var/run/postgresql are often unwritable to the build user.

Security: CVE-2014-0067
2014-06-14 09:41:13 -04:00
Peter Eisentraut 2f8e68bd7e Fix whitespace 2014-05-15 13:23:31 -04:00
Bruce Momjian 2e6602dfc2 pg_update: slight major version adjustment 2014-05-14 21:13:59 -04:00
Tom Lane 4456763c78 Fix version check for pg_upgrade line type crosscheck.
Per buildfarm.
2014-05-14 20:23:32 -04:00
Bruce Momjian bb797b6404 pg_upgrade: error out on 'line' data type usage
The data type internal format changed in 9.4.  Also mention this in the
9.4 release notes.
2014-05-14 16:26:13 -04:00
Bruce Momjian 0a78320057 pgindent run for 9.4
This includes removing tabs after periods in C comments, which was
applied to back branches, so this change should not effect backpatching.
2014-05-06 12:12:18 -04:00
Heikki Linnakangas a692ee5870 Replace SYSTEMQUOTEs with Windows-specific wrapper functions.
It's easy to forget using SYSTEMQUOTEs when constructing command strings
for system() or popen(). Even if we fix all the places missing it now, it is
bound to be forgotten again in the future. Introduce wrapper functions that
do the the extra quoting for you, and get rid of SYSTEMQUOTEs in all the
callers.

We previosly used SYSTEMQUOTEs in all the hard-coded command strings, and
this doesn't change the behavior of those. But user-supplied commands, like
archive_command, restore_command, COPY TO/FROM PROGRAM calls, as well as
pgbench's \shell, will now gain an extra pair of quotes. That is desirable,
but if you have existing scripts or config files that include an extra
pair of quotes, those might need to be adjusted.

Reviewed by Amit Kapila and Tom Lane
2014-05-05 16:07:40 +03:00