Commit Graph

322 Commits

Author SHA1 Message Date
Bruce Momjian
6f1b9e4efd Add pg_upgrade --jobs parameter
Add pg_upgrade --jobs, which allows parallel dump/restore of databases,
which improves performance.
2012-12-26 19:26:30 -05:00
Bruce Momjian
dc9896a245 Avoid using NAMEDATALEN in pg_upgrade
Because the client encoding might not match the server encoding,
pg_upgrade can't allocate NAMEDATALEN bytes for storage of database,
relation, and namespace identifiers.  Instead pg_strdup() the memory and
free it.

Also add C comment in initdb.c about safe NAMEDATALEN usage.
2012-12-20 13:56:31 -05:00
Bruce Momjian
345fb82f16 Add pg_upgrade comment about mismatch error
Add comment stating that constraint and index names must match.
2012-12-20 07:37:27 -05:00
Bruce Momjian
e95c4bd113 Fix pg_upgrade for invalid indexes
All versions of pg_upgrade upgraded invalid indexes caused by CREATE
INDEX CONCURRENTLY failures and marked them as valid.  The patch adds a
check to all pg_upgrade versions and throws an error during upgrade or
--check.

Backpatch to 9.2, 9.1, 9.0.  Patch slightly adjusted.
2012-12-11 15:09:22 -05:00
Bruce Momjian
acdb8c2259 Fix pg_upgrade -O/-o options
Fix previous commit that added synchronous_commit=off, but broke -O/-o
due to missing space in argument passing.

Backpatch to 9.2.
2012-12-10 23:03:25 -05:00
Bruce Momjian
6dd9584507 Improve pg_upgrade's status display
Pg_upgrade displays file names during copy and database names during
dump/restore.  Andrew Dunstan identified three bugs:

*  long file names were being truncated to 60 _leading_ characters, which
   often do not change for long file names

*  file names were truncated to 60 characters in log files

*  carriage returns were being output to log files

This commit fixes these --- it prints 60 _trailing_ characters to the
status display, and full path names without carriage returns to log
files.  It also suppresses status output to the log file unless verbose
mode is used.
2012-12-07 12:26:13 -05:00
Bruce Momjian
c47d261c07 In pg_upgrade testing script, turn off command echo at the end so status
report is clearer.
2012-12-04 08:17:45 -05:00
Bruce Momjian
2f22765607 Restore set -x in pg_upgrade/test.sh, so the user can see what is being
executed.
2012-12-03 23:44:23 -05:00
Bruce Momjian
630cd14426 Add initdb --sync-only option to sync the data directory to durable
storage.

Have pg_upgrade use it, and enable server options fsync=off and
full_page_writes=off.

Document that users turning fsync from off to on should run initdb
--sync-only.

[ Previous commit was incorrectly applied as a git merge. ]
2012-12-03 22:47:59 -05:00
Bruce Momjian
25d1ed04a2 Revert initdb --sync-only patch that had incorrect commit messages. 2012-12-03 22:46:51 -05:00
Bruce Momjian
4d88bc8f2b dummy commit 2012-12-03 22:45:02 -05:00
Bruce Momjian
cd7569a546 dummy commit 2012-12-03 22:45:02 -05:00
Bruce Momjian
452739df82 In pg_upgrade, remove 'set -x' from test script. 2012-12-01 12:07:14 -05:00
Bruce Momjian
1c59e37665 Revert:
In pg_upgrade, remove pg_restore's --single-transaction option,
    as it throws errors in certain cases.
2012-12-01 10:21:45 -05:00
Bruce Momjian
209772350b Remove pg_restore's --single-transaction option, as it throws errors in
certain cases.
2012-12-01 09:58:00 -05:00
Bruce Momjian
5eeab9c85c In pg_upgrade, improve status wording now that we have per-database
status output for dump/restore.
2012-11-30 22:32:25 -05:00
Bruce Momjian
12ee6ec71f In pg_upgrade, dump each database separately and use
--single-transaction to restore each database schema.  This yields
performance improvements for databases with many tables.  Also, remove
split_old_dump() as it is no longer needed.
2012-11-30 16:30:13 -05:00
Andrew Dunstan
abece8af17 Clean environment for pg_upgrade test.
This removes exisiting PG settings from the environment for
pg_upgrade tests, just like pg_regress does.
2012-11-30 07:54:24 -05:00
Bruce Momjian
6b711cf37c In pg_upgrade, simplify function copy_file() by using pg_malloc() and
centralizing error/shutdown code.
2012-11-24 22:39:03 -05:00
Bruce Momjian
16e1ae77f9 In pg_upgrade, fix a few place that used maloc/free rather than
pg_malloc/pg_free.
2012-11-24 22:12:39 -05:00
Bruce Momjian
b55743a5df In pg_upgrade, report errno string if file existence check returns an
error and errno != ENOENT.
2012-11-19 16:41:58 -05:00
Bruce Momjian
546d65d55f In pg_upgrade, add third meaningless parameter to open(). 2012-11-14 19:01:29 -05:00
Bruce Momjian
29add0de49 In pg_upgrade, copy fsm, vm, and extent files by checking for file
existence via open(), rather than collecting a directory listing and
looking up matching relfilenode files with sequential scans of the
array.  This speeds up pg_upgrade by 2x for a large number of tables,
e.g. 16k.

Per observation by Ants Aasma.
2012-11-14 17:32:07 -05:00
Bruce Momjian
dec10ba4c5 Mark pg_upgrade's free_db_and_rel_infos() as a static function. 2012-11-13 21:10:40 -05:00
Bruce Momjian
ed5699dd1b In pg_upgrade, set synchronous_commit=off for the new cluster, to
improve performance when restoring the schema from the old cluster.

Backpatch to 9.2.
2012-11-06 14:28:57 -05:00
Andrew Dunstan
2f2be7473b Use a more portable platform test. 2012-10-18 16:14:11 -04:00
Bruce Momjian
a9701a1d7d In pg_upgrade, issue proper error message when we can't open PG_VERSION.
Backpatch to 9.2.
2012-10-10 13:53:03 -04:00
Bruce Momjian
ce75457949 In pg_upgrade, use full path name for analyze_new_cluster.sh script.
Backpatch to 9.2.
2012-10-02 21:18:43 -04:00
Tom Lane
09ac603c36 Work around unportable behavior of malloc(0) and realloc(NULL, 0).
On some platforms these functions return NULL, rather than the more common
practice of returning a pointer to a zero-sized block of memory.  Hack our
various wrapper functions to hide the difference by substituting a size
request of 1.  This is probably not so important for the callers, who
should never touch the block anyway if they asked for size 0 --- but it's
important for the wrapper functions themselves, which mistakenly treated
the NULL result as an out-of-memory failure.  This broke at least pg_dump
for the case of no user-defined aggregates, as per report from
Matthew Carrington.

Back-patch to 9.2 to fix the pg_dump issue.  Given the lack of previous
complaints, it seems likely that there is no live bug in previous releases,
even though some of these functions were in place before that.
2012-10-02 17:32:42 -04:00
Bruce Momjian
8a7598091a In pg_upgrade, improve error reporting when the number of relation
objects does not match between the old and new clusters.

Backpatch to 9.2.
2012-10-02 11:53:45 -04:00
Bruce Momjian
ac96b851ec Adjust pg_upgrade query so toast tables related to system catalog schema
entries are not dumped.   This fixes an error caused by
droping/recreating the information_schema, but other failures were also
possible.

Backpatch to 9.2.
2012-10-02 11:46:08 -04:00
Bruce Momjian
b61837a49f In pg_upgrade, try to convert the locale names to canonical form before
comparison;  also report the old/new values if they don't match.

Backpatch to 9.2.
2012-10-02 11:42:34 -04:00
Peter Eisentraut
10bfe81dee pg_upgrade test: Disable fsync in initdb and postgres calls
This mirrors the behavior of pg_regress and makes the test run much
faster.
2012-09-26 22:41:57 -04:00
Peter Eisentraut
5cfd5bb15b pg_upgrade: Remove check for pg_config
It is no longer used, but was still being checked for.

bug #7548 from Reinhard Max
2012-09-18 22:04:28 -04:00
Andrew Dunstan
f8c81c5dde In pg_upgrade, try a few times to open a log file.
If we call pg_ctl stop, the server might continue and thus
hold a log file for a short time after it has deleted its pid file,
(which is when pg_ctl will exit), and so a subsequent attempt to
open the log file might fail.

We therefore try to open it a few times, sleeping one second between
tries, to give the server time to exit.

This corrects an error that was observed on the buildfarm.

Backpatched to 9.2,
2012-09-05 23:14:49 -04:00
Andrew Dunstan
f8f5cf33a3 Fix pg_upgrade test script's line end handling on Windows.
Call pg_dumpall using -f switch instead of redirection, to avoid
writing the output in text mode and generating spurious carriage
returns. Remove to carriage return ignoring hack introduced by
commit e442b0f0c6.

Backpatch to 9.2.
2012-09-05 18:00:31 -04:00
Andrew Dunstan
ea0b414a0d Fix line end mishandling in pg_upgrade on Windows.
pg_upgrade opened the output from pg_dumpall in text mode and
wrote the split files in text mode. This caused unwanted eating
of intended carriage returns on input and production of spurious
carriage returns on output. To avoid this, open all these files
in binary mode. On non-Windows platforms, this change has no
effect.

Backpatch to 9.0. On 9.0 and 9.1, we also switch from redirecting
pg_dumpall's output to using pg_dumpall's -f switch, for the same
reason.
2012-09-05 17:41:43 -04:00
Tom Lane
b98fd52a55 Silence -Wunused-result warning in contrib/pg_upgrade.
This is just neatnik-ism, but since we do it for comparable code in elog.c,
we may as well do it here.
2012-09-05 14:37:22 -04:00
Bruce Momjian
022cd22f0f In pg_upgrade, document why we can't issue \n\n in the command logfile
on Windows.  Slightly cleanup log output on Windows given this
restriction.

Backpatch to 9.2.
2012-09-05 00:01:13 -04:00
Andrew Dunstan
2042185baf Fix transcription error. 2012-09-04 09:39:49 -04:00
Andrew Dunstan
0829c7087e Fix command echoing in pg_upgade's analyze script for Windows. 2012-09-04 05:49:22 -04:00
Andrew Dunstan
2f0c7d5854 Indent fix_path_separator() header properly. 2012-09-03 22:59:19 -04:00
Andrew Dunstan
e442b0f0c6 Allow pg_upgrade "make check" to run on Windows/MSys.
Backpatch to 9.2.
2012-09-03 22:32:17 -04:00
Bruce Momjian
a80b8037cf In pg_upgrade, pull the port number from postmaster.pid, like we do for
socket location.  Also, prevent putting the socket in the current
directory for pre-9.1 servers in live check and non-live check mode,
because pre-9.1 pg_ctl -w can't handle it.

Backpatch to 9.2.
2012-09-03 22:15:09 -04:00
Andrew Dunstan
504aeea692 Use correct path separator for Windows builtin commands.
pg_upgrade produces a platform-specific script to remove the old
directory, but on Windows it has not been making sure that the
paths it writes as arguments for rmdir and del use the backslash
path separator, which will cause these scripts to fail.

The fix is backpatched to Release 9.0.
2012-09-03 18:06:47 -04:00
Andrew Dunstan
6259678f8f Fix bugs in exec.c that prevented pg_upgrade working in Windows.
Backpatch to 9.2 - code before that is quite different and should
not have these defects.
2012-09-03 15:31:26 -04:00
Tom Lane
f763b77193 Fix pg_upgrade to cope with non-default unix_socket_directory scenarios.
When starting either an old or new postmaster, force it to place its Unix
socket in the current directory.  This makes it even harder for accidental
connections to occur during pg_upgrade, and also works around some
scenarios where the default socket location isn't usable.  (For example,
if the default location is something other than "/tmp", it might not exist
during "make check".)

When checking an already-running old postmaster, find out its actual socket
directory location from postmaster.pid, if possible.  This dodges problems
with an old postmaster having a configured location different from the
default built into pg_upgrade's libpq.  We can't find that out if the old
postmaster is pre-9.1, so also document how to cope with such scenarios
manually.

In support of this, centralize handling of the connection-related command
line options passed to pg_upgrade's subsidiary programs, such as pg_dump.
This should make future changes easier.

Bruce Momjian and Tom Lane
2012-09-03 13:52:40 -04:00
Alvaro Herrera
088c065ce8 pg_upgrade: Fix exec_prog API to be less flaky
The previous signature made it very easy to pass something other than
the printf-format specifier in the corresponding position, without any
warning from the compiler.

While at it, move some of the escaping, redirecting and quoting
responsibilities from the callers into exec_prog() itself.  This makes
the callsites cleaner.
2012-08-27 14:29:14 -04:00
Alvaro Herrera
34c02044ed Fix thinko in comment
It was confusing symlinks with hard links.
2012-08-27 14:29:14 -04:00
Alvaro Herrera
ab577e63fa Remove analyze_new_cluster.sh on make clean, too 2012-08-27 14:29:14 -04:00