2015-03-23 18:47:52 +01:00
|
|
|
<!--
|
|
|
|
doc/src/sgml/ref/pg_rewind.sgml
|
|
|
|
PostgreSQL documentation
|
|
|
|
-->
|
|
|
|
|
|
|
|
<refentry id="app-pgrewind">
|
|
|
|
<indexterm zone="app-pgrewind">
|
|
|
|
<primary>pg_rewind</primary>
|
|
|
|
</indexterm>
|
|
|
|
|
|
|
|
<refmeta>
|
|
|
|
<refentrytitle><application>pg_rewind</application></refentrytitle>
|
|
|
|
<manvolnum>1</manvolnum>
|
|
|
|
<refmiscinfo>Application</refmiscinfo>
|
|
|
|
</refmeta>
|
|
|
|
|
|
|
|
<refnamediv>
|
|
|
|
<refname>pg_rewind</refname>
|
2016-08-01 18:52:22 +02:00
|
|
|
<refpurpose>synchronize a <productname>PostgreSQL</productname> data directory with another data directory that was forked from it</refpurpose>
|
2015-03-23 18:47:52 +01:00
|
|
|
</refnamediv>
|
|
|
|
|
|
|
|
<refsynopsisdiv>
|
|
|
|
<cmdsynopsis>
|
|
|
|
<command>pg_rewind</command>
|
|
|
|
<arg rep="repeat"><replaceable>option</replaceable></arg>
|
|
|
|
<group choice="plain">
|
|
|
|
<group choice="req">
|
|
|
|
<arg choice="plain"><option>-D </option></arg>
|
|
|
|
<arg choice="plain"><option>--target-pgdata</option></arg>
|
|
|
|
</group>
|
|
|
|
<replaceable> directory</replaceable>
|
|
|
|
<group choice="req">
|
|
|
|
<arg choice="plain"><option>--source-pgdata=<replaceable>directory</replaceable></option></arg>
|
|
|
|
<arg choice="plain"><option>--source-server=<replaceable>connstr</replaceable></option></arg>
|
|
|
|
</group>
|
|
|
|
</group>
|
|
|
|
</cmdsynopsis>
|
|
|
|
</refsynopsisdiv>
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
<title>Description</title>
|
|
|
|
|
|
|
|
<para>
|
2017-10-09 03:44:17 +02:00
|
|
|
<application>pg_rewind</application> is a tool for synchronizing a PostgreSQL cluster
|
2015-03-23 18:47:52 +01:00
|
|
|
with another copy of the same cluster, after the clusters' timelines have
|
|
|
|
diverged. A typical scenario is to bring an old master server back online
|
2016-08-01 18:52:22 +02:00
|
|
|
after failover as a standby that follows the new master.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
The result is equivalent to replacing the target data directory with the
|
2016-08-01 18:52:22 +02:00
|
|
|
source one. Only changed blocks from relation files are copied;
|
|
|
|
all other files are copied in full, including configuration files. The
|
2017-10-09 03:44:17 +02:00
|
|
|
advantage of <application>pg_rewind</application> over taking a new base backup, or
|
|
|
|
tools like <application>rsync</application>, is that <application>pg_rewind</application> does
|
2016-08-01 18:52:22 +02:00
|
|
|
not require reading through unchanged blocks in the cluster. This makes
|
|
|
|
it a lot faster when the database is large and only a small
|
|
|
|
fraction of blocks differ between the clusters.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2017-10-09 03:44:17 +02:00
|
|
|
<application>pg_rewind</application> examines the timeline histories of the source
|
2015-03-23 18:47:52 +01:00
|
|
|
and target clusters to determine the point where they diverged, and
|
2017-10-09 03:44:17 +02:00
|
|
|
expects to find WAL in the target cluster's <filename>pg_wal</filename> directory
|
2015-12-01 16:56:44 +01:00
|
|
|
reaching all the way back to the point of divergence. The point of divergence
|
2016-08-01 18:52:22 +02:00
|
|
|
can be found either on the target timeline, the source timeline, or their common
|
2015-12-01 16:56:44 +01:00
|
|
|
ancestor. In the typical failover scenario where the target cluster was
|
2016-08-01 18:52:22 +02:00
|
|
|
shut down soon after the divergence, this is not a problem, but if the
|
|
|
|
target cluster ran for a long time after the divergence, the old WAL
|
|
|
|
files might no longer be present. In that case, they can be manually
|
2017-10-09 03:44:17 +02:00
|
|
|
copied from the WAL archive to the <filename>pg_wal</filename> directory, or
|
2018-11-25 16:31:16 +01:00
|
|
|
fetched on startup by configuring <xref linkend="guc-primary-conninfo"/> or
|
|
|
|
<xref linkend="guc-restore-command"/>. The use of
|
2017-10-09 03:44:17 +02:00
|
|
|
<application>pg_rewind</application> is not limited to failover, e.g. a standby
|
2016-08-01 18:52:22 +02:00
|
|
|
server can be promoted, run some write transactions, and then rewinded
|
|
|
|
to become a standby again.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2016-08-01 18:52:22 +02:00
|
|
|
When the target server is started for the first time after running
|
2017-10-09 03:44:17 +02:00
|
|
|
<application>pg_rewind</application>, it will go into recovery mode and replay all
|
2015-03-23 18:47:52 +01:00
|
|
|
WAL generated in the source server after the point of divergence.
|
|
|
|
If some of the WAL was no longer available in the source server when
|
2017-10-09 03:44:17 +02:00
|
|
|
<application>pg_rewind</application> was run, and therefore could not be copied by the
|
|
|
|
<application>pg_rewind</application> session, it must be made available when the
|
2016-08-01 18:52:22 +02:00
|
|
|
target server is started. This can be done by creating a
|
2018-11-25 16:31:16 +01:00
|
|
|
<filename>recovery.signal</filename> file in the target data directory
|
|
|
|
and configuring suitable <xref linkend="guc-restore-command"/>
|
|
|
|
in <filename>postgresql.conf</filename>.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
2015-09-18 02:56:58 +02:00
|
|
|
|
|
|
|
<para>
|
2017-10-09 03:44:17 +02:00
|
|
|
<application>pg_rewind</application> requires that the target server either has
|
2017-11-23 15:39:47 +01:00
|
|
|
the <xref linkend="guc-wal-log-hints"/> option enabled
|
2017-10-09 03:44:17 +02:00
|
|
|
in <filename>postgresql.conf</filename> or data checksums enabled when
|
|
|
|
the cluster was initialized with <application>initdb</application>. Neither of these
|
2017-11-23 15:39:47 +01:00
|
|
|
are currently on by default. <xref linkend="guc-full-page-writes"/>
|
2017-10-09 03:44:17 +02:00
|
|
|
must also be set to <literal>on</literal>, but is enabled by default.
|
2015-09-18 02:56:58 +02:00
|
|
|
</para>
|
2018-07-07 01:10:10 +02:00
|
|
|
|
|
|
|
<warning>
|
|
|
|
<para>
|
|
|
|
If <application>pg_rewind</application> fails while processing, then
|
|
|
|
the data folder of the target is likely not in a state that can be
|
|
|
|
recovered. In such a case, taking a new fresh backup is recommended.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
<application>pg_rewind</application> will fail immediately if it finds
|
|
|
|
files it cannot write directly to. This can happen for example when
|
|
|
|
the source and the target server use the same file mapping for read-only
|
|
|
|
SSL keys and certificates. If such files are present on the target server
|
|
|
|
it is recommended to remove them before running
|
|
|
|
<application>pg_rewind</application>. After doing the rewind, some of
|
|
|
|
those files may have been copied from the source, in which case it may
|
|
|
|
be necessary to remove the data copied and restore back the set of links
|
|
|
|
used before the rewind.
|
|
|
|
</para>
|
|
|
|
</warning>
|
2015-03-23 18:47:52 +01:00
|
|
|
</refsect1>
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
<title>Options</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
<application>pg_rewind</application> accepts the following command-line
|
|
|
|
arguments:
|
|
|
|
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
2015-04-03 05:21:16 +02:00
|
|
|
<term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
|
|
|
|
<term><option>--target-pgdata=<replaceable class="parameter">directory</replaceable></option></term>
|
2015-03-23 18:47:52 +01:00
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
This option specifies the target data directory that is synchronized
|
2016-08-01 18:52:22 +02:00
|
|
|
with the source. The target server must be shut down cleanly before
|
2015-03-23 18:47:52 +01:00
|
|
|
running <application>pg_rewind</application>
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
2015-04-03 05:21:16 +02:00
|
|
|
<term><option>--source-pgdata=<replaceable class="parameter">directory</replaceable></option></term>
|
2015-03-23 18:47:52 +01:00
|
|
|
<listitem>
|
|
|
|
<para>
|
2016-08-01 18:52:22 +02:00
|
|
|
Specifies the file system path to the data directory of the source
|
|
|
|
server to synchronize the target with. This option requires the
|
|
|
|
source server to be cleanly shut down.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
2015-04-03 05:21:16 +02:00
|
|
|
<term><option>--source-server=<replaceable class="parameter">connstr</replaceable></option></term>
|
2015-03-23 18:47:52 +01:00
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Specifies a libpq connection string to connect to the source
|
2017-10-09 03:44:17 +02:00
|
|
|
<productname>PostgreSQL</productname> server to synchronize the target with.
|
2016-05-31 19:56:25 +02:00
|
|
|
The connection must be a normal (non-replication) connection
|
2016-08-01 18:52:22 +02:00
|
|
|
with superuser access. This option requires the source
|
|
|
|
server to be running and not in recovery mode.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term><option>-n</option></term>
|
|
|
|
<term><option>--dry-run</option></term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Do everything except actually modifying the target directory.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
2018-07-10 01:51:10 +02:00
|
|
|
<varlistentry>
|
|
|
|
<term><option>-N</option></term>
|
|
|
|
<term><option>--no-sync</option></term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
By default, <command>pg_rewind</command> will wait for all files
|
|
|
|
to be written safely to disk. This option causes
|
|
|
|
<command>pg_rewind</command> to return without waiting, which is
|
|
|
|
faster, but means that a subsequent operating system crash can leave
|
|
|
|
the synchronized data folder corrupt. Generally, this option is
|
|
|
|
useful for testing but should not be used when creating a production
|
|
|
|
installation.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
2015-03-23 18:47:52 +01:00
|
|
|
<varlistentry>
|
|
|
|
<term><option>-P</option></term>
|
|
|
|
<term><option>--progress</option></term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Enables progress reporting. Turning this on will deliver an approximate
|
2016-08-01 18:52:22 +02:00
|
|
|
progress report while copying data from the source cluster.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term><option>--debug</option></term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Print verbose debugging output that is mostly useful for developers
|
2017-10-09 03:44:17 +02:00
|
|
|
debugging <application>pg_rewind</application>.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term><option>-V</option></term>
|
|
|
|
<term><option>--version</option></term>
|
2015-09-18 02:56:58 +02:00
|
|
|
<listitem><para>Display version information, then exit.</para></listitem>
|
2015-03-23 18:47:52 +01:00
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term><option>-?</option></term>
|
|
|
|
<term><option>--help</option></term>
|
2015-09-18 02:56:58 +02:00
|
|
|
<listitem><para>Show help, then exit.</para></listitem>
|
2015-03-23 18:47:52 +01:00
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
</variablelist>
|
|
|
|
</para>
|
|
|
|
</refsect1>
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
<title>Environment</title>
|
|
|
|
|
|
|
|
<para>
|
2017-10-09 03:44:17 +02:00
|
|
|
When <option>--source-server</option> option is used,
|
2015-03-23 18:47:52 +01:00
|
|
|
<application>pg_rewind</application> also uses the environment variables
|
2017-11-23 15:39:47 +01:00
|
|
|
supported by <application>libpq</application> (see <xref linkend="libpq-envars"/>).
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
|
|
|
|
<para>
|
|
|
|
The environment variable <envar>PG_COLOR</envar> specifies whether to use
|
|
|
|
color in diagnostics messages. Possible values are
|
|
|
|
<literal>always</literal>, <literal>auto</literal>,
|
|
|
|
<literal>never</literal>.
|
|
|
|
</para>
|
2015-03-23 18:47:52 +01:00
|
|
|
</refsect1>
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
<title>Notes</title>
|
|
|
|
|
|
|
|
<refsect2>
|
|
|
|
<title>How it works</title>
|
|
|
|
|
|
|
|
<para>
|
2016-08-01 18:52:22 +02:00
|
|
|
The basic idea is to copy all file system-level changes from the source
|
|
|
|
cluster to the target cluster:
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<procedure>
|
|
|
|
<step>
|
|
|
|
<para>
|
2016-08-01 18:52:22 +02:00
|
|
|
Scan the WAL log of the target cluster, starting from the last
|
|
|
|
checkpoint before the point where the source cluster's timeline
|
|
|
|
history forked off from the target cluster. For each WAL record,
|
|
|
|
record each data block that was touched. This yields a list of all
|
|
|
|
the data blocks that were changed in the target cluster, after the
|
|
|
|
source cluster forked off.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</step>
|
|
|
|
<step>
|
|
|
|
<para>
|
2016-08-01 18:52:22 +02:00
|
|
|
Copy all those changed blocks from the source cluster to
|
|
|
|
the target cluster, either using direct file system access
|
2017-10-09 03:44:17 +02:00
|
|
|
(<option>--source-pgdata</option>) or SQL (<option>--source-server</option>).
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</step>
|
|
|
|
<step>
|
|
|
|
<para>
|
2017-03-17 14:46:58 +01:00
|
|
|
Copy all other files such as <filename>pg_xact</filename> and
|
2016-08-01 18:52:22 +02:00
|
|
|
configuration files from the source cluster to the target cluster
|
2018-03-28 21:56:52 +02:00
|
|
|
(everything except the relation files). Similarly to base backups,
|
|
|
|
the contents of the directories <filename>pg_dynshmem/</filename>,
|
|
|
|
<filename>pg_notify/</filename>, <filename>pg_replslot/</filename>,
|
|
|
|
<filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>,
|
|
|
|
<filename>pg_stat_tmp/</filename>, and
|
|
|
|
<filename>pg_subtrans/</filename> are omitted from the data copied
|
|
|
|
from the source cluster. Any file or directory beginning with
|
|
|
|
<filename>pgsql_tmp</filename> is omitted, as well as are
|
|
|
|
<filename>backup_label</filename>,
|
|
|
|
<filename>tablespace_map</filename>,
|
|
|
|
<filename>pg_internal.init</filename>,
|
|
|
|
<filename>postmaster.opts</filename> and
|
|
|
|
<filename>postmaster.pid</filename>.
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</step>
|
|
|
|
<step>
|
|
|
|
<para>
|
2016-08-01 18:52:22 +02:00
|
|
|
Apply the WAL from the source cluster, starting from the checkpoint
|
2017-10-09 03:44:17 +02:00
|
|
|
created at failover. (Strictly speaking, <application>pg_rewind</application>
|
2016-08-01 18:52:22 +02:00
|
|
|
doesn't apply the WAL, it just creates a backup label file that
|
2017-10-09 03:44:17 +02:00
|
|
|
makes <productname>PostgreSQL</productname> start by replaying all WAL from
|
2016-08-01 18:52:22 +02:00
|
|
|
that checkpoint forward.)
|
2015-03-23 18:47:52 +01:00
|
|
|
</para>
|
|
|
|
</step>
|
|
|
|
</procedure>
|
|
|
|
</refsect2>
|
|
|
|
</refsect1>
|
|
|
|
|
|
|
|
</refentry>
|