rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/bin/pg_basebackup/pg_basebackup.c

1956 lines
48 KiB
C
Raw Normal View History

Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*-------------------------------------------------------------------------
*
* pg_basebackup.c - receive a base backup using streaming replication protocol
*
* Author: Magnus Hagander <magnus@hagander.net>
*
Update copyright for 2014 Update all files in head, and files COPYRIGHT and legal.sgml in all back branches.
2014-01-07 22:05:30 +01:00
* Portions Copyright (c) 1996-2014, PostgreSQL Global Development Group
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
*
* IDENTIFICATION
* src/bin/pg_basebackup/pg_basebackup.c
*-------------------------------------------------------------------------
*/
Make xlog_internal.h includable in frontend context. This makes unnecessary the ugly hack used to #include postgres.h in pg_basebackup. Based on Alvaro Herrera's patch
2012-12-13 13:59:13 +01:00
#include "postgres_fe.h"
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#include "libpq-fe.h"
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
#include "pqexpbuffer.h"
#include "pgtar.h"
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#include <unistd.h>
#include <dirent.h>
#include <sys/stat.h>
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
#include <sys/types.h>
#include <sys/wait.h>
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
#include <time.h>
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#ifdef HAVE_LIBZ
#include <zlib.h>
#endif
#include "getopt_long.h"
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
#include "receivelog.h"
#include "streamutil.h"
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/* Global options */
Mark variables 'static' where possible. Move GinFuzzySearchLimit to ginget.c Per "clang -Wmissing-variable-declarations" output, posted by Andres Freund. I didn't silence all those warnings, though, only the most obvious cases.
2013-12-16 10:27:30 +01:00
static char *basedir = NULL;
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
static char *xlog_dir = "";
Mark variables 'static' where possible. Move GinFuzzySearchLimit to ginget.c Per "clang -Wmissing-variable-declarations" output, posted by Andres Freund. I didn't silence all those warnings, though, only the most obvious cases.
2013-12-16 10:27:30 +01:00
static char format = 'p'; /* p(lain)/t(ar) */
static char *label = "pg_basebackup base backup";
static bool showprogress = false;
static int verbose = 0;
static int compresslevel = 0;
static bool includewal = false;
static bool streamwal = false;
static bool fastcheckpoint = false;
static bool writerecoveryconf = false;
static int standby_message_timeout = 10 * 1000; /* 10 sec = default */
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/* Progress counters */
static uint64 totalsize;
static uint64 totaldone;
static int tablespacecount;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/* Pipe to communicate with background wal receiver process */
#ifndef WIN32
static int bgpipe[2] = {-1, -1};
#endif
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/* Handle to child process */
static pid_t bgchild = -1;
/* End position for xlog streaming, empty string if unknown yet */
static XLogRecPtr xlogendptr;
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
Miscellaneous cleanup to silence compiler warnings seen on Mingw. Remove some dead code, conditionally declare some items or call some code, and fix one or two declarations.
2011-12-11 00:15:15 +01:00
#ifndef WIN32
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
static int has_xlogendptr = 0;
Miscellaneous cleanup to silence compiler warnings seen on Mingw. Remove some dead code, conditionally declare some items or call some code, and fix one or two declarations.
2011-12-11 00:15:15 +01:00
#else
static volatile LONG has_xlogendptr = 0;
#endif
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/* Contents of recovery.conf to be generated */
static PQExpBuffer recoveryconfcontents = NULL;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
static void progress_report(int tablespacenum, const char *filename);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
static void GenerateRecoveryConf(PGconn *conn);
static void WriteRecoveryConf(void);
fix up a couple non-prototypes of the form foo() to be foo(void) -- found using -Wstrict-prototypes
2011-03-27 22:31:41 +02:00
static void BaseBackup(void);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#ifdef HAVE_LIBZ
static const char *
Fix incorrect uses of gzFile gzFile is already a pointer, so code like gzFile *handle = gzopen(...) is wrong. This used to pass silently because gzFile used to be defined as void*, and you can assign a void* to a void**. But somewhere between zlib versions 1.2.3.4 and 1.2.6, the definition of gzFile was changed to struct gzFile_s *, and with that new definition this usage causes compiler warnings. So remove all those extra pointer decorations. There is a related issue in pg_backup_archiver.h, where FILE *FH; /* General purpose file handle */ is used throughout pg_dump as sometimes a real FILE* and sometimes a gzFile handle, which also causes warnings now. This is not yet fixed here, because it might need more code restructuring.
2012-03-02 21:30:01 +01:00
get_gz_error(gzFile gzf)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
int errnum;
const char *errmsg;
errmsg = gzerror(gzf, &errnum);
if (errnum == Z_ERRNO)
return strerror(errno);
else
return errmsg;
}
#endif
static void
usage(void)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
printf(_("%s takes a base backup of a running PostgreSQL server.\n\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname);
printf(_("Usage:\n"));
printf(_(" %s [OPTION]...\n"), progname);
printf(_("\nOptions controlling the output:\n"));
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
printf(_(" -R, --write-recovery-conf\n"
" write recovery.conf after backup\n"));
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
Revert behaviour of -x/--xlog to 9.1 semantics To replace it, add -X/--xlog-method that allows the specification of fetch or stream. Do this to avoid unnecessary backwards-incompatiblity. Spotted and suggested by Peter Eisentraut.
2012-06-10 13:43:51 +02:00
printf(_(" -X, --xlog-method=fetch|stream\n"
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
" include required WAL files with specified method\n"));
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
printf(_(" --xlogdir=XLOGDIR location for the transaction log directory\n"));
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
printf(_(" -z, --gzip compress tar output\n"));
printf(_(" -Z, --compress=0-9 compress tar output with given compression level\n"));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
printf(_("\nGeneral options:\n"));
printf(_(" -c, --checkpoint=fast|spread\n"
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
" set fast or spread checkpointing\n"));
printf(_(" -l, --label=LABEL set backup label\n"));
printf(_(" -P, --progress show progress information\n"));
printf(_(" -v, --verbose output verbose messages\n"));
printf(_(" -V, --version output version information, then exit\n"));
printf(_(" -?, --help show this help, then exit\n"));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
printf(_("\nConnection options:\n"));
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
printf(_(" -d, --dbname=CONNSTR connection string\n"));
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
printf(_(" -h, --host=HOSTNAME database server host or socket directory\n"));
printf(_(" -p, --port=PORT database server port number\n"));
Put options on man page and in help output in slightly better order
2012-08-24 06:06:59 +02:00
printf(_(" -s, --status-interval=INTERVAL\n"
" time between status packets sent to server (in seconds)\n"));
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
printf(_(" -U, --username=NAME connect as specified database user\n"));
printf(_(" -w, --no-password never prompt for password\n"));
printf(_(" -W, --password force password prompt (should happen automatically)\n"));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
printf(_("\nReport bugs to <pgsql-bugs@postgresql.org>.\n"));
}
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/*
Fix base backup streaming xlog from standby When backing up from a standby server, the backup process will not automatically switch xlog segment. So we must accept a partially transferred xlog file in this case, but rename it into position anyway. In passing, merge the two callbacks for segment end and stop stream into a single callback, since their implementations were close to identical, and rename this callback to reflect that it stops streaming rather than continues it. Patch by Magnus Hagander, review by Fujii Masao
2012-05-25 11:36:22 +02:00
* Called in the background process every time data is received.
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
* On Unix, we check to see if there is any data on our pipe
* (which would mean we have a stop position), and if it is, check if
* it is time to stop.
* On Windows, we are in a single process, so we can just check if it's
* time to stop.
*/
static bool
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
if (!has_xlogendptr)
{
#ifndef WIN32
fd_set fds;
struct timeval tv;
int r;
/*
* Don't have the end pointer yet - check our pipe to see if it has
* been sent yet.
*/
FD_ZERO(&fds);
FD_SET(bgpipe[0], &fds);
MemSet(&tv, 0, sizeof(tv));
r = select(bgpipe[0] + 1, &fds, NULL, NULL, &tv);
if (r == 1)
{
char xlogend[64];
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
uint32 hi,
lo;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
MemSet(xlogend, 0, sizeof(xlogend));
Correct off-by-one when reading from pipe In pg_basebackup.c:reached_end_position(), we're reading from an internal pipe with our own background process but we're possibly reading more bytes than will actually fit into our buffer due to an off-by-one error. As we're reading from an internal pipe there's no real risk here, but it's good form to not depend on such convenient arrangements. Bug spotted by the Coverity scanner. Back-patch to 9.2 where this showed up.
2013-07-15 16:42:27 +02:00
r = read(bgpipe[0], xlogend, sizeof(xlogend)-1);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
if (r < 0)
{
fprintf(stderr, _("%s: could not read from ready pipe: %s\n"),
progname, strerror(errno));
exit(1);
}
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
if (sscanf(xlogend, "%X/%X", &hi, &lo) != 2)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not parse transaction log location \"%s\"\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, xlogend);
exit(1);
}
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
xlogendptr = ((uint64) hi) << 32 | lo;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
has_xlogendptr = 1;
/*
* Fall through to check if we've reached the point further
* already.
*/
}
else
{
/*
* No data received on the pipe means we don't know the end
* position yet - so just say it's not time to stop yet.
*/
return false;
}
#else
/*
* On win32, has_xlogendptr is set by the main thread, so if it's not
* set here, we just go back and wait until it shows up.
*/
return false;
#endif
}
/*
* At this point we have an end pointer, so compare it to the current
* position to figure out if it's time to stop.
*/
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
if (segendpos >= xlogendptr)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
return true;
/*
* Have end pointer, but haven't reached it yet - so tell the caller to
* keep streaming.
*/
return false;
}
typedef struct
{
PGconn *bgconn;
XLogRecPtr startptr;
char xlogdir[MAXPGPATH];
char *sysidentifier;
int timeline;
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
} logstreamer_param;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
static int
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
LogStreamerMain(logstreamer_param *param)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
if (!ReceiveXlogStream(param->bgconn, param->startptr, param->timeline,
param->sysidentifier, param->xlogdir,
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
reached_end_position, standby_message_timeout,
Make pg_receivexlog and pg_basebackup -X stream work across timeline switches. This mirrors the changes done earlier to the server in standby mode. When receivelog reaches the end of a timeline, as reported by the server, it fetches the timeline history file of the next timeline, and restarts streaming from the new timeline by issuing a new START_STREAMING command. When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the last segment on the old timeline. This helps you to tell apart a partial segment left in the directory because of a timeline switch, and a completed segment. If you just follow a single server, it won't make a difference, but it can be significant in more complicated scenarios where new WAL is still generated on the old timeline. This includes two small changes to the streaming replication protocol: First, when you reach the end of timeline while streaming, the server now sends the TLI of the next timeline in the server's history to the client. pg_receivexlog uses that as the next timeline, so that it doesn't need to parse the timeline history file like a standby server does. Second, when BASE_BACKUP command sends the begin and end WAL positions, it now also sends the timeline IDs corresponding the positions.
2013-01-17 19:23:00 +01:00
NULL))
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/*
* Any errors will already have been reported in the function process,
* but we need to tell the parent that we didn't shutdown in a nice
* way.
*/
return 1;
PQfinish(param->bgconn);
return 0;
}
/*
* Initiate background process for receiving xlog during the backup.
* The background stream will use its own database connection so we can
* stream the logfile in parallel with the backups.
*/
static void
StartLogStreamer(char *startpos, uint32 timeline, char *sysidentifier)
{
logstreamer_param *param;
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
uint32 hi,
lo;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
param = pg_malloc0(sizeof(logstreamer_param));
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
param->timeline = timeline;
param->sysidentifier = sysidentifier;
/* Convert the starting position */
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
if (sscanf(startpos, "%X/%X", &hi, &lo) != 2)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not parse transaction log location \"%s\"\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, startpos);
disconnect_and_exit(1);
}
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
param->startptr = ((uint64) hi) << 32 | lo;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/* Round off to even segment position */
Replace XLogRecPtr struct with a 64-bit integer. This simplifies code that needs to do arithmetic on XLogRecPtrs. To avoid changing on-disk format of data pages, the LSN on data pages is still stored in the old format. That should keep pg_upgrade happy. However, we have XLogRecPtrs embedded in the control file, and in the structs that are sent over the replication protocol, so this changes breaks compatibility of pg_basebackup and server. I didn't do anything about this in this patch, per discussion on -hackers, the right thing to do would to be to change the replication protocol to be architecture-independent, so that you could use a newer version of pg_receivexlog, for example, against an older server version.
2012-06-24 17:51:37 +02:00
param->startptr -= param->startptr % XLOG_SEG_SIZE;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
#ifndef WIN32
/* Create our background pipe */
Remove now redundant pgpipe code.
2012-03-29 05:24:07 +02:00
if (pipe(bgpipe) < 0)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not create pipe for background process: %s\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, strerror(errno));
disconnect_and_exit(1);
}
#endif
/* Get a second connection */
param->bgconn = GetConnection();
Make pg_recievexlog by default loop on connection failures Avoids the need for an external script in the most common scenario. Behavior can be overridden using the -n/--noloop commandline parameter.
2012-05-27 11:05:24 +02:00
if (!param->bgconn)
/* Error message already written in GetConnection() */
exit(1);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/*
* Always in plain format, so we can write to basedir/pg_xlog. But the
* directory entry in the tar file may arrive later, so make sure it's
* created before we start.
*/
snprintf(param->xlogdir, sizeof(param->xlogdir), "%s/pg_xlog", basedir);
verify_dir_is_empty_or_create(param->xlogdir);
/*
* Start a child process and tell it to start streaming. On Unix, this is
* a fork(). On Windows, we create a thread.
*/
#ifndef WIN32
bgchild = fork();
if (bgchild == 0)
{
/* in child process */
exit(LogStreamerMain(param));
}
else if (bgchild < 0)
{
fprintf(stderr, _("%s: could not create background process: %s\n"),
progname, strerror(errno));
disconnect_and_exit(1);
}
/*
* Else we are in the parent process and all is well.
*/
#else /* WIN32 */
bgchild = _beginthreadex(NULL, 0, (void *) LogStreamerMain, param, 0, NULL);
if (bgchild == 0)
{
fprintf(stderr, _("%s: could not create background thread: %s\n"),
progname, strerror(errno));
disconnect_and_exit(1);
}
#endif
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* Verify that the given directory exists and is empty. If it does not
* exist, it is created. If it exists but is not empty, an error will
* be give and the process ended.
*/
static void
verify_dir_is_empty_or_create(char *dirname)
{
switch (pg_check_dir(dirname))
{
case 0:
/*
* Does not exist, so create
*/
if (pg_mkdir_p(dirname, S_IRWXU) == -1)
{
fprintf(stderr,
_("%s: could not create directory \"%s\": %s\n"),
progname, dirname, strerror(errno));
disconnect_and_exit(1);
}
return;
case 1:
/*
* Exists, empty
*/
return;
case 2:
Warn about initdb using mount-points Add code to detect and warn about trying to initdb or create pg_xlog on mount points.
2013-02-17 00:52:50 +01:00
case 3:
case 4:
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* Exists, not empty
*/
fprintf(stderr,
_("%s: directory \"%s\" exists but is not empty\n"),
progname, dirname);
disconnect_and_exit(1);
case -1:
/*
* Access problem
*/
fprintf(stderr, _("%s: could not access directory \"%s\": %s\n"),
progname, dirname, strerror(errno));
disconnect_and_exit(1);
}
}
/*
* Print a progress report based on the global variables. If verbose output
* is enabled, also print the current file name.
*/
static void
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progress_report(int tablespacenum, const char *filename)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
int percent = (int) ((totaldone / 1024) * 100 / totalsize);
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
char totaldone_str[32];
char totalsize_str[32];
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
Adjust total size in pg_basebackup progress report when reality changes When streaming including WAL, the size estimate will always be incorrect, since we don't know how much WAL is included. To make sure the output doesn't look completely unreasonable, this patch increases the total size whenever we go past the estimate, to make sure we never go above 100%.
2011-08-16 16:56:47 +02:00
/*
Fix pg_bsd_indent bug where newlines were not being trimmed from typedef lines. Update pg_bsd_indent required version to 1.1 (and update ftp site). Problem reported by Magnus.
2011-10-26 23:23:33 +02:00
* Avoid overflowing past 100% or the full size. This may make the total
* size number change as we approach the end of the backup (the estimate
* will always be wrong if WAL is included), but that's better than having
* the done column be bigger than the total.
Adjust total size in pg_basebackup progress report when reality changes When streaming including WAL, the size estimate will always be incorrect, since we don't know how much WAL is included. To make sure the output doesn't look completely unreasonable, this patch increases the total size whenever we go past the estimate, to make sure we never go above 100%.
2011-08-16 16:56:47 +02:00
*/
Add option to include WAL in base backup When included, this makes the base backup a complete working "clone" of the initial database, ready to have a postmaster started against it without the need to set up any log archiving or similar. Magnus Hagander, reviewed by Fujii Masao and Heikki Linnakangas
2011-01-30 21:30:09 +01:00
if (percent > 100)
percent = 100;
Adjust total size in pg_basebackup progress report when reality changes When streaming including WAL, the size estimate will always be incorrect, since we don't know how much WAL is included. To make sure the output doesn't look completely unreasonable, this patch increases the total size whenever we go past the estimate, to make sure we never go above 100%.
2011-08-16 16:56:47 +02:00
if (totaldone / 1024 > totalsize)
totalsize = totaldone / 1024;
Add option to include WAL in base backup When included, this makes the base backup a complete working "clone" of the initial database, ready to have a postmaster started against it without the need to set up any log archiving or similar. Magnus Hagander, reviewed by Fujii Masao and Heikki Linnakangas
2011-01-30 21:30:09 +01:00
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
/*
Fix pg_bsd_indent bug where newlines were not being trimmed from typedef lines. Update pg_bsd_indent required version to 1.1 (and update ftp site). Problem reported by Magnus.
2011-10-26 23:23:33 +02:00
* Separate step to keep platform-dependent format code out of
* translatable strings. And we only test for INT64_FORMAT availability
* in snprintf, not fprintf.
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
*/
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
snprintf(totaldone_str, sizeof(totaldone_str), INT64_FORMAT,
totaldone / 1024);
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
snprintf(totalsize_str, sizeof(totalsize_str), INT64_FORMAT, totalsize);
Truncate filenames in the leadning end in pg_basebackup verbose output When truncating at the end, like before, the output would often end up just showing the path instead of the filename. Also increase the length of the filename by 5, which still keeps us at less than 80 characters in most outputs.
2013-01-17 14:38:49 +01:00
#define VERBOSE_FILENAME_LENGTH 35
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
if (verbose)
Properly clear the verbose output of pg_basebackup Don't leave partial filename information on the last status or dangling characters if the previous filename was longer than the current one.
2011-03-19 16:38:50 +01:00
{
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
if (!filename)
Properly clear the verbose output of pg_basebackup Don't leave partial filename information on the last status or dangling characters if the previous filename was longer than the current one.
2011-03-19 16:38:50 +01:00
/*
* No filename given, so clear the status line (used for last
* call)
*/
fprintf(stderr,
Make size-output fixed length in pg_basebackup verbose mode This way the line doesn't shift right as the amount of data processed increases.
2013-01-17 14:43:33 +01:00
ngettext("%*s/%s kB (100%%), %d/%d tablespace %*s",
"%*s/%s kB (100%%), %d/%d tablespaces %*s",
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
tablespacecount),
Silence compiler warnings
2013-01-17 16:10:16 +01:00
(int) strlen(totalsize_str),
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
totaldone_str, totalsize_str,
Truncate filenames in the leadning end in pg_basebackup verbose output When truncating at the end, like before, the output would often end up just showing the path instead of the filename. Also increase the length of the filename by 5, which still keeps us at less than 80 characters in most outputs.
2013-01-17 14:38:49 +01:00
tablespacenum, tablespacecount,
VERBOSE_FILENAME_LENGTH + 5, "");
Properly clear the verbose output of pg_basebackup Don't leave partial filename information on the last status or dangling characters if the previous filename was longer than the current one.
2011-03-19 16:38:50 +01:00
else
Truncate filenames in the leadning end in pg_basebackup verbose output When truncating at the end, like before, the output would often end up just showing the path instead of the filename. Also increase the length of the filename by 5, which still keeps us at less than 80 characters in most outputs.
2013-01-17 14:38:49 +01:00
{
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
bool truncate = (strlen(filename) > VERBOSE_FILENAME_LENGTH);
Truncate filenames in the leadning end in pg_basebackup verbose output When truncating at the end, like before, the output would often end up just showing the path instead of the filename. Also increase the length of the filename by 5, which still keeps us at less than 80 characters in most outputs.
2013-01-17 14:38:49 +01:00
Properly clear the verbose output of pg_basebackup Don't leave partial filename information on the last status or dangling characters if the previous filename was longer than the current one.
2011-03-19 16:38:50 +01:00
fprintf(stderr,
Make size-output fixed length in pg_basebackup verbose mode This way the line doesn't shift right as the amount of data processed increases.
2013-01-17 14:43:33 +01:00
ngettext("%*s/%s kB (%d%%), %d/%d tablespace (%s%-*.*s)",
"%*s/%s kB (%d%%), %d/%d tablespaces (%s%-*.*s)",
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
tablespacecount),
Silence compiler warnings
2013-01-17 16:10:16 +01:00
(int) strlen(totalsize_str),
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
totaldone_str, totalsize_str, percent,
Truncate filenames in the leadning end in pg_basebackup verbose output When truncating at the end, like before, the output would often end up just showing the path instead of the filename. Also increase the length of the filename by 5, which still keeps us at less than 80 characters in most outputs.
2013-01-17 14:38:49 +01:00
tablespacenum, tablespacecount,
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
/* Prefix with "..." if we do leading truncation */
Truncate filenames in the leadning end in pg_basebackup verbose output When truncating at the end, like before, the output would often end up just showing the path instead of the filename. Also increase the length of the filename by 5, which still keeps us at less than 80 characters in most outputs.
2013-01-17 14:38:49 +01:00
truncate ? "..." : "",
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
truncate ? VERBOSE_FILENAME_LENGTH - 3 : VERBOSE_FILENAME_LENGTH,
truncate ? VERBOSE_FILENAME_LENGTH - 3 : VERBOSE_FILENAME_LENGTH,
/* Truncate filename at beginning if it's too long */
Truncate filenames in the leadning end in pg_basebackup verbose output When truncating at the end, like before, the output would often end up just showing the path instead of the filename. Also increase the length of the filename by 5, which still keeps us at less than 80 characters in most outputs.
2013-01-17 14:38:49 +01:00
truncate ? filename + strlen(filename) - VERBOSE_FILENAME_LENGTH + 3 : filename);
}
Properly clear the verbose output of pg_basebackup Don't leave partial filename information on the last status or dangling characters if the previous filename was longer than the current one.
2011-03-19 16:38:50 +01:00
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
else
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
fprintf(stderr,
Make size-output fixed length in pg_basebackup verbose mode This way the line doesn't shift right as the amount of data processed increases.
2013-01-17 14:43:33 +01:00
ngettext("%*s/%s kB (%d%%), %d/%d tablespace",
"%*s/%s kB (%d%%), %d/%d tablespaces",
Make pg_basebackup progress report translatable Also fix a potential portability bug, because INT64_FORMAT is only guaranteed to be available with snprintf, not fprintf.
2011-08-16 10:24:08 +02:00
tablespacecount),
Silence compiler warnings
2013-01-17 16:10:16 +01:00
(int) strlen(totalsize_str),
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
totaldone_str, totalsize_str, percent,
tablespacenum, tablespacecount);
Move \r out of translatable strings The translation tools are very unhappy about seeing \r in translatable strings, so move it to a separate fprintf call.
2011-08-17 09:52:35 +02:00
fprintf(stderr, "\r");
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/*
* Write a piece of tar data
*/
static void
writeTarData(
#ifdef HAVE_LIBZ
gzFile ztarfile,
#endif
FILE *tarfile, char *buf, int r, char *current_file)
{
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
if (gzwrite(ztarfile, buf, r) != r)
{
fprintf(stderr,
_("%s: could not write to compressed file \"%s\": %s\n"),
progname, current_file, get_gz_error(ztarfile));
disconnect_and_exit(1);
}
}
else
#endif
{
if (fwrite(buf, r, 1, tarfile) != 1)
{
fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
progname, current_file, strerror(errno));
disconnect_and_exit(1);
}
}
}
#ifdef HAVE_LIBZ
#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
#else
#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
#endif
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
*
* The file will be named base.tar[.gz] if it's for the main data directory
* or <tablespaceoid>.tar[.gz] if it's for another tablespace.
*
* No attempt to inspect or validate the contents of the file is done.
*/
static void
ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
{
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
char filename[MAXPGPATH];
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
char *copybuf = NULL;
FILE *tarfile = NULL;
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
char tarhdr[512];
bool basetablespace = PQgetisnull(res, rownum, 0);
bool in_tarhdr = true;
bool skip_file = false;
size_t tarhdrsz = 0;
size_t filesz = 0;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#ifdef HAVE_LIBZ
Fix incorrect uses of gzFile gzFile is already a pointer, so code like gzFile *handle = gzopen(...) is wrong. This used to pass silently because gzFile used to be defined as void*, and you can assign a void* to a void**. But somewhere between zlib versions 1.2.3.4 and 1.2.6, the definition of gzFile was changed to struct gzFile_s *, and with that new definition this usage causes compiler warnings. So remove all those extra pointer decorations. There is a related issue in pg_backup_archiver.h, where FILE *FH; /* General purpose file handle */ is used throughout pg_dump as sometimes a real FILE* and sometimes a gzFile handle, which also causes warnings now. This is not yet fixed here, because it might need more code restructuring.
2012-03-02 21:30:01 +01:00
gzFile ztarfile = NULL;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#endif
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
if (basetablespace)
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
{
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* Base tablespaces
*/
if (strcmp(basedir, "-") == 0)
Allow pg_basebackup compressed tar output to stdout
2011-05-29 00:17:37 +02:00
{
#ifdef HAVE_LIBZ
Fix oversights in pg_basebackup's -z (compression) option. The short-form -z switch didn't work, for lack of telling getopt_long about it; and even if specified long-form, it failed to do anything, because the various tests elsewhere in the file would take Z_DEFAULT_COMPRESSION (which is -1) as meaning "don't compress". Per bug #6060 from Shigehiro Honda, though I editorialized on his patch a bit.
2011-06-15 19:15:05 +02:00
if (compresslevel != 0)
Allow pg_basebackup compressed tar output to stdout
2011-05-29 00:17:37 +02:00
{
ztarfile = gzdopen(dup(fileno(stdout)), "wb");
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
if (gzsetparams(ztarfile, compresslevel,
Z_DEFAULT_STRATEGY) != Z_OK)
Allow pg_basebackup compressed tar output to stdout
2011-05-29 00:17:37 +02:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not set compression level %d: %s\n"),
Allow pg_basebackup compressed tar output to stdout
2011-05-29 00:17:37 +02:00
progname, compresslevel, get_gz_error(ztarfile));
disconnect_and_exit(1);
}
}
else
#endif
tarfile = stdout;
Avoid printing uninitialized filename variable in verbose mode When using verbose mode for pg_basebackup, in tar format sent to stdout, we'd print an unitialized buffer as the filename. Reported by Pontus Lundkvist
2014-02-09 11:54:33 +01:00
strcpy(filename, "-");
Allow pg_basebackup compressed tar output to stdout
2011-05-29 00:17:37 +02:00
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
else
{
#ifdef HAVE_LIBZ
Fix oversights in pg_basebackup's -z (compression) option. The short-form -z switch didn't work, for lack of telling getopt_long about it; and even if specified long-form, it failed to do anything, because the various tests elsewhere in the file would take Z_DEFAULT_COMPRESSION (which is -1) as meaning "don't compress". Per bug #6060 from Shigehiro Honda, though I editorialized on his patch a bit.
2011-06-15 19:15:05 +02:00
if (compresslevel != 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
snprintf(filename, sizeof(filename), "%s/base.tar.gz", basedir);
ztarfile = gzopen(filename, "wb");
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
if (gzsetparams(ztarfile, compresslevel,
Z_DEFAULT_STRATEGY) != Z_OK)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not set compression level %d: %s\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, compresslevel, get_gz_error(ztarfile));
disconnect_and_exit(1);
}
}
else
#endif
{
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
snprintf(filename, sizeof(filename), "%s/base.tar", basedir);
tarfile = fopen(filename, "wb");
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
}
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
else
{
/*
* Specific tablespace
*/
#ifdef HAVE_LIBZ
Fix oversights in pg_basebackup's -z (compression) option. The short-form -z switch didn't work, for lack of telling getopt_long about it; and even if specified long-form, it failed to do anything, because the various tests elsewhere in the file would take Z_DEFAULT_COMPRESSION (which is -1) as meaning "don't compress". Per bug #6060 from Shigehiro Honda, though I editorialized on his patch a bit.
2011-06-15 19:15:05 +02:00
if (compresslevel != 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
snprintf(filename, sizeof(filename), "%s/%s.tar.gz", basedir,
PQgetvalue(res, rownum, 0));
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
ztarfile = gzopen(filename, "wb");
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
if (gzsetparams(ztarfile, compresslevel,
Z_DEFAULT_STRATEGY) != Z_OK)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not set compression level %d: %s\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, compresslevel, get_gz_error(ztarfile));
disconnect_and_exit(1);
}
}
else
#endif
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
snprintf(filename, sizeof(filename), "%s/%s.tar", basedir,
PQgetvalue(res, rownum, 0));
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
tarfile = fopen(filename, "wb");
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
}
#ifdef HAVE_LIBZ
Fix oversights in pg_basebackup's -z (compression) option. The short-form -z switch didn't work, for lack of telling getopt_long about it; and even if specified long-form, it failed to do anything, because the various tests elsewhere in the file would take Z_DEFAULT_COMPRESSION (which is -1) as meaning "don't compress". Per bug #6060 from Shigehiro Honda, though I editorialized on his patch a bit.
2011-06-15 19:15:05 +02:00
if (compresslevel != 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
if (!ztarfile)
{
/* Compression is in use */
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not create compressed file \"%s\": %s\n"),
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progname, filename, get_gz_error(ztarfile));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
}
else
#endif
{
/* Either no zlib support, or zlib support but compresslevel = 0 */
if (!tarfile)
{
fprintf(stderr, _("%s: could not create file \"%s\": %s\n"),
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progname, filename, strerror(errno));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
}
/*
* Get the COPY data stream
*/
res = PQgetResult(conn);
if (PQresultStatus(res) != PGRES_COPY_OUT)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not get COPY data stream: %s"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, PQerrorMessage(conn));
disconnect_and_exit(1);
}
while (1)
{
int r;
if (copybuf != NULL)
{
PQfreemem(copybuf);
copybuf = NULL;
}
r = PQgetCopyData(conn, &copybuf, 0);
if (r == -1)
{
/*
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
* End of chunk. If requested, and this is the base tablespace,
* write recovery.conf into the tarfile. When done, close the file
* (but not stdout).
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
char zerobuf[1024];
MemSet(zerobuf, 0, sizeof(zerobuf));
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
if (basetablespace && writerecoveryconf)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
char header[512];
int padding;
tarCreateHeader(header, "recovery.conf", NULL,
recoveryconfcontents->len,
0600, 04000, 02000,
time(NULL));
padding = ((recoveryconfcontents->len + 511) & ~511) - recoveryconfcontents->len;
WRITE_TAR_DATA(header, sizeof(header));
WRITE_TAR_DATA(recoveryconfcontents->data, recoveryconfcontents->len);
if (padding)
WRITE_TAR_DATA(zerobuf, padding);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/* 2 * 512 bytes empty data at end of file */
WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
Allow pg_basebackup compressed tar output to stdout
2011-05-29 00:17:37 +02:00
#ifdef HAVE_LIBZ
pg_basebackup: Error handling fixes. Thomas Ogrisegg and Fujii Masao
2012-03-28 18:19:22 +02:00
if (ztarfile != NULL)
{
if (gzclose(ztarfile) != 0)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not close compressed file \"%s\": %s\n"),
pg_basebackup: Error handling fixes. Thomas Ogrisegg and Fujii Masao
2012-03-28 18:19:22 +02:00
progname, filename, get_gz_error(ztarfile));
disconnect_and_exit(1);
}
Allow pg_basebackup compressed tar output to stdout
2011-05-29 00:17:37 +02:00
}
else
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#endif
pg_basebackup: Error handling fixes. Thomas Ogrisegg and Fujii Masao
2012-03-28 18:19:22 +02:00
{
if (strcmp(basedir, "-") != 0)
{
if (fclose(tarfile) != 0)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not close file \"%s\": %s\n"),
pg_basebackup: Error handling fixes. Thomas Ogrisegg and Fujii Masao
2012-03-28 18:19:22 +02:00
progname, filename, strerror(errno));
disconnect_and_exit(1);
}
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
break;
}
else if (r == -2)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not read COPY data: %s"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, PQerrorMessage(conn));
disconnect_and_exit(1);
}
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
if (!writerecoveryconf || !basetablespace)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/*
* When not writing recovery.conf, or when not working on the base
* tablespace, we never have to look for an existing recovery.conf
* file in the stream.
*/
WRITE_TAR_DATA(copybuf, r);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
else
{
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/*
* Look for a recovery.conf in the existing tar stream. If it's
* there, we must skip it so we can later overwrite it with our
* own version of the file.
*
* To do this, we have to process the individual files inside the
* TAR stream. The stream consists of a header and zero or more
* chunks, all 512 bytes long. The stream from the server is
* broken up into smaller pieces, so we have to track the size of
* the files to find the next header structure.
*/
int rr = r;
int pos = 0;
while (rr > 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
if (in_tarhdr)
{
/*
* We're currently reading a header structure inside the
* TAR stream, i.e. the file metadata.
*/
if (tarhdrsz < 512)
{
/*
* Copy the header structure into tarhdr in case the
* header is not aligned to 512 bytes or it's not
* returned in whole by the last PQgetCopyData call.
*/
int hdrleft;
int bytes2copy;
hdrleft = 512 - tarhdrsz;
bytes2copy = (rr > hdrleft ? hdrleft : rr);
memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
rr -= bytes2copy;
pos += bytes2copy;
tarhdrsz += bytes2copy;
}
else
{
/*
* We have the complete header structure in tarhdr,
* look at the file metadata: - the subsequent file
* contents have to be skipped if the filename is
* recovery.conf - find out the size of the file
* padded to the next multiple of 512
*/
int padding;
skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
sscanf(&tarhdr[124], "%11o", (unsigned int *) &filesz);
padding = ((filesz + 511) & ~511) - filesz;
filesz += padding;
/* Next part is the file, not the header */
in_tarhdr = false;
/*
* If we're not skipping the file, write the tar
* header unmodified.
*/
if (!skip_file)
WRITE_TAR_DATA(tarhdr, 512);
}
}
else
{
/*
* We're processing a file's contents.
*/
if (filesz > 0)
{
/*
* We still have data to read (and possibly write).
*/
int bytes2write;
bytes2write = (filesz > rr ? rr : filesz);
if (!skip_file)
WRITE_TAR_DATA(copybuf + pos, bytes2write);
rr -= bytes2write;
pos += bytes2write;
filesz -= bytes2write;
}
else
{
/*
* No more data in the current file, the next piece of
* data (if any) will be a new file header structure.
*/
in_tarhdr = true;
skip_file = false;
tarhdrsz = 0;
filesz = 0;
}
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
}
totaldone += r;
if (showprogress)
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progress_report(rownum, filename);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
} /* while (1) */
if (copybuf != NULL)
PQfreemem(copybuf);
}
/*
* Receive a tar format stream from the connection to the server, and unpack
* the contents of it into a directory. Only files, directories and
* symlinks are supported, no other kinds of special files.
*
* If the data is for the main data directory, it will be restored in the
* specified directory. If it's for another tablespace, it will be restored
* in the original directory, since relocation of tablespaces is not
* supported.
*/
static void
ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum)
{
char current_path[MAXPGPATH];
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
char filename[MAXPGPATH];
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
int current_len_left;
Suppress uninitialized-variable warning.
2011-01-23 19:06:38 +01:00
int current_padding = 0;
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
bool basetablespace = PQgetisnull(res, rownum, 0);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
char *copybuf = NULL;
FILE *file = NULL;
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
if (basetablespace)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
/*
* Get the COPY data
*/
res = PQgetResult(conn);
if (PQresultStatus(res) != PGRES_COPY_OUT)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not get COPY data stream: %s"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, PQerrorMessage(conn));
disconnect_and_exit(1);
}
while (1)
{
int r;
if (copybuf != NULL)
{
PQfreemem(copybuf);
copybuf = NULL;
}
r = PQgetCopyData(conn, &copybuf, 0);
if (r == -1)
{
/*
* End of chunk
*/
if (file)
fclose(file);
break;
}
else if (r == -2)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not read COPY data: %s"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, PQerrorMessage(conn));
disconnect_and_exit(1);
}
if (file == NULL)
{
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
int filemode;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* No current file, so this must be the header for a new file
*/
if (r != 512)
{
Replace printf format %i by %d They are identical, but the overwhelming majority of the code uses %d, so standardize on that.
2011-07-26 21:54:29 +02:00
fprintf(stderr, _("%s: invalid tar block header size: %d\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, r);
disconnect_and_exit(1);
}
totaldone += 512;
if (sscanf(copybuf + 124, "%11o", &current_len_left) != 1)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not parse file size\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname);
disconnect_and_exit(1);
}
/* Set permissions on the file */
if (sscanf(&copybuf[100], "%07o ", &filemode) != 1)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not parse file mode\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname);
disconnect_and_exit(1);
}
/*
* All files are padded up to 512 bytes
*/
current_padding =
((current_len_left + 511) & ~511) - current_len_left;
/*
* First part of header is zero terminated filename
*/
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
snprintf(filename, sizeof(filename), "%s/%s", current_path,
copybuf);
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
if (filename[strlen(filename) - 1] == '/')
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
/*
* Ends in a slash means directory or symlink to directory
*/
if (copybuf[156] == '5')
{
/*
* Directory
*/
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
filename[strlen(filename) - 1] = '\0'; /* Remove trailing slash */
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
if (mkdir(filename, S_IRWXU) != 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/*
* When streaming WAL, pg_xlog will have been created
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
* by the wal receiver process. Also, when transaction
* log directory location was specified, pg_xlog has
* already been created as a symbolic link before
* starting the actual backup. So just ignore failure
* on them.
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
*/
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
if ((!streamwal && (strcmp(xlog_dir, "") == 0))
|| strcmp(filename + strlen(filename) - 8, "/pg_xlog") != 0)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
fprintf(stderr,
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
_("%s: could not create directory \"%s\": %s\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, filename, strerror(errno));
disconnect_and_exit(1);
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
#ifndef WIN32
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
if (chmod(filename, (mode_t) filemode))
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not set permissions on directory \"%s\": %s\n"),
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progname, filename, strerror(errno));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#endif
}
else if (copybuf[156] == '2')
{
/*
* Symbolic link
*/
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
filename[strlen(filename) - 1] = '\0'; /* Remove trailing slash */
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
if (symlink(&copybuf[157], filename) != 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
fprintf(stderr,
Message improvements
2011-05-24 20:25:28 +02:00
_("%s: could not create symbolic link from \"%s\" to \"%s\": %s\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, filename, &copybuf[157], strerror(errno));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
}
else
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: unrecognized link indicator \"%c\"\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, copybuf[156]);
disconnect_and_exit(1);
}
continue; /* directory or link handled */
}
/*
* regular file
*/
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
file = fopen(filename, "wb");
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
if (!file)
{
fprintf(stderr, _("%s: could not create file \"%s\": %s\n"),
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progname, filename, strerror(errno));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
#ifndef WIN32
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
if (chmod(filename, (mode_t) filemode))
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
fprintf(stderr, _("%s: could not set permissions on file \"%s\": %s\n"),
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progname, filename, strerror(errno));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#endif
if (current_len_left == 0)
{
/*
* Done with this file, next one will be a new tar header
*/
fclose(file);
file = NULL;
continue;
}
} /* new file */
else
{
/*
* Continuing blocks in existing file
*/
if (current_len_left == 0 && r == current_padding)
{
/*
* Received the padding block for this file, ignore it and
* close the file, then move on to the next tar header.
*/
fclose(file);
file = NULL;
totaldone += r;
continue;
}
if (fwrite(copybuf, r, 1, file) != 1)
{
fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progname, filename, strerror(errno));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
totaldone += r;
if (showprogress)
Use less cryptic variable names
2011-08-16 10:19:50 +02:00
progress_report(rownum, filename);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
current_len_left -= r;
if (current_len_left == 0 && current_padding == 0)
{
/*
* Received the last block, and there is no padding to be
* expected. Close the file and move on to the next tar
* header.
*/
fclose(file);
file = NULL;
continue;
}
} /* continuing data in existing file */
} /* loop over all data blocks */
if (file != NULL)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: COPY stream ended before last file was finished\n"),
progname);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
if (copybuf != NULL)
PQfreemem(copybuf);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
Don't attempt to write recovery.conf when -R is not specified Fixes segmentation fault during regular use. Fujii Masao
2013-01-09 16:57:32 +01:00
if (basetablespace && writerecoveryconf)
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
WriteRecoveryConf();
}
/*
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
* Escape a parameter value so that it can be used as part of a libpq
* connection string, e.g. in:
*
* application_name=<value>
*
* The returned string is malloc'd. Return NULL on out-of-memory.
*/
static char *
escapeConnectionParameter(const char *src)
{
bool need_quotes = false;
bool need_escaping = false;
const char *p;
char *dstbuf;
char *dst;
/*
* First check if quoting is needed. Any quote (') or backslash (\)
* characters need to be escaped. Parameters are separated by whitespace,
* so any string containing whitespace characters need to be quoted. An
* empty string is represented by ''.
*/
if (strchr(src, '\'') != NULL || strchr(src, '\\') != NULL)
need_escaping = true;
for (p = src; *p; p++)
{
Fix unportable usage of isspace(). Must cast char argument to unsigned to avoid doing the wrong thing with high-bit-set characters. Oversight in commit 30b5ede7157e34e77c7914b8ecfd55aa8da6edc3.
2013-06-01 19:58:23 +02:00
if (isspace((unsigned char) *p))
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
{
need_quotes = true;
break;
}
}
if (*src == '\0')
return pg_strdup("''");
if (!need_quotes && !need_escaping)
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
return pg_strdup(src); /* no quoting or escaping needed */
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
/*
* Allocate a buffer large enough for the worst case that all the source
* characters need to be escaped, plus quotes.
*/
dstbuf = pg_malloc(strlen(src) * 2 + 2 + 1);
dst = dstbuf;
if (need_quotes)
*(dst++) = '\'';
for (; *src; src++)
{
if (*src == '\'' || *src == '\\')
*(dst++) = '\\';
*(dst++) = *src;
}
if (need_quotes)
*(dst++) = '\'';
*dst = '\0';
return dstbuf;
}
/*
* Escape a string so that it can be used as a value in a key-value pair
* a configuration file.
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
*/
static char *
escape_quotes(const char *src)
{
char *result = escape_single_quotes_ascii(src);
if (!result)
{
fprintf(stderr, _("%s: out of memory\n"), progname);
exit(1);
}
return result;
}
/*
* Create a recovery.conf file in memory using a PQExpBuffer
*/
static void
GenerateRecoveryConf(PGconn *conn)
{
PQconninfoOption *connOptions;
PQconninfoOption *option;
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
PQExpBufferData conninfo_buf;
char *escaped;
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
recoveryconfcontents = createPQExpBuffer();
if (!recoveryconfcontents)
{
pg_basebackup: Add missing newlines to several error messages
2013-03-10 21:53:04 +01:00
fprintf(stderr, _("%s: out of memory\n"), progname);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
disconnect_and_exit(1);
}
connOptions = PQconninfo(conn);
if (connOptions == NULL)
{
pg_basebackup: Add missing newlines to several error messages
2013-03-10 21:53:04 +01:00
fprintf(stderr, _("%s: out of memory\n"), progname);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
disconnect_and_exit(1);
}
appendPQExpBufferStr(recoveryconfcontents, "standby_mode = 'on'\n");
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
initPQExpBuffer(&conninfo_buf);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
for (option = connOptions; option && option->keyword; option++)
{
/*
* Do not emit this setting if: - the setting is "replication",
* "dbname" or "fallback_application_name", since these would be
* overridden by the libpqwalreceiver module anyway. - not set or
* empty.
*/
if (strcmp(option->keyword, "replication") == 0 ||
strcmp(option->keyword, "dbname") == 0 ||
strcmp(option->keyword, "fallback_application_name") == 0 ||
(option->val == NULL) ||
(option->val != NULL && option->val[0] == '\0'))
continue;
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
/* Separate key-value pairs with spaces */
if (conninfo_buf.len != 0)
appendPQExpBufferStr(&conninfo_buf, " ");
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/*
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
* Write "keyword=value" pieces, the value string is escaped and/or
* quoted if necessary.
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
*/
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
escaped = escapeConnectionParameter(option->val);
appendPQExpBuffer(&conninfo_buf, "%s=%s", option->keyword, escaped);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
free(escaped);
}
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
/*
* Escape the connection string, so that it can be put in the config file.
* Note that this is different from the escaping of individual connection
* options above!
*/
escaped = escape_quotes(conninfo_buf.data);
appendPQExpBuffer(recoveryconfcontents, "primary_conninfo = '%s'\n", escaped);
free(escaped);
if (PQExpBufferBroken(recoveryconfcontents) ||
PQExpBufferDataBroken(conninfo_buf))
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
{
pg_basebackup: Add missing newlines to several error messages
2013-03-10 21:53:04 +01:00
fprintf(stderr, _("%s: out of memory\n"), progname);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
disconnect_and_exit(1);
}
Fix escaping in generated recovery.conf file. In the primary_conninfo line that "pg_basebackup -R" generates, single quotes in parameter values need to be escaped into \\'; the libpq parser requires the quotes to be escaped into \', and recovery.conf parser requires the \ to be escaped into \\. Also, don't quote parameter values unnecessarily, to make the connection string prettier. Most options in a libpq connection string don't need quoting. Reported by Hari Babu, closer analysis by Zoltan Boszormenyi, although I didn't use his patch.
2013-05-20 18:34:27 +02:00
termPQExpBuffer(&conninfo_buf);
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
PQconninfoFree(connOptions);
}
/*
* Write a recovery.conf file into the directory specified in basedir,
* with the contents already collected in memory.
*/
static void
WriteRecoveryConf(void)
{
char filename[MAXPGPATH];
FILE *cf;
sprintf(filename, "%s/recovery.conf", basedir);
cf = fopen(filename, "w");
if (cf == NULL)
{
pg_basebackup: Add missing newlines to several error messages
2013-03-10 21:53:04 +01:00
fprintf(stderr, _("%s: could not create file \"%s\": %s\n"), progname, filename, strerror(errno));
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
disconnect_and_exit(1);
}
if (fwrite(recoveryconfcontents->data, recoveryconfcontents->len, 1, cf) != 1)
{
fprintf(stderr,
_("%s: could not write to file \"%s\": %s\n"),
progname, filename, strerror(errno));
disconnect_and_exit(1);
}
fclose(cf);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
static void
fix up a couple non-prototypes of the form foo() to be foo(void) -- found using -Wstrict-prototypes
2011-03-27 22:31:41 +02:00
BaseBackup(void)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
PGresult *res;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
char *sysidentifier;
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
uint32 latesttli;
Make pg_receivexlog and pg_basebackup -X stream work across timeline switches. This mirrors the changes done earlier to the server in standby mode. When receivelog reaches the end of a timeline, as reported by the server, it fetches the timeline history file of the next timeline, and restarts streaming from the new timeline by issuing a new START_STREAMING command. When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the last segment on the old timeline. This helps you to tell apart a partial segment left in the directory because of a timeline switch, and a completed segment. If you just follow a single server, it won't make a difference, but it can be significant in more complicated scenarios where new WAL is still generated on the old timeline. This includes two small changes to the streaming replication protocol: First, when you reach the end of timeline while streaming, the server now sends the TLI of the next timeline in the server's history to the client. pg_receivexlog uses that as the next timeline, so that it doesn't need to parse the timeline history file like a standby server does. Second, when BASE_BACKUP command sends the begin and end WAL positions, it now also sends the timeline IDs corresponding the positions.
2013-01-17 19:23:00 +01:00
uint32 starttli;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
char current_path[MAXPGPATH];
char escaped_label[MAXPGPATH];
int i;
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
char xlogstart[64];
char xlogend[64];
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
int minServerMajor,
maxServerMajor;
int serverMajor;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* Connect in replication mode to the server
*/
conn = GetConnection();
Make pg_recievexlog by default loop on connection failures Avoids the need for an external script in the most common scenario. Behavior can be overridden using the -n/--noloop commandline parameter.
2012-05-27 11:05:24 +02:00
if (!conn)
/* Error message already written in GetConnection() */
exit(1);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
/*
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
* Check server version. BASE_BACKUP command was introduced in 9.1, so we
* can't work with servers older than 9.1.
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
*/
minServerMajor = 901;
maxServerMajor = PG_VERSION_NUM / 100;
serverMajor = PQserverVersion(conn) / 100;
if (serverMajor < minServerMajor || serverMajor > maxServerMajor)
{
const char *serverver = PQparameterStatus(conn, "server_version");
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
fprintf(stderr, _("%s: incompatible server version %s\n"),
progname, serverver ? serverver : "'unknown'");
disconnect_and_exit(1);
}
/*
* If WAL streaming was requested, also check that the server is new
* enough for that.
*/
if (streamwal && !CheckServerVersionForStreaming(conn))
{
/* Error message already written in CheckServerVersionForStreaming() */
disconnect_and_exit(1);
}
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/*
* Build contents of recovery.conf if requested
*/
if (writerecoveryconf)
GenerateRecoveryConf(conn);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
res = PQexec(conn, "IDENTIFY_SYSTEM");
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr, _("%s: could not send replication command \"%s\": %s"),
progname, "IDENTIFY_SYSTEM", PQerrorMessage(conn));
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
disconnect_and_exit(1);
}
Check number of fields in IDENTIFY_SYSTEM response Jaime Casanova
2012-01-20 13:57:02 +01:00
if (PQntuples(res) != 1 || PQnfields(res) != 3)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not identify system: got %d rows and %d fields, expected %d rows and %d fields\n"),
progname, PQntuples(res), PQnfields(res), 1, 3);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
disconnect_and_exit(1);
}
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
sysidentifier = pg_strdup(PQgetvalue(res, 0, 0));
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
latesttli = atoi(PQgetvalue(res, 0, 1));
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
PQclear(res);
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
/*
* Start the actual backup
*/
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
PQescapeStringConn(conn, escaped_label, label, sizeof(escaped_label), &i);
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
snprintf(current_path, sizeof(current_path),
"BASE_BACKUP LABEL '%s' %s %s %s %s",
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
escaped_label,
showprogress ? "PROGRESS" : "",
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
includewal && !streamwal ? "WAL" : "",
Use NOWAIT when including WAL in base backup Avoids warning and waiting for the last segment to be archived, which isn't necessary when we're including the required WAL in the backup itself.
2011-02-10 12:10:13 +01:00
fastcheckpoint ? "FAST" : "",
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
includewal ? "NOWAIT" : "");
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
if (PQsendQuery(conn, current_path) == 0)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr, _("%s: could not send replication command \"%s\": %s"),
progname, "BASE_BACKUP", PQerrorMessage(conn));
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
/*
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
* Get the starting xlog position
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
*/
res = PQgetResult(conn);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not initiate base backup: %s"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, PQerrorMessage(conn));
disconnect_and_exit(1);
}
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
if (PQntuples(res) != 1)
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
{
Make pg_receivexlog and pg_basebackup -X stream work across timeline switches. This mirrors the changes done earlier to the server in standby mode. When receivelog reaches the end of a timeline, as reported by the server, it fetches the timeline history file of the next timeline, and restarts streaming from the new timeline by issuing a new START_STREAMING command. When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the last segment on the old timeline. This helps you to tell apart a partial segment left in the directory because of a timeline switch, and a completed segment. If you just follow a single server, it won't make a difference, but it can be significant in more complicated scenarios where new WAL is still generated on the old timeline. This includes two small changes to the streaming replication protocol: First, when you reach the end of timeline while streaming, the server now sends the TLI of the next timeline in the server's history to the client. pg_receivexlog uses that as the next timeline, so that it doesn't need to parse the timeline history file like a standby server does. Second, when BASE_BACKUP command sends the begin and end WAL positions, it now also sends the timeline IDs corresponding the positions.
2013-01-17 19:23:00 +01:00
fprintf(stderr,
_("%s: server returned unexpected response to BASE_BACKUP command; got %d rows and %d fields, expected %d rows and %d fields\n"),
progname, PQntuples(res), PQnfields(res), 1, 2);
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
disconnect_and_exit(1);
}
Make pg_receivexlog and pg_basebackup -X stream work across timeline switches. This mirrors the changes done earlier to the server in standby mode. When receivelog reaches the end of a timeline, as reported by the server, it fetches the timeline history file of the next timeline, and restarts streaming from the new timeline by issuing a new START_STREAMING command. When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the last segment on the old timeline. This helps you to tell apart a partial segment left in the directory because of a timeline switch, and a completed segment. If you just follow a single server, it won't make a difference, but it can be significant in more complicated scenarios where new WAL is still generated on the old timeline. This includes two small changes to the streaming replication protocol: First, when you reach the end of timeline while streaming, the server now sends the TLI of the next timeline in the server's history to the client. pg_receivexlog uses that as the next timeline, so that it doesn't need to parse the timeline history file like a standby server does. Second, when BASE_BACKUP command sends the begin and end WAL positions, it now also sends the timeline IDs corresponding the positions.
2013-01-17 19:23:00 +01:00
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
strcpy(xlogstart, PQgetvalue(res, 0, 0));
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
/*
* 9.3 and later sends the TLI of the starting point. With older servers,
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
* assume it's the same as the latest timeline reported by
* IDENTIFY_SYSTEM.
Make pg_basebackup work with pre-9.3 servers, and add server version check. A new 'starttli' field was added to the response of BASE_BACKUP command. Make pg_basebackup tolerate the case that it's missing, so that it still works with older servers. Add an explicit check for the server version, so that you get a nicer error message if you try to use it with a pre-9.1 server. The streaming protocol message format changed in 9.3, so -X stream still won't work with pre-9.3 servers. I added a version check to ReceiveXLogStream() earlier, but write that slightly differently, so that in 9.4, it will still work with a 9.3 server. (In 9.4, the error message needs to be adjusted to "9.3 or above", though). Also, if the version check fails, don't retry.
2013-03-22 12:02:59 +01:00
*/
if (PQnfields(res) >= 2)
starttli = atoi(PQgetvalue(res, 0, 1));
else
starttli = latesttli;
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
PQclear(res);
MemSet(xlogend, 0, sizeof(xlogend));
Make pg_receivexlog and pg_basebackup -X stream work across timeline switches. This mirrors the changes done earlier to the server in standby mode. When receivelog reaches the end of a timeline, as reported by the server, it fetches the timeline history file of the next timeline, and restarts streaming from the new timeline by issuing a new START_STREAMING command. When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the last segment on the old timeline. This helps you to tell apart a partial segment left in the directory because of a timeline switch, and a completed segment. If you just follow a single server, it won't make a difference, but it can be significant in more complicated scenarios where new WAL is still generated on the old timeline. This includes two small changes to the streaming replication protocol: First, when you reach the end of timeline while streaming, the server now sends the TLI of the next timeline in the server's history to the client. pg_receivexlog uses that as the next timeline, so that it doesn't need to parse the timeline history file like a standby server does. Second, when BASE_BACKUP command sends the begin and end WAL positions, it now also sends the timeline IDs corresponding the positions.
2013-01-17 19:23:00 +01:00
if (verbose && includewal)
fprintf(stderr, _("transaction log start point: %s on timeline %u\n"),
xlogstart, starttli);
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
/*
* Get the header
*/
res = PQgetResult(conn);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: could not get backup header: %s"),
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
progname, PQerrorMessage(conn));
disconnect_and_exit(1);
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
if (PQntuples(res) < 1)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: no data returned from server\n"), progname);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
disconnect_and_exit(1);
}
/*
* Sum up the total size, for progress reporting
*/
totalsize = totaldone = 0;
tablespacecount = PQntuples(res);
for (i = 0; i < PQntuples(res); i++)
{
if (showprogress)
totalsize += atol(PQgetvalue(res, i, 2));
/*
* Verify tablespace directories are empty. Don't bother with the
* first once since it can be relocated, and it will be checked before
* we do anything anyway.
*/
Add option to include WAL in base backup When included, this makes the base backup a complete working "clone" of the initial database, ready to have a postmaster started against it without the need to set up any log archiving or similar. Magnus Hagander, reviewed by Fujii Masao and Heikki Linnakangas
2011-01-30 21:30:09 +01:00
if (format == 'p' && !PQgetisnull(res, i, 1))
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
verify_dir_is_empty_or_create(PQgetvalue(res, i, 1));
}
/*
* When writing to stdout, require a single tablespace
*/
if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: can only write single tablespace to stdout, database has %d\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, PQntuples(res));
disconnect_and_exit(1);
}
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/*
* If we're streaming WAL, start the streaming session before we start
* receiving the actual data chunks.
*/
if (streamwal)
{
if (verbose)
fprintf(stderr, _("%s: starting background WAL receiver\n"),
progname);
Make pg_receivexlog and pg_basebackup -X stream work across timeline switches. This mirrors the changes done earlier to the server in standby mode. When receivelog reaches the end of a timeline, as reported by the server, it fetches the timeline history file of the next timeline, and restarts streaming from the new timeline by issuing a new START_STREAMING command. When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the last segment on the old timeline. This helps you to tell apart a partial segment left in the directory because of a timeline switch, and a completed segment. If you just follow a single server, it won't make a difference, but it can be significant in more complicated scenarios where new WAL is still generated on the old timeline. This includes two small changes to the streaming replication protocol: First, when you reach the end of timeline while streaming, the server now sends the TLI of the next timeline in the server's history to the client. pg_receivexlog uses that as the next timeline, so that it doesn't need to parse the timeline history file like a standby server does. Second, when BASE_BACKUP command sends the begin and end WAL positions, it now also sends the timeline IDs corresponding the positions.
2013-01-17 19:23:00 +01:00
StartLogStreamer(xlogstart, starttli, sysidentifier);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* Start receiving chunks
*/
for (i = 0; i < PQntuples(res); i++)
{
if (format == 't')
ReceiveTarFile(conn, res, i);
else
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
if (showprogress)
{
Properly clear the verbose output of pg_basebackup Don't leave partial filename information on the last status or dangling characters if the previous filename was longer than the current one.
2011-03-19 16:38:50 +01:00
progress_report(PQntuples(res), NULL);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
fprintf(stderr, "\n"); /* Need to move to next line */
}
PQclear(res);
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
/*
* Get the stop position
*/
res = PQgetResult(conn);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not get transaction log end position from server: %s"),
pg_basebackup: Error message improvements. Fujii Masao
2012-03-28 14:43:35 +02:00
progname, PQerrorMessage(conn));
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
disconnect_and_exit(1);
}
if (PQntuples(res) != 1)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: no transaction log end position returned from server\n"),
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
progname);
disconnect_and_exit(1);
}
strcpy(xlogend, PQgetvalue(res, 0, 0));
if (verbose && includewal)
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr, "transaction log end point: %s\n", xlogend);
Include more status information in walsender results Add the current xlog insert location to the response of IDENTIFY_SYSTEM, and adds result sets containing start and stop location of backups to BASE_BACKUP responses.
2011-02-03 13:46:23 +01:00
PQclear(res);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
res = PQgetResult(conn);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, _("%s: final receive failed: %s"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, PQerrorMessage(conn));
disconnect_and_exit(1);
}
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
if (bgchild > 0)
{
#ifndef WIN32
Miscellaneous cleanup to silence compiler warnings seen on Mingw. Remove some dead code, conditionally declare some items or call some code, and fix one or two declarations.
2011-12-11 00:15:15 +01:00
int status;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
int r;
Miscellaneous cleanup to silence compiler warnings seen on Mingw. Remove some dead code, conditionally declare some items or call some code, and fix one or two declarations.
2011-12-11 00:15:15 +01:00
#else
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
DWORD status;
I missed some references to xlogid/xrecoff in Win32-only code. Fix.
2012-06-24 21:14:31 +02:00
uint32 hi,
lo;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
#endif
if (verbose)
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
pg_basebackup: Small message punctuation improvements
2012-12-19 13:01:11 +01:00
_("%s: waiting for background process to finish streaming ...\n"), progname);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
#ifndef WIN32
Remove now redundant pgpipe code.
2012-03-29 05:24:07 +02:00
if (write(bgpipe[1], xlogend, strlen(xlogend)) != strlen(xlogend))
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not send command to background pipe: %s\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, strerror(errno));
disconnect_and_exit(1);
}
/* Just wait for the background process to exit */
r = waitpid(bgchild, &status, 0);
if (r == -1)
{
fprintf(stderr, _("%s: could not wait for child process: %s\n"),
progname, strerror(errno));
disconnect_and_exit(1);
}
if (r != bgchild)
{
Replace printf format %i by %d see also ce8d7bb6440710058503d213b2aafcdf56a5b481
2012-03-30 19:40:46 +02:00
fprintf(stderr, _("%s: child %d died, expected %d\n"),
Cast some printf arguments to avoid possibly-nonportable behavior. Per compiler warnings on buildfarm member black_firefly.
2012-03-24 01:18:04 +01:00
progname, r, (int) bgchild);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
disconnect_and_exit(1);
}
if (!WIFEXITED(status))
{
fprintf(stderr, _("%s: child process did not exit normally\n"),
progname);
disconnect_and_exit(1);
}
if (WEXITSTATUS(status) != 0)
{
Replace printf format %i by %d see also ce8d7bb6440710058503d213b2aafcdf56a5b481
2012-03-30 19:40:46 +02:00
fprintf(stderr, _("%s: child process exited with error %d\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, WEXITSTATUS(status));
disconnect_and_exit(1);
}
/* Exited normally, we're happy! */
#else /* WIN32 */
/*
* On Windows, since we are in the same process, we can just store the
* value directly in the variable, and then set the flag that says
* it's there.
*/
I missed some references to xlogid/xrecoff in Win32-only code. Fix.
2012-06-24 21:14:31 +02:00
if (sscanf(xlogend, "%X/%X", &hi, &lo) != 2)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: could not parse transaction log location \"%s\"\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, xlogend);
Check number of fields in IDENTIFY_SYSTEM response Jaime Casanova
2012-01-20 13:57:02 +01:00
disconnect_and_exit(1);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
}
I missed some references to xlogid/xrecoff in Win32-only code. Fix.
2012-06-24 21:14:31 +02:00
xlogendptr = ((uint64) hi) << 32 | lo;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
InterlockedIncrement(&has_xlogendptr);
/* First wait for the thread to exit */
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
if (WaitForSingleObjectEx((HANDLE) bgchild, INFINITE, FALSE) !=
WAIT_OBJECT_0)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
_dosmaperr(GetLastError());
fprintf(stderr, _("%s: could not wait for child thread: %s\n"),
progname, strerror(errno));
disconnect_and_exit(1);
}
if (GetExitCodeThread((HANDLE) bgchild, &status) == 0)
{
_dosmaperr(GetLastError());
fprintf(stderr, _("%s: could not get child thread exit status: %s\n"),
progname, strerror(errno));
disconnect_and_exit(1);
}
if (status != 0)
{
fprintf(stderr, _("%s: child thread exited with error %u\n"),
Miscellaneous cleanup to silence compiler warnings seen on Mingw. Remove some dead code, conditionally declare some items or call some code, and fix one or two declarations.
2011-12-11 00:15:15 +01:00
progname, (unsigned int) status);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
disconnect_and_exit(1);
}
/* Exited normally, we're happy */
#endif
}
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
/* Free the recovery.conf contents */
destroyPQExpBuffer(recoveryconfcontents);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
/*
* End of copy data. Final result is already checked inside the loop.
*/
Check number of fields in IDENTIFY_SYSTEM response Jaime Casanova
2012-01-20 13:57:02 +01:00
PQclear(res);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
PQfinish(conn);
if (verbose)
Message style cleanup
2011-05-04 19:56:52 +02:00
fprintf(stderr, "%s: base backup completed\n", progname);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
}
int
main(int argc, char **argv)
{
static struct option long_options[] = {
{"help", no_argument, NULL, '?'},
{"version", no_argument, NULL, 'V'},
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
{"write-recovery-conf", no_argument, NULL, 'R'},
Revert behaviour of -x/--xlog to 9.1 semantics To replace it, add -X/--xlog-method that allows the specification of fetch or stream. Do this to avoid unnecessary backwards-incompatiblity. Spotted and suggested by Peter Eisentraut.
2012-06-10 13:43:51 +02:00
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
Add pg_basebackup -z option for compression with default level
2011-05-30 00:02:02 +02:00
{"gzip", no_argument, NULL, 'z'},
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{"compress", required_argument, NULL, 'Z'},
{"label", required_argument, NULL, 'l'},
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
{"dbname", required_argument, NULL, 'd'},
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{"host", required_argument, NULL, 'h'},
{"port", required_argument, NULL, 'p'},
{"username", required_argument, NULL, 'U'},
{"no-password", no_argument, NULL, 'w'},
{"password", no_argument, NULL, 'W'},
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
{"status-interval", required_argument, NULL, 's'},
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{"verbose", no_argument, NULL, 'v'},
{"progress", no_argument, NULL, 'P'},
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
{"xlogdir", required_argument, NULL, 1},
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{NULL, 0, NULL, 0}
};
int c;
int option_index;
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_basebackup"));
if (argc > 1)
{
if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
{
usage();
exit(0);
}
else if (strcmp(argv[1], "-V") == 0
|| strcmp(argv[1], "--version") == 0)
{
puts("pg_basebackup (PostgreSQL) " PG_VERSION);
exit(0);
}
}
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:d:c:h:p:U:s:wWvP",
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
long_options, &option_index)) != -1)
{
switch (c)
{
case 'D':
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
basedir = pg_strdup(optarg);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
break;
case 'F':
if (strcmp(optarg, "p") == 0 || strcmp(optarg, "plain") == 0)
format = 'p';
else if (strcmp(optarg, "t") == 0 || strcmp(optarg, "tar") == 0)
format = 't';
else
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: invalid output format \"%s\", must be \"plain\" or \"tar\"\n"),
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
progname, optarg);
exit(1);
}
break;
Add support for generating minimal recovery.conf when doing base backups Adds commandline option -R to pg_basebackup that creates a recovery.conf which enables standby mode using the same parameters that pg_basebackup used to connect to the master, and writes it into the output directory (or injects it in the tar file when tar format is used). Zoltan Boszormenyi, modified by Magnus Hagander, reviewed by Amit Kapila & Fujii Masao
2013-01-05 16:54:06 +01:00
case 'R':
writerecoveryconf = true;
break;
Add option to include WAL in base backup When included, this makes the base backup a complete working "clone" of the initial database, ready to have a postmaster started against it without the need to set up any log archiving or similar. Magnus Hagander, reviewed by Fujii Masao and Heikki Linnakangas
2011-01-30 21:30:09 +01:00
case 'x':
Revert behaviour of -x/--xlog to 9.1 semantics To replace it, add -X/--xlog-method that allows the specification of fetch or stream. Do this to avoid unnecessary backwards-incompatiblity. Spotted and suggested by Peter Eisentraut.
2012-06-10 13:43:51 +02:00
if (includewal)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: cannot specify both --xlog and --xlog-method\n"),
Revert behaviour of -x/--xlog to 9.1 semantics To replace it, add -X/--xlog-method that allows the specification of fetch or stream. Do this to avoid unnecessary backwards-incompatiblity. Spotted and suggested by Peter Eisentraut.
2012-06-10 13:43:51 +02:00
progname);
exit(1);
}
includewal = true;
streamwal = false;
break;
case 'X':
if (includewal)
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
_("%s: cannot specify both --xlog and --xlog-method\n"),
Revert behaviour of -x/--xlog to 9.1 semantics To replace it, add -X/--xlog-method that allows the specification of fetch or stream. Do this to avoid unnecessary backwards-incompatiblity. Spotted and suggested by Peter Eisentraut.
2012-06-10 13:43:51 +02:00
progname);
exit(1);
}
Add option to include WAL in base backup When included, this makes the base backup a complete working "clone" of the initial database, ready to have a postmaster started against it without the need to set up any log archiving or similar. Magnus Hagander, reviewed by Fujii Masao and Heikki Linnakangas
2011-01-30 21:30:09 +01:00
includewal = true;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
if (strcmp(optarg, "f") == 0 ||
strcmp(optarg, "fetch") == 0)
streamwal = false;
else if (strcmp(optarg, "s") == 0 ||
strcmp(optarg, "stream") == 0)
streamwal = true;
else
{
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
fprintf(stderr,
pg_basebackup: Correct error message It still thought that the --xlog-method option argument could be empty, as in a previous version of this feature.
2012-08-27 06:49:39 +02:00
_("%s: invalid xlog-method option \"%s\", must be \"fetch\" or \"stream\"\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname, optarg);
exit(1);
}
Add option to include WAL in base backup When included, this makes the base backup a complete working "clone" of the initial database, ready to have a postmaster started against it without the need to set up any log archiving or similar. Magnus Hagander, reviewed by Fujii Masao and Heikki Linnakangas
2011-01-30 21:30:09 +01:00
break;
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
case 1:
xlog_dir = pg_strdup(optarg);
break;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
case 'l':
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
label = pg_strdup(optarg);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
break;
Add pg_basebackup -z option for compression with default level
2011-05-30 00:02:02 +02:00
case 'z':
#ifdef HAVE_LIBZ
compresslevel = Z_DEFAULT_COMPRESSION;
#else
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
compresslevel = 1; /* will be rejected below */
Add pg_basebackup -z option for compression with default level
2011-05-30 00:02:02 +02:00
#endif
break;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
case 'Z':
compresslevel = atoi(optarg);
if (compresslevel <= 0 || compresslevel > 9)
{
fprintf(stderr, _("%s: invalid compression level \"%s\"\n"),
progname, optarg);
exit(1);
}
break;
case 'c':
Use pg_strcasecmp instead of strcasecmp for portability Per buildfarm.
2011-01-23 17:35:02 +01:00
if (pg_strcasecmp(optarg, "fast") == 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
fastcheckpoint = true;
Use pg_strcasecmp instead of strcasecmp for portability Per buildfarm.
2011-01-23 17:35:02 +01:00
else if (pg_strcasecmp(optarg, "spread") == 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
fastcheckpoint = false;
else
{
fprintf(stderr, _("%s: invalid checkpoint argument \"%s\", must be \"fast\" or \"spread\"\n"),
progname, optarg);
exit(1);
}
break;
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
case 'd':
connection_string = pg_strdup(optarg);
break;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
case 'h':
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
dbhost = pg_strdup(optarg);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
break;
case 'p':
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
dbport = pg_strdup(optarg);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
break;
case 'U':
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
dbuser = pg_strdup(optarg);
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
break;
case 'w':
dbgetpassword = -1;
break;
case 'W':
dbgetpassword = 1;
break;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
case 's':
Fix pg_basebackup/pg_receivexlog for floating point timestamps Since the replication protocol deals with TimestampTz, we need to care for the floating point case as well in the frontend tools. Fujii Masao, with changes from Magnus Hagander
2012-06-10 12:12:36 +02:00
standby_message_timeout = atoi(optarg) * 1000;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
if (standby_message_timeout < 0)
{
fprintf(stderr, _("%s: invalid status interval \"%s\"\n"),
progname, optarg);
exit(1);
}
break;
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
case 'v':
verbose++;
break;
case 'P':
showprogress = true;
break;
default:
/*
* getopt_long already emitted a complaint
*/
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
}
}
/*
* Any non-option arguments?
*/
if (optind < argc)
{
fprintf(stderr,
_("%s: too many command-line arguments (first is \"%s\")\n"),
progname, argv[optind]);
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
}
/*
* Required arguments
*/
if (basedir == NULL)
{
fprintf(stderr, _("%s: no target directory specified\n"), progname);
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
}
/*
* Mutually exclusive arguments
*/
Fix oversights in pg_basebackup's -z (compression) option. The short-form -z switch didn't work, for lack of telling getopt_long about it; and even if specified long-form, it failed to do anything, because the various tests elsewhere in the file would take Z_DEFAULT_COMPRESSION (which is -1) as meaning "don't compress". Per bug #6060 from Shigehiro Honda, though I editorialized on his patch a bit.
2011-06-15 19:15:05 +02:00
if (format == 'p' && compresslevel != 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
fprintf(stderr,
_("%s: only tar mode backups can be compressed\n"),
progname);
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
}
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
if (format != 'p' && streamwal)
{
fprintf(stderr,
pg_basebackup: Small message punctuation improvements
2012-12-19 13:01:11 +01:00
_("%s: WAL streaming can only be used in plain mode\n"),
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
progname);
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
}
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
if (strcmp(xlog_dir, "") != 0)
{
if (format != 'p')
{
fprintf(stderr,
_("%s: transaction log directory location can only be specified in plain mode\n"),
progname);
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
}
/* clean up xlog directory name, check it's absolute */
canonicalize_path(xlog_dir);
if (!is_absolute_path(xlog_dir))
{
fprintf(stderr, _("%s: transaction log directory location must be "
"an absolute path\n"), progname);
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
}
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
#ifndef HAVE_LIBZ
Fix oversights in pg_basebackup's -z (compression) option. The short-form -z switch didn't work, for lack of telling getopt_long about it; and even if specified long-form, it failed to do anything, because the various tests elsewhere in the file would take Z_DEFAULT_COMPRESSION (which is -1) as meaning "don't compress". Per bug #6060 from Shigehiro Honda, though I editorialized on his patch a bit.
2011-06-15 19:15:05 +02:00
if (compresslevel != 0)
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
{
fprintf(stderr,
_("%s: this build does not support compression\n"),
progname);
exit(1);
}
#endif
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
* unless we are writing to stdout.
*/
if (format == 'p' || strcmp(basedir, "-") != 0)
verify_dir_is_empty_or_create(basedir);
Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me.
2013-11-27 06:00:16 +01:00
/* Create transaction log symlink, if required */
if (strcmp(xlog_dir, "") != 0)
{
char *linkloc;
verify_dir_is_empty_or_create(xlog_dir);
/* form name of the place where the symlink must go */
linkloc = psprintf("%s/pg_xlog", basedir);
#ifdef HAVE_SYMLINK
if (symlink(xlog_dir, linkloc) != 0)
{
fprintf(stderr, _("%s: could not create symbolic link \"%s\": %s\n"),
progname, linkloc, strerror(errno));
exit(1);
}
#else
fprintf(stderr, _("%s: symlinks are not supported on this platform"));
exit(1);
#endif
free(linkloc);
}
Add pg_basebackup tool for streaming base backups This tool makes it possible to do the pg_start_backup/ copy files/pg_stop_backup step in a single command. There are still some steps to be done before this is a complete backup solution, such as the ability to stream the required WAL logs, but it's still usable, and could do with some buildfarm coverage. In passing, make the checkpoint request optionally fast instead of hardcoding it. Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine
2011-01-23 12:21:23 +01:00
BaseBackup();
return 0;
}