rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/bin/pg_basebackup/streamutil.c

531 lines
12 KiB
C
Raw Normal View History

Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
/*-------------------------------------------------------------------------
*
* streamutil.c - utility functions for pg_basebackup and pg_receivelog
*
* Author: Magnus Hagander <magnus@hagander.net>
*
Update copyright via script for 2017
2017-01-03 19:48:53 +01:00
* Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
*
* IDENTIFICATION
* src/bin/pg_basebackup/streamutil.c
*-------------------------------------------------------------------------
*/
Make xlog_internal.h includable in frontend context. This makes unnecessary the ugly hack used to #include postgres.h in pg_basebackup. Based on Alvaro Herrera's patch
2012-12-13 13:59:13 +01:00
#include "postgres_fe.h"
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
#include <stdio.h>
#include <string.h>
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>
/* for ntohl/htonl */
#include <netinet/in.h>
#include <arpa/inet.h>
/* local includes */
#include "receivelog.h"
#include "streamutil.h"
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
#include "pqexpbuffer.h"
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
#include "common/fe_memutils.h"
#include "datatype/timestamp.h"
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
Optionally don't error out due to preexisting slots in commandline utilities. pg_receivexlog and pg_recvlogical error out when --create-slot is specified and a slot with the same name already exists. In some cases, especially with pg_receivexlog, that's rather annoying and requires additional scripting. Backpatch to 9.5 as slot control functions have newly been added to pg_receivexlog, and there doesn't seem much point leaving it in a less useful state. Discussion: 20150619144755.GG29350@alap3.anarazel.de
2015-07-12 22:06:27 +02:00
#define ERRCODE_DUPLICATE_OBJECT "42710"
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
const char *progname;
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
char *connection_string = NULL;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
char *dbhost = NULL;
char *dbuser = NULL;
char *dbport = NULL;
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
char *dbname = NULL;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
int dbgetpassword = 0; /* 0=auto, -1=never, 1=always */
Simplify correct use of simple_prompt(). The previous API for this function had it returning a malloc'd string. That meant that callers had to check for NULL return, which few of them were doing, and it also meant that callers had to remember to free() the string later, which required extra logic in most cases. Instead, make simple_prompt() write into a buffer supplied by the caller. Anywhere that the maximum required input length is reasonably small, which is almost all of the callers, we can just use a local or static array as the buffer instead of dealing with malloc/free. A fair number of callers used "pointer == NULL" as a proxy for "haven't requested the password yet". Maintaining the same behavior requires adding a separate boolean flag for that, which adds back some of the complexity we save by removing free()s. Nonetheless, this nets out at a small reduction in overall code size, and considerably less code than we would have had if we'd added the missing NULL-return checks everywhere they were needed. In passing, clean up the API comment for simple_prompt() and get rid of a very-unnecessary malloc/free in its Windows code path. This is nominally a bug fix, but it does not seem worth back-patching, because the actual risk of an OOM failure in any of these places seems pretty tiny, and all of them are client-side not server-side anyway. This patch is by me, but it owes a great deal to Michael Paquier who identified the problem and drafted a patch for fixing it the other way. Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com>
2016-08-30 23:02:02 +02:00
static bool have_password = false;
static char password[100];
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
PGconn *conn = NULL;
Make pg_recievexlog by default loop on connection failures Avoids the need for an external script in the most common scenario. Behavior can be overridden using the -n/--noloop commandline parameter.
2012-05-27 11:05:24 +02:00
/*
* Connect to the server. Returns a valid PGconn pointer if connected,
* or NULL on non-permanent error. On permanent error, the function will
* call exit(1) directly.
*/
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
PGconn *
GetConnection(void)
{
PGconn *tmpconn;
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
int argcount = 7; /* dbname, replication, fallback_app_name,
* host, user, port, password */
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
int i;
const char **keywords;
const char **values;
Prevent pg_basebackup when integer_datetimes flag doesn't match. Magnus Hagander, reviewed by Fujii Masao, with slight wording changes by me.
2012-05-22 16:02:47 +02:00
const char *tmpparam;
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
bool need_password;
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
PQconninfoOption *conn_opts = NULL;
PQconninfoOption *conn_opt;
char *err_msg = NULL;
Obstruct shell, SQL, and conninfo injection via database and role names. Due to simplistic quoting and confusion of database names with conninfo strings, roles with the CREATEDB or CREATEROLE option could escalate to superuser privileges when a superuser next ran certain maintenance commands. The new coding rule for PQconnectdbParams() calls, documented at conninfo_array_parse(), is to pass expand_dbname=true and wrap literal database names in a trivial connection string. Escape zero-length values in appendConnStrVal(). Back-patch to 9.1 (all supported versions). Nathan Bossart, Michael Paquier, and Noah Misch. Reviewed by Peter Eisentraut. Reported by Nathan Bossart. Security: CVE-2016-5424
2016-08-08 16:07:46 +02:00
/* pg_recvlogical uses dbname only; others use connection_string only. */
Assert(dbname == NULL || connection_string == NULL);
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
/*
* Merge the connection info inputs given in form of connection string,
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
* options and default values (dbname=replication, replication=true, etc.)
Obstruct shell, SQL, and conninfo injection via database and role names. Due to simplistic quoting and confusion of database names with conninfo strings, roles with the CREATEDB or CREATEROLE option could escalate to superuser privileges when a superuser next ran certain maintenance commands. The new coding rule for PQconnectdbParams() calls, documented at conninfo_array_parse(), is to pass expand_dbname=true and wrap literal database names in a trivial connection string. Escape zero-length values in appendConnStrVal(). Back-patch to 9.1 (all supported versions). Nathan Bossart, Michael Paquier, and Noah Misch. Reviewed by Peter Eisentraut. Reported by Nathan Bossart. Security: CVE-2016-5424
2016-08-08 16:07:46 +02:00
* Explicitly discard any dbname value in the connection string;
* otherwise, PQconnectdbParams() would interpret that value as being
* itself a connection string.
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
*/
i = 0;
if (connection_string)
{
conn_opts = PQconninfoParse(connection_string, &err_msg);
if (conn_opts == NULL)
{
Fix PQconninfoParse error message handling The returned error message already includes a newline, but the callers were adding their own when printing it out.
2013-07-16 02:04:14 +02:00
fprintf(stderr, "%s: %s", progname, err_msg);
pg_receivexlog - Exit on failure to parse In streamutil.c:GetConnection(), upgrade failure to parse the connection string to an exit(1) instead of simply returning NULL. Most callers already immediately exited, but pg_receivexlog would loop on this case, continually trying to re-parse the connection string (which can't be changed after pg_receivexlog has started). GetConnection() was already expected to exit(1) in some cases (eg: failure to allocate memory or if unable to determine the integer_datetimes flag), so this change shouldn't surprise anyone. Began looking at this due to the Coverity scanner complaining that we were leaking err_msg in this case- no longer an issue since we just exit(1) immediately.
2013-07-14 21:31:23 +02:00
exit(1);
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
}
for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
{
Obstruct shell, SQL, and conninfo injection via database and role names. Due to simplistic quoting and confusion of database names with conninfo strings, roles with the CREATEDB or CREATEROLE option could escalate to superuser privileges when a superuser next ran certain maintenance commands. The new coding rule for PQconnectdbParams() calls, documented at conninfo_array_parse(), is to pass expand_dbname=true and wrap literal database names in a trivial connection string. Escape zero-length values in appendConnStrVal(). Back-patch to 9.1 (all supported versions). Nathan Bossart, Michael Paquier, and Noah Misch. Reviewed by Peter Eisentraut. Reported by Nathan Bossart. Security: CVE-2016-5424
2016-08-08 16:07:46 +02:00
if (conn_opt->val != NULL && conn_opt->val[0] != '\0' &&
strcmp(conn_opt->keyword, "dbname") != 0)
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
argcount++;
}
keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
values = pg_malloc0((argcount + 1) * sizeof(*values));
for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
{
Obstruct shell, SQL, and conninfo injection via database and role names. Due to simplistic quoting and confusion of database names with conninfo strings, roles with the CREATEDB or CREATEROLE option could escalate to superuser privileges when a superuser next ran certain maintenance commands. The new coding rule for PQconnectdbParams() calls, documented at conninfo_array_parse(), is to pass expand_dbname=true and wrap literal database names in a trivial connection string. Escape zero-length values in appendConnStrVal(). Back-patch to 9.1 (all supported versions). Nathan Bossart, Michael Paquier, and Noah Misch. Reviewed by Peter Eisentraut. Reported by Nathan Bossart. Security: CVE-2016-5424
2016-08-08 16:07:46 +02:00
if (conn_opt->val != NULL && conn_opt->val[0] != '\0' &&
strcmp(conn_opt->keyword, "dbname") != 0)
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
{
keywords[i] = conn_opt->keyword;
values[i] = conn_opt->val;
i++;
}
}
}
else
{
keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
values = pg_malloc0((argcount + 1) * sizeof(*values));
}
keywords[i] = "dbname";
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
values[i] = dbname == NULL ? "replication" : dbname;
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
i++;
keywords[i] = "replication";
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
values[i] = dbname == NULL ? "true" : "database";
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
i++;
keywords[i] = "fallback_application_name";
values[i] = progname;
i++;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
if (dbhost)
{
keywords[i] = "host";
values[i] = dbhost;
i++;
}
if (dbuser)
{
keywords[i] = "user";
values[i] = dbuser;
i++;
}
if (dbport)
{
keywords[i] = "port";
values[i] = dbport;
i++;
}
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
/* If -W was given, force prompt for password, but only the first time */
Simplify correct use of simple_prompt(). The previous API for this function had it returning a malloc'd string. That meant that callers had to check for NULL return, which few of them were doing, and it also meant that callers had to remember to free() the string later, which required extra logic in most cases. Instead, make simple_prompt() write into a buffer supplied by the caller. Anywhere that the maximum required input length is reasonably small, which is almost all of the callers, we can just use a local or static array as the buffer instead of dealing with malloc/free. A fair number of callers used "pointer == NULL" as a proxy for "haven't requested the password yet". Maintaining the same behavior requires adding a separate boolean flag for that, which adds back some of the complexity we save by removing free()s. Nonetheless, this nets out at a small reduction in overall code size, and considerably less code than we would have had if we'd added the missing NULL-return checks everywhere they were needed. In passing, clean up the API comment for simple_prompt() and get rid of a very-unnecessary malloc/free in its Windows code path. This is nominally a bug fix, but it does not seem worth back-patching, because the actual risk of an OOM failure in any of these places seems pretty tiny, and all of them are client-side not server-side anyway. This patch is by me, but it owes a great deal to Michael Paquier who identified the problem and drafted a patch for fixing it the other way. Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com>
2016-08-30 23:02:02 +02:00
need_password = (dbgetpassword == 1 && !have_password);
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
Tweak streamutil.c further to avoid scan-build warning The previous change added a new scan-build warning about need_password assigned but not read.
2013-11-22 03:46:43 +01:00
do
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
/* Get a new password if appropriate */
if (need_password)
{
Simplify correct use of simple_prompt(). The previous API for this function had it returning a malloc'd string. That meant that callers had to check for NULL return, which few of them were doing, and it also meant that callers had to remember to free() the string later, which required extra logic in most cases. Instead, make simple_prompt() write into a buffer supplied by the caller. Anywhere that the maximum required input length is reasonably small, which is almost all of the callers, we can just use a local or static array as the buffer instead of dealing with malloc/free. A fair number of callers used "pointer == NULL" as a proxy for "haven't requested the password yet". Maintaining the same behavior requires adding a separate boolean flag for that, which adds back some of the complexity we save by removing free()s. Nonetheless, this nets out at a small reduction in overall code size, and considerably less code than we would have had if we'd added the missing NULL-return checks everywhere they were needed. In passing, clean up the API comment for simple_prompt() and get rid of a very-unnecessary malloc/free in its Windows code path. This is nominally a bug fix, but it does not seem worth back-patching, because the actual risk of an OOM failure in any of these places seems pretty tiny, and all of them are client-side not server-side anyway. This patch is by me, but it owes a great deal to Michael Paquier who identified the problem and drafted a patch for fixing it the other way. Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com>
2016-08-30 23:02:02 +02:00
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
need_password = false;
}
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
/* Use (or reuse, on a subsequent connection) password if we have it */
Simplify correct use of simple_prompt(). The previous API for this function had it returning a malloc'd string. That meant that callers had to check for NULL return, which few of them were doing, and it also meant that callers had to remember to free() the string later, which required extra logic in most cases. Instead, make simple_prompt() write into a buffer supplied by the caller. Anywhere that the maximum required input length is reasonably small, which is almost all of the callers, we can just use a local or static array as the buffer instead of dealing with malloc/free. A fair number of callers used "pointer == NULL" as a proxy for "haven't requested the password yet". Maintaining the same behavior requires adding a separate boolean flag for that, which adds back some of the complexity we save by removing free()s. Nonetheless, this nets out at a small reduction in overall code size, and considerably less code than we would have had if we'd added the missing NULL-return checks everywhere they were needed. In passing, clean up the API comment for simple_prompt() and get rid of a very-unnecessary malloc/free in its Windows code path. This is nominally a bug fix, but it does not seem worth back-patching, because the actual risk of an OOM failure in any of these places seems pretty tiny, and all of them are client-side not server-side anyway. This patch is by me, but it owes a great deal to Michael Paquier who identified the problem and drafted a patch for fixing it the other way. Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com>
2016-08-30 23:02:02 +02:00
if (have_password)
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
keywords[i] = "password";
Simplify correct use of simple_prompt(). The previous API for this function had it returning a malloc'd string. That meant that callers had to check for NULL return, which few of them were doing, and it also meant that callers had to remember to free() the string later, which required extra logic in most cases. Instead, make simple_prompt() write into a buffer supplied by the caller. Anywhere that the maximum required input length is reasonably small, which is almost all of the callers, we can just use a local or static array as the buffer instead of dealing with malloc/free. A fair number of callers used "pointer == NULL" as a proxy for "haven't requested the password yet". Maintaining the same behavior requires adding a separate boolean flag for that, which adds back some of the complexity we save by removing free()s. Nonetheless, this nets out at a small reduction in overall code size, and considerably less code than we would have had if we'd added the missing NULL-return checks everywhere they were needed. In passing, clean up the API comment for simple_prompt() and get rid of a very-unnecessary malloc/free in its Windows code path. This is nominally a bug fix, but it does not seem worth back-patching, because the actual risk of an OOM failure in any of these places seems pretty tiny, and all of them are client-side not server-side anyway. This patch is by me, but it owes a great deal to Michael Paquier who identified the problem and drafted a patch for fixing it the other way. Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com>
2016-08-30 23:02:02 +02:00
values[i] = password;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
}
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
else
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
{
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
keywords[i] = NULL;
values[i] = NULL;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
}
tmpconn = PQconnectdbParams(keywords, values, true);
Fix memory and file descriptor leaks in pg_receivexlog/pg_basebackup When the internal loop mode was added, freeing memory and closing filedescriptors before returning became important, and a few cases in the code missed that. Fujii Masao
2012-07-12 13:31:19 +02:00
/*
* If there is too little memory even to allocate the PGconn object
* and PQconnectdbParams returns NULL, we call exit(1) directly.
*/
if (!tmpconn)
{
fprintf(stderr, _("%s: could not connect to server\n"),
progname);
exit(1);
}
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
/* If we need a password and -w wasn't given, loop back and get one */
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
if (PQstatus(tmpconn) == CONNECTION_BAD &&
PQconnectionNeedsPassword(tmpconn) &&
dbgetpassword != -1)
{
Fix memory and file descriptor leaks in pg_receivexlog/pg_basebackup When the internal loop mode was added, freeing memory and closing filedescriptors before returning became important, and a few cases in the code missed that. Fujii Masao
2012-07-12 13:31:19 +02:00
PQfinish(tmpconn);
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
need_password = true;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
}
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
}
Tweak streamutil.c further to avoid scan-build warning The previous change added a new scan-build warning about need_password assigned but not read.
2013-11-22 03:46:43 +01:00
while (need_password);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
if (PQstatus(tmpconn) != CONNECTION_OK)
{
pg_basebackup: Remove redundant newline in error message
2015-06-23 02:39:41 +02:00
fprintf(stderr, _("%s: could not connect to server: %s"),
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
progname, PQerrorMessage(tmpconn));
PQfinish(tmpconn);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
free(values);
free(keywords);
Add -d option to pg_basebackup and pg_receivexlog, for connection string. Without this, there's no way to pass arbitrary libpq connection parameters to these applications. It's a bit strange that the option is called -d/--dbname, when in fact you can *not* pass a database name in it, but it's consistent with other client applications where a connection string is also passed using -d. Original patch by Amit Kapila, heavily modified by me.
2013-02-25 13:48:27 +01:00
if (conn_opts)
PQconninfoFree(conn_opts);
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
return NULL;
}
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
/* Connection ok! */
free(values);
free(keywords);
if (conn_opts)
PQconninfoFree(conn_opts);
/*
Remove now-dead code for !HAVE_INT64_TIMESTAMP. This is a basically mechanical removal of #ifdef HAVE_INT64_TIMESTAMP tests and the negative-case controlled code. Discussion: https://postgr.es/m/26788.1487455319@sss.pgh.pa.us
2017-02-23 20:04:43 +01:00
* Ensure we have the same value of integer_datetimes (now always "on") as
* the server we are connecting to.
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
*/
tmpparam = PQparameterStatus(tmpconn, "integer_datetimes");
if (!tmpparam)
{
fprintf(stderr,
_("%s: could not determine server setting for integer_datetimes\n"),
progname);
PQfinish(tmpconn);
exit(1);
}
Prevent pg_basebackup when integer_datetimes flag doesn't match. Magnus Hagander, reviewed by Fujii Masao, with slight wording changes by me.
2012-05-22 16:02:47 +02:00
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
if (strcmp(tmpparam, "on") != 0)
{
fprintf(stderr,
pg_basebackup: stylistic adjustments The most user-visible part of this is to change the long options --statusint and --noloop to --status-interval and --no-loop, respectively, per discussion. Also, consistently enclose file names in double quotes, per our conventions; and consistently use the term "transaction log file" to talk about WAL segments. (Someday we may need to go over this terminology and make it consistent across the whole source code.) Finally, reflow the code to better fit in 80 columns, and have pgindent fix it up some more.
2012-07-31 16:11:11 +02:00
_("%s: integer_datetimes compile flag does not match server\n"),
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
progname);
PQfinish(tmpconn);
exit(1);
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
}
Clean up password prompting logic in streamutil.c. The previous coding was fairly unreadable and drew double-free warnings from clang. I believe the double free was actually not reachable, because PQconnectionNeedsPassword is coded to not return true if a password was provided, so that the loop can't iterate more than twice. Nonetheless it seems worth rewriting. No back-patch since this is just cosmetic.
2013-11-15 23:27:41 +01:00
return tmpconn;
Implement streaming xlog for backup tools Add option for parallel streaming of the transaction log while a base backup is running, to get the logfiles before the server has removed them. Also add a tool called pg_receivexlog, which streams the transaction log into files, creating a log archive without having to wait for segments to complete, thus decreasing the window of data loss without having to waste space using archive_timeout. This works best in combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
}
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
/*
* Run IDENTIFY_SYSTEM through a given connection and give back to caller
* some result information if requested:
* - System identifier
Make pg_receivexlog silent with 9.3 and older servers A pointless and confusing error message is shown to the user when attempting to identify a 9.3 or older remote server with a 9.5/9.6 pg_receivexlog, because the return signature of IDENTIFY_SYSTEM was changed in 9.4. There's no good reason for the warning message, so shuffle code around to keep it quiet. (pg_recvlogical is also affected by this commit, but since it obviously cannot work with 9.3 that doesn't actually matter much.) Backpatch to 9.5. Reported by Marco Nenciarini, who also wrote the initial patch. Further tweaked by Robert Haas and Fujii Masao; reviewed by Michael Paquier and Craig Ringer.
2016-01-05 21:25:12 +01:00
* - Current timeline ID
* - Start LSN position
* - Database name (NULL in servers prior to 9.4)
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
*/
bool
RunIdentifySystem(PGconn *conn, char **sysid, TimeLineID *starttli,
XLogRecPtr *startpos, char **db_name)
{
PGresult *res;
pgindent run for 9.5
2015-05-24 03:35:49 +02:00
uint32 hi,
lo;
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
/* Check connection existence */
Assert(conn != NULL);
res = PQexec(conn, "IDENTIFY_SYSTEM");
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
fprintf(stderr, _("%s: could not send replication command \"%s\": %s"),
progname, "IDENTIFY_SYSTEM", PQerrorMessage(conn));
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
PQclear(res);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
return false;
}
if (PQntuples(res) != 1 || PQnfields(res) < 3)
{
fprintf(stderr,
_("%s: could not identify system: got %d rows and %d fields, expected %d rows and %d or more fields\n"),
progname, PQntuples(res), PQnfields(res), 1, 3);
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
PQclear(res);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
return false;
}
/* Get system identifier */
if (sysid != NULL)
*sysid = pg_strdup(PQgetvalue(res, 0, 0));
/* Get timeline ID to start streaming from */
if (starttli != NULL)
*starttli = atoi(PQgetvalue(res, 0, 1));
/* Get LSN start position if necessary */
if (startpos != NULL)
{
if (sscanf(PQgetvalue(res, 0, 2), "%X/%X", &hi, &lo) != 2)
{
fprintf(stderr,
pgindent run for 9.5
2015-05-24 03:35:49 +02:00
_("%s: could not parse transaction log location \"%s\"\n"),
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
progname, PQgetvalue(res, 0, 2));
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
PQclear(res);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
return false;
}
*startpos = ((uint64) hi) << 32 | lo;
}
/* Get database name, only available in 9.4 and newer versions */
pgindent run for 9.5
2015-05-24 03:35:49 +02:00
if (db_name != NULL)
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
{
Make pg_receivexlog silent with 9.3 and older servers A pointless and confusing error message is shown to the user when attempting to identify a 9.3 or older remote server with a 9.5/9.6 pg_receivexlog, because the return signature of IDENTIFY_SYSTEM was changed in 9.4. There's no good reason for the warning message, so shuffle code around to keep it quiet. (pg_recvlogical is also affected by this commit, but since it obviously cannot work with 9.3 that doesn't actually matter much.) Backpatch to 9.5. Reported by Marco Nenciarini, who also wrote the initial patch. Further tweaked by Robert Haas and Fujii Masao; reviewed by Michael Paquier and Craig Ringer.
2016-01-05 21:25:12 +01:00
*db_name = NULL;
if (PQserverVersion(conn) >= 90400)
{
if (PQnfields(res) < 4)
{
fprintf(stderr,
_("%s: could not identify system: got %d rows and %d fields, expected %d rows and %d or more fields\n"),
progname, PQntuples(res), PQnfields(res), 1, 4);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
Make pg_receivexlog silent with 9.3 and older servers A pointless and confusing error message is shown to the user when attempting to identify a 9.3 or older remote server with a 9.5/9.6 pg_receivexlog, because the return signature of IDENTIFY_SYSTEM was changed in 9.4. There's no good reason for the warning message, so shuffle code around to keep it quiet. (pg_recvlogical is also affected by this commit, but since it obviously cannot work with 9.3 that doesn't actually matter much.) Backpatch to 9.5. Reported by Marco Nenciarini, who also wrote the initial patch. Further tweaked by Robert Haas and Fujii Masao; reviewed by Michael Paquier and Craig Ringer.
2016-01-05 21:25:12 +01:00
PQclear(res);
return false;
}
if (!PQgetisnull(res, 0, 3))
*db_name = pg_strdup(PQgetvalue(res, 0, 3));
}
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
}
PQclear(res);
return true;
}
/*
* Create a replication slot for the given connection. This function
* returns true in case of success as well as the start position
* obtained after the slot creation.
*/
bool
CreateReplicationSlot(PGconn *conn, const char *slot_name, const char *plugin,
Optionally don't error out due to preexisting slots in commandline utilities. pg_receivexlog and pg_recvlogical error out when --create-slot is specified and a slot with the same name already exists. In some cases, especially with pg_receivexlog, that's rather annoying and requires additional scripting. Backpatch to 9.5 as slot control functions have newly been added to pg_receivexlog, and there doesn't seem much point leaving it in a less useful state. Discussion: 20150619144755.GG29350@alap3.anarazel.de
2015-07-12 22:06:27 +02:00
bool is_physical, bool slot_exists_ok)
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
{
PQExpBuffer query;
PGresult *res;
query = createPQExpBuffer();
Assert((is_physical && plugin == NULL) ||
(!is_physical && plugin != NULL));
Assert(slot_name != NULL);
/* Build query */
if (is_physical)
appendPQExpBuffer(query, "CREATE_REPLICATION_SLOT \"%s\" PHYSICAL",
slot_name);
else
appendPQExpBuffer(query, "CREATE_REPLICATION_SLOT \"%s\" LOGICAL \"%s\"",
slot_name, plugin);
res = PQexec(conn, query->data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
Optionally don't error out due to preexisting slots in commandline utilities. pg_receivexlog and pg_recvlogical error out when --create-slot is specified and a slot with the same name already exists. In some cases, especially with pg_receivexlog, that's rather annoying and requires additional scripting. Backpatch to 9.5 as slot control functions have newly been added to pg_receivexlog, and there doesn't seem much point leaving it in a less useful state. Discussion: 20150619144755.GG29350@alap3.anarazel.de
2015-07-12 22:06:27 +02:00
const char *sqlstate = PQresultErrorField(res, PG_DIAG_SQLSTATE);
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
Handle PQresultErrorField(PG_DIAG_SQLSTATE) returning NULL in streamutil.c. In ff27db5d I missed that PQresultErrorField() may return NULL if there's no sqlstate associated with an error. Spotted-By: Coverity Reported-By: Michael Paquier Discussion: CAB7nPqQ3o10SY6NVdU4pjq85GQTN5tbbkq2gnNUh2fBNU3rKyQ@mail.gmail.com Backpatch: 9.5, like ff27db5d
2015-08-12 17:35:50 +02:00
if (slot_exists_ok &&
sqlstate &&
strcmp(sqlstate, ERRCODE_DUPLICATE_OBJECT) == 0)
Optionally don't error out due to preexisting slots in commandline utilities. pg_receivexlog and pg_recvlogical error out when --create-slot is specified and a slot with the same name already exists. In some cases, especially with pg_receivexlog, that's rather annoying and requires additional scripting. Backpatch to 9.5 as slot control functions have newly been added to pg_receivexlog, and there doesn't seem much point leaving it in a less useful state. Discussion: 20150619144755.GG29350@alap3.anarazel.de
2015-07-12 22:06:27 +02:00
{
destroyPQExpBuffer(query);
PQclear(res);
return true;
}
else
{
fprintf(stderr, _("%s: could not send replication command \"%s\": %s"),
progname, query->data, PQerrorMessage(conn));
destroyPQExpBuffer(query);
PQclear(res);
return false;
}
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
}
if (PQntuples(res) != 1 || PQnfields(res) != 4)
{
fprintf(stderr,
_("%s: could not create replication slot \"%s\": got %d rows and %d fields, expected %d rows and %d fields\n"),
progname, slot_name,
PQntuples(res), PQnfields(res), 1, 4);
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
destroyPQExpBuffer(query);
PQclear(res);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
return false;
}
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
destroyPQExpBuffer(query);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
PQclear(res);
return true;
}
/*
* Drop a replication slot for the given connection. This function
* returns true in case of success.
*/
bool
DropReplicationSlot(PGconn *conn, const char *slot_name)
{
PQExpBuffer query;
PGresult *res;
Assert(slot_name != NULL);
query = createPQExpBuffer();
/* Build query */
appendPQExpBuffer(query, "DROP_REPLICATION_SLOT \"%s\"",
slot_name);
res = PQexec(conn, query->data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, _("%s: could not send replication command \"%s\": %s"),
progname, query->data, PQerrorMessage(conn));
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
destroyPQExpBuffer(query);
PQclear(res);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
return false;
}
if (PQntuples(res) != 0 || PQnfields(res) != 0)
{
fprintf(stderr,
_("%s: could not drop replication slot \"%s\": got %d rows and %d fields, expected %d rows and %d fields\n"),
progname, slot_name,
PQntuples(res), PQnfields(res), 0, 0);
Clean up Create/DropReplicationSlot query buffer CreateReplicationSlot() and DropReplicationSlot() were not cleaning up the query buffer in some cases (mostly error conditions) which meant a small leak. Not generally an issue as the error case would result in an immediate exit, but not difficult to fix either and reduces the number of false positives from code analyzers. In passing, also add appropriate PQclear() calls to RunIdentifySystem(). Pointed out by Coverity.
2014-10-06 17:18:13 +02:00
destroyPQExpBuffer(query);
PQclear(res);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
return false;
}
Fix assorted memory leaks. Per Coverity (not that any of these are so non-obvious that they should not have been caught before commit). The extent of leakage is probably minor to unnoticeable, but a leak is a leak. Back-patch as necessary. Michael Paquier
2015-07-12 22:25:51 +02:00
destroyPQExpBuffer(query);
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
PQclear(res);
return true;
}
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
/*
* Frontend version of GetCurrentTimestamp(), since we are not linked with
Refactor replication connection code of various pg_basebackup utilities. Move some more code to manage replication connection command to streamutil.c. A later patch will introduce replication slot via pg_receivexlog and this avoid duplicating relevant code between pg_receivexlog and pg_recvlogical. Author: Michael Paquier, with some editing by me.
2014-10-01 17:22:21 +02:00
* backend code. The replication protocol always uses integer timestamps,
* regardless of the server setting.
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
*/
int64
feGetCurrentTimestamp(void)
{
int64 result;
struct timeval tp;
gettimeofday(&tp, NULL);
result = (int64) tp.tv_sec -
((POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY);
result = (result * USECS_PER_SEC) + tp.tv_usec;
return result;
}
/*
* Frontend version of TimestampDifference(), since we are not linked with
* backend code.
*/
void
feTimestampDifference(int64 start_time, int64 stop_time,
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
long *secs, int *microsecs)
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
{
int64 diff = stop_time - start_time;
if (diff <= 0)
{
*secs = 0;
*microsecs = 0;
}
else
{
*secs = (long) (diff / USECS_PER_SEC);
*microsecs = (int) (diff % USECS_PER_SEC);
}
}
/*
* Frontend version of TimestampDifferenceExceeds(), since we are not
* linked with backend code.
*/
bool
feTimestampDifferenceExceeds(int64 start_time,
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
int64 stop_time,
int msec)
Add pg_recvlogical, a tool to receive data logical decoding data. This is fairly basic at the moment, but it's at least useful for testing and debugging, and possibly more. Andres Freund
2014-03-18 17:19:57 +01:00
{
int64 diff = stop_time - start_time;
return (diff >= msec * INT64CONST(1000));
}
/*
* Converts an int64 to network byte order.
*/
void
fe_sendint64(int64 i, char *buf)
{
uint32 n32;
/* High order half first, since we're doing MSB-first */
n32 = (uint32) (i >> 32);
n32 = htonl(n32);
memcpy(&buf[0], &n32, 4);
/* Now the low order half */
n32 = (uint32) i;
n32 = htonl(n32);
memcpy(&buf[4], &n32, 4);
}
/*
* Converts an int64 from network byte order to native format.
*/
int64
fe_recvint64(char *buf)
{
int64 result;
uint32 h32;
uint32 l32;
memcpy(&h32, buf, 4);
memcpy(&l32, buf + 4, 4);
h32 = ntohl(h32);
l32 = ntohl(l32);
result = h32;
result <<= 32;
result |= l32;
return result;
}