2000-07-21 13:43:26 +02:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
2001-01-12 05:32:07 +01:00
|
|
|
* pg_backup_db.c
|
|
|
|
*
|
|
|
|
* Implements the basic DB functions used by the archiver.
|
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
2010-09-20 22:08:53 +02:00
|
|
|
* src/bin/pg_dump/pg_backup_db.c
|
2002-01-18 18:13:51 +01:00
|
|
|
*
|
UUNET is looking into offering PostgreSQL as a part of a managed web
hosting product, on both shared and dedicated machines. We currently
offer Oracle and MySQL, and it would be a nice middle-ground.
However, as shipped, PostgreSQL lacks the following features we need
that MySQL has:
1. The ability to listen only on a particular IP address. Each
hosting customer has their own IP address, on which all of their
servers (http, ftp, real media, etc.) run.
2. The ability to place the Unix-domain socket in a mode 700 directory.
This allows us to automatically create an empty database, with an
empty DBA password, for new or upgrading customers without having
to interactively set a DBA password and communicate it to (or from)
the customer. This in turn cuts down our install and upgrade times.
3. The ability to connect to the Unix-domain socket from within a
change-rooted environment. We run CGI programs chrooted to the
user's home directory, which is another reason why we need to be
able to specify where the Unix-domain socket is, instead of /tmp.
4. The ability to, if run as root, open a pid file in /var/run as
root, and then setuid to the desired user. (mysqld -u can almost
do this; I had to patch it, too).
The patch below fixes problem 1-3. I plan to address #4, also, but
haven't done so yet. These diffs are big enough that they should give
the PG development team something to think about in the meantime :-)
Also, I'm about to leave for 2 weeks' vacation, so I thought I'd get
out what I have, which works (for the problems it tackles), now.
With these changes, we can set up and run PostgreSQL with scripts the
same way we can with apache or proftpd or mysql.
In summary, this patch makes the following enhancements:
1. Adds an environment variable PGUNIXSOCKET, analogous to MYSQL_UNIX_PORT,
and command line options -k --unix-socket to the relevant programs.
2. Adds a -h option to postmaster to set the hostname or IP address to
listen on instead of the default INADDR_ANY.
3. Extends some library interfaces to support the above.
4. Fixes a few memory leaks in PQconnectdb().
The default behavior is unchanged from stock 7.0.2; if you don't use
any of these new features, they don't change the operation.
David J. MacKenzie
2000-11-13 16:18:15 +01:00
|
|
|
*-------------------------------------------------------------------------
|
2000-07-21 13:43:26 +02:00
|
|
|
*/
|
2014-10-14 20:00:55 +02:00
|
|
|
#include "postgres_fe.h"
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2002-08-20 19:54:45 +02:00
|
|
|
#include <unistd.h>
|
2000-07-21 13:43:26 +02:00
|
|
|
#include <ctype.h>
|
|
|
|
#ifdef HAVE_TERMIOS_H
|
|
|
|
#include <termios.h>
|
|
|
|
#endif
|
|
|
|
|
2020-08-10 18:22:54 +02:00
|
|
|
#include "common/connect.h"
|
Remove arbitrary restrictions on password length.
This patch started out with the goal of harmonizing various arbitrary
limits on password length, but after awhile a better idea emerged:
let's just get rid of those fixed limits.
recv_password_packet() has an arbitrary limit on the packet size,
which we don't really need, so just drop it. (Note that this doesn't
really affect anything for MD5 or SCRAM password verification, since
those will hash the user's password to something shorter anyway.
It does matter for auth methods that require a cleartext password.)
Likewise remove the arbitrary error condition in pg_saslprep().
The remaining limits are mostly in client-side code that prompts
for passwords. To improve those, refactor simple_prompt() so that
it allocates its own result buffer that can be made as big as
necessary. Actually, it proves best to make a separate routine
pg_get_line() that has essentially the semantics of fgets(), except
that it allocates a suitable result buffer and hence will never
return a truncated line. (pg_get_line has a lot of potential
applications to replace randomly-sized fgets buffers elsewhere,
but I'll leave that for another patch.)
I built pg_get_line() atop stringinfo.c, which requires moving
that code to src/common/; but that seems fine since it was a poor
fit for src/port/ anyway.
This patch is mostly mine, but it owes a good deal to Nathan Bossart
who pressed for a solution to the password length problem and
created a predecessor patch. Also thanks to Peter Eisentraut and
Stephen Frost for ideas and discussion.
Discussion: https://postgr.es/m/09512C4F-8CB9-4021-B455-EF4C4F0D55A0@amazon.com
2020-09-04 02:09:18 +02:00
|
|
|
#include "common/string.h"
|
2019-10-23 06:08:53 +02:00
|
|
|
#include "dumputils.h"
|
|
|
|
#include "fe_utils/string_utils.h"
|
|
|
|
#include "parallel.h"
|
|
|
|
#include "pg_backup_archiver.h"
|
|
|
|
#include "pg_backup_db.h"
|
|
|
|
#include "pg_backup_utils.h"
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2008-04-13 05:49:22 +02:00
|
|
|
static void _check_database_version(ArchiveHandle *AH);
|
2001-08-12 21:02:39 +02:00
|
|
|
static void notice_processor(void *arg, const char *message);
|
They are two different problems; the TOC entry is important for any
multiline command or to rerun the command easily later.
Whereas displaying the failed SQL command is a matter of fixing the
error
messages.
The latter is complicated by failed COPY commands which, with
die-on-errors
off, results in the data being processed as a command, so dumping the
command will dump all of the data.
In the case of long commands, should the whole command be dumped? eg.
(eg.
several pages of function definition).
In the case of the COPY command, I'm not sure what to do. Obviously, it
would be best to avoid sending the data, but the data and command are
combined (from memory). Also, the 'data' may be in the form of INSERT
statements.
Attached patch produces the first 125 chars of the command:
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC Entry 26; 1255 16449270
FUNCTION
plpgsql_call_handler() pjw
pg_restore: [archiver (db)] could not execute query: ERROR: function
"plpgsql_call_handler" already exists with same argument types
Command was: CREATE FUNCTION plpgsql_call_handler() RETURNS
language_handler
AS '/var/lib/pgsql-8.0b1/lib/plpgsql', 'plpgsql_call_han...
pg_restore: [archiver (db)] Error from TOC Entry 27; 1255 16449271
FUNCTION
plpgsql_validator(oid) pjw
pg_restore: [archiver (db)] could not execute query: ERROR: function
"plpgsql_validator" already exists with same argument types
Command was: CREATE FUNCTION plpgsql_validator(oid) RETURNS void
AS '/var/lib/pgsql-8.0b1/lib/plpgsql', 'plpgsql_validator'
LANGU...
Philip Warner
2004-08-20 22:00:34 +02:00
|
|
|
|
2000-07-21 13:43:26 +02:00
|
|
|
static void
|
2008-04-13 05:49:22 +02:00
|
|
|
_check_database_version(ArchiveHandle *AH)
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
|
|
|
const char *remoteversion_str;
|
2001-04-25 09:03:20 +02:00
|
|
|
int remoteversion;
|
2016-05-26 22:14:23 +02:00
|
|
|
PGresult *res;
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2003-06-22 02:56:58 +02:00
|
|
|
remoteversion_str = PQparameterStatus(AH->connection, "server_version");
|
2013-03-26 14:21:57 +01:00
|
|
|
remoteversion = PQserverVersion(AH->connection);
|
|
|
|
if (remoteversion == 0 || !remoteversion_str)
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal("could not get server_version from libpq");
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2011-11-25 21:40:51 +01:00
|
|
|
AH->public.remoteVersionStr = pg_strdup(remoteversion_str);
|
2001-04-25 09:03:20 +02:00
|
|
|
AH->public.remoteVersion = remoteversion;
|
2010-02-24 03:42:55 +01:00
|
|
|
if (!AH->archiveRemoteVersion)
|
|
|
|
AH->archiveRemoteVersion = AH->public.remoteVersionStr;
|
2001-04-25 09:03:20 +02:00
|
|
|
|
2013-03-26 14:21:57 +01:00
|
|
|
if (remoteversion != PG_VERSION_NUM
|
2003-06-22 02:56:58 +02:00
|
|
|
&& (remoteversion < AH->public.minRemoteVersion ||
|
|
|
|
remoteversion > AH->public.maxRemoteVersion))
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_log_error("aborting because of server version mismatch");
|
|
|
|
pg_log_error_detail("server version: %s; %s version: %s",
|
|
|
|
remoteversion_str, progname, PG_VERSION);
|
|
|
|
exit(1);
|
2000-07-21 13:43:26 +02:00
|
|
|
}
|
2016-05-26 22:14:23 +02:00
|
|
|
|
|
|
|
/*
|
2021-12-16 18:01:59 +01:00
|
|
|
* Check if server is in recovery mode, which means we are on a hot
|
|
|
|
* standby.
|
2016-05-26 22:14:23 +02:00
|
|
|
*/
|
2021-12-16 18:01:59 +01:00
|
|
|
res = ExecuteSqlQueryForSingleRow((Archive *) AH,
|
|
|
|
"SELECT pg_catalog.pg_is_in_recovery()");
|
|
|
|
AH->public.isStandby = (strcmp(PQgetvalue(res, 0, 0), "t") == 0);
|
|
|
|
PQclear(res);
|
2000-07-21 13:43:26 +02:00
|
|
|
}
|
|
|
|
|
2000-08-01 17:51:45 +02:00
|
|
|
/*
|
2001-08-22 22:23:24 +02:00
|
|
|
* Reconnect to the server. If dbname is not NULL, use that database,
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
* else the one associated with the archive handle.
|
2000-08-01 17:51:45 +02:00
|
|
|
*/
|
In pg_dump, force reconnection after issuing ALTER DATABASE SET command(s).
The folly of not doing this was exposed by the buildfarm: in some cases,
the GUC settings applied through ALTER DATABASE SET may be essential to
interpreting the reloaded data correctly. Another argument why we can't
really get away with the scheme proposed in commit b3f840120 is that it
cannot work for parallel restore: even if the parent process manages to
hang onto the previous GUC state, worker processes would see the state
post-ALTER-DATABASE. (Perhaps we could have dodged that bullet by
delaying DATABASE PROPERTIES restoration to the end of the run, but
that does nothing for the data semantics problem.)
This leaves us with no solution for the default_transaction_read_only issue
that commit 4bd371f6f intended to work around, other than "you gotta remove
such settings before dumping/upgrading". However, in view of the fact that
parallel restore broke that hack years ago and no one has noticed, it's
fair to question how many people care. I'm unexcited about adding a large
dollop of new complexity to handle that corner case.
This would be a one-liner fix, except it turns out that ReconnectToServer
tries to optimize away "redundant" reconnections. While that may have been
valuable when coded, a quick survey of current callers shows that there are
no cases where that's actually useful, so just remove that check. While at
it, remove the function's useless return value.
Discussion: https://postgr.es/m/12453.1516655001@sss.pgh.pa.us
2018-01-23 16:55:08 +01:00
|
|
|
void
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
ReconnectToServer(ArchiveHandle *AH, const char *dbname)
|
2000-08-01 17:51:45 +02:00
|
|
|
{
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
PGconn *oldConn = AH->connection;
|
|
|
|
RestoreOptions *ropt = AH->public.ropt;
|
Remove arbitrary restrictions on password length.
This patch started out with the goal of harmonizing various arbitrary
limits on password length, but after awhile a better idea emerged:
let's just get rid of those fixed limits.
recv_password_packet() has an arbitrary limit on the packet size,
which we don't really need, so just drop it. (Note that this doesn't
really affect anything for MD5 or SCRAM password verification, since
those will hash the user's password to something shorter anyway.
It does matter for auth methods that require a cleartext password.)
Likewise remove the arbitrary error condition in pg_saslprep().
The remaining limits are mostly in client-side code that prompts
for passwords. To improve those, refactor simple_prompt() so that
it allocates its own result buffer that can be made as big as
necessary. Actually, it proves best to make a separate routine
pg_get_line() that has essentially the semantics of fgets(), except
that it allocates a suitable result buffer and hence will never
return a truncated line. (pg_get_line has a lot of potential
applications to replace randomly-sized fgets buffers elsewhere,
but I'll leave that for another patch.)
I built pg_get_line() atop stringinfo.c, which requires moving
that code to src/common/; but that seems fine since it was a poor
fit for src/port/ anyway.
This patch is mostly mine, but it owes a good deal to Nathan Bossart
who pressed for a solution to the password length problem and
created a predecessor patch. Also thanks to Peter Eisentraut and
Stephen Frost for ideas and discussion.
Discussion: https://postgr.es/m/09512C4F-8CB9-4021-B455-EF4C4F0D55A0@amazon.com
2020-09-04 02:09:18 +02:00
|
|
|
|
2015-12-23 20:25:31 +01:00
|
|
|
/*
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
* Save the dbname, if given, in override_dbname so that it will also
|
|
|
|
* affect any later reconnection attempt.
|
2015-12-23 20:25:31 +01:00
|
|
|
*/
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
if (dbname)
|
|
|
|
ropt->cparams.override_dbname = pg_strdup(dbname);
|
2001-05-17 23:12:49 +02:00
|
|
|
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
/*
|
|
|
|
* Note: we want to establish the new connection, and in particular update
|
|
|
|
* ArchiveHandle's connCancel, before closing old connection. Otherwise
|
|
|
|
* an ill-timed SIGINT could try to access a dead connection.
|
|
|
|
*/
|
|
|
|
AH->connection = NULL; /* dodge error check in ConnectDatabase */
|
2003-02-14 20:40:42 +01:00
|
|
|
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
ConnectDatabase((Archive *) AH, &ropt->cparams, true);
|
2001-08-12 21:02:39 +02:00
|
|
|
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
PQfinish(oldConn);
|
2000-07-24 08:24:26 +02:00
|
|
|
}
|
|
|
|
|
2001-05-17 23:12:49 +02:00
|
|
|
/*
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
* Make, or remake, a database connection with the given parameters.
|
|
|
|
*
|
|
|
|
* The resulting connection handle is stored in AHX->connection.
|
2009-02-02 21:07:37 +01:00
|
|
|
*
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
* An interactive password prompt is automatically issued if required.
|
|
|
|
* We store the results of that in AHX->savedPassword.
|
2009-02-02 21:07:37 +01:00
|
|
|
* Note: it's not really all that sensible to use a single-entry password
|
|
|
|
* cache if the username keeps changing. In current usage, however, the
|
|
|
|
* username never does change, so one savedPassword is sufficient.
|
2001-05-17 23:12:49 +02:00
|
|
|
*/
|
2012-02-16 19:00:24 +01:00
|
|
|
void
|
2000-07-21 13:43:26 +02:00
|
|
|
ConnectDatabase(Archive *AHX,
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
const ConnParams *cparams,
|
|
|
|
bool isReconnect)
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
trivalue prompt_password;
|
2015-12-23 20:25:31 +01:00
|
|
|
char *password;
|
2007-07-08 21:07:38 +02:00
|
|
|
bool new_pass;
|
2000-07-21 13:43:26 +02:00
|
|
|
|
|
|
|
if (AH->connection)
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal("already connected to a database");
|
2000-07-21 13:43:26 +02:00
|
|
|
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
/* Never prompt for a password during a reconnection */
|
|
|
|
prompt_password = isReconnect ? TRI_NO : cparams->promptPassword;
|
|
|
|
|
Simplify correct use of simple_prompt().
The previous API for this function had it returning a malloc'd string.
That meant that callers had to check for NULL return, which few of them
were doing, and it also meant that callers had to remember to free()
the string later, which required extra logic in most cases.
Instead, make simple_prompt() write into a buffer supplied by the caller.
Anywhere that the maximum required input length is reasonably small,
which is almost all of the callers, we can just use a local or static
array as the buffer instead of dealing with malloc/free.
A fair number of callers used "pointer == NULL" as a proxy for "haven't
requested the password yet". Maintaining the same behavior requires
adding a separate boolean flag for that, which adds back some of the
complexity we save by removing free()s. Nonetheless, this nets out
at a small reduction in overall code size, and considerably less code
than we would have had if we'd added the missing NULL-return checks
everywhere they were needed.
In passing, clean up the API comment for simple_prompt() and get rid
of a very-unnecessary malloc/free in its Windows code path.
This is nominally a bug fix, but it does not seem worth back-patching,
because the actual risk of an OOM failure in any of these places seems
pretty tiny, and all of them are client-side not server-side anyway.
This patch is by me, but it owes a great deal to Michael Paquier
who identified the problem and drafted a patch for fixing it the
other way.
Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com>
2016-08-30 23:02:02 +02:00
|
|
|
password = AH->savedPassword;
|
2015-12-23 20:25:31 +01:00
|
|
|
|
2009-02-26 17:02:39 +01:00
|
|
|
if (prompt_password == TRI_YES && password == NULL)
|
Remove arbitrary restrictions on password length.
This patch started out with the goal of harmonizing various arbitrary
limits on password length, but after awhile a better idea emerged:
let's just get rid of those fixed limits.
recv_password_packet() has an arbitrary limit on the packet size,
which we don't really need, so just drop it. (Note that this doesn't
really affect anything for MD5 or SCRAM password verification, since
those will hash the user's password to something shorter anyway.
It does matter for auth methods that require a cleartext password.)
Likewise remove the arbitrary error condition in pg_saslprep().
The remaining limits are mostly in client-side code that prompts
for passwords. To improve those, refactor simple_prompt() so that
it allocates its own result buffer that can be made as big as
necessary. Actually, it proves best to make a separate routine
pg_get_line() that has essentially the semantics of fgets(), except
that it allocates a suitable result buffer and hence will never
return a truncated line. (pg_get_line has a lot of potential
applications to replace randomly-sized fgets buffers elsewhere,
but I'll leave that for another patch.)
I built pg_get_line() atop stringinfo.c, which requires moving
that code to src/common/; but that seems fine since it was a poor
fit for src/port/ anyway.
This patch is mostly mine, but it owes a good deal to Nathan Bossart
who pressed for a solution to the password length problem and
created a predecessor patch. Also thanks to Peter Eisentraut and
Stephen Frost for ideas and discussion.
Discussion: https://postgr.es/m/09512C4F-8CB9-4021-B455-EF4C4F0D55A0@amazon.com
2020-09-04 02:09:18 +02:00
|
|
|
password = simple_prompt("Password: ", false);
|
|
|
|
|
2001-05-17 23:12:49 +02:00
|
|
|
/*
|
|
|
|
* Start the connection. Loop until we have a password if requested by
|
|
|
|
* backend.
|
|
|
|
*/
|
|
|
|
do
|
|
|
|
{
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
const char *keywords[8];
|
|
|
|
const char *values[8];
|
|
|
|
int i = 0;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If dbname is a connstring, its entries can override the other
|
|
|
|
* values obtained from cparams; but in turn, override_dbname can
|
|
|
|
* override the dbname component of it.
|
|
|
|
*/
|
|
|
|
keywords[i] = "host";
|
|
|
|
values[i++] = cparams->pghost;
|
|
|
|
keywords[i] = "port";
|
|
|
|
values[i++] = cparams->pgport;
|
|
|
|
keywords[i] = "user";
|
|
|
|
values[i++] = cparams->username;
|
|
|
|
keywords[i] = "password";
|
|
|
|
values[i++] = password;
|
|
|
|
keywords[i] = "dbname";
|
|
|
|
values[i++] = cparams->dbname;
|
|
|
|
if (cparams->override_dbname)
|
|
|
|
{
|
|
|
|
keywords[i] = "dbname";
|
|
|
|
values[i++] = cparams->override_dbname;
|
|
|
|
}
|
|
|
|
keywords[i] = "fallback_application_name";
|
|
|
|
values[i++] = progname;
|
|
|
|
keywords[i] = NULL;
|
|
|
|
values[i++] = NULL;
|
|
|
|
Assert(i <= lengthof(keywords));
|
2010-02-05 04:09:05 +01:00
|
|
|
|
2007-07-08 21:07:38 +02:00
|
|
|
new_pass = false;
|
2010-02-05 04:09:05 +01:00
|
|
|
AH->connection = PQconnectdbParams(keywords, values, true);
|
|
|
|
|
2001-05-17 23:12:49 +02:00
|
|
|
if (!AH->connection)
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal("could not connect to database");
|
2001-05-17 23:12:49 +02:00
|
|
|
|
|
|
|
if (PQstatus(AH->connection) == CONNECTION_BAD &&
|
2007-12-09 20:01:40 +01:00
|
|
|
PQconnectionNeedsPassword(AH->connection) &&
|
2009-02-26 17:02:39 +01:00
|
|
|
password == NULL &&
|
|
|
|
prompt_password != TRI_NO)
|
2001-05-17 23:12:49 +02:00
|
|
|
{
|
|
|
|
PQfinish(AH->connection);
|
Remove arbitrary restrictions on password length.
This patch started out with the goal of harmonizing various arbitrary
limits on password length, but after awhile a better idea emerged:
let's just get rid of those fixed limits.
recv_password_packet() has an arbitrary limit on the packet size,
which we don't really need, so just drop it. (Note that this doesn't
really affect anything for MD5 or SCRAM password verification, since
those will hash the user's password to something shorter anyway.
It does matter for auth methods that require a cleartext password.)
Likewise remove the arbitrary error condition in pg_saslprep().
The remaining limits are mostly in client-side code that prompts
for passwords. To improve those, refactor simple_prompt() so that
it allocates its own result buffer that can be made as big as
necessary. Actually, it proves best to make a separate routine
pg_get_line() that has essentially the semantics of fgets(), except
that it allocates a suitable result buffer and hence will never
return a truncated line. (pg_get_line has a lot of potential
applications to replace randomly-sized fgets buffers elsewhere,
but I'll leave that for another patch.)
I built pg_get_line() atop stringinfo.c, which requires moving
that code to src/common/; but that seems fine since it was a poor
fit for src/port/ anyway.
This patch is mostly mine, but it owes a good deal to Nathan Bossart
who pressed for a solution to the password length problem and
created a predecessor patch. Also thanks to Peter Eisentraut and
Stephen Frost for ideas and discussion.
Discussion: https://postgr.es/m/09512C4F-8CB9-4021-B455-EF4C4F0D55A0@amazon.com
2020-09-04 02:09:18 +02:00
|
|
|
password = simple_prompt("Password: ", false);
|
2007-07-08 21:07:38 +02:00
|
|
|
new_pass = true;
|
2001-05-17 23:12:49 +02:00
|
|
|
}
|
2007-07-08 21:07:38 +02:00
|
|
|
} while (new_pass);
|
2001-05-17 23:12:49 +02:00
|
|
|
|
2000-07-21 13:43:26 +02:00
|
|
|
/* check to see that the backend connection was successfully made */
|
|
|
|
if (PQstatus(AH->connection) == CONNECTION_BAD)
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
{
|
|
|
|
if (isReconnect)
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal("reconnection failed: %s",
|
|
|
|
PQerrorMessage(AH->connection));
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
else
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal("%s",
|
|
|
|
PQerrorMessage(AH->connection));
|
Fix handling of -d "connection string" in pg_dump/pg_restore.
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-09-25 00:19:38 +02:00
|
|
|
}
|
2000-07-21 13:43:26 +02:00
|
|
|
|
Empty search_path in Autovacuum and non-psql/pgbench clients.
This makes the client programs behave as documented regardless of the
connect-time search_path and regardless of user-created objects. Today,
a malicious user with CREATE permission on a search_path schema can take
control of certain of these clients' queries and invoke arbitrary SQL
functions under the client identity, often a superuser. This is
exploitable in the default configuration, where all users have CREATE
privilege on schema "public".
This changes behavior of user-defined code stored in the database, like
pg_index.indexprs and pg_extension_config_dump(). If they reach code
bearing unqualified names, "does not exist" or "no schema has been
selected to create in" errors might appear. Users may fix such errors
by schema-qualifying affected names. After upgrading, consider watching
server logs for these errors.
The --table arguments of src/bin/scripts clients have been lax; for
example, "vacuumdb -Zt pg_am\;CHECKPOINT" performed a checkpoint. That
now fails, but for now, "vacuumdb -Zt 'pg_am(amname);CHECKPOINT'" still
performs a checkpoint.
Back-patch to 9.3 (all supported versions).
Reviewed by Tom Lane, though this fix strategy was not his first choice.
Reported by Arseniy Sharoglazov.
Security: CVE-2018-1058
2018-02-26 16:39:44 +01:00
|
|
|
/* Start strict; later phases may override this. */
|
|
|
|
PQclear(ExecuteSqlQueryForSingleRow((Archive *) AH,
|
|
|
|
ALWAYS_SECURE_SEARCH_PATH_SQL));
|
|
|
|
|
Remove arbitrary restrictions on password length.
This patch started out with the goal of harmonizing various arbitrary
limits on password length, but after awhile a better idea emerged:
let's just get rid of those fixed limits.
recv_password_packet() has an arbitrary limit on the packet size,
which we don't really need, so just drop it. (Note that this doesn't
really affect anything for MD5 or SCRAM password verification, since
those will hash the user's password to something shorter anyway.
It does matter for auth methods that require a cleartext password.)
Likewise remove the arbitrary error condition in pg_saslprep().
The remaining limits are mostly in client-side code that prompts
for passwords. To improve those, refactor simple_prompt() so that
it allocates its own result buffer that can be made as big as
necessary. Actually, it proves best to make a separate routine
pg_get_line() that has essentially the semantics of fgets(), except
that it allocates a suitable result buffer and hence will never
return a truncated line. (pg_get_line has a lot of potential
applications to replace randomly-sized fgets buffers elsewhere,
but I'll leave that for another patch.)
I built pg_get_line() atop stringinfo.c, which requires moving
that code to src/common/; but that seems fine since it was a poor
fit for src/port/ anyway.
This patch is mostly mine, but it owes a good deal to Nathan Bossart
who pressed for a solution to the password length problem and
created a predecessor patch. Also thanks to Peter Eisentraut and
Stephen Frost for ideas and discussion.
Discussion: https://postgr.es/m/09512C4F-8CB9-4021-B455-EF4C4F0D55A0@amazon.com
2020-09-04 02:09:18 +02:00
|
|
|
if (password && password != AH->savedPassword)
|
|
|
|
free(password);
|
|
|
|
|
2015-12-23 20:25:31 +01:00
|
|
|
/*
|
|
|
|
* We want to remember connection's actual password, whether or not we got
|
|
|
|
* it by prompting. So we don't just store the password variable.
|
|
|
|
*/
|
|
|
|
if (PQconnectionUsedPassword(AH->connection))
|
|
|
|
{
|
2022-06-16 21:50:56 +02:00
|
|
|
free(AH->savedPassword);
|
2015-12-23 20:25:31 +01:00
|
|
|
AH->savedPassword = pg_strdup(PQpass(AH->connection));
|
|
|
|
}
|
|
|
|
|
2000-07-21 13:43:26 +02:00
|
|
|
/* check for version mismatch */
|
2008-04-13 05:49:22 +02:00
|
|
|
_check_database_version(AH);
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2001-08-12 21:02:39 +02:00
|
|
|
PQsetNoticeProcessor(AH->connection, notice_processor, NULL);
|
Redesign handling of SIGTERM/control-C in parallel pg_dump/pg_restore.
Formerly, Unix builds of pg_dump/pg_restore would trap SIGINT and similar
signals and set a flag that was tested in various data-transfer loops.
This was prone to errors of omission (cf commit 3c8aa6654); and even if
the client-side response was prompt, we did nothing that would cause
long-running SQL commands (e.g. CREATE INDEX) to terminate early.
Also, the master process would effectively do nothing at all upon receipt
of SIGINT; the only reason it seemed to work was that in typical scenarios
the signal would also be delivered to the child processes. We should
support termination when a signal is delivered only to the master process,
though.
Windows builds had no console interrupt handler, so they would just fall
over immediately at control-C, again leaving long-running SQL commands to
finish unmolested.
To fix, remove the flag-checking approach altogether. Instead, allow the
Unix signal handler to send a cancel request directly and then exit(1).
In the master process, also have it forward the signal to the children.
On Windows, add a console interrupt handler that behaves approximately
the same. The main difference is that a single execution of the Windows
handler can send all the cancel requests since all the info is available
in one process, whereas on Unix each process sends a cancel only for its
own database connection.
In passing, fix an old problem that DisconnectDatabase tends to send a
cancel request before exiting a parallel worker, even if nothing went
wrong. This is at least a waste of cycles, and could lead to unexpected
log messages, or maybe even data loss if it happened in pg_restore (though
in the current code the problem seems to affect only pg_dump). The cause
was that after a COPY step, pg_dump was leaving libpq in PGASYNC_BUSY
state, causing PQtransactionStatus() to report PQTRANS_ACTIVE. That's
normally harmless because the next PQexec() will silently clear the
PGASYNC_BUSY state; but in a parallel worker we might exit without any
additional SQL commands after a COPY step. So add an extra PQgetResult()
call after a COPY to allow libpq to return to PGASYNC_IDLE state.
This is a bug fix, IMO, so back-patch to 9.3 where parallel dump/restore
were introduced.
Thanks to Kyotaro Horiguchi for Windows testing and code suggestions.
Original-Patch: <7005.1464657274@sss.pgh.pa.us>
Discussion: <20160602.174941.256342236.horiguchi.kyotaro@lab.ntt.co.jp>
2016-06-02 19:27:53 +02:00
|
|
|
|
|
|
|
/* arrange for SIGINT to issue a query cancel on this connection */
|
|
|
|
set_archive_cancel_info(AH, AH->connection);
|
2000-07-21 13:43:26 +02:00
|
|
|
}
|
|
|
|
|
2013-03-24 16:27:20 +01:00
|
|
|
/*
|
|
|
|
* Close the connection to the database and also cancel off the query if we
|
|
|
|
* have one running.
|
|
|
|
*/
|
2012-02-16 17:49:20 +01:00
|
|
|
void
|
|
|
|
DisconnectDatabase(Archive *AHX)
|
|
|
|
{
|
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
2013-03-24 16:27:20 +01:00
|
|
|
char errbuf[1];
|
|
|
|
|
|
|
|
if (!AH->connection)
|
|
|
|
return;
|
2012-02-16 17:49:20 +01:00
|
|
|
|
Redesign handling of SIGTERM/control-C in parallel pg_dump/pg_restore.
Formerly, Unix builds of pg_dump/pg_restore would trap SIGINT and similar
signals and set a flag that was tested in various data-transfer loops.
This was prone to errors of omission (cf commit 3c8aa6654); and even if
the client-side response was prompt, we did nothing that would cause
long-running SQL commands (e.g. CREATE INDEX) to terminate early.
Also, the master process would effectively do nothing at all upon receipt
of SIGINT; the only reason it seemed to work was that in typical scenarios
the signal would also be delivered to the child processes. We should
support termination when a signal is delivered only to the master process,
though.
Windows builds had no console interrupt handler, so they would just fall
over immediately at control-C, again leaving long-running SQL commands to
finish unmolested.
To fix, remove the flag-checking approach altogether. Instead, allow the
Unix signal handler to send a cancel request directly and then exit(1).
In the master process, also have it forward the signal to the children.
On Windows, add a console interrupt handler that behaves approximately
the same. The main difference is that a single execution of the Windows
handler can send all the cancel requests since all the info is available
in one process, whereas on Unix each process sends a cancel only for its
own database connection.
In passing, fix an old problem that DisconnectDatabase tends to send a
cancel request before exiting a parallel worker, even if nothing went
wrong. This is at least a waste of cycles, and could lead to unexpected
log messages, or maybe even data loss if it happened in pg_restore (though
in the current code the problem seems to affect only pg_dump). The cause
was that after a COPY step, pg_dump was leaving libpq in PGASYNC_BUSY
state, causing PQtransactionStatus() to report PQTRANS_ACTIVE. That's
normally harmless because the next PQexec() will silently clear the
PGASYNC_BUSY state; but in a parallel worker we might exit without any
additional SQL commands after a COPY step. So add an extra PQgetResult()
call after a COPY to allow libpq to return to PGASYNC_IDLE state.
This is a bug fix, IMO, so back-patch to 9.3 where parallel dump/restore
were introduced.
Thanks to Kyotaro Horiguchi for Windows testing and code suggestions.
Original-Patch: <7005.1464657274@sss.pgh.pa.us>
Discussion: <20160602.174941.256342236.horiguchi.kyotaro@lab.ntt.co.jp>
2016-06-02 19:27:53 +02:00
|
|
|
if (AH->connCancel)
|
2013-03-24 16:27:20 +01:00
|
|
|
{
|
Redesign handling of SIGTERM/control-C in parallel pg_dump/pg_restore.
Formerly, Unix builds of pg_dump/pg_restore would trap SIGINT and similar
signals and set a flag that was tested in various data-transfer loops.
This was prone to errors of omission (cf commit 3c8aa6654); and even if
the client-side response was prompt, we did nothing that would cause
long-running SQL commands (e.g. CREATE INDEX) to terminate early.
Also, the master process would effectively do nothing at all upon receipt
of SIGINT; the only reason it seemed to work was that in typical scenarios
the signal would also be delivered to the child processes. We should
support termination when a signal is delivered only to the master process,
though.
Windows builds had no console interrupt handler, so they would just fall
over immediately at control-C, again leaving long-running SQL commands to
finish unmolested.
To fix, remove the flag-checking approach altogether. Instead, allow the
Unix signal handler to send a cancel request directly and then exit(1).
In the master process, also have it forward the signal to the children.
On Windows, add a console interrupt handler that behaves approximately
the same. The main difference is that a single execution of the Windows
handler can send all the cancel requests since all the info is available
in one process, whereas on Unix each process sends a cancel only for its
own database connection.
In passing, fix an old problem that DisconnectDatabase tends to send a
cancel request before exiting a parallel worker, even if nothing went
wrong. This is at least a waste of cycles, and could lead to unexpected
log messages, or maybe even data loss if it happened in pg_restore (though
in the current code the problem seems to affect only pg_dump). The cause
was that after a COPY step, pg_dump was leaving libpq in PGASYNC_BUSY
state, causing PQtransactionStatus() to report PQTRANS_ACTIVE. That's
normally harmless because the next PQexec() will silently clear the
PGASYNC_BUSY state; but in a parallel worker we might exit without any
additional SQL commands after a COPY step. So add an extra PQgetResult()
call after a COPY to allow libpq to return to PGASYNC_IDLE state.
This is a bug fix, IMO, so back-patch to 9.3 where parallel dump/restore
were introduced.
Thanks to Kyotaro Horiguchi for Windows testing and code suggestions.
Original-Patch: <7005.1464657274@sss.pgh.pa.us>
Discussion: <20160602.174941.256342236.horiguchi.kyotaro@lab.ntt.co.jp>
2016-06-02 19:27:53 +02:00
|
|
|
/*
|
2017-05-09 19:58:51 +02:00
|
|
|
* If we have an active query, send a cancel before closing, ignoring
|
|
|
|
* any errors. This is of no use for a normal exit, but might be
|
2022-04-08 20:55:14 +02:00
|
|
|
* helpful during pg_fatal().
|
Redesign handling of SIGTERM/control-C in parallel pg_dump/pg_restore.
Formerly, Unix builds of pg_dump/pg_restore would trap SIGINT and similar
signals and set a flag that was tested in various data-transfer loops.
This was prone to errors of omission (cf commit 3c8aa6654); and even if
the client-side response was prompt, we did nothing that would cause
long-running SQL commands (e.g. CREATE INDEX) to terminate early.
Also, the master process would effectively do nothing at all upon receipt
of SIGINT; the only reason it seemed to work was that in typical scenarios
the signal would also be delivered to the child processes. We should
support termination when a signal is delivered only to the master process,
though.
Windows builds had no console interrupt handler, so they would just fall
over immediately at control-C, again leaving long-running SQL commands to
finish unmolested.
To fix, remove the flag-checking approach altogether. Instead, allow the
Unix signal handler to send a cancel request directly and then exit(1).
In the master process, also have it forward the signal to the children.
On Windows, add a console interrupt handler that behaves approximately
the same. The main difference is that a single execution of the Windows
handler can send all the cancel requests since all the info is available
in one process, whereas on Unix each process sends a cancel only for its
own database connection.
In passing, fix an old problem that DisconnectDatabase tends to send a
cancel request before exiting a parallel worker, even if nothing went
wrong. This is at least a waste of cycles, and could lead to unexpected
log messages, or maybe even data loss if it happened in pg_restore (though
in the current code the problem seems to affect only pg_dump). The cause
was that after a COPY step, pg_dump was leaving libpq in PGASYNC_BUSY
state, causing PQtransactionStatus() to report PQTRANS_ACTIVE. That's
normally harmless because the next PQexec() will silently clear the
PGASYNC_BUSY state; but in a parallel worker we might exit without any
additional SQL commands after a COPY step. So add an extra PQgetResult()
call after a COPY to allow libpq to return to PGASYNC_IDLE state.
This is a bug fix, IMO, so back-patch to 9.3 where parallel dump/restore
were introduced.
Thanks to Kyotaro Horiguchi for Windows testing and code suggestions.
Original-Patch: <7005.1464657274@sss.pgh.pa.us>
Discussion: <20160602.174941.256342236.horiguchi.kyotaro@lab.ntt.co.jp>
2016-06-02 19:27:53 +02:00
|
|
|
*/
|
|
|
|
if (PQtransactionStatus(AH->connection) == PQTRANS_ACTIVE)
|
2017-05-09 19:58:51 +02:00
|
|
|
(void) PQcancel(AH->connCancel, errbuf, sizeof(errbuf));
|
Redesign handling of SIGTERM/control-C in parallel pg_dump/pg_restore.
Formerly, Unix builds of pg_dump/pg_restore would trap SIGINT and similar
signals and set a flag that was tested in various data-transfer loops.
This was prone to errors of omission (cf commit 3c8aa6654); and even if
the client-side response was prompt, we did nothing that would cause
long-running SQL commands (e.g. CREATE INDEX) to terminate early.
Also, the master process would effectively do nothing at all upon receipt
of SIGINT; the only reason it seemed to work was that in typical scenarios
the signal would also be delivered to the child processes. We should
support termination when a signal is delivered only to the master process,
though.
Windows builds had no console interrupt handler, so they would just fall
over immediately at control-C, again leaving long-running SQL commands to
finish unmolested.
To fix, remove the flag-checking approach altogether. Instead, allow the
Unix signal handler to send a cancel request directly and then exit(1).
In the master process, also have it forward the signal to the children.
On Windows, add a console interrupt handler that behaves approximately
the same. The main difference is that a single execution of the Windows
handler can send all the cancel requests since all the info is available
in one process, whereas on Unix each process sends a cancel only for its
own database connection.
In passing, fix an old problem that DisconnectDatabase tends to send a
cancel request before exiting a parallel worker, even if nothing went
wrong. This is at least a waste of cycles, and could lead to unexpected
log messages, or maybe even data loss if it happened in pg_restore (though
in the current code the problem seems to affect only pg_dump). The cause
was that after a COPY step, pg_dump was leaving libpq in PGASYNC_BUSY
state, causing PQtransactionStatus() to report PQTRANS_ACTIVE. That's
normally harmless because the next PQexec() will silently clear the
PGASYNC_BUSY state; but in a parallel worker we might exit without any
additional SQL commands after a COPY step. So add an extra PQgetResult()
call after a COPY to allow libpq to return to PGASYNC_IDLE state.
This is a bug fix, IMO, so back-patch to 9.3 where parallel dump/restore
were introduced.
Thanks to Kyotaro Horiguchi for Windows testing and code suggestions.
Original-Patch: <7005.1464657274@sss.pgh.pa.us>
Discussion: <20160602.174941.256342236.horiguchi.kyotaro@lab.ntt.co.jp>
2016-06-02 19:27:53 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Prevent signal handler from sending a cancel after this.
|
|
|
|
*/
|
|
|
|
set_archive_cancel_info(AH, NULL);
|
2013-03-24 16:27:20 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
PQfinish(AH->connection);
|
2012-02-16 17:49:20 +01:00
|
|
|
AH->connection = NULL;
|
|
|
|
}
|
|
|
|
|
2012-02-16 19:00:24 +01:00
|
|
|
PGconn *
|
|
|
|
GetConnection(Archive *AHX)
|
|
|
|
{
|
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
|
|
|
|
|
|
|
return AH->connection;
|
|
|
|
}
|
2001-08-12 21:02:39 +02:00
|
|
|
|
|
|
|
static void
|
|
|
|
notice_processor(void *arg, const char *message)
|
|
|
|
{
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_log_info("%s", message);
|
2001-08-12 21:02:39 +02:00
|
|
|
}
|
|
|
|
|
2022-04-08 20:55:14 +02:00
|
|
|
/* Like pg_fatal(), but with a complaint about a particular query. */
|
2012-03-20 22:38:11 +01:00
|
|
|
static void
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
die_on_query_failure(ArchiveHandle *AH, const char *query)
|
2012-03-20 22:38:11 +01:00
|
|
|
{
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
pg_log_error("query failed: %s",
|
2012-03-20 22:38:11 +01:00
|
|
|
PQerrorMessage(AH->connection));
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_log_error_detail("Query was: %s", query);
|
|
|
|
exit(1);
|
2012-03-20 22:38:11 +01:00
|
|
|
}
|
2001-08-12 21:02:39 +02:00
|
|
|
|
2012-02-07 16:07:02 +01:00
|
|
|
void
|
|
|
|
ExecuteSqlStatement(Archive *AHX, const char *query)
|
|
|
|
{
|
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
|
|
|
PGresult *res;
|
|
|
|
|
|
|
|
res = PQexec(AH->connection, query);
|
|
|
|
if (PQresultStatus(res) != PGRES_COMMAND_OK)
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
die_on_query_failure(AH, query);
|
2012-02-07 16:07:02 +01:00
|
|
|
PQclear(res);
|
|
|
|
}
|
|
|
|
|
|
|
|
PGresult *
|
|
|
|
ExecuteSqlQuery(Archive *AHX, const char *query, ExecStatusType status)
|
|
|
|
{
|
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
|
|
|
PGresult *res;
|
|
|
|
|
|
|
|
res = PQexec(AH->connection, query);
|
|
|
|
if (PQresultStatus(res) != status)
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
die_on_query_failure(AH, query);
|
2012-02-07 16:07:02 +01:00
|
|
|
return res;
|
|
|
|
}
|
|
|
|
|
2016-05-26 22:14:23 +02:00
|
|
|
/*
|
|
|
|
* Execute an SQL query and verify that we got exactly one row back.
|
|
|
|
*/
|
|
|
|
PGresult *
|
2017-10-31 15:34:31 +01:00
|
|
|
ExecuteSqlQueryForSingleRow(Archive *fout, const char *query)
|
2016-05-26 22:14:23 +02:00
|
|
|
{
|
|
|
|
PGresult *res;
|
|
|
|
int ntups;
|
|
|
|
|
|
|
|
res = ExecuteSqlQuery(fout, query, PGRES_TUPLES_OK);
|
|
|
|
|
|
|
|
/* Expecting a single result only */
|
|
|
|
ntups = PQntuples(res);
|
|
|
|
if (ntups != 1)
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal(ngettext("query returned %d row instead of one: %s",
|
|
|
|
"query returned %d rows instead of one: %s",
|
|
|
|
ntups),
|
|
|
|
ntups, query);
|
2016-05-26 22:14:23 +02:00
|
|
|
|
|
|
|
return res;
|
|
|
|
}
|
|
|
|
|
2011-07-28 20:06:57 +02:00
|
|
|
/*
|
|
|
|
* Convenience function to send a query.
|
|
|
|
* Monitors result to detect COPY statements
|
|
|
|
*/
|
2008-08-16 04:25:06 +02:00
|
|
|
static void
|
|
|
|
ExecuteSqlCommand(ArchiveHandle *AH, const char *qry, const char *desc)
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
2005-06-21 22:45:44 +02:00
|
|
|
PGconn *conn = AH->connection;
|
2000-07-21 13:43:26 +02:00
|
|
|
PGresult *res;
|
|
|
|
|
2008-08-16 04:25:06 +02:00
|
|
|
#ifdef NOT_USED
|
|
|
|
fprintf(stderr, "Executing: '%s'\n\n", qry);
|
|
|
|
#endif
|
|
|
|
res = PQexec(conn, qry);
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2008-08-16 04:25:06 +02:00
|
|
|
switch (PQresultStatus(res))
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
2008-08-16 04:25:06 +02:00
|
|
|
case PGRES_COMMAND_OK:
|
|
|
|
case PGRES_TUPLES_OK:
|
2011-07-28 20:06:57 +02:00
|
|
|
case PGRES_EMPTY_QUERY:
|
2008-08-16 04:25:06 +02:00
|
|
|
/* A-OK */
|
|
|
|
break;
|
|
|
|
case PGRES_COPY_IN:
|
|
|
|
/* Assume this is an expected result */
|
2006-02-05 21:58:47 +01:00
|
|
|
AH->pgCopyIn = true;
|
2008-08-16 04:25:06 +02:00
|
|
|
break;
|
|
|
|
default:
|
|
|
|
/* trouble */
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
warn_or_exit_horribly(AH, "%s: %sCommand was: %s",
|
2017-03-29 21:17:14 +02:00
|
|
|
desc, PQerrorMessage(conn), qry);
|
2008-08-16 04:25:06 +02:00
|
|
|
break;
|
2000-07-21 13:43:26 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
}
|
|
|
|
|
2011-07-28 20:06:57 +02:00
|
|
|
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
/*
|
|
|
|
* Process non-COPY table data (that is, INSERT commands).
|
|
|
|
*
|
|
|
|
* The commands have been run together as one long string for compressibility,
|
|
|
|
* and we are receiving them in bufferloads with arbitrary boundaries, so we
|
|
|
|
* have to locate command boundaries and save partial commands across calls.
|
|
|
|
* All state must be kept in AH->sqlparse, not in local variables of this
|
|
|
|
* routine. We assume that AH->sqlparse was filled with zeroes when created.
|
|
|
|
*
|
|
|
|
* We have to lex the data to the extent of identifying literals and quoted
|
|
|
|
* identifiers, so that we can recognize statement-terminating semicolons.
|
|
|
|
* We assume that INSERT data will not contain SQL comments, E'' literals,
|
|
|
|
* or dollar-quoted strings, so this is much simpler than a full SQL lexer.
|
2014-06-13 02:14:32 +02:00
|
|
|
*
|
|
|
|
* Note: when restoring from a pre-9.0 dump file, this code is also used to
|
|
|
|
* process BLOB COMMENTS data, which has the same problem of containing
|
|
|
|
* multiple SQL commands that might be split across bufferloads. Fortunately,
|
|
|
|
* that data won't contain anything complicated to lex either.
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
*/
|
|
|
|
static void
|
2014-06-13 02:14:32 +02:00
|
|
|
ExecuteSimpleCommands(ArchiveHandle *AH, const char *buf, size_t bufLen)
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
{
|
|
|
|
const char *qry = buf;
|
|
|
|
const char *eos = buf + bufLen;
|
|
|
|
|
|
|
|
/* initialize command buffer if first time through */
|
|
|
|
if (AH->sqlparse.curCmd == NULL)
|
|
|
|
AH->sqlparse.curCmd = createPQExpBuffer();
|
|
|
|
|
|
|
|
for (; qry < eos; qry++)
|
|
|
|
{
|
|
|
|
char ch = *qry;
|
|
|
|
|
|
|
|
/* For neatness, we skip any newlines between commands */
|
|
|
|
if (!(ch == '\n' && AH->sqlparse.curCmd->len == 0))
|
|
|
|
appendPQExpBufferChar(AH->sqlparse.curCmd, ch);
|
|
|
|
|
|
|
|
switch (AH->sqlparse.state)
|
|
|
|
{
|
|
|
|
case SQL_SCAN: /* Default state == 0, set in _allocAH */
|
|
|
|
if (ch == ';')
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* We've found the end of a statement. Send it and reset
|
|
|
|
* the buffer.
|
|
|
|
*/
|
|
|
|
ExecuteSqlCommand(AH, AH->sqlparse.curCmd->data,
|
|
|
|
"could not execute query");
|
|
|
|
resetPQExpBuffer(AH->sqlparse.curCmd);
|
|
|
|
}
|
|
|
|
else if (ch == '\'')
|
|
|
|
{
|
|
|
|
AH->sqlparse.state = SQL_IN_SINGLE_QUOTE;
|
|
|
|
AH->sqlparse.backSlash = false;
|
|
|
|
}
|
|
|
|
else if (ch == '"')
|
|
|
|
{
|
|
|
|
AH->sqlparse.state = SQL_IN_DOUBLE_QUOTE;
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
|
|
|
|
case SQL_IN_SINGLE_QUOTE:
|
|
|
|
/* We needn't handle '' specially */
|
|
|
|
if (ch == '\'' && !AH->sqlparse.backSlash)
|
|
|
|
AH->sqlparse.state = SQL_SCAN;
|
|
|
|
else if (ch == '\\' && !AH->public.std_strings)
|
|
|
|
AH->sqlparse.backSlash = !AH->sqlparse.backSlash;
|
|
|
|
else
|
|
|
|
AH->sqlparse.backSlash = false;
|
|
|
|
break;
|
|
|
|
|
|
|
|
case SQL_IN_DOUBLE_QUOTE:
|
|
|
|
/* We needn't handle "" specially */
|
|
|
|
if (ch == '"')
|
|
|
|
AH->sqlparse.state = SQL_SCAN;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2002-01-18 20:17:05 +01:00
|
|
|
/*
|
2011-07-28 20:06:57 +02:00
|
|
|
* Implement ahwrite() for direct-to-DB restore
|
2002-01-18 18:13:51 +01:00
|
|
|
*/
|
2011-07-28 20:06:57 +02:00
|
|
|
int
|
2014-10-14 20:00:55 +02:00
|
|
|
ExecuteSqlCommandBuf(Archive *AHX, const char *buf, size_t bufLen)
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
2014-10-14 20:00:55 +02:00
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
|
|
|
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
if (AH->outputKind == OUTPUT_COPYDATA)
|
2002-01-18 20:17:05 +01:00
|
|
|
{
|
2002-01-18 18:13:51 +01:00
|
|
|
/*
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
* COPY data.
|
|
|
|
*
|
2011-07-28 20:06:57 +02:00
|
|
|
* We drop the data on the floor if libpq has failed to enter COPY
|
|
|
|
* mode; this allows us to behave reasonably when trying to continue
|
|
|
|
* after an error in a COPY command.
|
2002-01-18 18:13:51 +01:00
|
|
|
*/
|
2011-07-28 20:06:57 +02:00
|
|
|
if (AH->pgCopyIn &&
|
|
|
|
PQputCopyData(AH->connection, buf, bufLen) <= 0)
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal("error returned by PQputCopyData: %s",
|
|
|
|
PQerrorMessage(AH->connection));
|
2011-07-28 20:06:57 +02:00
|
|
|
}
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
else if (AH->outputKind == OUTPUT_OTHERDATA)
|
|
|
|
{
|
|
|
|
/*
|
2014-06-13 02:14:32 +02:00
|
|
|
* Table data expressed as INSERT commands; or, in old dump files,
|
|
|
|
* BLOB COMMENTS data (which is expressed as COMMENT ON commands).
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
*/
|
2014-06-13 02:14:32 +02:00
|
|
|
ExecuteSimpleCommands(AH, buf, bufLen);
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
}
|
2011-07-28 20:06:57 +02:00
|
|
|
else
|
|
|
|
{
|
2002-01-18 18:13:51 +01:00
|
|
|
/*
|
Fix pg_restore's direct-to-database mode for INSERT-style table data.
In commit 6545a901aaf84cb05212bb6a7674059908f527c3, I removed the mini SQL
lexer that was in pg_backup_db.c, thinking that it had no real purpose
beyond separating COPY data from SQL commands, which purpose had been
obsoleted by long-ago fixes in pg_dump's archive file format.
Unfortunately this was in error: that code was also used to identify
command boundaries in INSERT-style table data, which is run together as a
single string in the archive file for better compressibility. As a result,
direct-to-database restores from archive files made with --inserts or
--column-inserts fail in our latest releases, as reported by Dick Visser.
To fix, restore the mini SQL lexer, but simplify it by adjusting the
calling logic so that it's only required to cope with INSERT-style table
data, not arbitrary SQL commands. This allows us to not have to deal with
SQL comments, E'' strings, or dollar-quoted strings, none of which have
ever been emitted by dumpTableData_insert.
Also, fix the lexer to cope with standard-conforming strings, which was the
actual bug that the previous patch was meant to solve.
Back-patch to all supported branches. The previous patch went back to 8.2,
which unfortunately means that the EOL release of 8.2 contains this bug,
but I don't think we're doing another 8.2 release just because of that.
2012-01-06 19:04:09 +01:00
|
|
|
* General SQL commands; we assume that commands will not be split
|
|
|
|
* across calls.
|
|
|
|
*
|
2011-07-28 20:06:57 +02:00
|
|
|
* In most cases the data passed to us will be a null-terminated
|
|
|
|
* string, but if it's not, we have to add a trailing null.
|
2002-01-18 18:13:51 +01:00
|
|
|
*/
|
2011-07-28 20:06:57 +02:00
|
|
|
if (buf[bufLen] == '\0')
|
|
|
|
ExecuteSqlCommand(AH, buf, "could not execute query");
|
|
|
|
else
|
2002-01-18 18:13:51 +01:00
|
|
|
{
|
2011-11-25 21:40:51 +01:00
|
|
|
char *str = (char *) pg_malloc(bufLen + 1);
|
2011-07-28 20:06:57 +02:00
|
|
|
|
|
|
|
memcpy(str, buf, bufLen);
|
|
|
|
str[bufLen] = '\0';
|
|
|
|
ExecuteSqlCommand(AH, str, "could not execute query");
|
|
|
|
free(str);
|
2002-01-18 18:13:51 +01:00
|
|
|
}
|
|
|
|
}
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2014-05-06 02:27:16 +02:00
|
|
|
return bufLen;
|
2011-07-28 20:06:57 +02:00
|
|
|
}
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2011-07-28 20:06:57 +02:00
|
|
|
/*
|
|
|
|
* Terminate a COPY operation during direct-to-DB restore
|
|
|
|
*/
|
|
|
|
void
|
2014-10-14 20:00:55 +02:00
|
|
|
EndDBCopyMode(Archive *AHX, const char *tocEntryTag)
|
2011-07-28 20:06:57 +02:00
|
|
|
{
|
2014-10-14 20:00:55 +02:00
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
|
|
|
|
2011-07-28 20:06:57 +02:00
|
|
|
if (AH->pgCopyIn)
|
2002-01-18 18:13:51 +01:00
|
|
|
{
|
2006-03-04 00:38:30 +01:00
|
|
|
PGresult *res;
|
|
|
|
|
|
|
|
if (PQputCopyEnd(AH->connection, NULL) <= 0)
|
2022-04-08 20:55:14 +02:00
|
|
|
pg_fatal("error returned by PQputCopyEnd: %s",
|
|
|
|
PQerrorMessage(AH->connection));
|
2000-07-21 13:43:26 +02:00
|
|
|
|
2006-03-04 00:38:30 +01:00
|
|
|
/* Check command status and return to normal libpq state */
|
|
|
|
res = PQgetResult(AH->connection);
|
|
|
|
if (PQresultStatus(res) != PGRES_COMMAND_OK)
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
warn_or_exit_horribly(AH, "COPY failed for table \"%s\": %s",
|
2014-10-14 20:00:55 +02:00
|
|
|
tocEntryTag, PQerrorMessage(AH->connection));
|
2006-03-04 00:38:30 +01:00
|
|
|
PQclear(res);
|
|
|
|
|
Redesign handling of SIGTERM/control-C in parallel pg_dump/pg_restore.
Formerly, Unix builds of pg_dump/pg_restore would trap SIGINT and similar
signals and set a flag that was tested in various data-transfer loops.
This was prone to errors of omission (cf commit 3c8aa6654); and even if
the client-side response was prompt, we did nothing that would cause
long-running SQL commands (e.g. CREATE INDEX) to terminate early.
Also, the master process would effectively do nothing at all upon receipt
of SIGINT; the only reason it seemed to work was that in typical scenarios
the signal would also be delivered to the child processes. We should
support termination when a signal is delivered only to the master process,
though.
Windows builds had no console interrupt handler, so they would just fall
over immediately at control-C, again leaving long-running SQL commands to
finish unmolested.
To fix, remove the flag-checking approach altogether. Instead, allow the
Unix signal handler to send a cancel request directly and then exit(1).
In the master process, also have it forward the signal to the children.
On Windows, add a console interrupt handler that behaves approximately
the same. The main difference is that a single execution of the Windows
handler can send all the cancel requests since all the info is available
in one process, whereas on Unix each process sends a cancel only for its
own database connection.
In passing, fix an old problem that DisconnectDatabase tends to send a
cancel request before exiting a parallel worker, even if nothing went
wrong. This is at least a waste of cycles, and could lead to unexpected
log messages, or maybe even data loss if it happened in pg_restore (though
in the current code the problem seems to affect only pg_dump). The cause
was that after a COPY step, pg_dump was leaving libpq in PGASYNC_BUSY
state, causing PQtransactionStatus() to report PQTRANS_ACTIVE. That's
normally harmless because the next PQexec() will silently clear the
PGASYNC_BUSY state; but in a parallel worker we might exit without any
additional SQL commands after a COPY step. So add an extra PQgetResult()
call after a COPY to allow libpq to return to PGASYNC_IDLE state.
This is a bug fix, IMO, so back-patch to 9.3 where parallel dump/restore
were introduced.
Thanks to Kyotaro Horiguchi for Windows testing and code suggestions.
Original-Patch: <7005.1464657274@sss.pgh.pa.us>
Discussion: <20160602.174941.256342236.horiguchi.kyotaro@lab.ntt.co.jp>
2016-06-02 19:27:53 +02:00
|
|
|
/* Do this to ensure we've pumped libpq back to idle state */
|
|
|
|
if (PQgetResult(AH->connection) != NULL)
|
Unified logging system for command-line programs
This unifies the various ad hoc logging (message printing, error
printing) systems used throughout the command-line programs.
Features:
- Program name is automatically prefixed.
- Message string does not end with newline. This removes a common
source of inconsistencies and omissions.
- Additionally, a final newline is automatically stripped, simplifying
use of PQerrorMessage() etc., another common source of mistakes.
- I converted error message strings to use %m where possible.
- As a result of the above several points, more translatable message
strings can be shared between different components and between
frontends and backend, without gratuitous punctuation or whitespace
differences.
- There is support for setting a "log level". This is not meant to be
user-facing, but can be used internally to implement debug or
verbose modes.
- Lazy argument evaluation, so no significant overhead if logging at
some level is disabled.
- Some color in the messages, similar to gcc and clang. Set
PG_COLOR=auto to try it out. Some colors are predefined, but can be
customized by setting PG_COLORS.
- Common files (common/, fe_utils/, etc.) can handle logging much more
simply by just using one API without worrying too much about the
context of the calling program, requiring callbacks, or having to
pass "progname" around everywhere.
- Some programs called setvbuf() to make sure that stderr is
unbuffered, even on Windows. But not all programs did that. This
is now done centrally.
Soft goals:
- Reduces vertical space use and visual complexity of error reporting
in the source code.
- Encourages more deliberate classification of messages. For example,
in some cases it wasn't clear without analyzing the surrounding code
whether a message was meant as an error or just an info.
- Concepts and terms are vaguely aligned with popular logging
frameworks such as log4j and Python logging.
This is all just about printing stuff out. Nothing affects program
flow (e.g., fatal exits). The uses are just too varied to do that.
Some existing code had wrappers that do some kind of print-and-exit,
and I adapted those.
I tried to keep the output mostly the same, but there is a lot of
historical baggage to unwind and special cases to consider, and I
might not always have succeeded. One significant change is that
pg_rewind used to write all error messages to stdout. That is now
changed to stderr.
Reviewed-by: Donald Dong <xdong@csumb.edu>
Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
|
|
|
pg_log_warning("unexpected extra results during COPY of table \"%s\"",
|
Redesign handling of SIGTERM/control-C in parallel pg_dump/pg_restore.
Formerly, Unix builds of pg_dump/pg_restore would trap SIGINT and similar
signals and set a flag that was tested in various data-transfer loops.
This was prone to errors of omission (cf commit 3c8aa6654); and even if
the client-side response was prompt, we did nothing that would cause
long-running SQL commands (e.g. CREATE INDEX) to terminate early.
Also, the master process would effectively do nothing at all upon receipt
of SIGINT; the only reason it seemed to work was that in typical scenarios
the signal would also be delivered to the child processes. We should
support termination when a signal is delivered only to the master process,
though.
Windows builds had no console interrupt handler, so they would just fall
over immediately at control-C, again leaving long-running SQL commands to
finish unmolested.
To fix, remove the flag-checking approach altogether. Instead, allow the
Unix signal handler to send a cancel request directly and then exit(1).
In the master process, also have it forward the signal to the children.
On Windows, add a console interrupt handler that behaves approximately
the same. The main difference is that a single execution of the Windows
handler can send all the cancel requests since all the info is available
in one process, whereas on Unix each process sends a cancel only for its
own database connection.
In passing, fix an old problem that DisconnectDatabase tends to send a
cancel request before exiting a parallel worker, even if nothing went
wrong. This is at least a waste of cycles, and could lead to unexpected
log messages, or maybe even data loss if it happened in pg_restore (though
in the current code the problem seems to affect only pg_dump). The cause
was that after a COPY step, pg_dump was leaving libpq in PGASYNC_BUSY
state, causing PQtransactionStatus() to report PQTRANS_ACTIVE. That's
normally harmless because the next PQexec() will silently clear the
PGASYNC_BUSY state; but in a parallel worker we might exit without any
additional SQL commands after a COPY step. So add an extra PQgetResult()
call after a COPY to allow libpq to return to PGASYNC_IDLE state.
This is a bug fix, IMO, so back-patch to 9.3 where parallel dump/restore
were introduced.
Thanks to Kyotaro Horiguchi for Windows testing and code suggestions.
Original-Patch: <7005.1464657274@sss.pgh.pa.us>
Discussion: <20160602.174941.256342236.horiguchi.kyotaro@lab.ntt.co.jp>
2016-06-02 19:27:53 +02:00
|
|
|
tocEntryTag);
|
|
|
|
|
2006-02-05 21:58:47 +01:00
|
|
|
AH->pgCopyIn = false;
|
2002-01-18 18:13:51 +01:00
|
|
|
}
|
2000-07-21 13:43:26 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
void
|
2014-10-14 20:00:55 +02:00
|
|
|
StartTransaction(Archive *AHX)
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
2014-10-14 20:00:55 +02:00
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
|
|
|
|
2008-08-16 04:25:06 +02:00
|
|
|
ExecuteSqlCommand(AH, "BEGIN", "could not start database transaction");
|
2000-07-21 13:43:26 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
void
|
2014-10-14 20:00:55 +02:00
|
|
|
CommitTransaction(Archive *AHX)
|
2000-07-21 13:43:26 +02:00
|
|
|
{
|
2014-10-14 20:00:55 +02:00
|
|
|
ArchiveHandle *AH = (ArchiveHandle *) AHX;
|
|
|
|
|
2008-08-16 04:25:06 +02:00
|
|
|
ExecuteSqlCommand(AH, "COMMIT", "could not commit database transaction");
|
2000-10-31 15:20:30 +01:00
|
|
|
}
|
2004-08-20 18:07:15 +02:00
|
|
|
|
2009-12-14 01:39:11 +01:00
|
|
|
void
|
2022-12-05 08:52:11 +01:00
|
|
|
DropLOIfExists(ArchiveHandle *AH, Oid oid)
|
2009-12-14 01:39:11 +01:00
|
|
|
{
|
2010-02-18 02:29:10 +01:00
|
|
|
/*
|
|
|
|
* If we are not restoring to a direct database connection, we have to
|
2022-12-05 08:52:11 +01:00
|
|
|
* guess about how to detect whether the LO exists. Assume new-style.
|
2010-02-18 02:29:10 +01:00
|
|
|
*/
|
|
|
|
if (AH->connection == NULL ||
|
|
|
|
PQserverVersion(AH->connection) >= 90000)
|
2009-12-14 01:39:11 +01:00
|
|
|
{
|
2010-02-18 02:29:10 +01:00
|
|
|
ahprintf(AH,
|
|
|
|
"SELECT pg_catalog.lo_unlink(oid) "
|
|
|
|
"FROM pg_catalog.pg_largeobject_metadata "
|
|
|
|
"WHERE oid = '%u';\n",
|
|
|
|
oid);
|
2009-12-14 01:39:11 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2010-02-18 02:29:10 +01:00
|
|
|
/* Restoring to pre-9.0 server, so do it the old way */
|
|
|
|
ahprintf(AH,
|
|
|
|
"SELECT CASE WHEN EXISTS("
|
|
|
|
"SELECT 1 FROM pg_catalog.pg_largeobject WHERE loid = '%u'"
|
|
|
|
") THEN pg_catalog.lo_unlink('%u') END;\n",
|
2009-12-14 01:39:11 +01:00
|
|
|
oid, oid);
|
|
|
|
}
|
|
|
|
}
|