postgresql/src/interfaces/libpq/fe-exec.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

4441 lines
109 KiB
C
Raw Normal View History

/*-------------------------------------------------------------------------
*
* fe-exec.c
* functions related to sending a query down to the backend
*
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
*
* IDENTIFICATION
2010-09-20 22:08:53 +02:00
* src/interfaces/libpq/fe-exec.c
*
*-------------------------------------------------------------------------
*/
#include "postgres_fe.h"
#include <ctype.h>
#include <fcntl.h>
#include <limits.h>
#ifdef WIN32
#include "win32.h"
1999-07-19 08:25:40 +02:00
#else
#include <unistd.h>
#endif
1997-11-10 06:10:50 +01:00
#include "libpq-fe.h"
#include "libpq-int.h"
#include "mb/pg_wchar.h"
/* keep this in same order as ExecStatusType in libpq-fe.h */
char *const pgresStatus[] = {
"PGRES_EMPTY_QUERY",
"PGRES_COMMAND_OK",
"PGRES_TUPLES_OK",
"PGRES_COPY_OUT",
"PGRES_COPY_IN",
"PGRES_BAD_RESPONSE",
"PGRES_NONFATAL_ERROR",
"PGRES_FATAL_ERROR",
"PGRES_COPY_BOTH",
"PGRES_SINGLE_TUPLE",
"PGRES_PIPELINE_SYNC",
"PGRES_PIPELINE_ABORTED"
};
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* We return this if we're unable to make a PGresult at all */
static const PGresult OOM_result = {
.resultStatus = PGRES_FATAL_ERROR,
.client_encoding = PG_SQL_ASCII,
.errMsg = "out of memory\n",
};
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
/*
* static state needed by PQescapeString and PQescapeBytea; initialize to
* values that result in backward-compatible behavior
*/
static int static_client_encoding = PG_SQL_ASCII;
static bool static_std_strings = false;
static PGEvent *dupEvents(PGEvent *events, int count, size_t *memSize);
static bool pqAddTuple(PGresult *res, PGresAttValue *tup,
const char **errmsgp);
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
static int PQsendQueryInternal(PGconn *conn, const char *query, bool newQuery);
static bool PQsendQueryStart(PGconn *conn, bool newQuery);
static int PQsendQueryGuts(PGconn *conn,
const char *command,
const char *stmtName,
int nParams,
const Oid *paramTypes,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
int resultFormat);
static void parseInput(PGconn *conn);
static PGresult *getCopyResult(PGconn *conn, ExecStatusType copytype);
static bool PQexecStart(PGconn *conn);
static PGresult *PQexecFinish(PGconn *conn);
static int PQsendDescribe(PGconn *conn, char desc_type,
const char *desc_target);
static int check_field_number(const PGresult *res, int field_num);
static void pqPipelineProcessQueue(PGconn *conn);
static int pqPipelineFlush(PGconn *conn);
2003-03-22 04:29:06 +01:00
/* ----------------
* Space management for PGresult.
*
* Formerly, libpq did a separate malloc() for each field of each tuple
* returned by a query. This was remarkably expensive --- malloc/free
* consumed a sizable part of the application's runtime. And there is
* no real need to keep track of the fields separately, since they will
* all be freed together when the PGresult is released. So now, we grab
* large blocks of storage from malloc and allocate space for query data
* within these blocks, using a trivially simple allocator. This reduces
* the number of malloc/free calls dramatically, and it also avoids
* fragmentation of the malloc storage arena.
* The PGresult structure itself is still malloc'd separately. We could
* combine it with the first allocation block, but that would waste space
* for the common case that no extra storage is actually needed (that is,
* the SQL command did not return tuples).
*
* We also malloc the top-level array of tuple pointers separately, because
* we need to be able to enlarge it via realloc, and our trivial space
* allocator doesn't handle that effectively. (Too bad the FE/BE protocol
* doesn't tell us up front how many tuples will be returned.)
* All other subsidiary storage for a PGresult is kept in PGresult_data blocks
* of size PGRESULT_DATA_BLOCKSIZE. The overhead at the start of each block
* is just a link to the next one, if any. Free-space management info is
* kept in the owning PGresult.
* A query returning a small amount of data will thus require three malloc
* calls: one for the PGresult, one for the tuples pointer array, and one
* PGresult_data block.
*
* Only the most recently allocated PGresult_data block is a candidate to
* have more stuff added to it --- any extra space left over in older blocks
* is wasted. We could be smarter and search the whole chain, but the point
* here is to be simple and fast. Typical applications do not keep a PGresult
* around very long anyway, so some wasted space within one is not a problem.
*
* Tuning constants for the space allocator are:
* PGRESULT_DATA_BLOCKSIZE: size of a standard allocation block, in bytes
* PGRESULT_ALIGN_BOUNDARY: assumed alignment requirement for binary data
* PGRESULT_SEP_ALLOC_THRESHOLD: objects bigger than this are given separate
* blocks, instead of being crammed into a regular allocation block.
* Requirements for correct function are:
* PGRESULT_ALIGN_BOUNDARY must be a multiple of the alignment requirements
* of all machine data types. (Currently this is set from configure
* tests, so it should be OK automatically.)
* PGRESULT_SEP_ALLOC_THRESHOLD + PGRESULT_BLOCK_OVERHEAD <=
* PGRESULT_DATA_BLOCKSIZE
* pqResultAlloc assumes an object smaller than the threshold will fit
* in a new block.
* The amount of space wasted at the end of a block could be as much as
* PGRESULT_SEP_ALLOC_THRESHOLD, so it doesn't pay to make that too large.
* ----------------
*/
#define PGRESULT_DATA_BLOCKSIZE 2048
#define PGRESULT_ALIGN_BOUNDARY MAXIMUM_ALIGNOF /* from configure */
#define PGRESULT_BLOCK_OVERHEAD Max(sizeof(PGresult_data), PGRESULT_ALIGN_BOUNDARY)
#define PGRESULT_SEP_ALLOC_THRESHOLD (PGRESULT_DATA_BLOCKSIZE / 2)
/*
* PQmakeEmptyPGresult
* returns a newly allocated, initialized PGresult with given status.
* If conn is not NULL and status indicates an error, the conn's
* errorMessage is copied. Also, any PGEvents are copied from the conn.
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
*
* Note: the logic to copy the conn's errorMessage is now vestigial;
* no internal caller uses it. However, that behavior is documented for
* outside callers, so we'd better keep it.
*/
PGresult *
PQmakeEmptyPGresult(PGconn *conn, ExecStatusType status)
{
PGresult *result;
result = (PGresult *) malloc(sizeof(PGresult));
if (!result)
return NULL;
result->ntups = 0;
result->numAttributes = 0;
result->attDescs = NULL;
result->tuples = NULL;
result->tupArrSize = 0;
result->numParameters = 0;
result->paramDescs = NULL;
result->resultStatus = status;
result->cmdStatus[0] = '\0';
result->binary = 0;
result->events = NULL;
result->nEvents = 0;
result->errMsg = NULL;
result->errFields = NULL;
result->errQuery = NULL;
result->null_field[0] = '\0';
result->curBlock = NULL;
result->curOffset = 0;
result->spaceLeft = 0;
result->memorySize = sizeof(PGresult);
if (conn)
{
/* copy connection data we might need for operations on PGresult */
result->noticeHooks = conn->noticeHooks;
result->client_encoding = conn->client_encoding;
/* consider copying conn's errorMessage */
switch (status)
{
case PGRES_EMPTY_QUERY:
case PGRES_COMMAND_OK:
case PGRES_TUPLES_OK:
case PGRES_COPY_OUT:
case PGRES_COPY_IN:
case PGRES_COPY_BOTH:
case PGRES_SINGLE_TUPLE:
/* non-error cases */
break;
default:
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* we intentionally do not use or modify errorReported here */
pqSetResultError(result, &conn->errorMessage, 0);
break;
}
/* copy events last; result must be valid if we need to PQclear */
if (conn->nEvents > 0)
{
result->events = dupEvents(conn->events, conn->nEvents,
&result->memorySize);
if (!result->events)
{
PQclear(result);
return NULL;
}
result->nEvents = conn->nEvents;
}
}
else
{
/* defaults... */
result->noticeHooks.noticeRec = NULL;
result->noticeHooks.noticeRecArg = NULL;
result->noticeHooks.noticeProc = NULL;
result->noticeHooks.noticeProcArg = NULL;
result->client_encoding = PG_SQL_ASCII;
}
return result;
}
/*
* PQsetResultAttrs
*
* Set the attributes for a given result. This function fails if there are
* already attributes contained in the provided result. The call is
* ignored if numAttributes is zero or attDescs is NULL. If the
* function fails, it returns zero. If the function succeeds, it
* returns a non-zero value.
*/
int
PQsetResultAttrs(PGresult *res, int numAttributes, PGresAttDesc *attDescs)
{
int i;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Fail if argument is NULL or OOM_result */
if (!res || (const PGresult *) res == &OOM_result)
return false;
/* If attrs already exist, they cannot be overwritten. */
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
if (res->numAttributes > 0)
return false;
/* ignore no-op request */
if (numAttributes <= 0 || !attDescs)
return true;
res->attDescs = (PGresAttDesc *)
PQresultAlloc(res, numAttributes * sizeof(PGresAttDesc));
if (!res->attDescs)
return false;
res->numAttributes = numAttributes;
memcpy(res->attDescs, attDescs, numAttributes * sizeof(PGresAttDesc));
/* deep-copy the attribute names, and determine format */
res->binary = 1;
for (i = 0; i < res->numAttributes; i++)
{
if (res->attDescs[i].name)
res->attDescs[i].name = pqResultStrdup(res, res->attDescs[i].name);
else
res->attDescs[i].name = res->null_field;
if (!res->attDescs[i].name)
return false;
if (res->attDescs[i].format == 0)
res->binary = 0;
}
return true;
}
/*
* PQcopyResult
*
* Returns a deep copy of the provided 'src' PGresult, which cannot be NULL.
* The 'flags' argument controls which portions of the result will or will
* NOT be copied. The created result is always put into the
* PGRES_TUPLES_OK status. The source result error message is not copied,
* although cmdStatus is.
*
* To set custom attributes, use PQsetResultAttrs. That function requires
* that there are no attrs contained in the result, so to use that
* function you cannot use the PG_COPYRES_ATTRS or PG_COPYRES_TUPLES
* options with this function.
*
* Options:
* PG_COPYRES_ATTRS - Copy the source result's attributes
*
* PG_COPYRES_TUPLES - Copy the source result's tuples. This implies
* copying the attrs, seeing how the attrs are needed by the tuples.
*
* PG_COPYRES_EVENTS - Copy the source result's events.
*
* PG_COPYRES_NOTICEHOOKS - Copy the source result's notice hooks.
*/
PGresult *
PQcopyResult(const PGresult *src, int flags)
{
PGresult *dest;
int i;
if (!src)
return NULL;
dest = PQmakeEmptyPGresult(NULL, PGRES_TUPLES_OK);
if (!dest)
return NULL;
/* Always copy these over. Is cmdStatus really useful here? */
dest->client_encoding = src->client_encoding;
strcpy(dest->cmdStatus, src->cmdStatus);
/* Wants attrs? */
if (flags & (PG_COPYRES_ATTRS | PG_COPYRES_TUPLES))
{
if (!PQsetResultAttrs(dest, src->numAttributes, src->attDescs))
{
PQclear(dest);
return NULL;
}
}
/* Wants to copy tuples? */
if (flags & PG_COPYRES_TUPLES)
{
int tup,
field;
for (tup = 0; tup < src->ntups; tup++)
{
for (field = 0; field < src->numAttributes; field++)
{
if (!PQsetvalue(dest, tup, field,
src->tuples[tup][field].value,
src->tuples[tup][field].len))
{
PQclear(dest);
return NULL;
}
}
}
}
/* Wants to copy notice hooks? */
if (flags & PG_COPYRES_NOTICEHOOKS)
dest->noticeHooks = src->noticeHooks;
/* Wants to copy PGEvents? */
if ((flags & PG_COPYRES_EVENTS) && src->nEvents > 0)
{
dest->events = dupEvents(src->events, src->nEvents,
&dest->memorySize);
if (!dest->events)
{
PQclear(dest);
return NULL;
}
dest->nEvents = src->nEvents;
}
/* Okay, trigger PGEVT_RESULTCOPY event */
for (i = 0; i < dest->nEvents; i++)
{
2022-02-18 17:37:27 +01:00
/* We don't fire events that had some previous failure */
if (src->events[i].resultInitialized)
{
PGEventResultCopy evt;
evt.src = src;
evt.dest = dest;
2022-02-18 17:37:27 +01:00
if (dest->events[i].proc(PGEVT_RESULTCOPY, &evt,
dest->events[i].passThrough))
dest->events[i].resultInitialized = true;
}
}
return dest;
}
/*
* Copy an array of PGEvents (with no extra space for more).
* Does not duplicate the event instance data, sets this to NULL.
* Also, the resultInitialized flags are all cleared.
* The total space allocated is added to *memSize.
*/
static PGEvent *
dupEvents(PGEvent *events, int count, size_t *memSize)
{
PGEvent *newEvents;
size_t msize;
int i;
if (!events || count <= 0)
return NULL;
msize = count * sizeof(PGEvent);
newEvents = (PGEvent *) malloc(msize);
if (!newEvents)
return NULL;
for (i = 0; i < count; i++)
{
newEvents[i].proc = events[i].proc;
newEvents[i].passThrough = events[i].passThrough;
newEvents[i].data = NULL;
newEvents[i].resultInitialized = false;
newEvents[i].name = strdup(events[i].name);
if (!newEvents[i].name)
{
while (--i >= 0)
free(newEvents[i].name);
free(newEvents);
return NULL;
}
msize += strlen(events[i].name) + 1;
}
*memSize += msize;
return newEvents;
}
/*
* Sets the value for a tuple field. The tup_num must be less than or
* equal to PQntuples(res). If it is equal, a new tuple is created and
* added to the result.
* Returns a non-zero value for success and zero for failure.
* (On failure, we report the specific problem via pqInternalNotice.)
*/
int
PQsetvalue(PGresult *res, int tup_num, int field_num, char *value, int len)
{
PGresAttValue *attval;
const char *errmsg = NULL;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Fail if argument is NULL or OOM_result */
if (!res || (const PGresult *) res == &OOM_result)
return false;
/* Invalid field_num? */
if (!check_field_number(res, field_num))
return false;
/* Invalid tup_num, must be <= ntups */
if (tup_num < 0 || tup_num > res->ntups)
{
pqInternalNotice(&res->noticeHooks,
"row number %d is out of range 0..%d",
tup_num, res->ntups);
return false;
}
/* need to allocate a new tuple? */
if (tup_num == res->ntups)
{
PGresAttValue *tup;
int i;
tup = (PGresAttValue *)
pqResultAlloc(res, res->numAttributes * sizeof(PGresAttValue),
true);
if (!tup)
goto fail;
/* initialize each column to NULL */
for (i = 0; i < res->numAttributes; i++)
{
tup[i].len = NULL_LEN;
tup[i].value = res->null_field;
}
/* add it to the array */
if (!pqAddTuple(res, tup, &errmsg))
goto fail;
}
attval = &res->tuples[tup_num][field_num];
/* treat either NULL_LEN or NULL value pointer as a NULL field */
if (len == NULL_LEN || value == NULL)
{
attval->len = NULL_LEN;
attval->value = res->null_field;
}
else if (len <= 0)
{
attval->len = 0;
attval->value = res->null_field;
}
else
{
attval->value = (char *) pqResultAlloc(res, len + 1, true);
if (!attval->value)
goto fail;
attval->len = len;
memcpy(attval->value, value, len);
attval->value[len] = '\0';
}
return true;
/*
* Report failure via pqInternalNotice. If preceding code didn't provide
* an error message, assume "out of memory" was meant.
*/
fail:
if (!errmsg)
errmsg = libpq_gettext("out of memory");
pqInternalNotice(&res->noticeHooks, "%s", errmsg);
return false;
}
/*
* pqResultAlloc - exported routine to allocate local storage in a PGresult.
*
* We force all such allocations to be maxaligned, since we don't know
* whether the value might be binary.
*/
void *
PQresultAlloc(PGresult *res, size_t nBytes)
{
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Fail if argument is NULL or OOM_result */
if (!res || (const PGresult *) res == &OOM_result)
return NULL;
return pqResultAlloc(res, nBytes, true);
}
/*
* pqResultAlloc -
* Allocate subsidiary storage for a PGresult.
*
* nBytes is the amount of space needed for the object.
* If isBinary is true, we assume that we need to align the object on
* a machine allocation boundary.
* If isBinary is false, we assume the object is a char string and can
* be allocated on any byte boundary.
*/
void *
pqResultAlloc(PGresult *res, size_t nBytes, bool isBinary)
{
char *space;
PGresult_data *block;
if (!res)
return NULL;
if (nBytes <= 0)
return res->null_field;
/*
* If alignment is needed, round up the current position to an alignment
* boundary.
*/
if (isBinary)
{
int offset = res->curOffset % PGRESULT_ALIGN_BOUNDARY;
1999-05-25 18:15:34 +02:00
if (offset)
{
res->curOffset += PGRESULT_ALIGN_BOUNDARY - offset;
res->spaceLeft -= PGRESULT_ALIGN_BOUNDARY - offset;
}
}
/* If there's enough space in the current block, no problem. */
if (nBytes <= (size_t) res->spaceLeft)
{
space = res->curBlock->space + res->curOffset;
res->curOffset += nBytes;
res->spaceLeft -= nBytes;
return space;
}
/*
* If the requested object is very large, give it its own block; this
* avoids wasting what might be most of the current block to start a new
* block. (We'd have to special-case requests bigger than the block size
* anyway.) The object is always given binary alignment in this case.
*/
if (nBytes >= PGRESULT_SEP_ALLOC_THRESHOLD)
{
size_t alloc_size = nBytes + PGRESULT_BLOCK_OVERHEAD;
block = (PGresult_data *) malloc(alloc_size);
if (!block)
return NULL;
res->memorySize += alloc_size;
space = block->space + PGRESULT_BLOCK_OVERHEAD;
if (res->curBlock)
{
/*
* Tuck special block below the active block, so that we don't
* have to waste the free space in the active block.
*/
block->next = res->curBlock->next;
res->curBlock->next = block;
}
else
{
/* Must set up the new block as the first active block. */
block->next = NULL;
res->curBlock = block;
res->spaceLeft = 0; /* be sure it's marked full */
}
return space;
}
/* Otherwise, start a new block. */
block = (PGresult_data *) malloc(PGRESULT_DATA_BLOCKSIZE);
if (!block)
return NULL;
res->memorySize += PGRESULT_DATA_BLOCKSIZE;
block->next = res->curBlock;
res->curBlock = block;
if (isBinary)
{
/* object needs full alignment */
res->curOffset = PGRESULT_BLOCK_OVERHEAD;
res->spaceLeft = PGRESULT_DATA_BLOCKSIZE - PGRESULT_BLOCK_OVERHEAD;
}
else
{
/* we can cram it right after the overhead pointer */
res->curOffset = sizeof(PGresult_data);
res->spaceLeft = PGRESULT_DATA_BLOCKSIZE - sizeof(PGresult_data);
}
space = block->space + res->curOffset;
res->curOffset += nBytes;
res->spaceLeft -= nBytes;
return space;
}
/*
* PQresultMemorySize -
* Returns total space allocated for the PGresult.
*/
size_t
PQresultMemorySize(const PGresult *res)
{
if (!res)
return 0;
return res->memorySize;
}
/*
* pqResultStrdup -
* Like strdup, but the space is subsidiary PGresult space.
*/
char *
pqResultStrdup(PGresult *res, const char *str)
{
char *space = (char *) pqResultAlloc(res, strlen(str) + 1, false);
1999-05-25 18:15:34 +02:00
if (space)
strcpy(space, str);
return space;
}
/*
* pqSetResultError -
* assign a new error message to a PGresult
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
*
* Copy text from errorMessage buffer beginning at given offset
* (it's caller's responsibility that offset is valid)
*/
void
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
pqSetResultError(PGresult *res, PQExpBuffer errorMessage, int offset)
{
char *msg;
if (!res)
return;
/*
* We handle two OOM scenarios here. The errorMessage buffer might be
* marked "broken" due to having previously failed to allocate enough
* memory for the message, or it might be fine but pqResultStrdup fails
* and returns NULL. In either case, just make res->errMsg point directly
* at a constant "out of memory" string.
*/
if (!PQExpBufferBroken(errorMessage))
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
msg = pqResultStrdup(res, errorMessage->data + offset);
else
msg = NULL;
if (msg)
res->errMsg = msg;
else
res->errMsg = libpq_gettext("out of memory\n");
}
/*
* PQclear -
* free's the memory associated with a PGresult
*/
void
PQclear(PGresult *res)
{
PGresult_data *block;
int i;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* As a convenience, do nothing for a NULL pointer */
if (!res)
return;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Also, do nothing if the argument is OOM_result */
if ((const PGresult *) res == &OOM_result)
return;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Close down any events we may have */
for (i = 0; i < res->nEvents; i++)
{
/* only send DESTROY to successfully-initialized event procs */
if (res->events[i].resultInitialized)
{
PGEventResultDestroy evt;
evt.result = res;
(void) res->events[i].proc(PGEVT_RESULTDESTROY, &evt,
res->events[i].passThrough);
}
free(res->events[i].name);
}
free(res->events);
/* Free all the subsidiary blocks */
while ((block = res->curBlock) != NULL)
{
res->curBlock = block->next;
free(block);
}
/* Free the top-level tuple pointer array */
free(res->tuples);
/* zero out the pointer fields to catch programming errors */
res->attDescs = NULL;
res->tuples = NULL;
res->paramDescs = NULL;
res->errFields = NULL;
res->events = NULL;
res->nEvents = 0;
/* res->curBlock was zeroed out earlier */
/* Free the PGresult structure itself */
free(res);
}
/*
* Handy subroutine to deallocate any partially constructed async result.
*
* Any "next" result gets cleared too.
*/
void
pqClearAsyncResult(PGconn *conn)
{
PQclear(conn->result);
conn->result = NULL;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
conn->error_result = false;
PQclear(conn->next_result);
conn->next_result = NULL;
}
/*
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
* pqSaveErrorResult -
* remember that we have an error condition
*
* In much of libpq, reporting an error just requires appending text to
* conn->errorMessage and returning a failure code to one's caller.
* Where returning a failure code is impractical, instead call this
* function to remember that an error needs to be reported.
*
* (It might seem that appending text to conn->errorMessage should be
* sufficient, but we can't rely on that working under out-of-memory
* conditions. The OOM hazard is also why we don't try to make a new
* PGresult right here.)
*/
void
pqSaveErrorResult(PGconn *conn)
{
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Drop any pending result ... */
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
pqClearAsyncResult(conn);
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* ... and set flag to remember to make an error result later */
conn->error_result = true;
}
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
/*
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
* pqSaveWriteError -
* report a write failure
*
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
* As above, after appending conn->write_err_msg to whatever other error we
* have. This is used when we've detected a write failure and have exhausted
* our chances of reporting something else instead.
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
*/
static void
pqSaveWriteError(PGconn *conn)
{
/*
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
* If write_err_msg is null because of previous strdup failure, do what we
* can. (It's likely our machinations here will get OOM failures as well,
* but might as well try.)
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
*/
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (conn->write_err_msg)
{
appendPQExpBufferStr(&conn->errorMessage, conn->write_err_msg);
/* Avoid possibly appending the same message twice */
conn->write_err_msg[0] = '\0';
}
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
else
libpq_append_conn_error(conn, "write to server failed");
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
pqSaveErrorResult(conn);
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
}
/*
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
* pqPrepareAsyncResult -
* prepare the current async result object for return to the caller
*
* If there is not already an async result object, build an error object
* using whatever is in conn->errorMessage. In any case, clear the async
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
* result storage, and update our notion of how much error text has been
* returned to the application.
*/
PGresult *
pqPrepareAsyncResult(PGconn *conn)
{
PGresult *res;
res = conn->result;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
if (res)
{
/*
* If the pre-existing result is an ERROR (presumably something
* received from the server), assume that it represents whatever is in
* conn->errorMessage, and advance errorReported.
*/
if (res->resultStatus == PGRES_FATAL_ERROR)
conn->errorReported = conn->errorMessage.len;
}
else
{
/*
* We get here after internal-to-libpq errors. We should probably
* always have error_result = true, but if we don't, gin up some error
* text.
*/
if (!conn->error_result)
libpq_append_conn_error(conn, "no error text available");
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Paranoia: be sure errorReported offset is sane */
if (conn->errorReported < 0 ||
conn->errorReported >= conn->errorMessage.len)
conn->errorReported = 0;
/*
* Make a PGresult struct for the error. We temporarily lie about the
* result status, so that PQmakeEmptyPGresult doesn't uselessly copy
* all of conn->errorMessage.
*/
res = PQmakeEmptyPGresult(conn, PGRES_EMPTY_QUERY);
if (res)
{
/*
* Report whatever new error text we have, and advance
* errorReported.
*/
res->resultStatus = PGRES_FATAL_ERROR;
pqSetResultError(res, &conn->errorMessage, conn->errorReported);
conn->errorReported = conn->errorMessage.len;
}
else
{
/*
* Ouch, not enough memory for a PGresult. Fortunately, we have a
* card up our sleeve: we can use the static OOM_result. Casting
* away const here is a bit ugly, but it seems best to declare
* OOM_result as const, in hopes it will be allocated in read-only
* storage.
*/
res = unconstify(PGresult *, &OOM_result);
/*
* Don't advance errorReported. Perhaps we'll be able to report
* the text later.
*/
}
}
/*
* Replace conn->result with next_result, if any. In the normal case
* there isn't a next result and we're just dropping ownership of the
* current result. In single-row mode this restores the situation to what
* it was before we created the current single-row result.
*/
conn->result = conn->next_result;
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
conn->error_result = false; /* next_result is never an error */
conn->next_result = NULL;
return res;
}
/*
* pqInternalNotice - produce an internally-generated notice message
*
* A format string and optional arguments can be passed. Note that we do
* libpq_gettext() here, so callers need not.
*
* The supplied text is taken as primary message (ie., it should not include
* a trailing newline, and should not be more than one line).
*/
void
pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
{
char msgBuf[1024];
va_list args;
PGresult *res;
if (hooks->noticeRec == NULL)
return; /* nobody home to receive notice? */
/* Format the message */
va_start(args, fmt);
vsnprintf(msgBuf, sizeof(msgBuf), libpq_gettext(fmt), args);
va_end(args);
msgBuf[sizeof(msgBuf) - 1] = '\0'; /* make real sure it's terminated */
/* Make a PGresult to pass to the notice receiver */
res = PQmakeEmptyPGresult(NULL, PGRES_NONFATAL_ERROR);
if (!res)
return;
res->noticeHooks = *hooks;
2003-08-04 02:43:34 +02:00
/*
* Set up fields of notice.
*/
pqSaveMessageField(res, PG_DIAG_MESSAGE_PRIMARY, msgBuf);
pqSaveMessageField(res, PG_DIAG_SEVERITY, libpq_gettext("NOTICE"));
pqSaveMessageField(res, PG_DIAG_SEVERITY_NONLOCALIZED, "NOTICE");
/* XXX should provide a SQLSTATE too? */
2003-08-04 02:43:34 +02:00
/*
* Result text is always just the primary message + newline. If we can't
* allocate it, substitute "out of memory", as in pqSetResultError.
*/
res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
if (res->errMsg)
sprintf(res->errMsg, "%s\n", msgBuf);
else
res->errMsg = libpq_gettext("out of memory\n");
2003-08-04 02:43:34 +02:00
/*
* Pass to receiver, then free it.
*/
res->noticeHooks.noticeRec(res->noticeHooks.noticeRecArg, res);
PQclear(res);
}
/*
* pqAddTuple
* add a row pointer to the PGresult structure, growing it if necessary
* Returns true if OK, false if an error prevented adding the row
*
* On error, *errmsgp can be set to an error string to be returned.
* If it is left NULL, the error is presumed to be "out of memory".
*/
static bool
pqAddTuple(PGresult *res, PGresAttValue *tup, const char **errmsgp)
{
if (res->ntups >= res->tupArrSize)
{
/*
* Try to grow the array.
*
* We can use realloc because shallow copying of the structure is
* okay. Note that the first time through, res->tuples is NULL. While
* ANSI says that realloc() should act like malloc() in that case,
* some old C libraries (like SunOS 4.1.x) coredump instead. On
* failure realloc is supposed to return NULL without damaging the
* existing allocation. Note that the positions beyond res->ntups are
* garbage, not necessarily NULL.
*/
int newSize;
PGresAttValue **newTuples;
1999-05-25 18:15:34 +02:00
/*
* Since we use integers for row numbers, we can't support more than
* INT_MAX rows. Make sure we allow that many, though.
*/
if (res->tupArrSize <= INT_MAX / 2)
newSize = (res->tupArrSize > 0) ? res->tupArrSize * 2 : 128;
else if (res->tupArrSize < INT_MAX)
newSize = INT_MAX;
else
{
*errmsgp = libpq_gettext("PGresult cannot support more than INT_MAX tuples");
return false;
}
/*
* Also, on 32-bit platforms we could, in theory, overflow size_t even
* before newSize gets to INT_MAX. (In practice we'd doubtless hit
* OOM long before that, but let's check.)
*/
#if INT_MAX >= (SIZE_MAX / 2)
if (newSize > SIZE_MAX / sizeof(PGresAttValue *))
{
*errmsgp = libpq_gettext("size_t overflow");
return false;
}
#endif
if (res->tuples == NULL)
newTuples = (PGresAttValue **)
malloc(newSize * sizeof(PGresAttValue *));
else
newTuples = (PGresAttValue **)
realloc(res->tuples, newSize * sizeof(PGresAttValue *));
if (!newTuples)
return false; /* malloc or realloc failed */
res->memorySize +=
(newSize - res->tupArrSize) * sizeof(PGresAttValue *);
res->tupArrSize = newSize;
res->tuples = newTuples;
}
res->tuples[res->ntups] = tup;
res->ntups++;
return true;
}
/*
* pqSaveMessageField - save one field of an error or notice message
*/
void
pqSaveMessageField(PGresult *res, char code, const char *value)
{
PGMessageField *pfield;
pfield = (PGMessageField *)
pqResultAlloc(res,
2015-02-21 07:46:43 +01:00
offsetof(PGMessageField, contents) +
strlen(value) + 1,
true);
if (!pfield)
return; /* out of memory? */
pfield->code = code;
strcpy(pfield->contents, value);
pfield->next = res->errFields;
res->errFields = pfield;
}
/*
* pqSaveParameterStatus - remember parameter status sent by backend
*/
void
pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
{
pgParameterStatus *pstatus;
pgParameterStatus *prev;
/*
* Forget any old information about the parameter
*/
for (pstatus = conn->pstatus, prev = NULL;
pstatus != NULL;
prev = pstatus, pstatus = pstatus->next)
{
if (strcmp(pstatus->name, name) == 0)
{
if (prev)
prev->next = pstatus->next;
else
conn->pstatus = pstatus->next;
free(pstatus); /* frees name and value strings too */
break;
}
}
2003-08-04 02:43:34 +02:00
/*
* Store new info as a single malloc block
*/
pstatus = (pgParameterStatus *) malloc(sizeof(pgParameterStatus) +
strlen(name) + strlen(value) + 2);
if (pstatus)
{
char *ptr;
ptr = ((char *) pstatus) + sizeof(pgParameterStatus);
pstatus->name = ptr;
strcpy(ptr, name);
ptr += strlen(name) + 1;
pstatus->value = ptr;
strcpy(ptr, value);
pstatus->next = conn->pstatus;
conn->pstatus = pstatus;
}
2003-08-04 02:43:34 +02:00
/*
* Save values of settings that are of interest to libpq in fields of the
* PGconn object. We keep client_encoding and standard_conforming_strings
* in static variables as well, so that PQescapeString and PQescapeBytea
* can behave somewhat sanely (at least in single-connection-using
* programs).
*/
if (strcmp(name, "client_encoding") == 0)
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
{
conn->client_encoding = pg_char_to_encoding(value);
Fix the inadvertent libpq ABI breakage discovered by Martin Pitt: the renumbering of encoding IDs done between 8.2 and 8.3 turns out to break 8.2 initdb and psql if they are run with an 8.3beta1 libpq.so. For the moment we can rearrange the order of enum pg_enc to keep the same number for everything except PG_JOHAB, which isn't a problem since there are no direct references to it in the 8.2 programs anyway. (This does force initdb unfortunately.) Going forward, we want to fix things so that encoding IDs can be changed without an ABI break, and this commit includes the changes needed to allow libpq's encoding IDs to be treated as fully independent of the backend's. The main issue is that libpq clients should not include pg_wchar.h or otherwise assume they know the specific values of libpq's encoding IDs, since they might encounter version skew between pg_wchar.h and the libpq.so they are using. To fix, have libpq officially export functions needed for encoding name<=>ID conversion and validity checking; it was doing this anyway unofficially. It's still the case that we can't renumber backend encoding IDs until the next bump in libpq's major version number, since doing so will break the 8.2-era client programs. However the code is now prepared to avoid this type of problem in future. Note that initdb is no longer a libpq client: we just pull in the two source files we need directly. The patch also fixes a few places that were being sloppy about checking for an unrecognized encoding name.
2007-10-13 22:18:42 +02:00
/* if we don't recognize the encoding name, fall back to SQL_ASCII */
if (conn->client_encoding < 0)
conn->client_encoding = PG_SQL_ASCII;
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
static_client_encoding = conn->client_encoding;
}
else if (strcmp(name, "standard_conforming_strings") == 0)
{
conn->std_strings = (strcmp(value, "on") == 0);
static_std_strings = conn->std_strings;
}
else if (strcmp(name, "server_version") == 0)
{
/* We convert the server version to numeric form. */
int cnt;
int vmaj,
vmin,
vrev;
cnt = sscanf(value, "%d.%d.%d", &vmaj, &vmin, &vrev);
if (cnt == 3)
{
/* old style, e.g. 9.6.1 */
conn->sversion = (100 * vmaj + vmin) * 100 + vrev;
}
else if (cnt == 2)
{
if (vmaj >= 10)
{
/* new style, e.g. 10.1 */
conn->sversion = 100 * 100 * vmaj + vmin;
}
else
{
/* old style without minor version, e.g. 9.6devel */
conn->sversion = (100 * vmaj + vmin) * 100;
}
}
else if (cnt == 1)
{
/* new style without minor version, e.g. 10devel */
conn->sversion = 100 * 100 * vmaj;
}
else
conn->sversion = 0; /* unknown */
}
else if (strcmp(name, "default_transaction_read_only") == 0)
{
conn->default_transaction_read_only =
(strcmp(value, "on") == 0) ? PG_BOOL_YES : PG_BOOL_NO;
}
else if (strcmp(name, "in_hot_standby") == 0)
{
conn->in_hot_standby =
(strcmp(value, "on") == 0) ? PG_BOOL_YES : PG_BOOL_NO;
}
Make SCRAM iteration count configurable Replace the hardcoded value with a GUC such that the iteration count can be raised in order to increase protection against brute-force attacks. The hardcoded value for SCRAM iteration count was defined to be 4096, which is taken from RFC 7677, so set the default for the GUC to 4096 to match. In RFC 7677 the recommendation is at least 15000 iterations but 4096 is listed as a SHOULD requirement given that it's estimated to yield a 0.5s processing time on a mobile handset of the time of RFC writing (late 2015). Raising the iteration count of SCRAM will make stored passwords more resilient to brute-force attacks at a higher computational cost during connection establishment. Lowering the count will reduce computational overhead during connections at the tradeoff of reducing strength against brute-force attacks. There are however platforms where even a modest iteration count yields a too high computational overhead, with weaker password encryption schemes chosen as a result. In these situations, SCRAM with a very low iteration count still gives benefits over weaker schemes like md5, so we allow the iteration count to be set to one at the low end. The new GUC is intentionally generically named such that it can be made to support future SCRAM standards should they emerge. At that point the value can be made into key:value pairs with an undefined key as a default which will be backwards compatible with this. Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Jonathan S. Katz <jkatz@postgresql.org> Discussion: https://postgr.es/m/F72E7BC7-189F-4B17-BF47-9735EB72C364@yesql.se
2023-03-27 09:46:29 +02:00
else if (strcmp(name, "scram_iterations") == 0)
{
conn->scram_sha_256_iterations = atoi(value);
}
}
/*
* pqRowProcessor
* Add the received row to the current async result (conn->result).
* Returns 1 if OK, 0 if error occurred.
*
* On error, *errmsgp can be set to an error string to be returned.
* (Such a string should already be translated via libpq_gettext().)
* If it is left NULL, the error is presumed to be "out of memory".
*
* In single-row mode, we create a new result holding just the current row,
* stashing the previous result in conn->next_result so that it becomes
* active again after pqPrepareAsyncResult(). This allows the result metadata
* (column descriptions) to be carried forward to each result row.
*/
int
pqRowProcessor(PGconn *conn, const char **errmsgp)
{
PGresult *res = conn->result;
int nfields = res->numAttributes;
const PGdataValue *columns = conn->rowBuf;
PGresAttValue *tup;
int i;
/*
* In single-row mode, make a new PGresult that will hold just this one
* row; the original conn->result is left unchanged so that it can be used
* again as the template for future rows.
*/
if (conn->singleRowMode)
{
/* Copy everything that should be in the result at this point */
res = PQcopyResult(res,
PG_COPYRES_ATTRS | PG_COPYRES_EVENTS |
PG_COPYRES_NOTICEHOOKS);
if (!res)
return 0;
}
/*
* Basically we just allocate space in the PGresult for each field and
* copy the data over.
*
* Note: on malloc failure, we return 0 leaving *errmsgp still NULL, which
* caller will take to mean "out of memory". This is preferable to trying
* to set up such a message here, because evidently there's not enough
* memory for gettext() to do anything.
*/
tup = (PGresAttValue *)
pqResultAlloc(res, nfields * sizeof(PGresAttValue), true);
if (tup == NULL)
goto fail;
for (i = 0; i < nfields; i++)
{
int clen = columns[i].len;
if (clen < 0)
{
/* null field */
tup[i].len = NULL_LEN;
tup[i].value = res->null_field;
}
else
{
bool isbinary = (res->attDescs[i].format != 0);
char *val;
val = (char *) pqResultAlloc(res, clen + 1, isbinary);
if (val == NULL)
goto fail;
/* copy and zero-terminate the data (even if it's binary) */
memcpy(val, columns[i].value, clen);
val[clen] = '\0';
tup[i].len = clen;
tup[i].value = val;
}
}
/* And add the tuple to the PGresult's tuple array */
if (!pqAddTuple(res, tup, errmsgp))
goto fail;
/*
* Success. In single-row mode, make the result available to the client
* immediately.
*/
if (conn->singleRowMode)
{
/* Change result status to special single-row value */
res->resultStatus = PGRES_SINGLE_TUPLE;
/* Stash old result for re-use later */
conn->next_result = conn->result;
conn->result = res;
/* And mark the result ready to return */
conn->asyncStatus = PGASYNC_READY_MORE;
}
return 1;
fail:
/* release locally allocated PGresult, if we made one */
if (res != conn->result)
PQclear(res);
return 0;
}
/*
* pqAllocCmdQueueEntry
* Get a command queue entry for caller to fill.
*
* If the recycle queue has a free element, that is returned; if not, a
* fresh one is allocated. Caller is responsible for adding it to the
* command queue (pqAppendCmdQueueEntry) once the struct is filled in, or
* releasing the memory (pqRecycleCmdQueueEntry) if an error occurs.
*
* If allocation fails, sets the error message and returns NULL.
*/
static PGcmdQueueEntry *
pqAllocCmdQueueEntry(PGconn *conn)
{
PGcmdQueueEntry *entry;
if (conn->cmd_queue_recycle == NULL)
{
entry = (PGcmdQueueEntry *) malloc(sizeof(PGcmdQueueEntry));
if (entry == NULL)
{
libpq_append_conn_error(conn, "out of memory");
return NULL;
}
}
else
{
entry = conn->cmd_queue_recycle;
conn->cmd_queue_recycle = entry->next;
}
entry->next = NULL;
entry->query = NULL;
return entry;
}
/*
* pqAppendCmdQueueEntry
* Append a caller-allocated entry to the command queue, and update
* conn->asyncStatus to account for it.
*
* The query itself must already have been put in the output buffer by the
* caller.
*/
static void
pqAppendCmdQueueEntry(PGconn *conn, PGcmdQueueEntry *entry)
{
Assert(entry->next == NULL);
if (conn->cmd_queue_head == NULL)
conn->cmd_queue_head = entry;
else
conn->cmd_queue_tail->next = entry;
conn->cmd_queue_tail = entry;
switch (conn->pipelineStatus)
{
case PQ_PIPELINE_OFF:
case PQ_PIPELINE_ON:
/*
* When not in pipeline aborted state, if there's a result ready
* to be consumed, let it be so (that is, don't change away from
* READY or READY_MORE); otherwise set us busy to wait for
* something to arrive from the server.
*/
if (conn->asyncStatus == PGASYNC_IDLE)
conn->asyncStatus = PGASYNC_BUSY;
break;
case PQ_PIPELINE_ABORTED:
/*
* In aborted pipeline state, we don't expect anything from the
* server (since we don't send any queries that are queued).
* Therefore, if IDLE then do what PQgetResult would do to let
* itself consume commands from the queue; if we're in any other
* state, we don't have to do anything.
*/
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
if (conn->asyncStatus == PGASYNC_IDLE ||
conn->asyncStatus == PGASYNC_PIPELINE_IDLE)
pqPipelineProcessQueue(conn);
break;
}
}
/*
* pqRecycleCmdQueueEntry
* Push a command queue entry onto the freelist.
*/
static void
pqRecycleCmdQueueEntry(PGconn *conn, PGcmdQueueEntry *entry)
{
if (entry == NULL)
return;
/* recyclable entries should not have a follow-on command */
Assert(entry->next == NULL);
if (entry->query)
{
free(entry->query);
entry->query = NULL;
}
entry->next = conn->cmd_queue_recycle;
conn->cmd_queue_recycle = entry;
}
/*
* PQsendQuery
* Submit a query, but don't wait for it to finish
*
* Returns: 1 if successfully submitted
* 0 if error (conn->errorMessage is set)
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
*
* PQsendQueryContinue is a non-exported version that behaves identically
* except that it doesn't reset conn->errorMessage.
*/
int
PQsendQuery(PGconn *conn, const char *query)
{
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
return PQsendQueryInternal(conn, query, true);
}
int
PQsendQueryContinue(PGconn *conn, const char *query)
{
return PQsendQueryInternal(conn, query, false);
}
static int
PQsendQueryInternal(PGconn *conn, const char *query, bool newQuery)
{
PGcmdQueueEntry *entry = NULL;
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (!PQsendQueryStart(conn, newQuery))
return 0;
/* check the argument */
if (!query)
{
libpq_append_conn_error(conn, "command string is a null pointer");
return 0;
}
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "%s not allowed in pipeline mode",
"PQsendQuery");
return 0;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
}
entry = pqAllocCmdQueueEntry(conn);
if (entry == NULL)
return 0; /* error msg already set */
/* Send the query message(s) */
/* construct the outgoing Query message */
if (pqPutMsgStart('Q', conn) < 0 ||
pqPuts(query, conn) < 0 ||
pqPutMsgEnd(conn) < 0)
{
/* error message should be set up already */
pqRecycleCmdQueueEntry(conn, entry);
return 0;
}
/* remember we are using simple query protocol */
entry->queryclass = PGQUERY_SIMPLE;
/* and remember the query text too, if possible */
entry->query = strdup(query);
/*
* Give the data a push. In nonblock mode, don't complain if we're unable
* to send it all; PQgetResult() will do any additional flushing needed.
*/
if (pqFlush(conn) < 0)
goto sendFailed;
/* OK, it's launched! */
pqAppendCmdQueueEntry(conn, entry);
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
return 1;
sendFailed:
pqRecycleCmdQueueEntry(conn, entry);
/* error message should be set up already */
return 0;
}
/*
* PQsendQueryParams
* Like PQsendQuery, but use extended query protocol so we can pass parameters
*/
int
PQsendQueryParams(PGconn *conn,
const char *command,
int nParams,
const Oid *paramTypes,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
{
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (!PQsendQueryStart(conn, true))
return 0;
/* check the arguments */
if (!command)
{
libpq_append_conn_error(conn, "command string is a null pointer");
return 0;
}
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
return PQsendQueryGuts(conn,
command,
"", /* use unnamed statement */
nParams,
paramTypes,
paramValues,
paramLengths,
paramFormats,
resultFormat);
}
/*
* PQsendPrepare
* Submit a Parse message, but don't wait for it to finish
*
* Returns: 1 if successfully submitted
* 0 if error (conn->errorMessage is set)
*/
int
PQsendPrepare(PGconn *conn,
const char *stmtName, const char *query,
int nParams, const Oid *paramTypes)
{
PGcmdQueueEntry *entry = NULL;
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (!PQsendQueryStart(conn, true))
return 0;
/* check the arguments */
if (!stmtName)
{
libpq_append_conn_error(conn, "statement name is a null pointer");
return 0;
}
if (!query)
{
libpq_append_conn_error(conn, "command string is a null pointer");
return 0;
}
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
entry = pqAllocCmdQueueEntry(conn);
if (entry == NULL)
return 0; /* error msg already set */
/* construct the Parse message */
if (pqPutMsgStart('P', conn) < 0 ||
pqPuts(stmtName, conn) < 0 ||
pqPuts(query, conn) < 0)
goto sendFailed;
if (nParams > 0 && paramTypes)
{
int i;
if (pqPutInt(nParams, 2, conn) < 0)
goto sendFailed;
for (i = 0; i < nParams; i++)
{
if (pqPutInt(paramTypes[i], 4, conn) < 0)
goto sendFailed;
}
}
else
{
if (pqPutInt(0, 2, conn) < 0)
goto sendFailed;
}
if (pqPutMsgEnd(conn) < 0)
goto sendFailed;
/* Add a Sync, unless in pipeline mode. */
if (conn->pipelineStatus == PQ_PIPELINE_OFF)
{
if (pqPutMsgStart('S', conn) < 0 ||
pqPutMsgEnd(conn) < 0)
goto sendFailed;
}
/* remember we are doing just a Parse */
entry->queryclass = PGQUERY_PREPARE;
/* and remember the query text too, if possible */
/* if insufficient memory, query just winds up NULL */
entry->query = strdup(query);
/*
* Give the data a push (in pipeline mode, only if we're past the size
* threshold). In nonblock mode, don't complain if we're unable to send
* it all; PQgetResult() will do any additional flushing needed.
*/
if (pqPipelineFlush(conn) < 0)
goto sendFailed;
/* OK, it's launched! */
pqAppendCmdQueueEntry(conn, entry);
return 1;
sendFailed:
pqRecycleCmdQueueEntry(conn, entry);
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
/* error message should be set up already */
return 0;
}
/*
* PQsendQueryPrepared
* Like PQsendQuery, but execute a previously prepared statement,
* using extended query protocol so we can pass parameters
*/
int
PQsendQueryPrepared(PGconn *conn,
const char *stmtName,
int nParams,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
{
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (!PQsendQueryStart(conn, true))
return 0;
/* check the arguments */
if (!stmtName)
{
libpq_append_conn_error(conn, "statement name is a null pointer");
return 0;
}
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
return PQsendQueryGuts(conn,
NULL, /* no command to parse */
stmtName,
nParams,
NULL, /* no param types */
paramValues,
paramLengths,
paramFormats,
resultFormat);
}
/*
* PQsendQueryStart
* Common startup code for PQsendQuery and sibling routines
*/
static bool
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
PQsendQueryStart(PGconn *conn, bool newQuery)
{
if (!conn)
return false;
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
/*
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
* If this is the beginning of a query cycle, reset the error state.
* However, in pipeline mode with something already queued, the error
* buffer belongs to that command and we shouldn't clear it.
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
*/
if (newQuery && conn->cmd_queue_head == NULL)
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
pqClearConnErrorState(conn);
/* Don't try to send if we know there's no live connection. */
if (conn->status != CONNECTION_OK)
{
libpq_append_conn_error(conn, "no connection to the server");
return false;
}
/* Can't send while already busy, either, unless enqueuing for later */
if (conn->asyncStatus != PGASYNC_IDLE &&
conn->pipelineStatus == PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "another command is already in progress");
return false;
}
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
{
/*
* When enqueuing commands we don't change much of the connection
* state since it's already in use for the current command. The
* connection state will get updated when pqPipelineProcessQueue()
* advances to start processing the queued message.
*
* Just make sure we can safely enqueue given the current connection
* state. We can enqueue behind another queue item, or behind a
* non-queue command (one that sends its own sync), but we can't
* enqueue if the connection is in a copy state.
*/
switch (conn->asyncStatus)
{
case PGASYNC_IDLE:
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
case PGASYNC_PIPELINE_IDLE:
case PGASYNC_READY:
case PGASYNC_READY_MORE:
case PGASYNC_BUSY:
/* ok to queue */
break;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
case PGASYNC_COPY_IN:
case PGASYNC_COPY_OUT:
case PGASYNC_COPY_BOTH:
libpq_append_conn_error(conn, "cannot queue commands during COPY");
return false;
}
}
else
{
/*
* This command's results will come in immediately. Initialize async
* result-accumulation state
*/
pqClearAsyncResult(conn);
/* reset single-row processing mode */
conn->singleRowMode = false;
}
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* ready to send command message */
return true;
}
/*
* PQsendQueryGuts
* Common code for sending a query with extended query protocol
* PQsendQueryStart should be done already
*
* command may be NULL to indicate we use an already-prepared statement
*/
static int
PQsendQueryGuts(PGconn *conn,
const char *command,
const char *stmtName,
int nParams,
const Oid *paramTypes,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
{
int i;
PGcmdQueueEntry *entry;
entry = pqAllocCmdQueueEntry(conn);
if (entry == NULL)
return 0; /* error msg already set */
/*
* We will send Parse (if needed), Bind, Describe Portal, Execute, Sync
* (if not in pipeline mode), using specified statement name and the
* unnamed portal.
*/
if (command)
{
/* construct the Parse message */
if (pqPutMsgStart('P', conn) < 0 ||
pqPuts(stmtName, conn) < 0 ||
pqPuts(command, conn) < 0)
goto sendFailed;
if (nParams > 0 && paramTypes)
{
if (pqPutInt(nParams, 2, conn) < 0)
goto sendFailed;
for (i = 0; i < nParams; i++)
{
if (pqPutInt(paramTypes[i], 4, conn) < 0)
goto sendFailed;
}
}
else
{
if (pqPutInt(0, 2, conn) < 0)
goto sendFailed;
}
if (pqPutMsgEnd(conn) < 0)
goto sendFailed;
}
/* Construct the Bind message */
if (pqPutMsgStart('B', conn) < 0 ||
pqPuts("", conn) < 0 ||
pqPuts(stmtName, conn) < 0)
goto sendFailed;
2006-08-05 00:20:06 +02:00
/* Send parameter formats */
if (nParams > 0 && paramFormats)
{
if (pqPutInt(nParams, 2, conn) < 0)
goto sendFailed;
for (i = 0; i < nParams; i++)
{
if (pqPutInt(paramFormats[i], 2, conn) < 0)
goto sendFailed;
}
}
else
{
if (pqPutInt(0, 2, conn) < 0)
goto sendFailed;
}
2006-08-05 00:20:06 +02:00
if (pqPutInt(nParams, 2, conn) < 0)
goto sendFailed;
2006-08-05 00:20:06 +02:00
/* Send parameters */
for (i = 0; i < nParams; i++)
{
if (paramValues && paramValues[i])
{
int nbytes;
if (paramFormats && paramFormats[i] != 0)
{
/* binary parameter */
if (paramLengths)
nbytes = paramLengths[i];
else
{
libpq_append_conn_error(conn, "length must be given for binary parameter");
goto sendFailed;
}
}
else
{
/* text parameter, do not use paramLengths */
nbytes = strlen(paramValues[i]);
}
if (pqPutInt(nbytes, 4, conn) < 0 ||
pqPutnchar(paramValues[i], nbytes, conn) < 0)
goto sendFailed;
}
else
{
/* take the param as NULL */
if (pqPutInt(-1, 4, conn) < 0)
goto sendFailed;
}
}
if (pqPutInt(1, 2, conn) < 0 ||
pqPutInt(resultFormat, 2, conn))
goto sendFailed;
if (pqPutMsgEnd(conn) < 0)
goto sendFailed;
/* construct the Describe Portal message */
if (pqPutMsgStart('D', conn) < 0 ||
pqPutc('P', conn) < 0 ||
pqPuts("", conn) < 0 ||
pqPutMsgEnd(conn) < 0)
goto sendFailed;
/* construct the Execute message */
if (pqPutMsgStart('E', conn) < 0 ||
pqPuts("", conn) < 0 ||
pqPutInt(0, 4, conn) < 0 ||
pqPutMsgEnd(conn) < 0)
goto sendFailed;
/* construct the Sync message if not in pipeline mode */
if (conn->pipelineStatus == PQ_PIPELINE_OFF)
{
if (pqPutMsgStart('S', conn) < 0 ||
pqPutMsgEnd(conn) < 0)
goto sendFailed;
}
/* remember we are using extended query protocol */
entry->queryclass = PGQUERY_EXTENDED;
/* and remember the query text too, if possible */
/* if insufficient memory, query just winds up NULL */
if (command)
entry->query = strdup(command);
/*
* Give the data a push (in pipeline mode, only if we're past the size
* threshold). In nonblock mode, don't complain if we're unable to send
* it all; PQgetResult() will do any additional flushing needed.
*/
if (pqPipelineFlush(conn) < 0)
goto sendFailed;
/* OK, it's launched! */
pqAppendCmdQueueEntry(conn, entry);
return 1;
sendFailed:
pqRecycleCmdQueueEntry(conn, entry);
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
/* error message should be set up already */
return 0;
}
/*
* Select row-by-row processing mode
*/
int
PQsetSingleRowMode(PGconn *conn)
{
/*
* Only allow setting the flag when we have launched a query and not yet
* received any results.
*/
if (!conn)
return 0;
if (conn->asyncStatus != PGASYNC_BUSY)
return 0;
if (!conn->cmd_queue_head ||
(conn->cmd_queue_head->queryclass != PGQUERY_SIMPLE &&
conn->cmd_queue_head->queryclass != PGQUERY_EXTENDED))
return 0;
if (pgHavePendingResult(conn))
return 0;
/* OK, set flag */
conn->singleRowMode = true;
return 1;
}
/*
* Consume any available input from the backend
* 0 return: some kind of trouble
* 1 return: no problem
*/
int
PQconsumeInput(PGconn *conn)
{
if (!conn)
return 0;
/*
* for non-blocking connections try to flush the send-queue, otherwise we
* may never get a response for something that may not have already been
* sent because it's in our write buffer!
*/
if (pqIsnonblocking(conn))
{
if (pqFlush(conn) < 0)
return 0;
}
/*
* Load more data, if available. We do this no matter what state we are
* in, since we are probably getting called because the application wants
* to get rid of a read-select condition. Note that we will NOT block
* waiting for more input.
*/
if (pqReadData(conn) < 0)
return 0;
/* Parsing of the data waits till later. */
return 1;
}
/*
* parseInput: if appropriate, parse input data from backend
* until input is exhausted or a stopping state is reached.
* Note that this function will NOT attempt to read more data from the backend.
*/
static void
parseInput(PGconn *conn)
{
pqParseInput3(conn);
}
/*
* PQisBusy
* Return true if PQgetResult would block waiting for input.
*/
int
PQisBusy(PGconn *conn)
{
if (!conn)
return false;
/* Parse any available data, if our state permits. */
parseInput(conn);
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
/*
Fix thinko in PQisBusy(). In commit 1f39a1c06 I made PQisBusy consider conn->write_failed, but that is now looking like complete brain fade. In the first place, the logic is quite wrong: it ought to be like "and not" rather than "or". This meant that once we'd gotten into a write_failed state, PQisBusy would always return true, probably causing the calling application to iterate its loop until PQconsumeInput returns a hard failure thanks to connection loss. That's not what we want: the intended behavior is to return an error PGresult, which the application probably has much cleaner support for. But in the second place, checking write_failed here seems like the wrong thing anyway. The idea of the write_failed mechanism is to postpone handling of a write failure until we've read all we can from the server; so that flag should not interfere with input-processing behavior. (Compare 7247e243a.) What we *should* check for is status = CONNECTION_BAD, ie, socket already closed. (Most places that close the socket don't touch asyncStatus, but they do reset status.) This primarily ensures that if PQisBusy() returns true then there is an open socket, which is assumed by several call sites in our own code, and probably other applications too. While at it, fix a nearby thinko in libpq's my_sock_write: we should only consult errno for res < 0, not res == 0. This is harmless since pqsecure_raw_write would force errno to zero in such a case, but it still could confuse readers. Noted by Andres Freund. Backpatch to v12 where 1f39a1c06 came in. Discussion: https://postgr.es/m/20220211011025.ek7exh6owpzjyudn@alap3.anarazel.de
2022-02-12 19:23:20 +01:00
* PQgetResult will return immediately in all states except BUSY. Also,
* if we've detected read EOF and dropped the connection, we can expect
* that PQgetResult will fail immediately. Note that we do *not* check
* conn->write_failed here --- once that's become set, we know we have
* trouble, but we need to keep trying to read until we have a complete
* server message or detect read EOF.
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
*/
Fix thinko in PQisBusy(). In commit 1f39a1c06 I made PQisBusy consider conn->write_failed, but that is now looking like complete brain fade. In the first place, the logic is quite wrong: it ought to be like "and not" rather than "or". This meant that once we'd gotten into a write_failed state, PQisBusy would always return true, probably causing the calling application to iterate its loop until PQconsumeInput returns a hard failure thanks to connection loss. That's not what we want: the intended behavior is to return an error PGresult, which the application probably has much cleaner support for. But in the second place, checking write_failed here seems like the wrong thing anyway. The idea of the write_failed mechanism is to postpone handling of a write failure until we've read all we can from the server; so that flag should not interfere with input-processing behavior. (Compare 7247e243a.) What we *should* check for is status = CONNECTION_BAD, ie, socket already closed. (Most places that close the socket don't touch asyncStatus, but they do reset status.) This primarily ensures that if PQisBusy() returns true then there is an open socket, which is assumed by several call sites in our own code, and probably other applications too. While at it, fix a nearby thinko in libpq's my_sock_write: we should only consult errno for res < 0, not res == 0. This is harmless since pqsecure_raw_write would force errno to zero in such a case, but it still could confuse readers. Noted by Andres Freund. Backpatch to v12 where 1f39a1c06 came in. Discussion: https://postgr.es/m/20220211011025.ek7exh6owpzjyudn@alap3.anarazel.de
2022-02-12 19:23:20 +01:00
return conn->asyncStatus == PGASYNC_BUSY && conn->status != CONNECTION_BAD;
}
/*
* PQgetResult
* Get the next PGresult produced by a query. Returns NULL if no
* query work remains or an error has occurred (e.g. out of
* memory).
*
* In pipeline mode, once all the result of a query have been returned,
* PQgetResult returns NULL to let the user know that the next
* query is being processed. At the end of the pipeline, returns a
* result with PQresultStatus(result) == PGRES_PIPELINE_SYNC.
*/
PGresult *
PQgetResult(PGconn *conn)
{
PGresult *res;
if (!conn)
return NULL;
/* Parse any available data, if our state permits. */
parseInput(conn);
/* If not ready to return something, block until we are. */
while (conn->asyncStatus == PGASYNC_BUSY)
{
int flushResult;
/*
* If data remains unsent, send it. Else we might be waiting for the
* result of a command the backend hasn't even got yet.
*/
while ((flushResult = pqFlush(conn)) > 0)
{
if (pqWait(false, true, conn))
{
flushResult = -1;
break;
}
}
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
/*
* Wait for some more data, and load it. (Note: if the connection has
* been lost, pqWait should return immediately because the socket
* should be read-ready, either with the last server data or with an
* EOF indication. We expect therefore that this won't result in any
* undue delay in reporting a previous write failure.)
*/
if (flushResult ||
pqWait(true, false, conn) ||
pqReadData(conn) < 0)
{
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
/* Report the error saved by pqWait or pqReadData */
pqSaveErrorResult(conn);
conn->asyncStatus = PGASYNC_IDLE;
return pqPrepareAsyncResult(conn);
}
/* Parse it. */
parseInput(conn);
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
/*
* If we had a write error, but nothing above obtained a query result
* or detected a read error, report the write error.
*/
if (conn->write_failed && conn->asyncStatus == PGASYNC_BUSY)
{
pqSaveWriteError(conn);
conn->asyncStatus = PGASYNC_IDLE;
return pqPrepareAsyncResult(conn);
}
}
/* Return the appropriate thing. */
switch (conn->asyncStatus)
{
case PGASYNC_IDLE:
res = NULL; /* query is complete */
break;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
case PGASYNC_PIPELINE_IDLE:
Assert(conn->pipelineStatus != PQ_PIPELINE_OFF);
/*
* We're about to return the NULL that terminates the round of
* results from the current query; prepare to send the results of
* the next query, if any, when we're called next. If there's no
* next element in the command queue, this gets us in IDLE state.
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
*/
pqPipelineProcessQueue(conn);
res = NULL; /* query is complete */
break;
case PGASYNC_READY:
/*
* For any query type other than simple query protocol, we advance
* the command queue here. This is because for simple query
* protocol we can get the READY state multiple times before the
* command is actually complete, since the command string can
* contain many queries. In simple query protocol, the queue
* advance is done by fe-protocol3 when it receives ReadyForQuery.
*/
if (conn->cmd_queue_head &&
conn->cmd_queue_head->queryclass != PGQUERY_SIMPLE)
pqCommandQueueAdvance(conn);
res = pqPrepareAsyncResult(conn);
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
{
/*
* We're about to send the results of the current query. Set
* us idle now, and ...
*/
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
conn->asyncStatus = PGASYNC_PIPELINE_IDLE;
/*
* ... in cases when we're sending a pipeline-sync result,
* move queue processing forwards immediately, so that next
* time we're called, we're prepared to return the next result
* received from the server. In all other cases, leave the
* queue state change for next time, so that a terminating
* NULL result is sent.
*
* (In other words: we don't return a NULL after a pipeline
* sync.)
*/
if (res && res->resultStatus == PGRES_PIPELINE_SYNC)
pqPipelineProcessQueue(conn);
}
else
{
/* Set the state back to BUSY, allowing parsing to proceed. */
conn->asyncStatus = PGASYNC_BUSY;
}
break;
case PGASYNC_READY_MORE:
res = pqPrepareAsyncResult(conn);
/* Set the state back to BUSY, allowing parsing to proceed. */
conn->asyncStatus = PGASYNC_BUSY;
break;
case PGASYNC_COPY_IN:
res = getCopyResult(conn, PGRES_COPY_IN);
break;
case PGASYNC_COPY_OUT:
res = getCopyResult(conn, PGRES_COPY_OUT);
break;
case PGASYNC_COPY_BOTH:
res = getCopyResult(conn, PGRES_COPY_BOTH);
break;
default:
libpq_append_conn_error(conn, "unexpected asyncStatus: %d", (int) conn->asyncStatus);
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
pqSaveErrorResult(conn);
conn->asyncStatus = PGASYNC_IDLE; /* try to restore valid state */
res = pqPrepareAsyncResult(conn);
break;
}
2022-02-18 17:37:27 +01:00
/* Time to fire PGEVT_RESULTCREATE events, if there are any */
if (res && res->nEvents > 0)
(void) PQfireResultCreateEvents(conn, res);
return res;
}
/*
* getCopyResult
* Helper for PQgetResult: generate result for COPY-in-progress cases
*/
static PGresult *
getCopyResult(PGconn *conn, ExecStatusType copytype)
{
/*
* If the server connection has been lost, don't pretend everything is
* hunky-dory; instead return a PGRES_FATAL_ERROR result, and reset the
* asyncStatus to idle (corresponding to what we'd do if we'd detected I/O
* error in the earlier steps in PQgetResult). The text returned in the
* result is whatever is in conn->errorMessage; we hope that was filled
* with something relevant when the lost connection was detected.
*/
if (conn->status != CONNECTION_OK)
{
pqSaveErrorResult(conn);
conn->asyncStatus = PGASYNC_IDLE;
return pqPrepareAsyncResult(conn);
}
/* If we have an async result for the COPY, return that */
if (conn->result && conn->result->resultStatus == copytype)
return pqPrepareAsyncResult(conn);
/* Otherwise, invent a suitable PGresult */
return PQmakeEmptyPGresult(conn, copytype);
}
/*
* PQexec
* send a query to the backend and package up the result in a PGresult
*
* If the query was not even sent, return NULL; conn->errorMessage is set to
* a relevant message.
* If the query was sent, a new PGresult is returned (which could indicate
* either success or failure).
* The user is responsible for freeing the PGresult via PQclear()
* when done with it.
*/
PGresult *
PQexec(PGconn *conn, const char *query)
{
if (!PQexecStart(conn))
return NULL;
if (!PQsendQuery(conn, query))
return NULL;
return PQexecFinish(conn);
}
/*
* PQexecParams
* Like PQexec, but use extended query protocol so we can pass parameters
*/
PGresult *
PQexecParams(PGconn *conn,
const char *command,
int nParams,
const Oid *paramTypes,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
{
if (!PQexecStart(conn))
return NULL;
if (!PQsendQueryParams(conn, command,
nParams, paramTypes, paramValues, paramLengths,
paramFormats, resultFormat))
return NULL;
return PQexecFinish(conn);
}
/*
* PQprepare
* Creates a prepared statement by issuing a Parse message.
*
* If the query was not even sent, return NULL; conn->errorMessage is set to
* a relevant message.
* If the query was sent, a new PGresult is returned (which could indicate
* either success or failure).
* The user is responsible for freeing the PGresult via PQclear()
* when done with it.
*/
PGresult *
PQprepare(PGconn *conn,
const char *stmtName, const char *query,
int nParams, const Oid *paramTypes)
{
if (!PQexecStart(conn))
return NULL;
if (!PQsendPrepare(conn, stmtName, query, nParams, paramTypes))
return NULL;
return PQexecFinish(conn);
}
/*
* PQexecPrepared
* Like PQexec, but execute a previously prepared statement,
* using extended query protocol so we can pass parameters
*/
PGresult *
PQexecPrepared(PGconn *conn,
const char *stmtName,
int nParams,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
{
if (!PQexecStart(conn))
return NULL;
if (!PQsendQueryPrepared(conn, stmtName,
nParams, paramValues, paramLengths,
paramFormats, resultFormat))
return NULL;
return PQexecFinish(conn);
}
/*
* Common code for PQexec and sibling routines: prepare to send command
*/
static bool
PQexecStart(PGconn *conn)
{
PGresult *result;
if (!conn)
return false;
/*
* Since this is the beginning of a query cycle, reset the error state.
* However, in pipeline mode with something already queued, the error
* buffer belongs to that command and we shouldn't clear it.
*/
if (conn->cmd_queue_head == NULL)
pqClearConnErrorState(conn);
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "synchronous command execution functions are not allowed in pipeline mode");
return false;
}
/*
* Silently discard any prior query result that application didn't eat.
* This is probably poor design, but it's here for backward compatibility.
*/
while ((result = PQgetResult(conn)) != NULL)
{
ExecStatusType resultStatus = result->resultStatus;
PQclear(result); /* only need its status */
if (resultStatus == PGRES_COPY_IN)
{
/* get out of a COPY IN state */
if (PQputCopyEnd(conn,
libpq_gettext("COPY terminated by new PQexec")) < 0)
return false;
/* keep waiting to swallow the copy's failure message */
}
else if (resultStatus == PGRES_COPY_OUT)
{
/*
* Get out of a COPY OUT state: we just switch back to BUSY and
* allow the remaining COPY data to be dropped on the floor.
*/
conn->asyncStatus = PGASYNC_BUSY;
/* keep waiting to swallow the copy's completion message */
}
else if (resultStatus == PGRES_COPY_BOTH)
{
/* We don't allow PQexec during COPY BOTH */
libpq_append_conn_error(conn, "PQexec not allowed during COPY BOTH");
return false;
}
/* check for loss of connection, too */
if (conn->status == CONNECTION_BAD)
return false;
}
/* OK to send a command */
return true;
}
/*
* Common code for PQexec and sibling routines: wait for command result
*/
static PGresult *
PQexecFinish(PGconn *conn)
{
PGresult *result;
PGresult *lastResult;
/*
* For backwards compatibility, return the last result if there are more
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
* than one. (We used to have logic here to concatenate successive error
* messages, but now that happens automatically, since conn->errorMessage
* will continue to accumulate errors throughout this loop.)
*
* We have to stop if we see copy in/out/both, however. We will resume
* parsing after application performs the data transfer.
*
* Also stop if the connection is lost (else we'll loop infinitely).
*/
lastResult = NULL;
while ((result = PQgetResult(conn)) != NULL)
{
PQclear(lastResult);
lastResult = result;
if (result->resultStatus == PGRES_COPY_IN ||
result->resultStatus == PGRES_COPY_OUT ||
result->resultStatus == PGRES_COPY_BOTH ||
conn->status == CONNECTION_BAD)
break;
}
return lastResult;
}
/*
* PQdescribePrepared
* Obtain information about a previously prepared statement
*
* If the query was not even sent, return NULL; conn->errorMessage is set to
* a relevant message.
* If the query was sent, a new PGresult is returned (which could indicate
* either success or failure). On success, the PGresult contains status
* PGRES_COMMAND_OK, and its parameter and column-heading fields describe
* the statement's inputs and outputs respectively.
* The user is responsible for freeing the PGresult via PQclear()
* when done with it.
*/
PGresult *
PQdescribePrepared(PGconn *conn, const char *stmt)
{
if (!PQexecStart(conn))
return NULL;
if (!PQsendDescribe(conn, 'S', stmt))
return NULL;
return PQexecFinish(conn);
}
/*
* PQdescribePortal
* Obtain information about a previously created portal
*
* This is much like PQdescribePrepared, except that no parameter info is
* returned. Note that at the moment, libpq doesn't really expose portals
* to the client; but this can be used with a portal created by a SQL
* DECLARE CURSOR command.
*/
PGresult *
PQdescribePortal(PGconn *conn, const char *portal)
{
if (!PQexecStart(conn))
return NULL;
if (!PQsendDescribe(conn, 'P', portal))
return NULL;
return PQexecFinish(conn);
}
/*
* PQsendDescribePrepared
* Submit a Describe Statement command, but don't wait for it to finish
*
* Returns: 1 if successfully submitted
* 0 if error (conn->errorMessage is set)
*/
int
PQsendDescribePrepared(PGconn *conn, const char *stmt)
{
return PQsendDescribe(conn, 'S', stmt);
}
/*
* PQsendDescribePortal
* Submit a Describe Portal command, but don't wait for it to finish
*
* Returns: 1 if successfully submitted
* 0 if error (conn->errorMessage is set)
*/
int
PQsendDescribePortal(PGconn *conn, const char *portal)
{
return PQsendDescribe(conn, 'P', portal);
}
/*
* PQsendDescribe
* Common code to send a Describe command
*
* Available options for desc_type are
* 'S' to describe a prepared statement; or
* 'P' to describe a portal.
* Returns 1 on success and 0 on failure.
*/
static int
PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
{
PGcmdQueueEntry *entry = NULL;
/* Treat null desc_target as empty string */
if (!desc_target)
desc_target = "";
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (!PQsendQueryStart(conn, true))
return 0;
entry = pqAllocCmdQueueEntry(conn);
if (entry == NULL)
return 0; /* error msg already set */
/* construct the Describe message */
if (pqPutMsgStart('D', conn) < 0 ||
pqPutc(desc_type, conn) < 0 ||
pqPuts(desc_target, conn) < 0 ||
pqPutMsgEnd(conn) < 0)
goto sendFailed;
/* construct the Sync message */
if (conn->pipelineStatus == PQ_PIPELINE_OFF)
{
if (pqPutMsgStart('S', conn) < 0 ||
pqPutMsgEnd(conn) < 0)
goto sendFailed;
}
/* remember we are doing a Describe */
entry->queryclass = PGQUERY_DESCRIBE;
/*
* Give the data a push (in pipeline mode, only if we're past the size
* threshold). In nonblock mode, don't complain if we're unable to send
* it all; PQgetResult() will do any additional flushing needed.
*/
if (pqPipelineFlush(conn) < 0)
goto sendFailed;
/* OK, it's launched! */
pqAppendCmdQueueEntry(conn, entry);
return 1;
sendFailed:
pqRecycleCmdQueueEntry(conn, entry);
Restructure libpq's handling of send failures. Originally, if libpq got a failure (e.g., ECONNRESET) while trying to send data to the server, it would just report that and wash its hands of the matter. It was soon found that that wasn't a very pleasant way of coping with server-initiated disconnections, so we introduced a hack (pqHandleSendFailure) in the code that sends queries to make it peek ahead for server error reports before reporting the send failure. It now emerges that related cases can occur during connection setup; in particular, as of TLS 1.3 it's unsafe to assume that SSL connection failures will be reported by SSL_connect rather than during our first send attempt. We could have fixed that in a hacky way by applying pqHandleSendFailure after a startup packet send failure, but (a) pqHandleSendFailure explicitly disclaims suitability for use in any state except query startup, and (b) the problem still potentially exists for other send attempts in libpq. Instead, let's fix this in a more general fashion by eliminating pqHandleSendFailure altogether, and instead arranging to postpone all reports of send failures in libpq until after we've made an attempt to read and process server messages. The send failure won't be reported at all if we find a server message or detect input EOF. (Note: this removes one of the reasons why libpq typically overwrites, rather than appending to, conn->errorMessage: pqHandleSendFailure needed that behavior so that the send failure report would be replaced if we got a server message or read failure report. Eventually I'd like to get rid of that overwrite behavior altogether, but today is not that day. For the moment, pqSendSome is assuming that its callees will overwrite not append to conn->errorMessage.) Possibly this change should get back-patched someday; but it needs testing first, so let's not consider that till after v12 beta. Discussion: https://postgr.es/m/CAEepm=2n6Nv+5tFfe8YnkUm1fXgvxR0Mm1FoD+QKG-vLNGLyKg@mail.gmail.com
2019-03-19 21:20:20 +01:00
/* error message should be set up already */
return 0;
}
/*
* PQnotifies
* returns a PGnotify* structure of the latest async notification
* that has not yet been handled
*
* returns NULL, if there is currently
* no unhandled async notification from the backend
*
* the CALLER is responsible for FREE'ing the structure returned
*
* Note that this function does not read any new data from the socket;
* so usually, caller should call PQconsumeInput() first.
*/
PGnotify *
PQnotifies(PGconn *conn)
{
PGnotify *event;
if (!conn)
return NULL;
/* Parse any available data to see if we can extract NOTIFY messages. */
parseInput(conn);
event = conn->notifyHead;
if (event)
{
conn->notifyHead = event->next;
if (!conn->notifyHead)
conn->notifyTail = NULL;
event->next = NULL; /* don't let app see the internal state */
}
return event;
}
/*
* PQputCopyData - send some data to the backend during COPY IN or COPY BOTH
*
* Returns 1 if successful, 0 if data could not be sent (only possible
* in nonblock mode), or -1 if an error occurs.
*/
int
PQputCopyData(PGconn *conn, const char *buffer, int nbytes)
{
if (!conn)
return -1;
if (conn->asyncStatus != PGASYNC_COPY_IN &&
conn->asyncStatus != PGASYNC_COPY_BOTH)
{
libpq_append_conn_error(conn, "no COPY in progress");
return -1;
}
/*
* Process any NOTICE or NOTIFY messages that might be pending in the
* input buffer. Since the server might generate many notices during the
* COPY, we want to clean those out reasonably promptly to prevent
* indefinite expansion of the input buffer. (Note: the actual read of
* input data into the input buffer happens down inside pqSendSome, but
* it's not authorized to get rid of the data again.)
*/
parseInput(conn);
if (nbytes > 0)
{
/*
* Try to flush any previously sent data in preference to growing the
* output buffer. If we can't enlarge the buffer enough to hold the
* data, return 0 in the nonblock case, else hard error. (For
* simplicity, always assume 5 bytes of overhead.)
*/
if ((conn->outBufSize - conn->outCount - 5) < nbytes)
{
if (pqFlush(conn) < 0)
return -1;
if (pqCheckOutBufferSpace(conn->outCount + 5 + (size_t) nbytes,
conn))
return pqIsnonblocking(conn) ? 0 : -1;
}
/* Send the data (too simple to delegate to fe-protocol files) */
if (pqPutMsgStart('d', conn) < 0 ||
pqPutnchar(buffer, nbytes, conn) < 0 ||
pqPutMsgEnd(conn) < 0)
return -1;
}
return 1;
}
/*
* PQputCopyEnd - send EOF indication to the backend during COPY IN
*
* After calling this, use PQgetResult() to check command completion status.
*
* Returns 1 if successful, 0 if data could not be sent (only possible
* in nonblock mode), or -1 if an error occurs.
*/
int
PQputCopyEnd(PGconn *conn, const char *errormsg)
{
if (!conn)
return -1;
Allow a streaming replication standby to follow a timeline switch. Before this patch, streaming replication would refuse to start replicating if the timeline in the primary doesn't exactly match the standby. The situation where it doesn't match is when you have a master, and two standbys, and you promote one of the standbys to become new master. Promoting bumps up the timeline ID, and after that bump, the other standby would refuse to continue. There's significantly more timeline related logic in streaming replication now. First of all, when a standby connects to primary, it will ask the primary for any timeline history files that are missing from the standby. The missing files are sent using a new replication command TIMELINE_HISTORY, and stored in standby's pg_xlog directory. Using the timeline history files, the standby can follow the latest timeline present in the primary (recovery_target_timeline='latest'), just as it can follow new timelines appearing in an archive directory. START_REPLICATION now takes a TIMELINE parameter, to specify exactly which timeline to stream WAL from. This allows the standby to request the primary to send over WAL that precedes the promotion. The replication protocol is changed slightly (in a backwards-compatible way although there's little hope of streaming replication working across major versions anyway), to allow replication to stop when the end of timeline reached, putting the walsender back into accepting a replication command. Many thanks to Amit Kapila for testing and reviewing various versions of this patch.
2012-12-13 18:00:00 +01:00
if (conn->asyncStatus != PGASYNC_COPY_IN &&
conn->asyncStatus != PGASYNC_COPY_BOTH)
{
libpq_append_conn_error(conn, "no COPY in progress");
return -1;
}
2003-08-04 02:43:34 +02:00
/*
* Send the COPY END indicator. This is simple enough that we don't
* bother delegating it to the fe-protocol files.
*/
if (errormsg)
{
/* Send COPY FAIL */
if (pqPutMsgStart('f', conn) < 0 ||
pqPuts(errormsg, conn) < 0 ||
pqPutMsgEnd(conn) < 0)
return -1;
}
else
{
/* Send COPY DONE */
if (pqPutMsgStart('c', conn) < 0 ||
pqPutMsgEnd(conn) < 0)
return -1;
}
/*
* If we sent the COPY command in extended-query mode, we must issue a
* Sync as well.
*/
if (conn->cmd_queue_head &&
conn->cmd_queue_head->queryclass != PGQUERY_SIMPLE)
{
if (pqPutMsgStart('S', conn) < 0 ||
pqPutMsgEnd(conn) < 0)
return -1;
}
/* Return to active duty */
Allow a streaming replication standby to follow a timeline switch. Before this patch, streaming replication would refuse to start replicating if the timeline in the primary doesn't exactly match the standby. The situation where it doesn't match is when you have a master, and two standbys, and you promote one of the standbys to become new master. Promoting bumps up the timeline ID, and after that bump, the other standby would refuse to continue. There's significantly more timeline related logic in streaming replication now. First of all, when a standby connects to primary, it will ask the primary for any timeline history files that are missing from the standby. The missing files are sent using a new replication command TIMELINE_HISTORY, and stored in standby's pg_xlog directory. Using the timeline history files, the standby can follow the latest timeline present in the primary (recovery_target_timeline='latest'), just as it can follow new timelines appearing in an archive directory. START_REPLICATION now takes a TIMELINE parameter, to specify exactly which timeline to stream WAL from. This allows the standby to request the primary to send over WAL that precedes the promotion. The replication protocol is changed slightly (in a backwards-compatible way although there's little hope of streaming replication working across major versions anyway), to allow replication to stop when the end of timeline reached, putting the walsender back into accepting a replication command. Many thanks to Amit Kapila for testing and reviewing various versions of this patch.
2012-12-13 18:00:00 +01:00
if (conn->asyncStatus == PGASYNC_COPY_BOTH)
conn->asyncStatus = PGASYNC_COPY_OUT;
else
conn->asyncStatus = PGASYNC_BUSY;
/* Try to flush data */
if (pqFlush(conn) < 0)
return -1;
return 1;
}
/*
* PQgetCopyData - read a row of data from the backend during COPY OUT
* or COPY BOTH
*
* If successful, sets *buffer to point to a malloc'd row of data, and
* returns row length (always > 0) as result.
* Returns 0 if no row available yet (only possible if async is true),
* -1 if end of copy (consult PQgetResult), or -2 if error (consult
* PQerrorMessage).
*/
int
PQgetCopyData(PGconn *conn, char **buffer, int async)
{
*buffer = NULL; /* for all failure cases */
if (!conn)
return -2;
if (conn->asyncStatus != PGASYNC_COPY_OUT &&
conn->asyncStatus != PGASYNC_COPY_BOTH)
{
libpq_append_conn_error(conn, "no COPY in progress");
return -2;
}
return pqGetCopyData3(conn, buffer, async);
}
/*
* PQgetline - gets a newline-terminated string from the backend.
*
* Chiefly here so that applications can use "COPY <rel> to stdout"
* and read the output string. Returns a null-terminated string in `buffer`.
*
* XXX this routine is now deprecated, because it can't handle binary data.
* If called during a COPY BINARY we return EOF.
*
* PQgetline reads up to `length`-1 characters (like fgets(3)) but strips
* the terminating \n (like gets(3)).
*
* CAUTION: the caller is responsible for detecting the end-of-copy signal
* (a line containing just "\.") when using this routine.
*
* RETURNS:
* EOF if error (eg, invalid arguments are given)
* 0 if EOL is reached (i.e., \n has been read)
* (this is required for backward-compatibility -- this
* routine used to always return EOF or 0, assuming that
* the line ended within `length` bytes.)
* 1 in other cases (i.e., the buffer was filled before \n is reached)
*/
int
PQgetline(PGconn *conn, char *buffer, int length)
{
if (!buffer || length <= 0)
return EOF;
*buffer = '\0';
/* length must be at least 3 to hold the \. terminator! */
if (length < 3)
return EOF;
if (!conn)
return EOF;
return pqGetline3(conn, buffer, length);
}
/*
* PQgetlineAsync - gets a COPY data row without blocking.
*
* This routine is for applications that want to do "COPY <rel> to stdout"
* asynchronously, that is without blocking. Having issued the COPY command
* and gotten a PGRES_COPY_OUT response, the app should call PQconsumeInput
* and this routine until the end-of-data signal is detected. Unlike
* PQgetline, this routine takes responsibility for detecting end-of-data.
*
* On each call, PQgetlineAsync will return data if a complete data row
* is available in libpq's input buffer. Otherwise, no data is returned
* until the rest of the row arrives.
*
* If -1 is returned, the end-of-data signal has been recognized (and removed
* from libpq's input buffer). The caller *must* next call PQendcopy and
* then return to normal processing.
*
* RETURNS:
* -1 if the end-of-copy-data marker has been recognized
* 0 if no data is available
* >0 the number of bytes returned.
*
* The data returned will not extend beyond a data-row boundary. If possible
* a whole row will be returned at one time. But if the buffer offered by
* the caller is too small to hold a row sent by the backend, then a partial
* data row will be returned. In text mode this can be detected by testing
* whether the last returned byte is '\n' or not.
*
* The returned data is *not* null-terminated.
*/
int
PQgetlineAsync(PGconn *conn, char *buffer, int bufsize)
{
if (!conn)
return -1;
return pqGetlineAsync3(conn, buffer, bufsize);
}
/*
* PQputline -- sends a string to the backend during COPY IN.
* Returns 0 if OK, EOF if not.
*
* This is deprecated primarily because the return convention doesn't allow
* caller to tell the difference between a hard error and a nonblock-mode
* send failure.
*/
int
PQputline(PGconn *conn, const char *string)
{
return PQputnbytes(conn, string, strlen(string));
}
/*
* PQputnbytes -- like PQputline, but buffer need not be null-terminated.
* Returns 0 if OK, EOF if not.
*/
int
PQputnbytes(PGconn *conn, const char *buffer, int nbytes)
{
if (PQputCopyData(conn, buffer, nbytes) > 0)
return 0;
else
return EOF;
}
/*
* PQendcopy
* After completing the data transfer portion of a copy in/out,
* the application must call this routine to finish the command protocol.
*
* This is deprecated; it's cleaner to use PQgetResult to get the transfer
* status.
*
* RETURNS:
* 0 on success
* 1 on failure
*/
int
PQendcopy(PGconn *conn)
{
if (!conn)
return 0;
return pqEndcopy3(conn);
}
/* ----------------
* PQfn - Send a function call to the POSTGRES backend.
*
* conn : backend connection
* fnid : OID of function to be called
* result_buf : pointer to result buffer
* result_len : actual length of result is returned here
* result_is_int : If the result is an integer, this must be 1,
* otherwise this should be 0
* args : pointer to an array of function arguments
* (each has length, if integer, and value/pointer)
* nargs : # of arguments in args array.
*
* RETURNS
* PGresult with status = PGRES_COMMAND_OK if successful.
* *result_len is > 0 if there is a return value, 0 if not.
* PGresult with status = PGRES_FATAL_ERROR if backend returns an error.
* NULL on communications failure. conn->errorMessage will be set.
* ----------------
*/
PGresult *
PQfn(PGconn *conn,
int fnid,
int *result_buf,
int *result_len,
int result_is_int,
const PQArgBlock *args,
int nargs)
{
*result_len = 0;
if (!conn)
return NULL;
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
/*
Rearrange libpq's error reporting to avoid duplicated error text. Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18 21:35:15 +01:00
* Since this is the beginning of a query cycle, reset the error state.
* However, in pipeline mode with something already queued, the error
* buffer belongs to that command and we shouldn't clear it.
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
*/
if (conn->cmd_queue_head == NULL)
pqClearConnErrorState(conn);
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "%s not allowed in pipeline mode", "PQfn");
return NULL;
}
if (conn->sock == PGINVALID_SOCKET || conn->asyncStatus != PGASYNC_IDLE ||
pgHavePendingResult(conn))
{
libpq_append_conn_error(conn, "connection in wrong state");
return NULL;
}
return pqFunctionCall3(conn, fnid,
result_buf, result_len,
result_is_int,
args, nargs);
}
/* ====== Pipeline mode support ======== */
/*
* PQenterPipelineMode
* Put an idle connection in pipeline mode.
*
* Returns 1 on success. On failure, errorMessage is set and 0 is returned.
*
* Commands submitted after this can be pipelined on the connection;
* there's no requirement to wait for one to finish before the next is
* dispatched.
*
* Queuing of a new query or syncing during COPY is not allowed.
*
* A set of commands is terminated by a PQpipelineSync. Multiple sync
* points can be established while in pipeline mode. Pipeline mode can
* be exited by calling PQexitPipelineMode() once all results are processed.
*
* This doesn't actually send anything on the wire, it just puts libpq
* into a state where it can pipeline work.
*/
int
PQenterPipelineMode(PGconn *conn)
{
if (!conn)
return 0;
/* succeed with no action if already in pipeline mode */
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
return 1;
if (conn->asyncStatus != PGASYNC_IDLE)
{
libpq_append_conn_error(conn, "cannot enter pipeline mode, connection not idle");
return 0;
}
conn->pipelineStatus = PQ_PIPELINE_ON;
return 1;
}
/*
* PQexitPipelineMode
* End pipeline mode and return to normal command mode.
*
* Returns 1 in success (pipeline mode successfully ended, or not in pipeline
* mode).
*
* Returns 0 if in pipeline mode and cannot be ended yet. Error message will
* be set.
*/
int
PQexitPipelineMode(PGconn *conn)
{
if (!conn)
return 0;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
if (conn->pipelineStatus == PQ_PIPELINE_OFF &&
(conn->asyncStatus == PGASYNC_IDLE ||
conn->asyncStatus == PGASYNC_PIPELINE_IDLE) &&
conn->cmd_queue_head == NULL)
return 1;
switch (conn->asyncStatus)
{
case PGASYNC_READY:
case PGASYNC_READY_MORE:
/* there are some uncollected results */
libpq_append_conn_error(conn, "cannot exit pipeline mode with uncollected results");
return 0;
case PGASYNC_BUSY:
libpq_append_conn_error(conn, "cannot exit pipeline mode while busy");
return 0;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
case PGASYNC_IDLE:
case PGASYNC_PIPELINE_IDLE:
/* OK */
break;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
case PGASYNC_COPY_IN:
case PGASYNC_COPY_OUT:
case PGASYNC_COPY_BOTH:
libpq_append_conn_error(conn, "cannot exit pipeline mode while in COPY");
}
/* still work to process */
if (conn->cmd_queue_head != NULL)
{
libpq_append_conn_error(conn, "cannot exit pipeline mode with uncollected results");
return 0;
}
conn->pipelineStatus = PQ_PIPELINE_OFF;
conn->asyncStatus = PGASYNC_IDLE;
/* Flush any pending data in out buffer */
if (pqFlush(conn) < 0)
return 0; /* error message is setup already */
return 1;
}
/*
* pqCommandQueueAdvance
* Remove one query from the command queue, when we receive
* all results from the server that pertain to it.
*/
void
pqCommandQueueAdvance(PGconn *conn)
{
PGcmdQueueEntry *prevquery;
if (conn->cmd_queue_head == NULL)
return;
/* delink from queue */
prevquery = conn->cmd_queue_head;
conn->cmd_queue_head = conn->cmd_queue_head->next;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
/* If the queue is now empty, reset the tail too */
if (conn->cmd_queue_head == NULL)
conn->cmd_queue_tail = NULL;
/* and make it recyclable */
prevquery->next = NULL;
pqRecycleCmdQueueEntry(conn, prevquery);
}
/*
* pqPipelineProcessQueue: subroutine for PQgetResult
* In pipeline mode, start processing the results of the next query in the queue.
*/
static void
pqPipelineProcessQueue(PGconn *conn)
{
switch (conn->asyncStatus)
{
case PGASYNC_COPY_IN:
case PGASYNC_COPY_OUT:
case PGASYNC_COPY_BOTH:
case PGASYNC_READY:
case PGASYNC_READY_MORE:
case PGASYNC_BUSY:
/* client still has to process current query or results */
return;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
case PGASYNC_IDLE:
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
/*
* If we're in IDLE mode and there's some command in the queue,
* get us into PIPELINE_IDLE mode and process normally. Otherwise
* there's nothing for us to do.
*/
if (conn->cmd_queue_head != NULL)
{
conn->asyncStatus = PGASYNC_PIPELINE_IDLE;
break;
}
return;
case PGASYNC_PIPELINE_IDLE:
Assert(conn->pipelineStatus != PQ_PIPELINE_OFF);
/* next query please */
break;
}
/*
* Reset single-row processing mode. (Client has to set it up for each
* query, if desired.)
*/
conn->singleRowMode = false;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
/*
* If there are no further commands to process in the queue, get us in
* "real idle" mode now.
*/
if (conn->cmd_queue_head == NULL)
{
conn->asyncStatus = PGASYNC_IDLE;
return;
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
}
/*
* Reset the error state. This and the next couple of steps correspond to
* what PQsendQueryStart didn't do for this query.
*/
pqClearConnErrorState(conn);
/* Initialize async result-accumulation state */
pqClearAsyncResult(conn);
if (conn->pipelineStatus == PQ_PIPELINE_ABORTED &&
conn->cmd_queue_head->queryclass != PGQUERY_SYNC)
{
/*
* In an aborted pipeline we don't get anything from the server for
* each result; we're just discarding commands from the queue until we
* get to the next sync from the server.
*
* The PGRES_PIPELINE_ABORTED results tell the client that its queries
* got aborted.
*/
conn->result = PQmakeEmptyPGresult(conn, PGRES_PIPELINE_ABORTED);
if (!conn->result)
{
libpq_append_conn_error(conn, "out of memory");
pqSaveErrorResult(conn);
return;
}
conn->asyncStatus = PGASYNC_READY;
}
else
{
/* allow parsing to continue */
conn->asyncStatus = PGASYNC_BUSY;
}
}
/*
* PQpipelineSync
* Send a Sync message as part of a pipeline, and flush to server
*
* It's legal to start submitting more commands in the pipeline immediately,
* without waiting for the results of the current pipeline. There's no need to
* end pipeline mode and start it again.
*
* If a command in a pipeline fails, every subsequent command up to and including
* the result to the Sync message sent by PQpipelineSync gets set to
* PGRES_PIPELINE_ABORTED state. If the whole pipeline is processed without
* error, a PGresult with PGRES_PIPELINE_SYNC is produced.
*
* Queries can already have been sent before PQpipelineSync is called, but
* PQpipelineSync need to be called before retrieving command results.
*
* The connection will remain in pipeline mode and unavailable for new
* synchronous command execution functions until all results from the pipeline
* are processed by the client.
*/
int
PQpipelineSync(PGconn *conn)
{
PGcmdQueueEntry *entry;
if (!conn)
return 0;
if (conn->pipelineStatus == PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "cannot send pipeline when not in pipeline mode");
return 0;
}
switch (conn->asyncStatus)
{
case PGASYNC_COPY_IN:
case PGASYNC_COPY_OUT:
case PGASYNC_COPY_BOTH:
/* should be unreachable */
appendPQExpBufferStr(&conn->errorMessage,
"internal error: cannot send pipeline while in COPY\n");
return 0;
case PGASYNC_READY:
case PGASYNC_READY_MORE:
case PGASYNC_BUSY:
case PGASYNC_IDLE:
libpq: Improve idle state handling in pipeline mode We were going into IDLE state too soon when executing queries via PQsendQuery in pipeline mode, causing several scenarios to misbehave in different ways -- most notably, as reported by Daniele Varrazzo, that a warning message is produced by libpq: message type 0x33 arrived from server while idle But it is also possible, if queries are sent and results consumed not in lockstep, for the expected mediating NULL result values from PQgetResult to be lost (a problem which has not been reported, but which is more serious). Fix this by introducing two new concepts: one is a command queue element PGQUERY_CLOSE to tell libpq to wait for the CloseComplete server response to the Close message that is sent by PQsendQuery. Because the application is not expecting any PGresult from this, the mechanism to consume it is a bit hackish. The other concept, authored by Horiguchi-san, is a PGASYNC_PIPELINE_IDLE state for libpq's state machine to differentiate "really idle" from merely "the idle state that occurs in between reading results from the server for elements in the pipeline". This makes libpq not go fully IDLE when the libpq command queue contains entries; in normal cases, we only go IDLE once at the end of the pipeline, when the server response to the final SYNC message is received. (However, there are corner cases it doesn't fix, such as terminating the query sequence by PQsendFlushRequest instead of PQpipelineSync; this sort of scenario is what requires PGQUERY_CLOSE bit above.) This last bit helps make the libpq state machine clearer; in particular we can get rid of an ugly hack in pqParseInput3 to avoid considering IDLE as such when the command queue contains entries. A new test mode is added to libpq_pipeline.c to tickle some related problematic cases. Reported-by: Daniele Varrazzo <daniele.varrazzo@gmail.com> Co-authored-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA+mi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw@mail.gmail.com
2022-07-05 14:21:20 +02:00
case PGASYNC_PIPELINE_IDLE:
/* OK to send sync */
break;
}
entry = pqAllocCmdQueueEntry(conn);
if (entry == NULL)
return 0; /* error msg already set */
entry->queryclass = PGQUERY_SYNC;
entry->query = NULL;
/* construct the Sync message */
if (pqPutMsgStart('S', conn) < 0 ||
pqPutMsgEnd(conn) < 0)
goto sendFailed;
/*
* Give the data a push. In nonblock mode, don't complain if we're unable
* to send it all; PQgetResult() will do any additional flushing needed.
*/
if (PQflush(conn) < 0)
goto sendFailed;
/* OK, it's launched! */
pqAppendCmdQueueEntry(conn, entry);
return 1;
sendFailed:
pqRecycleCmdQueueEntry(conn, entry);
/* error message should be set up already */
return 0;
}
/*
* PQsendFlushRequest
* Send request for server to flush its buffer. Useful in pipeline
* mode when a sync point is not desired.
*/
int
PQsendFlushRequest(PGconn *conn)
{
if (!conn)
return 0;
/* Don't try to send if we know there's no live connection. */
if (conn->status != CONNECTION_OK)
{
libpq_append_conn_error(conn, "no connection to the server");
return 0;
}
/* Can't send while already busy, either, unless enqueuing for later */
if (conn->asyncStatus != PGASYNC_IDLE &&
conn->pipelineStatus == PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "another command is already in progress");
return 0;
}
if (pqPutMsgStart('H', conn) < 0 ||
pqPutMsgEnd(conn) < 0)
{
return 0;
}
return 1;
}
/* ====== accessor funcs for PGresult ======== */
ExecStatusType
PQresultStatus(const PGresult *res)
{
if (!res)
return PGRES_FATAL_ERROR;
return res->resultStatus;
}
char *
PQresStatus(ExecStatusType status)
{
if ((unsigned int) status >= lengthof(pgresStatus))
return libpq_gettext("invalid ExecStatusType code");
return pgresStatus[status];
}
char *
PQresultErrorMessage(const PGresult *res)
{
if (!res || !res->errMsg)
return "";
return res->errMsg;
}
char *
PQresultVerboseErrorMessage(const PGresult *res,
PGVerbosity verbosity,
PGContextVisibility show_context)
{
PQExpBufferData workBuf;
/*
* Because the caller is expected to free the result string, we must
* strdup any constant result. We use plain strdup and document that
* callers should expect NULL if out-of-memory.
*/
if (!res ||
(res->resultStatus != PGRES_FATAL_ERROR &&
res->resultStatus != PGRES_NONFATAL_ERROR))
return strdup(libpq_gettext("PGresult is not an error result\n"));
initPQExpBuffer(&workBuf);
pqBuildErrorMessage3(&workBuf, res, verbosity, show_context);
/* If insufficient memory to format the message, fail cleanly */
if (PQExpBufferDataBroken(workBuf))
{
termPQExpBuffer(&workBuf);
return strdup(libpq_gettext("out of memory\n"));
}
return workBuf.data;
}
char *
PQresultErrorField(const PGresult *res, int fieldcode)
{
PGMessageField *pfield;
if (!res)
return NULL;
for (pfield = res->errFields; pfield != NULL; pfield = pfield->next)
{
if (pfield->code == fieldcode)
return pfield->contents;
}
return NULL;
}
int
PQntuples(const PGresult *res)
{
if (!res)
return 0;
return res->ntups;
}
int
PQnfields(const PGresult *res)
{
if (!res)
return 0;
return res->numAttributes;
}
int
PQbinaryTuples(const PGresult *res)
{
if (!res)
return 0;
return res->binary;
}
/*
* Helper routines to range-check field numbers and tuple numbers.
* Return true if OK, false if not
*/
static int
check_field_number(const PGresult *res, int field_num)
{
if (!res)
return false; /* no way to display error message... */
if (field_num < 0 || field_num >= res->numAttributes)
{
pqInternalNotice(&res->noticeHooks,
"column number %d is out of range 0..%d",
field_num, res->numAttributes - 1);
return false;
}
return true;
}
static int
check_tuple_field_number(const PGresult *res,
int tup_num, int field_num)
{
if (!res)
return false; /* no way to display error message... */
if (tup_num < 0 || tup_num >= res->ntups)
{
pqInternalNotice(&res->noticeHooks,
"row number %d is out of range 0..%d",
tup_num, res->ntups - 1);
return false;
}
if (field_num < 0 || field_num >= res->numAttributes)
{
pqInternalNotice(&res->noticeHooks,
"column number %d is out of range 0..%d",
field_num, res->numAttributes - 1);
return false;
}
return true;
}
static int
check_param_number(const PGresult *res, int param_num)
{
if (!res)
return false; /* no way to display error message... */
if (param_num < 0 || param_num >= res->numParameters)
{
pqInternalNotice(&res->noticeHooks,
"parameter number %d is out of range 0..%d",
param_num, res->numParameters - 1);
return false;
}
2006-10-04 02:30:14 +02:00
return true;
}
/*
* returns NULL if the field_num is invalid
*/
char *
PQfname(const PGresult *res, int field_num)
{
if (!check_field_number(res, field_num))
return NULL;
if (res->attDescs)
return res->attDescs[field_num].name;
else
return NULL;
}
/*
* PQfnumber: find column number given column name
*
* The column name is parsed as if it were in a SQL statement, including
* case-folding and double-quote processing. But note a possible gotcha:
* downcasing in the frontend might follow different locale rules than
* downcasing in the backend...
*
* Returns -1 if no match. In the present backend it is also possible
* to have multiple matches, in which case the first one is found.
*/
int
PQfnumber(const PGresult *res, const char *field_name)
{
1997-11-10 06:10:50 +01:00
char *field_case;
bool in_quotes;
bool all_lower = true;
const char *iptr;
char *optr;
int i;
if (!res)
return -1;
/*
* Note: it is correct to reject a zero-length input string; the proper
* input to match a zero-length field name would be "".
*/
if (field_name == NULL ||
field_name[0] == '\0' ||
res->attDescs == NULL)
return -1;
/*
* Check if we can avoid the strdup() and related work because the
* passed-in string wouldn't be changed before we do the check anyway.
*/
for (iptr = field_name; *iptr; iptr++)
{
char c = *iptr;
if (c == '"' || c != pg_tolower((unsigned char) c))
{
all_lower = false;
break;
}
}
if (all_lower)
for (i = 0; i < res->numAttributes; i++)
if (strcmp(field_name, res->attDescs[i].name) == 0)
return i;
2014-03-03 20:05:33 +01:00
/* Fall through to the normal check if that didn't work out. */
/*
* Note: this code will not reject partially quoted strings, eg
* foo"BAR"foo will become fooBARfoo when it probably ought to be an error
* condition.
*/
1997-11-10 06:10:50 +01:00
field_case = strdup(field_name);
if (field_case == NULL)
return -1; /* grotty */
in_quotes = false;
optr = field_case;
for (iptr = field_case; *iptr; iptr++)
1997-11-10 06:10:50 +01:00
{
char c = *iptr;
if (in_quotes)
{
if (c == '"')
{
if (iptr[1] == '"')
{
/* doubled quotes become a single quote */
*optr++ = '"';
iptr++;
}
else
in_quotes = false;
}
else
*optr++ = c;
}
else if (c == '"')
in_quotes = true;
else
{
c = pg_tolower((unsigned char) c);
*optr++ = c;
}
1997-11-10 06:10:50 +01:00
}
*optr = '\0';
1997-11-10 06:10:50 +01:00
for (i = 0; i < res->numAttributes; i++)
{
if (strcmp(field_case, res->attDescs[i].name) == 0)
1997-11-10 06:10:50 +01:00
{
free(field_case);
return i;
1997-11-10 06:10:50 +01:00
}
}
1997-11-10 06:10:50 +01:00
free(field_case);
return -1;
}
Oid
PQftable(const PGresult *res, int field_num)
{
if (!check_field_number(res, field_num))
return InvalidOid;
if (res->attDescs)
return res->attDescs[field_num].tableid;
else
return InvalidOid;
}
int
PQftablecol(const PGresult *res, int field_num)
{
if (!check_field_number(res, field_num))
return 0;
if (res->attDescs)
return res->attDescs[field_num].columnid;
else
return 0;
}
int
PQfformat(const PGresult *res, int field_num)
{
if (!check_field_number(res, field_num))
return 0;
if (res->attDescs)
return res->attDescs[field_num].format;
else
return 0;
}
Oid
PQftype(const PGresult *res, int field_num)
{
if (!check_field_number(res, field_num))
return InvalidOid;
if (res->attDescs)
return res->attDescs[field_num].typid;
else
return InvalidOid;
}
int
PQfsize(const PGresult *res, int field_num)
{
if (!check_field_number(res, field_num))
return 0;
if (res->attDescs)
return res->attDescs[field_num].typlen;
else
return 0;
}
int
PQfmod(const PGresult *res, int field_num)
{
if (!check_field_number(res, field_num))
return 0;
if (res->attDescs)
return res->attDescs[field_num].atttypmod;
else
return 0;
}
char *
PQcmdStatus(PGresult *res)
{
if (!res)
return NULL;
return res->cmdStatus;
}
/*
* PQoidStatus -
* if the last command was an INSERT, return the oid string
* if not, return ""
*/
char *
PQoidStatus(const PGresult *res)
{
/*
* This must be enough to hold the result. Don't laugh, this is better
* than what this function used to do.
*/
static char buf[24];
size_t len;
if (!res || strncmp(res->cmdStatus, "INSERT ", 7) != 0)
return "";
len = strspn(res->cmdStatus + 7, "0123456789");
if (len > sizeof(buf) - 1)
len = sizeof(buf) - 1;
memcpy(buf, res->cmdStatus + 7, len);
buf[len] = '\0';
return buf;
}
/*
* PQoidValue -
* a perhaps preferable form of the above which just returns
* an Oid type
*/
Oid
PQoidValue(const PGresult *res)
{
char *endptr = NULL;
unsigned long result;
if (!res ||
strncmp(res->cmdStatus, "INSERT ", 7) != 0 ||
res->cmdStatus[7] < '0' ||
res->cmdStatus[7] > '9')
return InvalidOid;
result = strtoul(res->cmdStatus + 7, &endptr, 10);
if (!endptr || (*endptr != ' ' && *endptr != '\0'))
return InvalidOid;
else
return (Oid) result;
}
/*
* PQcmdTuples -
Add support for MERGE SQL command MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <pavan.deolasee@gmail.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Amit Langote <amitlangote09@gmail.com> Author: Simon Riggs <simon.riggs@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions) Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions) Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:45:58 +02:00
* If the last command was INSERT/UPDATE/DELETE/MERGE/MOVE/FETCH/COPY,
* return a string containing the number of inserted/affected tuples.
* If not, return "".
*
* XXX: this should probably return an int
*/
char *
PQcmdTuples(PGresult *res)
{
char *p,
*c;
if (!res)
return "";
if (strncmp(res->cmdStatus, "INSERT ", 7) == 0)
{
p = res->cmdStatus + 7;
/* INSERT: skip oid and space */
while (*p && *p != ' ')
p++;
if (*p == 0)
goto interpret_error; /* no space? */
p++;
}
else if (strncmp(res->cmdStatus, "SELECT ", 7) == 0 ||
strncmp(res->cmdStatus, "DELETE ", 7) == 0 ||
strncmp(res->cmdStatus, "UPDATE ", 7) == 0)
p = res->cmdStatus + 7;
Add support for MERGE SQL command MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <pavan.deolasee@gmail.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Amit Langote <amitlangote09@gmail.com> Author: Simon Riggs <simon.riggs@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions) Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions) Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:45:58 +02:00
else if (strncmp(res->cmdStatus, "FETCH ", 6) == 0 ||
strncmp(res->cmdStatus, "MERGE ", 6) == 0)
p = res->cmdStatus + 6;
else if (strncmp(res->cmdStatus, "MOVE ", 5) == 0 ||
strncmp(res->cmdStatus, "COPY ", 5) == 0)
p = res->cmdStatus + 5;
else
return "";
/* check that we have an integer (at least one digit, nothing else) */
for (c = p; *c; c++)
{
if (!isdigit((unsigned char) *c))
goto interpret_error;
}
if (c == p)
goto interpret_error;
return p;
2006-10-04 02:30:14 +02:00
interpret_error:
pqInternalNotice(&res->noticeHooks,
"could not interpret result from server: %s",
res->cmdStatus);
return "";
}
/*
* PQgetvalue:
* return the value of field 'field_num' of row 'tup_num'
*/
char *
PQgetvalue(const PGresult *res, int tup_num, int field_num)
{
if (!check_tuple_field_number(res, tup_num, field_num))
return NULL;
return res->tuples[tup_num][field_num].value;
}
/* PQgetlength:
* returns the actual length of a field value in bytes.
*/
int
PQgetlength(const PGresult *res, int tup_num, int field_num)
{
if (!check_tuple_field_number(res, tup_num, field_num))
return 0;
if (res->tuples[tup_num][field_num].len != NULL_LEN)
return res->tuples[tup_num][field_num].len;
else
return 0;
}
/* PQgetisnull:
* returns the null status of a field value.
*/
int
PQgetisnull(const PGresult *res, int tup_num, int field_num)
{
if (!check_tuple_field_number(res, tup_num, field_num))
return 1; /* pretend it is null */
if (res->tuples[tup_num][field_num].len == NULL_LEN)
return 1;
else
return 0;
}
/* PQnparams:
* returns the number of input parameters of a prepared statement.
*/
int
PQnparams(const PGresult *res)
{
if (!res)
return 0;
return res->numParameters;
}
/* PQparamtype:
* returns type Oid of the specified statement parameter.
*/
Oid
PQparamtype(const PGresult *res, int param_num)
{
if (!check_param_number(res, param_num))
return InvalidOid;
if (res->paramDescs)
return res->paramDescs[param_num].typid;
else
return InvalidOid;
}
/* PQsetnonblocking:
* sets the PGconn's database connection non-blocking if the arg is true
* or makes it blocking if the arg is false, this will not protect
* you from PQexec(), you'll only be safe when using the non-blocking API.
* Needs to be called only on a connected database connection.
*/
int
PQsetnonblocking(PGconn *conn, int arg)
{
bool barg;
if (!conn || conn->status == CONNECTION_BAD)
return -1;
barg = (arg ? true : false);
/* early out if the socket is already in the state requested */
if (barg == conn->nonblocking)
return 0;
/*
* to guarantee constancy for flushing/query/result-polling behavior we
* need to flush the send queue at this point in order to guarantee proper
* behavior. this is ok because either they are making a transition _from_
* or _to_ blocking mode, either way we can block them.
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
*
* Clear error state in case pqFlush adds to it, unless we're actively
* pipelining, in which case it seems best not to.
*/
if (conn->cmd_queue_head == NULL)
pqClearConnErrorState(conn);
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
/* if we are going from blocking to non-blocking flush here */
if (pqFlush(conn))
return -1;
conn->nonblocking = barg;
return 0;
}
/*
* return the blocking status of the database connection
* true == nonblocking, false == blocking
*/
int
PQisnonblocking(const PGconn *conn)
{
if (!conn || conn->status == CONNECTION_BAD)
return false;
return pqIsnonblocking(conn);
}
/* libpq is thread-safe? */
int
PQisthreadsafe(void)
{
#ifdef ENABLE_THREAD_SAFETY
return true;
#else
return false;
#endif
}
/* try to force data out, really only useful for non-blocking users */
int
PQflush(PGconn *conn)
{
if (!conn || conn->status == CONNECTION_BAD)
return -1;
return pqFlush(conn);
}
Here's a patch against 7.1.3 that fixes a problem with sending larger queries over non-blocking connections with libpq. "Larger" here basically means that it doesn't fit into the output buffer. The basic strategy is to fix pqFlush and pqPutBytes. The problem with pqFlush as it stands now is that it returns EOF when an error occurs or when not all data could be sent. The latter case is clearly not an error for a non-blocking connection but the caller can't distringuish it from an error very well. The first part of the fix is therefore to fix pqFlush. This is done by to renaming it to pqSendSome which only differs from pqFlush in its return values to allow the caller to make the above distinction and a new pqFlush which is implemented in terms of pqSendSome and behaves exactly like the old pqFlush. The second part of the fix modifies pqPutBytes to use pqSendSome instead of pqFlush and to either send all the data or if not all data can be sent on a non-blocking connection to at least put all data into the output buffer, enlarging it if necessary. The callers of pqPutBytes don't have to be changed because from their point of view pqPutBytes behaves like before. It either succeeds in queueing all output data or fails with an error. I've also added a new API function PQsendSome which analogously to PQflush just calls pqSendSome. Programs using non-blocking queries should use this new function. The main difference is that this function will have to be called repeatedly (calling select() properly in between) until all data has been written. AFAICT, the code in CVS HEAD hasn't changed with respect to non-blocking queries and this fix should work there, too, but I haven't tested that yet. Bernhard Herzog
2002-03-05 06:20:12 +01:00
/*
* pqPipelineFlush
*
* In pipeline mode, data will be flushed only when the out buffer reaches the
* threshold value. In non-pipeline mode, it behaves as stock pqFlush.
*
* Returns 0 on success.
*/
static int
pqPipelineFlush(PGconn *conn)
{
if ((conn->pipelineStatus != PQ_PIPELINE_ON) ||
(conn->outCount >= OUTBUFFER_THRESHOLD))
return pqFlush(conn);
return 0;
}
/*
* PQfreemem - safely frees memory allocated
*
* Needed mostly by Win32, unless multithreaded DLL (/MD in VC6)
* Used for freeing memory from PQescapeBytea()/PQunescapeBytea()
*/
void
PQfreemem(void *ptr)
{
free(ptr);
}
/*
* PQfreeNotify - free's the memory associated with a PGnotify
*
* This function is here only for binary backward compatibility.
* New code should use PQfreemem(). A macro will automatically map
* calls to PQfreemem. It should be removed in the future. bjm 2003-03-24
*/
#undef PQfreeNotify
void PQfreeNotify(PGnotify *notify);
void
PQfreeNotify(PGnotify *notify)
{
PQfreemem(notify);
}
/*
* Escaping arbitrary strings to get valid SQL literal strings.
*
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
* Replaces "'" with "''", and if not std_strings, replaces "\" with "\\".
*
* length is the length of the source string. (Note: if a terminating NUL
* is encountered sooner, PQescapeString stops short of "length"; the behavior
* is thus rather like strncpy.)
*
* For safety the buffer at "to" must be at least 2*length + 1 bytes long.
* A terminating NUL character is added to the output string, whether the
* input is NUL-terminated or not.
*
* Returns the actual length of the output (not counting the terminating NUL).
*/
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
static size_t
PQescapeStringInternal(PGconn *conn,
char *to, const char *from, size_t length,
int *error,
int encoding, bool std_strings)
{
const char *source = from;
char *target = to;
size_t remaining = length;
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
if (error)
*error = 0;
while (remaining > 0 && *source != '\0')
{
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
char c = *source;
int len;
int i;
/* Fast path for plain ASCII */
if (!IS_HIGHBIT_SET(c))
{
/* Apply quoting if needed */
if (SQL_STR_DOUBLE(c, !std_strings))
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
*target++ = c;
/* Copy the character */
*target++ = c;
source++;
remaining--;
continue;
}
/* Slow path for possible multibyte characters */
len = pg_encoding_mblen(encoding, source);
/* Copy the character */
for (i = 0; i < len; i++)
{
if (remaining == 0 || *source == '\0')
break;
*target++ = *source++;
remaining--;
}
/*
* If we hit premature end of string (ie, incomplete multibyte
* character), try to pad out to the correct length with spaces. We
* may not be able to pad completely, but we will always be able to
* insert at least one pad space (since we'd not have quoted a
* multibyte character). This should be enough to make a string that
* the server will error out on.
*/
if (i < len)
{
if (error)
*error = 1;
if (conn)
libpq_append_conn_error(conn, "incomplete multibyte character");
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
for (; i < len; i++)
{
if (((size_t) (target - to)) / 2 >= length)
break;
*target++ = ' ';
}
break;
}
}
/* Write the terminating NUL character. */
*target = '\0';
return target - to;
}
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
size_t
PQescapeStringConn(PGconn *conn,
char *to, const char *from, size_t length,
int *error)
{
if (!conn)
{
/* force empty-string result */
*to = '\0';
if (error)
*error = 1;
return 0;
}
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (conn->cmd_queue_head == NULL)
pqClearConnErrorState(conn);
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
return PQescapeStringInternal(conn, to, from, length, error,
conn->client_encoding,
conn->std_strings);
}
size_t
PQescapeString(char *to, const char *from, size_t length)
{
return PQescapeStringInternal(NULL, to, from, length, NULL,
static_client_encoding,
static_std_strings);
}
/*
* Escape arbitrary strings. If as_ident is true, we escape the result
* as an identifier; if false, as a literal. The result is returned in
* a newly allocated buffer. If we fail due to an encoding violation or out
* of memory condition, we return NULL, storing an error message into conn.
*/
static char *
PQescapeInternal(PGconn *conn, const char *str, size_t len, bool as_ident)
{
const char *s;
char *result;
char *rp;
int num_quotes = 0; /* single or double, depending on as_ident */
int num_backslashes = 0;
int input_len;
int result_size;
char quote_char = as_ident ? '"' : '\'';
/* We must have a connection, else fail immediately. */
if (!conn)
return NULL;
if (conn->cmd_queue_head == NULL)
pqClearConnErrorState(conn);
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
/* Scan the string for characters that must be escaped. */
for (s = str; (s - str) < len && *s != '\0'; ++s)
{
if (*s == quote_char)
++num_quotes;
else if (*s == '\\')
++num_backslashes;
else if (IS_HIGHBIT_SET(*s))
{
int charlen;
/* Slow path for possible multibyte characters */
charlen = pg_encoding_mblen(conn->client_encoding, s);
/* Multibyte character overruns allowable length. */
if ((s - str) + charlen > len || memchr(s, 0, charlen) != NULL)
{
libpq_append_conn_error(conn, "incomplete multibyte character");
return NULL;
}
/* Adjust s, bearing in mind that for loop will increment it. */
s += charlen - 1;
}
}
/* Allocate output buffer. */
input_len = s - str;
result_size = input_len + num_quotes + 3; /* two quotes, plus a NUL */
if (!as_ident && num_backslashes > 0)
result_size += num_backslashes + 2;
result = rp = (char *) malloc(result_size);
if (rp == NULL)
{
libpq_append_conn_error(conn, "out of memory");
return NULL;
}
/*
* If we are escaping a literal that contains backslashes, we use the
* escape string syntax so that the result is correct under either value
* of standard_conforming_strings. We also emit a leading space in this
* case, to guard against the possibility that the result might be
* interpolated immediately following an identifier.
*/
if (!as_ident && num_backslashes > 0)
{
*rp++ = ' ';
*rp++ = 'E';
}
/* Opening quote. */
*rp++ = quote_char;
/*
* Use fast path if possible.
*
* We've already verified that the input string is well-formed in the
* current encoding. If it contains no quotes and, in the case of
* literal-escaping, no backslashes, then we can just copy it directly to
* the output buffer, adding the necessary quotes.
*
* If not, we must rescan the input and process each character
* individually.
*/
if (num_quotes == 0 && (num_backslashes == 0 || as_ident))
{
memcpy(rp, str, input_len);
rp += input_len;
}
else
{
for (s = str; s - str < input_len; ++s)
{
if (*s == quote_char || (!as_ident && *s == '\\'))
{
*rp++ = *s;
*rp++ = *s;
}
else if (!IS_HIGHBIT_SET(*s))
*rp++ = *s;
else
{
int i = pg_encoding_mblen(conn->client_encoding, s);
2010-02-26 03:01:40 +01:00
while (1)
{
*rp++ = *s;
if (--i == 0)
break;
++s; /* for loop will provide the final increment */
}
}
}
}
/* Closing quote and terminating NUL. */
*rp++ = quote_char;
*rp = '\0';
return result;
}
char *
PQescapeLiteral(PGconn *conn, const char *str, size_t len)
{
return PQescapeInternal(conn, str, len, false);
}
char *
PQescapeIdentifier(PGconn *conn, const char *str, size_t len)
{
return PQescapeInternal(conn, str, len, true);
}
/* HEX encoding support for bytea */
static const char hextbl[] = "0123456789abcdef";
static const int8 hexlookup[128] = {
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, -1, -1, -1, -1, -1, -1,
-1, 10, 11, 12, 13, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, 10, 11, 12, 13, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
};
static inline char
get_hex(char c)
{
int res = -1;
if (c > 0 && c < 127)
res = hexlookup[(unsigned char) c];
return (char) res;
}
/*
* PQescapeBytea - converts from binary string to the
* minimal encoding necessary to include the string in an SQL
* INSERT statement with a bytea type column as the target.
*
* We can use either hex or escape (traditional) encoding.
* In escape mode, the following transformations are applied:
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
* '\0' == ASCII 0 == \000
* '\'' == ASCII 39 == ''
* '\\' == ASCII 92 == \\
* anything < 0x20, or > 0x7e ---> \ooo
* (where ooo is an octal expression)
*
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
* If not std_strings, all backslashes sent to the output are doubled.
*/
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
static unsigned char *
PQescapeByteaInternal(PGconn *conn,
const unsigned char *from, size_t from_length,
size_t *to_length, bool std_strings, bool use_hex)
{
const unsigned char *vp;
unsigned char *rp;
unsigned char *result;
size_t i;
size_t len;
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
size_t bslash_len = (std_strings ? 1 : 2);
/*
* empty string has 1 char ('\0')
*/
len = 1;
if (use_hex)
{
len += bslash_len + 1 + 2 * from_length;
}
else
{
vp = from;
for (i = from_length; i > 0; i--, vp++)
{
if (*vp < 0x20 || *vp > 0x7e)
len += bslash_len + 3;
else if (*vp == '\'')
len += 2;
else if (*vp == '\\')
len += bslash_len + bslash_len;
else
len++;
}
}
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
*to_length = len;
rp = result = (unsigned char *) malloc(len);
if (rp == NULL)
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
{
if (conn)
libpq_append_conn_error(conn, "out of memory");
return NULL;
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
}
if (use_hex)
{
if (!std_strings)
*rp++ = '\\';
*rp++ = '\\';
*rp++ = 'x';
}
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
vp = from;
for (i = from_length; i > 0; i--, vp++)
{
unsigned char c = *vp;
if (use_hex)
{
*rp++ = hextbl[(c >> 4) & 0xF];
*rp++ = hextbl[c & 0xF];
}
else if (c < 0x20 || c > 0x7e)
{
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
if (!std_strings)
*rp++ = '\\';
*rp++ = '\\';
*rp++ = (c >> 6) + '0';
*rp++ = ((c >> 3) & 07) + '0';
*rp++ = (c & 07) + '0';
}
else if (c == '\'')
{
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
*rp++ = '\'';
*rp++ = '\'';
}
else if (c == '\\')
{
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
if (!std_strings)
{
*rp++ = '\\';
*rp++ = '\\';
}
*rp++ = '\\';
*rp++ = '\\';
}
else
*rp++ = c;
}
*rp = '\0';
return result;
}
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
unsigned char *
PQescapeByteaConn(PGconn *conn,
const unsigned char *from, size_t from_length,
size_t *to_length)
{
if (!conn)
return NULL;
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
if (conn->cmd_queue_head == NULL)
pqClearConnErrorState(conn);
In libpq, always append new error messages to conn->errorMessage. Previously, we had an undisciplined mish-mash of printfPQExpBuffer and appendPQExpBuffer calls to report errors within libpq. This commit establishes a uniform rule that appendPQExpBuffer[Str] should be used. conn->errorMessage is reset only at the start of an application request, and then accumulates messages till we're done. We can remove no less than three different ad-hoc mechanisms that were used to get the effect of concatenation of error messages within a sequence of operations. Although this makes things quite a bit cleaner conceptually, the main reason to do it is to make the world safer for the multiple-target-host feature that was added awhile back. Previously, there were many cases in which an error occurring during an individual host connection attempt would wipe out the record of what had happened during previous attempts. (The reporting is still inadequate, in that it can be hard to tell which host got the failure, but that seems like a matter for a separate commit.) Currently, lo_import and lo_export contain exceptions to the "never use printfPQExpBuffer" rule. If we changed them, we'd risk reporting an incidental lo_close failure before the actual read or write failure, which would be confusing, not least because lo_close happened after the main failure. We could improve this by inventing an internal version of lo_close that doesn't reset the errorMessage; but we'd also need a version of PQfn() that does that, and it didn't quite seem worth the trouble for now. Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
2021-01-11 19:12:09 +01:00
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
return PQescapeByteaInternal(conn, from, from_length, to_length,
conn->std_strings,
(conn->sversion >= 90000));
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
}
unsigned char *
PQescapeBytea(const unsigned char *from, size_t from_length, size_t *to_length)
{
return PQescapeByteaInternal(NULL, from, from_length, to_length,
static_std_strings,
false /* can't use hex */ );
}
#define ISFIRSTOCTDIGIT(CH) ((CH) >= '0' && (CH) <= '3')
#define ISOCTDIGIT(CH) ((CH) >= '0' && (CH) <= '7')
#define OCTVAL(CH) ((CH) - '0')
/*
* PQunescapeBytea - converts the null terminated string representation
* of a bytea, strtext, into binary, filling a buffer. It returns a
* pointer to the buffer (or NULL on error), and the size of the
* buffer in retbuflen. The pointer may subsequently be used as an
Modify libpq's string-escaping routines to be aware of encoding considerations and standard_conforming_strings. The encoding changes are needed for proper escaping in multibyte encodings, as per the SQL-injection vulnerabilities noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied to the server to ensure that it rejects queries that may have been corrupted by attempted SQL injection, but this merely guarantees that unpatched clients will fail rather than allow injection. An actual fix requires changing the client-side code. While at it we have also fixed these routines to understand about standard_conforming_strings, so that the upcoming changeover to SQL-spec string syntax can be somewhat transparent to client code. Since the existing API of PQescapeString and PQescapeBytea provides no way to inform them which settings are in use, these functions are now deprecated in favor of new functions PQescapeStringConn and PQescapeByteaConn. The new functions take the PGconn to which the string will be sent as an additional parameter, and look inside the connection structure to determine what to do. So as to provide some functionality for clients using the old functions, libpq stores the latest encoding and standard_conforming_strings values received from the backend in static variables, and the old functions consult these variables. This will work reliably in clients using only one Postgres connection at a time, or even multiple connections if they all use the same encoding and string syntax settings; which should cover many practical scenarios. Clients that use homebrew escaping methods, such as PHP's addslashes() function or even hardwired regexp substitution, will require extra effort to fix :-(. It is strongly recommended that such code be replaced by use of PQescapeStringConn/PQescapeByteaConn if at all feasible.
2006-05-21 22:19:23 +02:00
* argument to the function PQfreemem.
*
* The following transformations are made:
* \\ == ASCII 92 == \
* \ooo == a byte whose value = ooo (ooo is an octal number)
* \x == x (x is any character not matched by the above transformations)
*/
unsigned char *
PQunescapeBytea(const unsigned char *strtext, size_t *retbuflen)
{
size_t strtextlen,
buflen;
unsigned char *buffer,
*tmpbuf;
size_t i,
j;
if (strtext == NULL)
return NULL;
strtextlen = strlen((const char *) strtext);
2004-08-29 07:07:03 +02:00
if (strtext[0] == '\\' && strtext[1] == 'x')
{
const unsigned char *s;
unsigned char *p;
buflen = (strtextlen - 2) / 2;
/* Avoid unportable malloc(0) */
buffer = (unsigned char *) malloc(buflen > 0 ? buflen : 1);
if (buffer == NULL)
return NULL;
s = strtext + 2;
p = buffer;
while (*s)
{
char v1,
v2;
/*
* Bad input is silently ignored. Note that this includes
* whitespace between hex pairs, which is allowed by byteain.
*/
v1 = get_hex(*s++);
if (!*s || v1 == (char) -1)
continue;
v2 = get_hex(*s++);
if (v2 != (char) -1)
*p++ = (v1 << 4) | v2;
}
buflen = p - buffer;
}
else
{
/*
* Length of input is max length of output, but add one to avoid
* unportable malloc(0) if input is zero-length.
*/
buffer = (unsigned char *) malloc(strtextlen + 1);
if (buffer == NULL)
return NULL;
for (i = j = 0; i < strtextlen;)
{
switch (strtext[i])
2010-02-26 03:01:40 +01:00
{
case '\\':
i++;
if (strtext[i] == '\\')
buffer[j++] = strtext[i++];
else
{
if ((ISFIRSTOCTDIGIT(strtext[i])) &&
(ISOCTDIGIT(strtext[i + 1])) &&
(ISOCTDIGIT(strtext[i + 2])))
2010-02-26 03:01:40 +01:00
{
int byte;
2010-02-26 03:01:40 +01:00
byte = OCTVAL(strtext[i++]);
byte = (byte << 3) + OCTVAL(strtext[i++]);
byte = (byte << 3) + OCTVAL(strtext[i++]);
buffer[j++] = byte;
2010-02-26 03:01:40 +01:00
}
}
2004-08-29 07:07:03 +02:00
/*
* Note: if we see '\' followed by something that isn't a
* recognized escape sequence, we loop around having done
* nothing except advance i. Therefore the something will
* be emitted as ordinary data on the next cycle. Corner
* case: '\' at end of string will just be discarded.
*/
break;
default:
buffer[j++] = strtext[i++];
break;
2010-02-26 03:01:40 +01:00
}
}
buflen = j; /* buflen is the length of the dequoted data */
}
2003-10-02 16:47:44 +02:00
/* Shrink the buffer to be no larger than necessary */
/* +1 avoids unportable behavior when buflen==0 */
tmpbuf = realloc(buffer, buflen + 1);
2003-10-02 16:47:44 +02:00
/* It would only be a very brain-dead realloc that could fail, but... */
if (!tmpbuf)
{
free(buffer);
2003-10-02 16:47:44 +02:00
return NULL;
}
*retbuflen = buflen;
return tmpbuf;
}