Another round of protocol changes. Backend-to-frontend messages now all

have length words.  COPY OUT reimplemented per new protocol: it doesn't
need \. anymore, thank goodness.  COPY BINARY to/from frontend works,
at least as far as the backend is concerned --- libpq's PQgetline API
is not up to snuff, and will have to be replaced with something that is
null-safe.  libpq uses message length words for performance improvement
(no cycles wasted rescanning long messages), but not yet for error
recovery.
This commit is contained in:
Tom Lane 2003-04-22 00:08:07 +00:00
parent ca944bd2d4
commit 5ed27e35f3
22 changed files with 782 additions and 357 deletions

View File

@ -1,5 +1,5 @@
<!--
$Header: /cvsroot/pgsql/doc/src/sgml/libpq.sgml,v 1.119 2003/04/19 00:02:29 tgl Exp $
$Header: /cvsroot/pgsql/doc/src/sgml/libpq.sgml,v 1.120 2003/04/22 00:08:06 tgl Exp $
-->
<chapter id="libpq">
@ -175,7 +175,8 @@ PGconn *PQconnectdb(const char *conninfo);
<term><literal>connect_timeout</literal></term>
<listitem>
<para>
Time space in seconds given to connection function. Zero or not set means infinite.
Maximum wait for connection, in seconds (write as a decimal integer
string). Zero or not specified means infinite.
</para>
</listitem>
</varlistentry>
@ -184,7 +185,7 @@ PGconn *PQconnectdb(const char *conninfo);
<term><literal>options</literal></term>
<listitem>
<para>
Configuration options to be sent to the server.
Command-line options to be sent to the server.
</para>
</listitem>
</varlistentry>
@ -252,8 +253,9 @@ PGconn *PQsetdbLogin(const char *pghost,
</para>
<para>
This is the predecessor of <function>PQconnectdb</function> with a fixed number
of parameters but the same functionality.
This is the predecessor of <function>PQconnectdb</function> with a fixed
number of parameters. It has the same functionality except that the
missing parameters cannot be specified in the call.
</para>
</listitem>
</varlistentry>
@ -274,8 +276,8 @@ PGconn *PQsetdb(char *pghost,
<para>
This is a macro that calls <function>PQsetdbLogin</function> with null pointers
for the <parameter>login</> and <parameter>pwd</> parameters. It is provided primarily
for backward compatibility with old programs.
for the <parameter>login</> and <parameter>pwd</> parameters. It is provided
for backward compatibility with very old programs.
</para>
</listitem>
</varlistentry>
@ -454,7 +456,7 @@ switch(PQstatus(conn))
</para>
<para>
Finally, these functions leave the socket in a nonblocking state as if
Finally, these functions leave the connection in a nonblocking state as if
<function>PQsetnonblocking</function> had been called.
</para>
</listitem>
@ -486,8 +488,6 @@ typedef struct
</para>
<para>
converts an escaped string representation of binary data into binary
data --- the reverse of <function>PQescapeBytea</function>.
Returns a connection options array. This may
be used to determine all possible <function>PQconnectdb</function> options and their
current default values. The return value points to an array of
@ -683,7 +683,7 @@ char *PQtty(const PGconn *conn);
<term><function>PQoptions</function></term>
<listitem>
<para>
Returns the configuration options passed in the connection request.
Returns the command-line options passed in the connection request.
<synopsis>
char *PQoptions(const PGconn *conn);
</synopsis>
@ -2047,13 +2047,13 @@ contains example functions that correctly handle the <command>COPY</command> pro
<term><function>PQgetlineAsync</function></term>
<listitem>
<para>
Reads a newline-terminated line of characters
Reads a row of COPY data
(transmitted by the server) into a buffer
without blocking.
<synopsis>
int PQgetlineAsync(PGconn *conn,
char *buffer,
int length);
int bufsize);
</synopsis>
</para>
@ -2070,24 +2070,27 @@ end-of-data signal is detected.
<para>
Unlike <function>PQgetline</function>, this function takes
responsibility for detecting end-of-data.
On each call, <function>PQgetlineAsync</function> will return data if a complete newline-
terminated data line is available in <application>libpq</>'s input buffer, or if the
incoming data line is too long to fit in the buffer offered by the caller.
Otherwise, no data is returned until the rest of the line arrives.
</para>
<para>
On each call, <function>PQgetlineAsync</function> will return data if a
complete data row is available in <application>libpq</>'s input buffer.
Otherwise, no data is returned until the rest of the row arrives.
The function returns -1 if the end-of-copy-data marker has been recognized,
or 0 if no data is available, or a positive number giving the number of
bytes of data returned. If -1 is returned, the caller must next call
<function>PQendcopy</function>, and then return to normal processing.
</para>
<para>
The data returned will not extend beyond a newline character. If possible
a whole line will be returned at one time. But if the buffer offered by
the caller is too small to hold a line sent by the server, then a partial
data line will be returned. This can be detected by testing whether the
last returned byte is <literal>\n</literal> or not.
The data returned will not extend beyond a data-row boundary. If possible
a whole row will be returned at one time. But if the buffer offered by
the caller is too small to hold a row sent by the server, then a partial
data row will be returned. With textual data this can be detected by testing
whether the last returned byte is <literal>\n</literal> or not. (In a binary
COPY, actual parsing of the COPY data format will be needed to make the
equivalent determination.)
The returned string is not null-terminated. (If you want to add a
terminating null, be sure to pass a <parameter>length</parameter> one smaller than the room
actually available.)
terminating null, be sure to pass a <parameter>bufsize</parameter> one smaller
than the room actually available.)
</para>
</listitem>
</varlistentry>
@ -2105,10 +2108,24 @@ int PQputline(PGconn *conn,
</para>
<para>
Note the application must explicitly send the two
characters <literal>\.</literal> on a final line to indicate to
the server that it has finished sending its data.
The COPY datastream sent by a series of calls to
<function>PQputline</function> has the same format as that returned by
<function>PQgetlineAsync</function>, except that applications are not
obliged to send exactly one data row per <function>PQputline</function>
call; it is okay to send a partial line or multiple lines per call.
</para>
<note>
<para>
Before <productname>PostgreSQL</productname> 7.4, it was necessary for the
application to explicitly send the two characters <literal>\.</literal> as a
final line to indicate to the server that it had finished sending COPY data.
While this still works, it is deprecated and the special meaning of
<literal>\.</literal> can be expected to be removed in a future release.
It is sufficient to call <function>PQendcopy</function> after having sent the
actual data.
</para>
</note>
</listitem>
</varlistentry>
@ -2126,9 +2143,9 @@ int PQputnbytes(PGconn *conn,
</para>
<para>
This is exactly like <function>PQputline</function>, except that the data buffer need
not be null-terminated since the number of bytes to send is
specified directly.
This is exactly like <function>PQputline</function>, except that the data
buffer need not be null-terminated since the number of bytes to send is
specified directly. Use this procedure when sending binary data.
</para>
</listitem>
</varlistentry>
@ -2147,11 +2164,12 @@ int PQendcopy(PGconn *conn);
sent to the server using <function>PQputline</function> or when the
last string has been received from the server
using <function>PGgetline</function>. It must be issued or the server
may get <quote>out of sync</quote> with the client. Upon
will get <quote>out of sync</quote> with the client. Upon
return from this function, the server is ready to
receive the next SQL command.
The return value is 0 on successful completion,
nonzero otherwise.
nonzero otherwise. (Use <function>PQerrorMessage</function> to retrieve
details if the return value is nonzero.)
</para>
<para>
@ -2187,7 +2205,6 @@ PQexec(conn, "COPY foo FROM STDIN;");
PQputline(conn, "3\thello world\t4.5\n");
PQputline(conn, "4\tgoodbye world\t7.11\n");
...
PQputline(conn, "\\.\n");
PQendcopy(conn);
</programlisting>
</para>

View File

@ -1,4 +1,4 @@
<!-- $Header: /cvsroot/pgsql/doc/src/sgml/protocol.sgml,v 1.28 2003/04/19 00:02:29 tgl Exp $ -->
<!-- $Header: /cvsroot/pgsql/doc/src/sgml/protocol.sgml,v 1.29 2003/04/22 00:08:06 tgl Exp $ -->
<chapter id="protocol">
<title>Frontend/Backend Protocol</title>
@ -3691,7 +3691,8 @@ Terminate (F)
<para>
This section describes the fields that may appear in ErrorResponse and
NoticeResponse messages. Each field type has a single-byte identification
token.
token. Note that any given field type should appear at most once per
message.
</para>
<VariableList>
@ -3863,7 +3864,29 @@ PasswordMessage now has a type byte.
<para>
COPY data is now encapsulated into CopyData and CopyDone messages. There
is a well-defined way to recover from errors during COPY.
is a well-defined way to recover from errors during COPY. The special
<quote><literal>\.</></quote> last line is not needed anymore, and is not sent
during COPY OUT.
(It is still recognized as a terminator during COPY IN, but its use is
deprecated and will eventually be removed.) Binary COPY is supported.
The CopyInResponse and CopyOutResponse messages carry a field indicating
whether the COPY operation is text or binary.
</para>
<para>
The CursorResponse ('<literal>P</>') message is no longer generated by
the backend.
</para>
<para>
The NotificationResponse ('<literal>A</>') message has an additional string
field, which is presently empty but may someday carry additional data passed
from the NOTIFY event sender.
</para>
<para>
The EmptyQueryResponse ('<literal>I</>') message used to include an empty
string parameter; this has been removed.
</para>
<note>

View File

@ -9,7 +9,7 @@
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/access/common/printtup.c,v 1.65 2002/09/04 20:31:08 momjian Exp $
* $Header: /cvsroot/pgsql/src/backend/access/common/printtup.c,v 1.66 2003/04/22 00:08:06 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -77,15 +77,18 @@ static void
printtup_setup(DestReceiver *self, int operation,
const char *portalName, TupleDesc typeinfo)
{
/*
* Send portal name to frontend.
*
* If portal name not specified, use "blank" portal.
*/
if (portalName == NULL)
portalName = "blank";
if (PG_PROTOCOL_MAJOR(FrontendProtocol) < 3)
{
/*
* Send portal name to frontend (obsolete cruft, gone in proto 3.0)
*
* If portal name not specified, use "blank" portal.
*/
if (portalName == NULL)
portalName = "blank";
pq_puttextmessage('P', portalName);
pq_puttextmessage('P', portalName);
}
/*
* if this is a retrieve, then we send back the tuple descriptor of
@ -98,8 +101,7 @@ printtup_setup(DestReceiver *self, int operation,
int i;
StringInfoData buf;
pq_beginmessage(&buf);
pq_sendbyte(&buf, 'T'); /* tuple descriptor message type */
pq_beginmessage(&buf, 'T'); /* tuple descriptor message type */
pq_sendint(&buf, natts, 2); /* # of attrs in tuples */
for (i = 0; i < natts; ++i)
@ -174,8 +176,7 @@ printtup(HeapTuple tuple, TupleDesc typeinfo, DestReceiver *self)
/*
* tell the frontend to expect new tuple data (in ASCII style)
*/
pq_beginmessage(&buf);
pq_sendbyte(&buf, 'D');
pq_beginmessage(&buf, 'D');
/*
* send a bitmap of which attributes are not null
@ -388,8 +389,7 @@ printtup_internal(HeapTuple tuple, TupleDesc typeinfo, DestReceiver *self)
/*
* tell the frontend to expect new tuple data (in binary style)
*/
pq_beginmessage(&buf);
pq_sendbyte(&buf, 'B');
pq_beginmessage(&buf, 'B');
/*
* send a bitmap of which attributes are not null

View File

@ -7,7 +7,7 @@
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/commands/async.c,v 1.92 2003/02/18 02:53:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/commands/async.c,v 1.93 2003/04/22 00:08:06 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -847,10 +847,14 @@ NotifyMyFrontEnd(char *relname, int32 listenerPID)
{
StringInfoData buf;
pq_beginmessage(&buf);
pq_sendbyte(&buf, 'A');
pq_beginmessage(&buf, 'A');
pq_sendint(&buf, listenerPID, sizeof(int32));
pq_sendstring(&buf, relname);
if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 3)
{
/* XXX Add parameter string here later */
pq_sendstring(&buf, "");
}
pq_endmessage(&buf);
/*

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/commands/copy.c,v 1.194 2003/04/19 20:36:03 momjian Exp $
* $Header: /cvsroot/pgsql/src/backend/commands/copy.c,v 1.195 2003/04/22 00:08:06 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -50,13 +50,6 @@
#define ISOCTAL(c) (((c) >= '0') && ((c) <= '7'))
#define OCTVALUE(c) ((c) - '0')
/* Default line termination */
#ifndef WIN32
#define PGEOL "\n"
#else
#define PGEOL "\r\n"
#endif
/*
* Represents the different source/dest cases we need to worry about at
* the bottom level
@ -92,7 +85,7 @@ typedef enum EolType
/* non-export function prototypes */
static void CopyTo(Relation rel, List *attnumlist, bool binary, bool oids,
bool pipe, char *delim, char *null_print);
char *delim, char *null_print);
static void CopyFrom(Relation rel, List *attnumlist, bool binary, bool oids,
char *delim, char *null_print);
static Oid GetInputFunction(Oid type);
@ -101,8 +94,7 @@ static char *CopyReadAttribute(const char *delim, CopyReadResult *result);
static void CopyAttributeOut(char *string, char *delim);
static List *CopyGetAttnums(Relation rel, List *attnamelist);
/* The trailing null is part of the signature */
static const char BinarySignature[] = "PGBCOPY\n\377\r\n";
static const char BinarySignature[12] = "PGBCOPY\n\377\r\n\0";
/*
* Static communication variables ... pretty grotty, but COPY has
@ -135,10 +127,11 @@ static int server_encoding;
*/
static void SendCopyBegin(bool binary);
static void ReceiveCopyBegin(bool binary);
static void SendCopyEnd(bool binary, bool pipe);
static void SendCopyEnd(bool binary);
static void CopySendData(void *databuf, int datasize);
static void CopySendString(const char *str);
static void CopySendChar(char c);
static void CopySendEndOfRow(bool binary);
static void CopyGetData(void *databuf, int datasize);
static int CopyGetChar(void);
#define CopyGetEof() (fe_eof)
@ -154,22 +147,32 @@ SendCopyBegin(bool binary)
{
if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 3)
{
pq_putbytes("H", 1); /* new way */
/* XXX grottiness needed for old protocol */
pq_startcopyout();
/* new way */
StringInfoData buf;
pq_beginmessage(&buf, 'H');
pq_sendbyte(&buf, binary ? 1 : 0);
pq_endmessage(&buf);
copy_dest = COPY_NEW_FE;
copy_msgbuf = makeStringInfo();
}
else if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 2)
{
pq_putbytes("H", 1); /* old way */
/* grottiness needed for old protocol */
/* old way */
if (binary)
elog(ERROR, "COPY BINARY is not supported to stdout or from stdin");
pq_putemptymessage('H');
/* grottiness needed for old COPY OUT protocol */
pq_startcopyout();
copy_dest = COPY_OLD_FE;
}
else
{
pq_putbytes("B", 1); /* very old way */
/* grottiness needed for old protocol */
/* very old way */
if (binary)
elog(ERROR, "COPY BINARY is not supported to stdout or from stdin");
pq_putemptymessage('B');
/* grottiness needed for old COPY OUT protocol */
pq_startcopyout();
copy_dest = COPY_OLD_FE;
}
@ -180,18 +183,29 @@ ReceiveCopyBegin(bool binary)
{
if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 3)
{
pq_putbytes("G", 1); /* new way */
/* new way */
StringInfoData buf;
pq_beginmessage(&buf, 'G');
pq_sendbyte(&buf, binary ? 1 : 0);
pq_endmessage(&buf);
copy_dest = COPY_NEW_FE;
copy_msgbuf = makeStringInfo();
}
else if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 2)
{
pq_putbytes("G", 1); /* old way */
/* old way */
if (binary)
elog(ERROR, "COPY BINARY is not supported to stdout or from stdin");
pq_putemptymessage('G');
copy_dest = COPY_OLD_FE;
}
else
{
pq_putbytes("D", 1); /* very old way */
/* very old way */
if (binary)
elog(ERROR, "COPY BINARY is not supported to stdout or from stdin");
pq_putemptymessage('D');
copy_dest = COPY_OLD_FE;
}
/* We *must* flush here to ensure FE knows it can send. */
@ -199,22 +213,39 @@ ReceiveCopyBegin(bool binary)
}
static void
SendCopyEnd(bool binary, bool pipe)
SendCopyEnd(bool binary)
{
if (!binary)
if (copy_dest == COPY_NEW_FE)
{
CopySendString("\\.");
CopySendString(!pipe ? PGEOL : "\n");
if (binary)
{
/* Need to flush out file trailer word */
CopySendEndOfRow(true);
}
else
{
/* Shouldn't have any unsent data */
Assert(copy_msgbuf->len == 0);
}
/* Send Copy Done message */
pq_putemptymessage('c');
}
else
{
/* The FE/BE protocol uses \n as newline for all platforms */
CopySendData("\\.\n", 3);
pq_endcopyout(false);
}
pq_endcopyout(false);
}
/*
/*----------
* CopySendData sends output data to the destination (file or frontend)
* CopySendString does the same for null-terminated strings
* CopySendChar does the same for single characters
* CopySendEndOfRow does the appropriate thing at end of each data row
*
* NB: no data conversion is applied by these functions
*----------
*/
static void
CopySendData(void *databuf, int datasize)
@ -228,12 +259,13 @@ CopySendData(void *databuf, int datasize)
break;
case COPY_OLD_FE:
if (pq_putbytes((char *) databuf, datasize))
fe_eof = true;
{
/* no hope of recovering connection sync, so FATAL */
elog(FATAL, "CopySendData: connection lost");
}
break;
case COPY_NEW_FE:
/* XXX fix later */
if (pq_putbytes((char *) databuf, datasize))
fe_eof = true;
appendBinaryStringInfo(copy_msgbuf, (char *) databuf, datasize);
break;
}
}
@ -250,6 +282,40 @@ CopySendChar(char c)
CopySendData(&c, 1);
}
static void
CopySendEndOfRow(bool binary)
{
switch (copy_dest)
{
case COPY_FILE:
if (!binary)
{
/* Default line termination depends on platform */
#ifndef WIN32
CopySendChar('\n');
#else
CopySendString("\r\n");
#endif
}
break;
case COPY_OLD_FE:
/* The FE/BE protocol uses \n as newline for all platforms */
if (!binary)
CopySendChar('\n');
break;
case COPY_NEW_FE:
/* The FE/BE protocol uses \n as newline for all platforms */
if (!binary)
CopySendChar('\n');
/* Dump the accumulated row as one CopyData message */
(void) pq_putmessage('d', copy_msgbuf->data, copy_msgbuf->len);
/* Reset copy_msgbuf to empty */
copy_msgbuf->len = 0;
copy_msgbuf->data[0] = '\0';
break;
}
}
/*
* CopyGetData reads data from the source (file or frontend)
* CopyGetChar does the same for single characters
@ -568,13 +634,6 @@ DoCopy(const CopyStmt *stmt)
"directly to or from a file. Anyone can COPY to stdout or "
"from stdin. Psql's \\copy command also works for anyone.");
/*
* This restriction is unfortunate, but necessary until the frontend
* COPY protocol is redesigned to be binary-safe...
*/
if (pipe && binary)
elog(ERROR, "COPY BINARY is not supported to stdout or from stdin");
/*
* Presently, only single-character delimiter strings are supported.
*/
@ -698,13 +757,13 @@ DoCopy(const CopyStmt *stmt)
elog(ERROR, "COPY: %s is a directory", filename);
}
}
CopyTo(rel, attnumlist, binary, oids, pipe, delim, null_print);
CopyTo(rel, attnumlist, binary, oids, delim, null_print);
}
if (!pipe)
FreeFile(copy_file);
else if (IsUnderPostmaster && !is_from)
SendCopyEnd(binary, pipe);
SendCopyEnd(binary);
pfree(attribute_buf.data);
/*
@ -721,7 +780,7 @@ DoCopy(const CopyStmt *stmt)
* Copy from relation TO file.
*/
static void
CopyTo(Relation rel, List *attnumlist, bool binary, bool oids, bool pipe,
CopyTo(Relation rel, List *attnumlist, bool binary, bool oids,
char *delim, char *null_print)
{
HeapTuple tuple;
@ -786,7 +845,7 @@ CopyTo(Relation rel, List *attnumlist, bool binary, bool oids, bool pipe,
int32 tmp;
/* Signature */
CopySendData((char *) BinarySignature, sizeof(BinarySignature));
CopySendData((char *) BinarySignature, 12);
/* Integer layout field */
tmp = 0x01020304;
CopySendData(&tmp, sizeof(int32));
@ -918,8 +977,7 @@ CopyTo(Relation rel, List *attnumlist, bool binary, bool oids, bool pipe,
}
}
if (!binary)
CopySendString(!pipe ? PGEOL : "\n");
CopySendEndOfRow(binary);
MemoryContextSwitchTo(oldcontext);
}
@ -1100,8 +1158,7 @@ CopyFrom(Relation rel, List *attnumlist, bool binary, bool oids,
/* Signature */
CopyGetData(readSig, 12);
if (CopyGetEof() || memcmp(readSig, BinarySignature,
sizeof(BinarySignature)) != 0)
if (CopyGetEof() || memcmp(readSig, BinarySignature, 12) != 0)
elog(ERROR, "COPY BINARY: file signature not recognized");
/* Integer layout field */
CopyGetData(&tmp, sizeof(int32));

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/libpq/auth.c,v 1.99 2003/04/19 00:02:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/libpq/auth.c,v 1.100 2003/04/22 00:08:06 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -512,8 +512,7 @@ sendAuthRequest(Port *port, AuthRequest areq)
{
StringInfoData buf;
pq_beginmessage(&buf);
pq_sendbyte(&buf, 'R');
pq_beginmessage(&buf, 'R');
pq_sendint(&buf, (int32) areq, sizeof(int32));
/* Add the salt for encrypted passwords. */

View File

@ -12,15 +12,16 @@
* No other messages can be sent while COPY OUT is in progress; and if the
* copy is aborted by an elog(ERROR), we need to close out the copy so that
* the frontend gets back into sync. Therefore, these routines have to be
* aware of COPY OUT state.
* aware of COPY OUT state. (New COPY-OUT is message-based and does *not*
* set the DoingCopyOut flag.)
*
* NOTE: generally, it's a bad idea to emit outgoing messages directly with
* pq_putbytes(), especially if the message would require multiple calls
* to send. Instead, use the routines in pqformat.c to construct the message
* in a buffer and then emit it in one call to pq_putmessage. This helps
* ensure that the channel will not be clogged by an incomplete message
* if execution is aborted by elog(ERROR) partway through the message.
* The only non-libpq code that should call pq_putbytes directly is COPY OUT.
* in a buffer and then emit it in one call to pq_putmessage. This ensures
* that the channel will not be clogged by an incomplete message if execution
* is aborted by elog(ERROR) partway through the message. The only non-libpq
* code that should call pq_putbytes directly is old-style COPY OUT.
*
* At one time, libpq was shared between frontend and backend, but now
* the backend's "backend/libpq" is quite separate from "interfaces/libpq".
@ -29,7 +30,7 @@
* Portions Copyright (c) 1996-2002, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $Header: /cvsroot/pgsql/src/backend/libpq/pqcomm.c,v 1.150 2003/04/19 00:02:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/libpq/pqcomm.c,v 1.151 2003/04/22 00:08:06 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -846,13 +847,17 @@ pq_flush(void)
* pq_putmessage - send a normal message (suppressed in COPY OUT mode)
*
* If msgtype is not '\0', it is a message type code to place before
* the message body (len counts only the body size!).
* If msgtype is '\0', then the buffer already includes the type code.
* the message body. If msgtype is '\0', then the message has no type
* code (this is only valid in pre-3.0 protocols).
*
* All normal messages are suppressed while COPY OUT is in progress.
* (In practice only a few messages might get emitted then; dropping
* them is annoying, but at least they will still appear in the
* postmaster log.)
* len is the length of the message body data at *s. In protocol 3.0
* and later, a message length word (equal to len+4 because it counts
* itself too) is inserted by this routine.
*
* All normal messages are suppressed while old-style COPY OUT is in
* progress. (In practice only a few notice messages might get emitted
* then; dropping them is annoying, but at least they will still appear
* in the postmaster log.)
*
* returns 0 if OK, EOF if trouble
* --------------------------------
@ -865,6 +870,14 @@ pq_putmessage(char msgtype, const char *s, size_t len)
if (msgtype)
if (pq_putbytes(&msgtype, 1))
return EOF;
if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 3)
{
uint32 n32;
n32 = htonl((uint32) (len + 4));
if (pq_putbytes((char *) &n32, 4))
return EOF;
}
return pq_putbytes(s, len);
}
@ -880,12 +893,13 @@ pq_startcopyout(void)
}
/* --------------------------------
* pq_endcopyout - end a COPY OUT transfer
* pq_endcopyout - end an old-style COPY OUT transfer
*
* If errorAbort is indicated, we are aborting a COPY OUT due to an error,
* and must send a terminator line. Since a partial data line might have
* been emitted, send a couple of newlines first (the first one could
* get absorbed by a backslash...)
* get absorbed by a backslash...) Note that old-style COPY OUT does
* not allow binary transfers, so a textual terminator is always correct.
* --------------------------------
*/
void
@ -893,8 +907,8 @@ pq_endcopyout(bool errorAbort)
{
if (!DoingCopyOut)
return;
DoingCopyOut = false;
if (errorAbort)
pq_putbytes("\n\n\\.\n", 5);
/* in non-error case, copy.c will have emitted the terminator line */
DoingCopyOut = false;
}

View File

@ -18,7 +18,7 @@
* Portions Copyright (c) 1996-2002, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $Header: /cvsroot/pgsql/src/backend/libpq/pqformat.c,v 1.27 2003/04/19 00:02:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/libpq/pqformat.c,v 1.28 2003/04/22 00:08:06 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -38,6 +38,7 @@
*
* Special-case message output:
* pq_puttextmessage - generate a character set-converted message in one step
* pq_putemptymessage - convenience routine for message with empty body
*
* Message parsing after input:
* pq_getmsgbyte - get a raw byte from a message buffer
@ -63,6 +64,22 @@
#include "mb/pg_wchar.h"
/* --------------------------------
* pq_beginmessage - initialize for sending a message
* --------------------------------
*/
void
pq_beginmessage(StringInfo buf, char msgtype)
{
initStringInfo(buf);
/*
* We stash the message type into the buffer's cursor field, expecting
* that the pq_sendXXX routines won't touch it. We could alternatively
* make it the first byte of the buffer contents, but this seems easier.
*/
buf->cursor = msgtype;
}
/* --------------------------------
* pq_sendbyte - append a raw byte to a StringInfo buffer
* --------------------------------
@ -176,7 +193,8 @@ pq_sendint(StringInfo buf, int i, int b)
void
pq_endmessage(StringInfo buf)
{
(void) pq_putmessage('\0', buf->data, buf->len);
/* msgtype was saved in cursor field */
(void) pq_putmessage(buf->cursor, buf->data, buf->len);
/* no need to complain about any failure, since pqcomm.c already did */
pfree(buf->data);
buf->data = NULL;
@ -188,11 +206,9 @@ pq_endmessage(StringInfo buf)
* This is the same as the pqcomm.c routine pq_putmessage, except that
* the message body is a null-terminated string to which encoding
* conversion applies.
*
* returns 0 if OK, EOF if trouble
* --------------------------------
*/
int
void
pq_puttextmessage(char msgtype, const char *str)
{
int slen = strlen(str);
@ -201,12 +217,22 @@ pq_puttextmessage(char msgtype, const char *str)
p = (char *) pg_server_to_client((unsigned char *) str, slen);
if (p != str) /* actual conversion has been done? */
{
int result = pq_putmessage(msgtype, p, strlen(p) + 1);
(void) pq_putmessage(msgtype, p, strlen(p) + 1);
pfree(p);
return result;
return;
}
return pq_putmessage(msgtype, str, slen + 1);
(void) pq_putmessage(msgtype, str, slen + 1);
}
/* --------------------------------
* pq_putemptymessage - convenience routine for message with empty body
* --------------------------------
*/
void
pq_putemptymessage(char msgtype)
{
(void) pq_putmessage(msgtype, NULL, 0);
}

View File

@ -37,7 +37,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/postmaster/postmaster.c,v 1.313 2003/04/19 00:02:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/postmaster/postmaster.c,v 1.314 2003/04/22 00:08:06 tgl Exp $
*
* NOTES
*
@ -1118,7 +1118,13 @@ ProcessStartupPacket(Port *port, bool SSLdone)
if (pq_getbytes((char *) &len, 4) == EOF)
{
elog(COMMERROR, "incomplete startup packet");
/*
* EOF after SSLdone probably means the client didn't like our
* response to NEGOTIATE_SSL_CODE. That's not an error condition,
* so don't clutter the log with a complaint.
*/
if (!SSLdone)
elog(COMMERROR, "incomplete startup packet");
return STATUS_ERROR;
}
@ -1127,7 +1133,10 @@ ProcessStartupPacket(Port *port, bool SSLdone)
if (len < (int32) sizeof(ProtocolVersion) ||
len > MAX_STARTUP_PACKET_LENGTH)
elog(FATAL, "invalid length of startup packet");
{
elog(COMMERROR, "invalid length of startup packet");
return STATUS_ERROR;
}
/*
* Allocate at least the size of an old-style startup packet, plus one
@ -1173,7 +1182,7 @@ ProcessStartupPacket(Port *port, bool SSLdone)
#endif
if (send(port->sock, &SSLok, 1, 0) != 1)
{
elog(LOG, "failed to send SSL negotiation response: %m");
elog(COMMERROR, "failed to send SSL negotiation response: %m");
return STATUS_ERROR; /* close the connection */
}
@ -1188,6 +1197,11 @@ ProcessStartupPacket(Port *port, bool SSLdone)
/* Could add additional special packet types here */
/*
* Set FrontendProtocol now so that elog() knows what format to send
* if we fail during startup.
*/
FrontendProtocol = proto;
/*
* XXX temporary for 3.0 protocol development: we are using the minor

View File

@ -8,7 +8,7 @@
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/tcop/dest.c,v 1.52 2003/04/19 00:02:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/tcop/dest.c,v 1.53 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -141,7 +141,9 @@ EndCommand(const char *commandTag, CommandDest dest)
* libpq's crufty way of determining whether a multiple-command
* query string is done. In protocol 2.0 it's probably not really
* necessary to distinguish empty queries anymore, but we still do it
* for backwards compatibility with 1.0.
* for backwards compatibility with 1.0. In protocol 3.0 it has some
* use again, since it ensures that there will be a recognizable end
* to the response to an Execute message.
* ----------------
*/
void
@ -153,9 +155,13 @@ NullCommand(CommandDest dest)
case Remote:
/*
* tell the fe that we saw an empty query string
* tell the fe that we saw an empty query string. In protocols
* before 3.0 this has a useless empty-string message body.
*/
pq_putbytes("I", 2); /* note we send I and \0 */
if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 3)
pq_putemptymessage('I');
else
pq_puttextmessage('I', "");
break;
case Debug:
@ -184,7 +190,7 @@ ReadyForQuery(CommandDest dest)
case RemoteInternal:
case Remote:
if (PG_PROTOCOL_MAJOR(FrontendProtocol) >= 2)
pq_putbytes("Z", 1);
pq_putemptymessage('Z');
/* Flush output at end of cycle in any case. */
pq_flush();
break;

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/tcop/fastpath.c,v 1.58 2003/04/19 00:02:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/tcop/fastpath.c,v 1.59 2003/04/22 00:08:07 tgl Exp $
*
* NOTES
* This cruft is the server side of PQfn.
@ -119,8 +119,7 @@ SendFunctionResult(Datum retval, bool retbyval, int retlen)
{
StringInfoData buf;
pq_beginmessage(&buf);
pq_sendbyte(&buf, 'V');
pq_beginmessage(&buf, 'V');
if (retlen != 0)
{

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/tcop/postgres.c,v 1.322 2003/04/19 00:02:29 tgl Exp $
* $Header: /cvsroot/pgsql/src/backend/tcop/postgres.c,v 1.323 2003/04/22 00:08:07 tgl Exp $
*
* NOTES
* this is the "main" module of the postgres backend and
@ -1821,8 +1821,7 @@ PostgresMain(int argc, char *argv[], const char *username)
{
StringInfoData buf;
pq_beginmessage(&buf);
pq_sendbyte(&buf, 'K');
pq_beginmessage(&buf, 'K');
pq_sendint(&buf, (int32) MyProcPid, sizeof(int32));
pq_sendint(&buf, (int32) MyCancelKey, sizeof(int32));
pq_endmessage(&buf);
@ -1832,7 +1831,7 @@ PostgresMain(int argc, char *argv[], const char *username)
if (!IsUnderPostmaster)
{
puts("\nPOSTGRES backend interactive interface ");
puts("$Revision: 1.322 $ $Date: 2003/04/19 00:02:29 $\n");
puts("$Revision: 1.323 $ $Date: 2003/04/22 00:08:07 $\n");
}
/*

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/utils/error/elog.c,v 1.107 2003/03/20 03:34:56 momjian Exp $
* $Header: /cvsroot/pgsql/src/backend/utils/error/elog.c,v 1.108 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -406,20 +406,19 @@ elog(int lev, const char *fmt,...)
*/
oldcxt = MemoryContextSwitchTo(ErrorContext);
if (lev <= WARNING)
/* exclude the timestamp from msg sent to frontend */
send_message_to_frontend(lev, msg_buf + timestamp_size);
else
if (lev >= ERROR)
{
/*
* Abort any COPY OUT in progress when an error is detected.
* This hack is necessary because of poor design of copy
* protocol.
* This hack is necessary because of poor design of old-style
* copy protocol.
*/
pq_endcopyout(true);
send_message_to_frontend(ERROR, msg_buf + timestamp_size);
}
/* Exclude the timestamp from msg sent to frontend */
send_message_to_frontend(lev, msg_buf + timestamp_size);
MemoryContextSwitchTo(oldcxt);
}
@ -745,11 +744,9 @@ send_message_to_frontend(int type, const char *msg)
{
StringInfoData buf;
AssertArg(type <= ERROR);
pq_beginmessage(&buf);
/* 'N' (Notice) is for nonfatal conditions, 'E' is for errors */
pq_sendbyte(&buf, type < ERROR ? 'N' : 'E');
pq_beginmessage(&buf, (type < ERROR) ? 'N' : 'E');
/* XXX more to do here */
pq_sendstring(&buf, msg);
pq_endmessage(&buf);

View File

@ -9,7 +9,7 @@
* Portions Copyright (c) 1996-2002, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $Id: pqcomm.h,v 1.77 2003/04/19 00:02:29 tgl Exp $
* $Id: pqcomm.h,v 1.78 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -106,7 +106,7 @@ typedef union SockAddr
/* The earliest and latest frontend/backend protocol version supported. */
#define PG_PROTOCOL_EARLIEST PG_PROTOCOL(1,0)
#define PG_PROTOCOL_LATEST PG_PROTOCOL(3,101) /* XXX temporary value */
#define PG_PROTOCOL_LATEST PG_PROTOCOL(3,102) /* XXX temporary value */
typedef uint32 ProtocolVersion; /* FE/BE protocol version number */

View File

@ -6,7 +6,7 @@
* Portions Copyright (c) 1996-2002, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $Id: pqformat.h,v 1.14 2003/04/19 00:02:29 tgl Exp $
* $Id: pqformat.h,v 1.15 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -15,8 +15,7 @@
#include "lib/stringinfo.h"
#define pq_beginmessage(buf) initStringInfo(buf)
extern void pq_beginmessage(StringInfo buf, char msgtype);
extern void pq_sendbyte(StringInfo buf, int byt);
extern void pq_sendbytes(StringInfo buf, const char *data, int datalen);
extern void pq_sendcountedtext(StringInfo buf, const char *str, int slen);
@ -24,7 +23,8 @@ extern void pq_sendstring(StringInfo buf, const char *str);
extern void pq_sendint(StringInfo buf, int i, int b);
extern void pq_endmessage(StringInfo buf);
extern int pq_puttextmessage(char msgtype, const char *str);
extern void pq_puttextmessage(char msgtype, const char *str);
extern void pq_putemptymessage(char msgtype);
extern int pq_getmsgbyte(StringInfo msg);
extern unsigned int pq_getmsgint(StringInfo msg, int b);

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-connect.c,v 1.233 2003/04/19 00:02:30 tgl Exp $
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-connect.c,v 1.234 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -1328,6 +1328,8 @@ keep_going: /* We will come back to here until there
case CONNECTION_AWAITING_RESPONSE:
{
char beresp;
int msgLength;
int avail;
AuthRequest areq;
/*
@ -1337,12 +1339,93 @@ keep_going: /* We will come back to here until there
*/
conn->inCursor = conn->inStart;
/* Read type byte */
if (pqGetc(&beresp, conn))
{
/* We'll come back when there is more data */
return PGRES_POLLING_READING;
}
/*
* Validate message type: we expect only an authentication
* request or an error here. Anything else probably means
* it's not Postgres on the other end at all.
*/
if (!(beresp == 'R' || beresp == 'E'))
{
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext(
"expected authentication request from "
"server, but received %c\n"
),
beresp);
goto error_return;
}
/* Read message length word */
if (pqGetInt(&msgLength, 4, conn))
{
/* We'll come back when there is more data */
return PGRES_POLLING_READING;
}
/*
* Try to validate message length before using it.
* Authentication requests can't be very large. Errors
* can be a little larger, but not huge. If we see a large
* apparent length in an error, it means we're really talking
* to a pre-3.0-protocol server; cope.
*/
if (beresp == 'R' && (msgLength < 8 || msgLength > 100))
{
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext(
"expected authentication request from "
"server, but received %c\n"
),
beresp);
goto error_return;
}
if (beresp == 'E' && (msgLength < 8 || msgLength > 30000))
{
/* Handle error from a pre-3.0 server */
conn->inCursor = conn->inStart + 1; /* reread data */
if (pqGets(&conn->errorMessage, conn))
{
/* We'll come back when there is more data */
return PGRES_POLLING_READING;
}
/* OK, we read the message; mark data consumed */
conn->inStart = conn->inCursor;
/*
* The postmaster typically won't end its message with
* a newline, so add one to conform to libpq
* conventions.
*/
appendPQExpBufferChar(&conn->errorMessage, '\n');
goto error_return;
}
/*
* Can't process if message body isn't all here yet.
*/
msgLength -= 4;
avail = conn->inEnd - conn->inCursor;
if (avail < msgLength)
{
/*
* Before returning, try to enlarge the input buffer if
* needed to hold the whole message; see notes in
* parseInput.
*/
if (pqCheckInBufferSpace(conn->inCursor + msgLength, conn))
goto error_return;
/* We'll come back when there is more data */
return PGRES_POLLING_READING;
}
/* Handle errors. */
if (beresp == 'E')
{
@ -1363,18 +1446,7 @@ keep_going: /* We will come back to here until there
goto error_return;
}
/* Otherwise it should be an authentication request. */
if (beresp != 'R')
{
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext(
"expected authentication request from "
"server, but received %c\n"
),
beresp);
goto error_return;
}
/* It is an authentication request. */
/* Get the type of request. */
if (pqGetInt((int *) &areq, 4, conn))
{

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v 1.129 2003/04/19 00:02:30 tgl Exp $
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v 1.130 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -51,6 +51,7 @@ static PGresult *prepareAsyncResult(PGconn *conn);
static int addTuple(PGresult *res, PGresAttValue * tup);
static void parseInput(PGconn *conn);
static void handleSendFailure(PGconn *conn);
static void handleSyncLoss(PGconn *conn, char id, int msgLength);
static int getRowDescriptions(PGconn *conn);
static int getAnotherTuple(PGconn *conn, int binary);
static int getNotify(PGconn *conn);
@ -866,6 +867,8 @@ static void
parseInput(PGconn *conn)
{
char id;
int msgLength;
int avail;
char noticeWorkspace[128];
/*
@ -874,25 +877,63 @@ parseInput(PGconn *conn)
for (;;)
{
/*
* Quit if in COPY_OUT state: we expect raw data from the server
* until PQendcopy is called. Don't try to parse it according to
* the normal protocol. (This is bogus. The data lines ought to
* be part of the protocol and have identifying leading
* characters.)
*/
if (conn->asyncStatus == PGASYNC_COPY_OUT)
return;
/*
* OK to try to read a message type code.
* Try to read a message. First get the type code and length.
* Return if not enough data.
*/
conn->inCursor = conn->inStart;
if (pqGetc(&id, conn))
return;
if (pqGetInt(&msgLength, 4, conn))
return;
/*
* NOTIFY and NOTICE messages can happen in any state besides
* COPY OUT; always process them right away.
* Try to validate message type/length here. A length less than 4
* is definitely broken. Large lengths should only be believed
* for a few message types.
*/
if (msgLength < 4)
{
handleSyncLoss(conn, id, msgLength);
return;
}
if (msgLength > 30000 &&
!(id == 'T' || id == 'D' || id == 'B' || id == 'd'))
{
handleSyncLoss(conn, id, msgLength);
return;
}
/*
* Can't process if message body isn't all here yet.
*/
msgLength -= 4;
avail = conn->inEnd - conn->inCursor;
if (avail < msgLength)
{
/*
* Before returning, enlarge the input buffer if needed to hold
* the whole message. This is better than leaving it to
* pqReadData because we can avoid multiple cycles of realloc()
* when the message is large; also, we can implement a reasonable
* recovery strategy if we are unable to make the buffer big
* enough.
*/
if (pqCheckInBufferSpace(conn->inCursor + msgLength, conn))
{
/*
* XXX add some better recovery code... plan is to skip
* over the message using its length, then report an error.
* For the moment, just treat this like loss of sync (which
* indeed it might be!)
*/
handleSyncLoss(conn, id, msgLength);
}
return;
}
/*
* NOTIFY and NOTICE messages can happen in any state; always process
* them right away.
*
* Most other messages should only be processed while in BUSY state.
* (In particular, in READY state we hold off further parsing
@ -936,9 +977,8 @@ parseInput(PGconn *conn)
libpq_gettext("message type 0x%02x arrived from server while idle\n"),
id);
DONOTICE(conn, noticeWorkspace);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
break;
/* Discard the unexpected message */
conn->inCursor += msgLength;
}
}
else
@ -969,16 +1009,6 @@ parseInput(PGconn *conn)
conn->asyncStatus = PGASYNC_IDLE;
break;
case 'I': /* empty query */
/* read and throw away the closing '\0' */
if (pqGetc(&id, conn))
return;
if (id != '\0')
{
snprintf(noticeWorkspace, sizeof(noticeWorkspace),
libpq_gettext("unexpected character %c following empty query response (\"I\" message)\n"),
id);
DONOTICE(conn, noticeWorkspace);
}
if (conn->result == NULL)
conn->result = PQmakeEmptyPGresult(conn,
PGRES_EMPTY_QUERY);
@ -996,11 +1026,6 @@ parseInput(PGconn *conn)
if (pqGetInt(&(conn->be_key), 4, conn))
return;
break;
case 'P': /* synchronous (normal) portal */
if (pqGets(&conn->workBuffer, conn))
return;
/* We pretty much ignore this message type... */
break;
case 'T': /* row descriptions (start of query
* results) */
if (conn->result == NULL)
@ -1034,9 +1059,8 @@ parseInput(PGconn *conn)
snprintf(noticeWorkspace, sizeof(noticeWorkspace),
libpq_gettext("server sent data (\"D\" message) without prior row description (\"T\" message)\n"));
DONOTICE(conn, noticeWorkspace);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
return;
/* Discard the unexpected message */
conn->inCursor += msgLength;
}
break;
case 'B': /* Binary data tuple */
@ -1051,16 +1075,36 @@ parseInput(PGconn *conn)
snprintf(noticeWorkspace, sizeof(noticeWorkspace),
libpq_gettext("server sent binary data (\"B\" message) without prior row description (\"T\" message)\n"));
DONOTICE(conn, noticeWorkspace);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
return;
/* Discard the unexpected message */
conn->inCursor += msgLength;
}
break;
case 'G': /* Start Copy In */
if (pqGetc(&conn->copy_is_binary, conn))
return;
conn->asyncStatus = PGASYNC_COPY_IN;
break;
case 'H': /* Start Copy Out */
if (pqGetc(&conn->copy_is_binary, conn))
return;
conn->asyncStatus = PGASYNC_COPY_OUT;
conn->copy_already_done = 0;
break;
case 'd': /* Copy Data */
/*
* If we see Copy Data, just silently drop it. This
* would only occur if application exits COPY OUT mode
* too early.
*/
conn->inCursor += msgLength;
break;
case 'c': /* Copy Done */
/*
* If we see Copy Done, just silently drop it. This
* is the normal case during PQendcopy. We will keep
* swallowing data, expecting to see command-complete
* for the COPY command.
*/
break;
default:
printfPQExpBuffer(&conn->errorMessage,
@ -1069,17 +1113,54 @@ parseInput(PGconn *conn)
id);
/* build an error result holding the error message */
saveErrorResult(conn);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
conn->asyncStatus = PGASYNC_READY;
return;
/* Discard the unexpected message */
conn->inCursor += msgLength;
break;
} /* switch on protocol character */
}
/* Successfully consumed this message */
conn->inStart = conn->inCursor;
if (conn->inCursor == conn->inStart + 5 + msgLength)
{
/* Normal case: parsing agrees with specified length */
conn->inStart = conn->inCursor;
}
else
{
/* Trouble --- report it */
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext("Message contents do not agree with length in message type \"%c\"\n"),
id);
/* build an error result holding the error message */
saveErrorResult(conn);
conn->asyncStatus = PGASYNC_READY;
/* trust the specified message length as what to skip */
conn->inStart += 5 + msgLength;
}
}
}
/*
* handleSyncLoss: clean up after loss of message-boundary sync
*
* There isn't really a lot we can do here except abandon the connection.
*/
static void
handleSyncLoss(PGconn *conn, char id, int msgLength)
{
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext(
"lost synchronization with server: got message type \"%c\", length %d\n"),
id, msgLength);
conn->status = CONNECTION_BAD; /* No more connection to backend */
pqsecure_close(conn);
#ifdef WIN32
closesocket(conn->sock);
#else
close(conn->sock);
#endif
conn->sock = -1;
}
/*
* parseInput subroutine to read a 'T' (row descriptions) message.
@ -1100,7 +1181,7 @@ getRowDescriptions(PGconn *conn)
result = PQmakeEmptyPGresult(conn, PGRES_TUPLES_OK);
/* parseInput already read the 'T' label. */
/* parseInput already read the 'T' label and message length. */
/* the next two bytes are the number of fields */
if (pqGetInt(&(result->numAttributes), 2, conn))
{
@ -1461,7 +1542,7 @@ errout:
/*
* Attempt to read a Notice response message.
* This is possible in several places, so we break it out as a subroutine.
* Entry: 'N' flag character has already been consumed.
* Entry: 'N' message type and length have already been consumed.
* Exit: returns 0 if successfully consumed Notice message.
* returns EOF if not enough data.
*/
@ -1489,7 +1570,7 @@ getNotice(PGconn *conn)
/*
* Attempt to read a Notify response message.
* This is possible in several places, so we break it out as a subroutine.
* Entry: 'A' flag character has already been consumed.
* Entry: 'A' message type and length have already been consumed.
* Exit: returns 0 if successfully consumed Notify message.
* returns EOF if not enough data.
*/
@ -1511,10 +1592,18 @@ getNotify(PGconn *conn)
*/
newNotify = (PGnotify *) malloc(sizeof(PGnotify) +
strlen(conn->workBuffer.data) +1);
newNotify->relname = (char *) newNotify + sizeof(PGnotify);
strcpy(newNotify->relname, conn->workBuffer.data);
newNotify->be_pid = be_pid;
DLAddTail(conn->notifyList, DLNewElem(newNotify));
if (newNotify)
{
newNotify->relname = (char *) newNotify + sizeof(PGnotify);
strcpy(newNotify->relname, conn->workBuffer.data);
newNotify->be_pid = be_pid;
DLAddTail(conn->notifyList, DLNewElem(newNotify));
}
/* Swallow extra string (not presently used) */
if (pqGets(&conn->workBuffer, conn))
return EOF;
return 0;
}
@ -1556,6 +1645,9 @@ PQnotifies(PGconn *conn)
* Chiefly here so that applications can use "COPY <rel> to stdout"
* and read the output string. Returns a null-terminated string in s.
*
* XXX this routine is now deprecated, because it can't handle binary data.
* If called during a COPY BINARY we return EOF.
*
* PQgetline reads up to maxlen-1 characters (like fgets(3)) but strips
* the terminating \n (like gets(3)).
*
@ -1563,7 +1655,7 @@ PQnotifies(PGconn *conn)
* (a line containing just "\.") when using this routine.
*
* RETURNS:
* EOF if it is detected or invalid arguments are given
* EOF if error (eg, invalid arguments are given)
* 0 if EOL is reached (i.e., \n has been read)
* (this is required for backward-compatibility -- this
* routine used to always return EOF or 0, assuming that
@ -1573,53 +1665,55 @@ PQnotifies(PGconn *conn)
int
PQgetline(PGconn *conn, char *s, int maxlen)
{
int result = 1; /* return value if buffer overflows */
int status;
if (!s || maxlen <= 0)
/* maxlen must be at least 3 to hold the \. terminator! */
if (!conn || !s || maxlen < 3)
return EOF;
if (!conn || conn->sock < 0)
if (conn->sock < 0 ||
conn->asyncStatus != PGASYNC_COPY_OUT ||
conn->copy_is_binary)
{
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext("PQgetline: not doing text COPY OUT\n"));
*s = '\0';
return EOF;
}
/*
* Since this is a purely synchronous routine, we don't bother to
* maintain conn->inCursor; there is no need to back up.
*/
while (maxlen > 1)
while ((status = PQgetlineAsync(conn, s, maxlen-1)) == 0)
{
if (conn->inStart < conn->inEnd)
/* need to load more data */
if (pqWait(TRUE, FALSE, conn) ||
pqReadData(conn) < 0)
{
char c = conn->inBuffer[conn->inStart++];
if (c == '\n')
{
result = 0; /* success exit */
break;
}
*s++ = c;
maxlen--;
}
else
{
/* need to load more data */
if (pqWait(TRUE, FALSE, conn) ||
pqReadData(conn) < 0)
{
result = EOF;
break;
}
*s = '\0';
return EOF;
}
}
*s = '\0';
return result;
if (status < 0)
{
/* End of copy detected; gin up old-style terminator */
strcpy(s, "\\.");
return 0;
}
/* Add null terminator, and strip trailing \n if present */
if (s[status-1] == '\n')
{
s[status-1] = '\0';
return 0;
}
else
{
s[status] = '\0';
return 1;
}
}
/*
* PQgetlineAsync - gets a newline-terminated string without blocking.
* PQgetlineAsync - gets a COPY data row without blocking.
*
* This routine is for applications that want to do "COPY <rel> to stdout"
* asynchronously, that is without blocking. Having issued the COPY command
@ -1627,10 +1721,9 @@ PQgetline(PGconn *conn, char *s, int maxlen)
* and this routine until the end-of-data signal is detected. Unlike
* PQgetline, this routine takes responsibility for detecting end-of-data.
*
* On each call, PQgetlineAsync will return data if a complete newline-
* terminated data line is available in libpq's input buffer, or if the
* incoming data line is too long to fit in the buffer offered by the caller.
* Otherwise, no data is returned until the rest of the line arrives.
* On each call, PQgetlineAsync will return data if a complete data row
* is available in libpq's input buffer. Otherwise, no data is returned
* until the rest of the row arrives.
*
* If -1 is returned, the end-of-data signal has been recognized (and removed
* from libpq's input buffer). The caller *must* next call PQendcopy and
@ -1640,66 +1733,73 @@ PQgetline(PGconn *conn, char *s, int maxlen)
* -1 if the end-of-copy-data marker has been recognized
* 0 if no data is available
* >0 the number of bytes returned.
* The data returned will not extend beyond a newline character. If possible
* a whole line will be returned at one time. But if the buffer offered by
* the caller is too small to hold a line sent by the backend, then a partial
* data line will be returned. This can be detected by testing whether the
* last returned byte is '\n' or not.
* The returned string is *not* null-terminated.
*
* The data returned will not extend beyond a data-row boundary. If possible
* a whole row will be returned at one time. But if the buffer offered by
* the caller is too small to hold a row sent by the backend, then a partial
* data row will be returned. In text mode this can be detected by testing
* whether the last returned byte is '\n' or not.
*
* The returned data is *not* null-terminated.
*/
int
PQgetlineAsync(PGconn *conn, char *buffer, int bufsize)
{
char id;
int msgLength;
int avail;
if (!conn || conn->asyncStatus != PGASYNC_COPY_OUT)
return -1; /* we are not doing a copy... */
/*
* Move data from libpq's buffer to the caller's. We want to accept
* data only in units of whole lines, not partial lines. This ensures
* that we can recognize the terminator line "\\.\n". (Otherwise, if
* it happened to cross a packet/buffer boundary, we might hand the
* first one or two characters off to the caller, which we shouldn't.)
* Recognize the next input message. To make life simpler for async
* callers, we keep returning 0 until the next message is fully available
* even if it is not Copy Data. This should keep PQendcopy from blocking.
*/
conn->inCursor = conn->inStart;
avail = bufsize;
while (avail > 0 && conn->inCursor < conn->inEnd)
{
char c = conn->inBuffer[conn->inCursor++];
*buffer++ = c;
--avail;
if (c == '\n')
{
/* Got a complete line; mark the data removed from libpq */
conn->inStart = conn->inCursor;
/* Is it the endmarker line? */
if (bufsize - avail == 3 && buffer[-3] == '\\' && buffer[-2] == '.')
return -1;
/* No, return the data line to the caller */
return bufsize - avail;
}
}
if (pqGetc(&id, conn))
return 0;
if (pqGetInt(&msgLength, 4, conn))
return 0;
avail = conn->inEnd - conn->inCursor;
if (avail < msgLength - 4)
return 0;
/*
* We don't have a complete line. We'd prefer to leave it in libpq's
* buffer until the rest arrives, but there is a special case: what if
* the line is longer than the buffer the caller is offering us? In
* that case we'd better hand over a partial line, else we'd get into
* an infinite loop. Do this in a way that ensures we can't
* misrecognize a terminator line later: leave last 3 characters in
* libpq buffer.
* Cannot proceed unless it's a Copy Data message. Anything else means
* end of copy mode.
*/
if (avail == 0 && bufsize > 3)
if (id != 'd')
return -1;
/*
* Move data from libpq's buffer to the caller's. In the case where
* a prior call found the caller's buffer too small, we use
* conn->copy_already_done to remember how much of the row was already
* returned to the caller.
*/
conn->inCursor += conn->copy_already_done;
avail = msgLength - 4 - conn->copy_already_done;
if (avail <= bufsize)
{
conn->inStart = conn->inCursor - 3;
return bufsize - 3;
/* Able to consume the whole message */
memcpy(buffer, &conn->inBuffer[conn->inCursor], avail);
/* Mark message consumed */
conn->inStart = conn->inCursor + avail;
/* Reset state for next time */
conn->copy_already_done = 0;
return avail;
}
else
{
/* We must return a partial message */
memcpy(buffer, &conn->inBuffer[conn->inCursor], bufsize);
/* The message is NOT consumed from libpq's buffer */
conn->copy_already_done += bufsize;
return bufsize;
}
return 0;
}
/*
@ -1774,14 +1874,21 @@ PQendcopy(PGconn *conn)
if (pqFlush(conn) && pqIsnonblocking(conn))
return (1);
/* non blocking connections may have to abort at this point. */
if (pqIsnonblocking(conn) && PQisBusy(conn))
return (1);
/* Return to active duty */
conn->asyncStatus = PGASYNC_BUSY;
resetPQExpBuffer(&conn->errorMessage);
/*
* Non blocking connections may have to abort at this point. If everyone
* played the game there should be no problem, but in error scenarios
* the expected messages may not have arrived yet. (We are assuming that
* the backend's packetizing will ensure that CommandComplete arrives
* along with the CopyDone; are there corner cases where that doesn't
* happen?)
*/
if (pqIsnonblocking(conn) && PQisBusy(conn))
return (1);
/* Wait for the completion response */
result = PQgetResult(conn);
@ -1793,26 +1900,16 @@ PQendcopy(PGconn *conn)
}
/*
* Trouble. The worst case is that we've lost sync with the backend
* entirely due to application screwup of the copy in/out protocol. To
* recover, reset the connection (talk about using a sledgehammer...)
* Trouble. For backwards-compatibility reasons, we issue the error
* message as if it were a notice (would be nice to get rid of this
* silliness, but too many apps probably don't handle errors from
* PQendcopy reasonably). Note that the app can still obtain the
* error status from the PGconn object.
*/
PQclear(result);
if (conn->errorMessage.len > 0)
DONOTICE(conn, conn->errorMessage.data);
DONOTICE(conn, libpq_gettext("lost synchronization with server, resetting connection\n"));
/*
* Users doing non-blocking connections need to handle the reset
* themselves, they'll need to check the connection status if we
* return an error.
*/
if (pqIsnonblocking(conn))
PQresetStart(conn);
else
PQreset(conn);
PQclear(result);
return 1;
}
@ -1853,6 +1950,8 @@ PQfn(PGconn *conn,
bool needInput = false;
ExecStatusType status = PGRES_FATAL_ERROR;
char id;
int msgLength;
int avail;
int i;
*actual_result_len = 0;
@ -1927,11 +2026,55 @@ PQfn(PGconn *conn,
* Scan the message. If we run out of data, loop around to try
* again.
*/
conn->inCursor = conn->inStart;
needInput = true;
conn->inCursor = conn->inStart;
if (pqGetc(&id, conn))
continue;
if (pqGetInt(&msgLength, 4, conn))
continue;
/*
* Try to validate message type/length here. A length less than 4
* is definitely broken. Large lengths should only be believed
* for a few message types.
*/
if (msgLength < 4)
{
handleSyncLoss(conn, id, msgLength);
break;
}
if (msgLength > 30000 &&
!(id == 'T' || id == 'D' || id == 'B' || id == 'd' || id == 'V'))
{
handleSyncLoss(conn, id, msgLength);
break;
}
/*
* Can't process if message body isn't all here yet.
*/
msgLength -= 4;
avail = conn->inEnd - conn->inCursor;
if (avail < msgLength)
{
/*
* Before looping, enlarge the input buffer if needed to hold
* the whole message. See notes in parseInput.
*/
if (pqCheckInBufferSpace(conn->inCursor + msgLength, conn))
{
/*
* XXX add some better recovery code... plan is to skip
* over the message using its length, then report an error.
* For the moment, just treat this like loss of sync (which
* indeed it might be!)
*/
handleSyncLoss(conn, id, msgLength);
break;
}
continue;
}
/*
* We should see V or E response to the command, but might get N
@ -1975,7 +2118,7 @@ PQfn(PGconn *conn,
libpq_gettext("protocol error: id=0x%x\n"),
id);
saveErrorResult(conn);
conn->inStart = conn->inCursor;
conn->inStart += 5 + msgLength;
return prepareAsyncResult(conn);
}
break;
@ -1998,7 +2141,8 @@ PQfn(PGconn *conn,
break;
case 'Z': /* backend is ready for new query */
/* consume the message and exit */
conn->inStart = conn->inCursor;
conn->inStart += 5 + msgLength;
/* XXX expect additional fields here */
/* if we saved a result object (probably an error), use it */
if (conn->result)
return prepareAsyncResult(conn);
@ -2009,11 +2153,13 @@ PQfn(PGconn *conn,
libpq_gettext("protocol error: id=0x%x\n"),
id);
saveErrorResult(conn);
conn->inStart = conn->inCursor;
/* trust the specified message length as what to skip */
conn->inStart += 5 + msgLength;
return prepareAsyncResult(conn);
}
/* Completed this message, keep going */
conn->inStart = conn->inCursor;
/* trust the specified message length as what to skip */
conn->inStart += 5 + msgLength;
needInput = false;
}

View File

@ -23,7 +23,7 @@
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-misc.c,v 1.89 2003/04/19 00:02:30 tgl Exp $
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-misc.c,v 1.90 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -277,12 +277,12 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
/*
* Make sure conn's output buffer can hold bytes_needed bytes (caller must
* include existing outCount into the value!)
* include already-stored data into the value!)
*
* Returns 0 on success, EOF on error
* Returns 0 on success, EOF if failed to enlarge buffer
*/
static int
checkOutBufferSpace(int bytes_needed, PGconn *conn)
pqCheckOutBufferSpace(int bytes_needed, PGconn *conn)
{
int newsize = conn->outBufSize;
char *newbuf;
@ -335,6 +335,66 @@ checkOutBufferSpace(int bytes_needed, PGconn *conn)
return EOF;
}
/*
* Make sure conn's input buffer can hold bytes_needed bytes (caller must
* include already-stored data into the value!)
*
* Returns 0 on success, EOF if failed to enlarge buffer
*/
int
pqCheckInBufferSpace(int bytes_needed, PGconn *conn)
{
int newsize = conn->inBufSize;
char *newbuf;
if (bytes_needed <= newsize)
return 0;
/*
* If we need to enlarge the buffer, we first try to double it in size;
* if that doesn't work, enlarge in multiples of 8K. This avoids
* thrashing the malloc pool by repeated small enlargements.
*
* Note: tests for newsize > 0 are to catch integer overflow.
*/
do {
newsize *= 2;
} while (bytes_needed > newsize && newsize > 0);
if (bytes_needed <= newsize)
{
newbuf = realloc(conn->inBuffer, newsize);
if (newbuf)
{
/* realloc succeeded */
conn->inBuffer = newbuf;
conn->inBufSize = newsize;
return 0;
}
}
newsize = conn->inBufSize;
do {
newsize += 8192;
} while (bytes_needed > newsize && newsize > 0);
if (bytes_needed <= newsize)
{
newbuf = realloc(conn->inBuffer, newsize);
if (newbuf)
{
/* realloc succeeded */
conn->inBuffer = newbuf;
conn->inBufSize = newsize;
return 0;
}
}
/* realloc failed. Probably out of memory */
printfPQExpBuffer(&conn->errorMessage,
"cannot allocate memory for input buffer\n");
return EOF;
}
/*
* pqPutMsgStart: begin construction of a message to the server
*
@ -364,7 +424,7 @@ pqPutMsgStart(char msg_type, PGconn *conn)
else
lenPos = conn->outCount;
/* make sure there is room for it */
if (checkOutBufferSpace(lenPos + 4, conn))
if (pqCheckOutBufferSpace(lenPos + 4, conn))
return EOF;
/* okay, save the message type byte if any */
if (msg_type)
@ -390,7 +450,7 @@ static int
pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
{
/* make sure there is room for it */
if (checkOutBufferSpace(conn->outMsgEnd + len, conn))
if (pqCheckOutBufferSpace(conn->outMsgEnd + len, conn))
return EOF;
/* okay, save the data */
memcpy(conn->outBuffer + conn->outMsgEnd, buf, len);
@ -486,13 +546,13 @@ pqReadData(PGconn *conn)
*/
if (conn->inBufSize - conn->inEnd < 8192)
{
int newSize = conn->inBufSize * 2;
char *newBuf = (char *) realloc(conn->inBuffer, newSize);
if (newBuf)
if (pqCheckInBufferSpace(conn->inEnd + 8192, conn))
{
conn->inBuffer = newBuf;
conn->inBufSize = newSize;
/*
* We don't insist that the enlarge worked, but we need some room
*/
if (conn->inBufSize - conn->inEnd < 100)
return -1; /* errorMessage already set */
}
}

View File

@ -12,7 +12,7 @@
* Portions Copyright (c) 1996-2002, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $Id: libpq-int.h,v 1.62 2003/04/19 00:02:30 tgl Exp $
* $Id: libpq-int.h,v 1.63 2003/04/22 00:08:07 tgl Exp $
*
*-------------------------------------------------------------------------
*/
@ -56,7 +56,7 @@ typedef int ssize_t; /* ssize_t doesn't exist in VC (atleast
* pqcomm.h describe what the backend knows, not what libpq knows.
*/
#define PG_PROTOCOL_LIBPQ PG_PROTOCOL(3,101) /* XXX temporary value */
#define PG_PROTOCOL_LIBPQ PG_PROTOCOL(3,102) /* XXX temporary value */
/*
* POSTGRES backend dependent Constants.
@ -216,7 +216,8 @@ struct pg_conn
* is listening on; if NULL, uses a
* default constructed from pgport */
char *pgtty; /* tty on which the backend messages is
* displayed (NOT ACTUALLY USED???) */
* displayed (OBSOLETE, NOT USED) */
char *connect_timeout; /* connection timeout (numeric string) */
char *pgoptions; /* options to start the backend with */
char *dbName; /* database name */
char *pguser; /* Postgres username and password, if any */
@ -232,6 +233,10 @@ struct pg_conn
/* Status indicators */
ConnStatusType status;
PGAsyncStatusType asyncStatus;
char copy_is_binary; /* 1 = copy binary, 0 = copy text */
int copy_already_done; /* # bytes already returned in COPY OUT */
int nonblocking; /* whether this connection is using a
* blocking socket to the backend or not */
Dllist *notifyList; /* Notify msgs not yet handed to
* application */
@ -246,6 +251,7 @@ struct pg_conn
int be_key; /* key of backend --- needed for cancels */
char md5Salt[4]; /* password salt received from backend */
char cryptSalt[2]; /* password salt received from backend */
int client_encoding; /* encoding id */
PGlobjfuncs *lobjfuncs; /* private state for large-object access
* fns */
@ -258,9 +264,6 @@ struct pg_conn
int inEnd; /* offset to first position after avail
* data */
int nonblocking; /* whether this connection is using a
* blocking socket to the backend or not */
/* Buffer for data not yet sent to backend */
char *outBuffer; /* currently allocated buffer */
int outBufSize; /* allocated size of buffer */
@ -291,10 +294,6 @@ struct pg_conn
/* Buffer for receiving various parts of messages */
PQExpBufferData workBuffer; /* expansible string */
int client_encoding; /* encoding id */
char *connect_timeout;
};
/* String descriptions of the ExecStatusTypes.
@ -330,6 +329,7 @@ extern void pqClearAsyncResult(PGconn *conn);
* for Get, EOF merely means the buffer is exhausted, not that there is
* necessarily any error.
*/
extern int pqCheckInBufferSpace(int bytes_needed, PGconn *conn);
extern int pqGetc(char *result, PGconn *conn);
extern int pqPutc(char c, PGconn *conn);
extern int pqGets(PQExpBuffer buf, PGconn *conn);

View File

@ -995,7 +995,6 @@ copy test("........pg.dropped.1........") to stdout;
ERROR: Relation "test" has no column "........pg.dropped.1........"
copy test from stdin;
ERROR: copy: line 1, Extra data after last expected column
lost synchronization with server, resetting connection
SET autocommit TO 'on';
select * from test;
b | c

View File

@ -35,17 +35,13 @@ ERROR: Attribute "d" specified more than once
-- missing data: should fail
COPY x from stdin;
ERROR: copy: line 1, pg_atoi: zero-length string
lost synchronization with server, resetting connection
COPY x from stdin;
ERROR: copy: line 1, Missing data for column "e"
lost synchronization with server, resetting connection
COPY x from stdin;
ERROR: copy: line 1, Missing data for column "e"
lost synchronization with server, resetting connection
-- extra data: should fail
COPY x from stdin;
ERROR: copy: line 1, Extra data after last expected column
lost synchronization with server, resetting connection
SET autocommit TO 'on';
-- various COPY options: delimiters, oids, NULL string
COPY x (b, c, d, e) from stdin with oids delimiter ',' null 'x';

View File

@ -40,7 +40,6 @@ INSERT INTO basictest values ('88', 'haha', 'short', '123.1212'); -- Truncate
-- Test copy
COPY basictest (testvarchar) FROM stdin; -- fail
ERROR: copy: line 1, value too long for type character varying(5)
lost synchronization with server, resetting connection
SET autocommit TO 'on';
COPY basictest (testvarchar) FROM stdin;
select * from basictest;
@ -128,12 +127,10 @@ INSERT INTO nulltest values ('a', 'b', 'c', NULL, 'd'); -- Good
-- Test copy
COPY nulltest FROM stdin; --fail
ERROR: copy: line 1, Domain dcheck does not allow NULL values
lost synchronization with server, resetting connection
SET autocommit TO 'on';
-- Last row is bad
COPY nulltest FROM stdin;
ERROR: copy: line 3, CopyFrom: rejected due to CHECK constraint "nulltest_col5" on "nulltest"
lost synchronization with server, resetting connection
select * from nulltest;
col1 | col2 | col3 | col4 | col5
------+------+------+------+------