rdelaage/postgresql
1
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2024-10-01 20:51:21 +02:00
Code Issues Packages Projects Releases Wiki Activity
fb85cd4320
postgresql/src/backend/access/common/printtup.c

640 lines
17 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* printtup.c
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
* Routines to print out tuples to the destination (both frontend
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
* clients and standalone backends are supported here).
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Update copyright for 2014 Update all files in head, and files COPYRIGHT and legal.sgml in all back branches.
2014-01-07 22:05:30 +01:00
* Portions Copyright (c) 1996-2014, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/access/common/printtup.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*-------------------------------------------------------------------------
*/
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
#include "postgres.h"
#include "access/printtup.h"
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
#include "libpq/libpq.h"
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
#include "libpq/pqformat.h"
Fix the mechanism for reporting the original table OID and column number of columns of a query result so that it can "see through" cursors and prepared statements. Per gripe a couple months back from John DeSoi.
2005-06-22 19:45:46 +02:00
#include "tcop/pquery.h"
Add a bunch of pseudo-types to replace the behavior formerly associated with OPAQUE, as per recent pghackers discussion. I still want to do some more work on the 'cstring' pseudo-type, but I'm going to commit the bulk of the changes now before the tree starts shifting under me ...
2002-08-22 02:01:51 +02:00
#include "utils/lsyscache.h"
Cooperate with the Valgrind instrumentation framework. Valgrind "client requests" in aset.c and mcxt.c teach Valgrind and its Memcheck tool about the PostgreSQL allocator. This makes Valgrind roughly as sensitive to memory errors involving palloc chunks as it is to memory errors involving malloc chunks. Further client requests in PageAddItem() and printtup() verify that all bits being added to a buffer page or furnished to an output function are predictably-defined. Those tests catch failures of C-language functions to fully initialize the bits of a Datum, which in turn stymie optimizations that rely on _equalConst(). Define the USE_VALGRIND symbol in pg_config_manual.h to enable these additions. An included "suppression file" silences nominal errors we don't plan to fix. Reviewed in earlier versions by Peter Geoghegan and Korry Douglas.
2013-06-27 02:00:08 +02:00
#include "utils/memdebug.h"
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
#include "utils/memutils.h"
Add a bunch of pseudo-types to replace the behavior formerly associated with OPAQUE, as per recent pghackers discussion. I still want to do some more work on the 'cstring' pseudo-type, but I'm going to commit the bulk of the changes now before the tree starts shifting under me ...
2002-08-22 02:01:51 +02:00
Other then: indextuple.c:159: warning: `bp' might be used uninitialized in this function this directory passes -Wall -Werror under FreeBSD
1996-10-21 00:04:49 +02:00
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
static void printtup_startup(DestReceiver *self, int operation,
pgindent run.
2003-08-04 02:43:34 +02:00
TupleDesc typeinfo);
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
static void printtup(TupleTableSlot *slot, DestReceiver *self);
static void printtup_20(TupleTableSlot *slot, DestReceiver *self);
static void printtup_internal_20(TupleTableSlot *slot, DestReceiver *self);
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
static void printtup_shutdown(DestReceiver *self);
static void printtup_destroy(DestReceiver *self);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/* ----------------------------------------------------------------
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* printtup / debugtup support
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------------------------------------------------------
*/
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
* Private state for a printtup destination object
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
*
* NOTE: finfo is the lookup info for either typoutput or typsend, whichever
* we are using for this column.
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* ----------------
*/
pgindent run over code.
1999-05-25 18:15:34 +02:00
typedef struct
{ /* Per-attribute information */
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
Oid typoutput; /* Oid for the type's text output fn */
Oid typsend; /* Oid for the type's binary output fn */
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
bool typisvarlena; /* is it varlena (ie possibly toastable)? */
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
int16 format; /* format code for this column */
FmgrInfo finfo; /* Precomputed call info for output fn */
Another pgindent run. Sorry folks.
1999-05-26 00:43:53 +02:00
} PrinttupAttrInfo;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
typedef struct
{
DestReceiver pub; /* publicly-known function pointers */
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
Portal portal; /* the Portal we are printing from */
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
bool sendDescrip; /* send RowDescription at startup? */
pgindent run over code.
1999-05-25 18:15:34 +02:00
TupleDesc attrinfo; /* The attr info we are set up for */
int nattrs;
PrinttupAttrInfo *myinfo; /* Cached info about each attr */
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext tmpcontext; /* Memory context for per-row workspace */
Another pgindent run. Sorry folks.
1999-05-26 00:43:53 +02:00
} DR_printtup;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
* Initialize: create a DestReceiver for printtup
* ----------------
*/
pgindent run over code.
1999-05-25 18:15:34 +02:00
DestReceiver *
Clean up the API for DestReceiver objects by eliminating the assumption that a Portal is a useful and sufficient additional argument for CreateDestReceiver --- it just isn't, in most cases. Instead formalize the approach of passing any needed parameters to the receiver separately. One unexpected benefit of this change is that we can declare typedef Portal in a less surprising location. This patch is just code rearrangement and doesn't change any functionality. I'll tackle the HOLD-cursor-vs-toast problem in a follow-on patch.
2008-11-30 21:51:25 +01:00
printtup_create_DR(CommandDest dest)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
Clean up the API for DestReceiver objects by eliminating the assumption that a Portal is a useful and sufficient additional argument for CreateDestReceiver --- it just isn't, in most cases. Instead formalize the approach of passing any needed parameters to the receiver separately. One unexpected benefit of this change is that we can declare typedef Portal in a less surprising location. This patch is just code rearrangement and doesn't change any functionality. I'll tackle the HOLD-cursor-vs-toast problem in a follow-on patch.
2008-11-30 21:51:25 +01:00
DR_printtup *self = (DR_printtup *) palloc0(sizeof(DR_printtup));
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
self->pub.receiveSlot = printtup; /* might get changed later */
Rename fields of DestReceiver to avoid collisions with (ill-considered) macros in some platforms' sys/socket.h.
2003-08-06 19:46:46 +02:00
self->pub.rStartup = printtup_startup;
self->pub.rShutdown = printtup_shutdown;
self->pub.rDestroy = printtup_destroy;
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
self->pub.mydest = dest;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Rename the members of CommandDest enum so they don't collide with other uses of those names. (Debug and None were pretty bad names anyway.) I hope I catched all uses of the names in comments too.
2005-11-03 18:11:40 +01:00
/*
* Send T message automatically if DestRemote, but not if
* DestRemoteExecute
*/
self->sendDescrip = (dest == DestRemote);
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
self->attrinfo = NULL;
self->nattrs = 0;
self->myinfo = NULL;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
self->tmpcontext = NULL;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
return (DestReceiver *) self;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
}
Clean up the API for DestReceiver objects by eliminating the assumption that a Portal is a useful and sufficient additional argument for CreateDestReceiver --- it just isn't, in most cases. Instead formalize the approach of passing any needed parameters to the receiver separately. One unexpected benefit of this change is that we can declare typedef Portal in a less surprising location. This patch is just code rearrangement and doesn't change any functionality. I'll tackle the HOLD-cursor-vs-toast problem in a follow-on patch.
2008-11-30 21:51:25 +01:00
/*
* Set parameters for a DestRemote (or DestRemoteExecute) receiver
*/
void
SetRemoteDestReceiverParams(DestReceiver *self, Portal portal)
{
DR_printtup *myState = (DR_printtup *) self;
Assert(myState->pub.mydest == DestRemote ||
myState->pub.mydest == DestRemoteExecute);
myState->portal = portal;
if (PG_PROTOCOL_MAJOR(FrontendProtocol) < 3)
{
/*
* In protocol 2.0 the Bind message does not exist, so there is no way
* for the columns to have different print formats; it's sufficient to
* look at the first one.
*/
if (portal->formats && portal->formats[0] != 0)
myState->pub.receiveSlot = printtup_internal_20;
else
myState->pub.receiveSlot = printtup_20;
}
}
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
static void
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
printtup_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
DR_printtup *myState = (DR_printtup *) self;
pgindent run.
2003-08-04 02:43:34 +02:00
Portal portal = myState->portal;
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/*
* Create a temporary memory context that we can reset once per row to
* recover palloc'd memory. This avoids any problems with leaks inside
* datatype output routines, and should be faster than retail pfree's
* anyway.
*/
myState->tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
"printtup",
ALLOCSET_DEFAULT_MINSIZE,
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
if (PG_PROTOCOL_MAJOR(FrontendProtocol) < 3)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Send portal name to frontend (obsolete cruft, gone in proto 3.0)
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
*
* If portal name not specified, use "blank" portal.
*/
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
const char *portalName = portal->name;
if (portalName == NULL || portalName[0] == '\0')
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
portalName = "blank";
pq_puttextmessage('P', portalName);
}
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* If we are supposed to emit row descriptions, then send the tuple
* descriptor of the tuples.
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
*/
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
2006-08-12 04:52:06 +02:00
if (myState->sendDescrip)
Fix the mechanism for reporting the original table OID and column number of columns of a query result so that it can "see through" cursors and prepared statements. Per gripe a couple months back from John DeSoi.
2005-06-22 19:45:46 +02:00
SendRowDescriptionMessage(typeinfo,
FetchPortalTargetList(portal),
portal->formats);
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
* We could set up the derived attr info at this time, but we postpone it
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
* until the first call of printtup, for 2 reasons:
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* 1. We don't waste time (compared to the old way) if there are no
pgindent run over code.
1999-05-25 18:15:34 +02:00
* tuples at all to output.
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* 2. Checking in printtup allows us to handle the case that the tuples
pgindent run over code.
1999-05-25 18:15:34 +02:00
* change type midway through (although this probably can't happen in
* the current executor).
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* ----------------
*/
}
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
/*
* SendRowDescriptionMessage --- send a RowDescription message to the frontend
Implement feature of new FE/BE protocol whereby RowDescription identifies the column by table OID and column number, if it's a simple column reference. Along the way, get rid of reskey/reskeyop fields in Resdoms. Turns out that representation was not convenient for either the planner or the executor; we can make the planner deliver exactly what the executor wants with no more effort. initdb forced due to change in stored rule representation.
2003-05-06 02:20:33 +02:00
*
* Notes: the TupleDesc has typically been manufactured by ExecTypeFromTL()
* or some similar function; it does not contain a full set of fields.
* The targetlist will be NIL when executing a utility function that does
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* not have a plan. If the targetlist isn't NIL then it is a Query node's
pgindent run.
2003-08-04 02:43:34 +02:00
* targetlist; it is up to us to ignore resjunk columns in it. The formats[]
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* array pointer might be NULL (if we are doing Describe on a prepared stmt);
* send zeroes for the format codes in that case.
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
*/
void
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
SendRowDescriptionMessage(TupleDesc typeinfo, List *targetlist, int16 *formats)
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
{
Form_pg_attribute *attrs = typeinfo->attrs;
int natts = typeinfo->natts;
int proto = PG_PROTOCOL_MAJOR(FrontendProtocol);
int i;
StringInfoData buf;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
ListCell *tlist_item = list_head(targetlist);
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
pgindent run.
2003-08-04 02:43:34 +02:00
pq_beginmessage(&buf, 'T'); /* tuple descriptor message type */
pq_sendint(&buf, natts, 2); /* # of attrs in tuples */
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
for (i = 0; i < natts; ++i)
{
pgindent run.
2003-08-04 02:43:34 +02:00
Oid atttypid = attrs[i]->atttypid;
int32 atttypmod = attrs[i]->atttypmod;
In RowDescription messages, report columns of domain datatypes as having the type OID and typmod of the underlying base type. Per discussions a few weeks ago with Andreas Pflug and others. Note that this behavioral change affects both old- and new-protocol clients.
2003-05-13 20:39:50 +02:00
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
pq_sendstring(&buf, NameStr(attrs[i]->attname));
/* column ID info appears in protocol 3.0 and up */
if (proto >= 3)
{
Implement feature of new FE/BE protocol whereby RowDescription identifies the column by table OID and column number, if it's a simple column reference. Along the way, get rid of reskey/reskeyop fields in Resdoms. Turns out that representation was not convenient for either the planner or the executor; we can make the planner deliver exactly what the executor wants with no more effort. initdb forced due to change in stored rule representation.
2003-05-06 02:20:33 +02:00
/* Do we have a non-resjunk tlist item? */
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
while (tlist_item &&
Merge Resdom nodes into TargetEntry nodes to simplify code and save a few palloc's. I also chose to eliminate the restype and restypmod fields entirely, since they are redundant with information stored in the node's contained expression; re-examining the expression at need seems simpler and more reliable than trying to keep restype/restypmod up to date. initdb forced due to change in contents of stored rules.
2005-04-06 18:34:07 +02:00
((TargetEntry *) lfirst(tlist_item))->resjunk)
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
tlist_item = lnext(tlist_item);
if (tlist_item)
Implement feature of new FE/BE protocol whereby RowDescription identifies the column by table OID and column number, if it's a simple column reference. Along the way, get rid of reskey/reskeyop fields in Resdoms. Turns out that representation was not convenient for either the planner or the executor; we can make the planner deliver exactly what the executor wants with no more effort. initdb forced due to change in stored rule representation.
2003-05-06 02:20:33 +02:00
{
Merge Resdom nodes into TargetEntry nodes to simplify code and save a few palloc's. I also chose to eliminate the restype and restypmod fields entirely, since they are redundant with information stored in the node's contained expression; re-examining the expression at need seems simpler and more reliable than trying to keep restype/restypmod up to date. initdb forced due to change in contents of stored rules.
2005-04-06 18:34:07 +02:00
TargetEntry *tle = (TargetEntry *) lfirst(tlist_item);
Implement feature of new FE/BE protocol whereby RowDescription identifies the column by table OID and column number, if it's a simple column reference. Along the way, get rid of reskey/reskeyop fields in Resdoms. Turns out that representation was not convenient for either the planner or the executor; we can make the planner deliver exactly what the executor wants with no more effort. initdb forced due to change in stored rule representation.
2003-05-06 02:20:33 +02:00
Merge Resdom nodes into TargetEntry nodes to simplify code and save a few palloc's. I also chose to eliminate the restype and restypmod fields entirely, since they are redundant with information stored in the node's contained expression; re-examining the expression at need seems simpler and more reliable than trying to keep restype/restypmod up to date. initdb forced due to change in contents of stored rules.
2005-04-06 18:34:07 +02:00
pq_sendint(&buf, tle->resorigtbl, 4);
pq_sendint(&buf, tle->resorigcol, 2);
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
tlist_item = lnext(tlist_item);
Implement feature of new FE/BE protocol whereby RowDescription identifies the column by table OID and column number, if it's a simple column reference. Along the way, get rid of reskey/reskeyop fields in Resdoms. Turns out that representation was not convenient for either the planner or the executor; we can make the planner deliver exactly what the executor wants with no more effort. initdb forced due to change in stored rule representation.
2003-05-06 02:20:33 +02:00
}
else
{
/* No info available, so send zeroes */
pq_sendint(&buf, 0, 4);
pq_sendint(&buf, 0, 2);
}
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
}
In RowDescription messages, report columns of domain datatypes as having the type OID and typmod of the underlying base type. Per discussions a few weeks ago with Andreas Pflug and others. Note that this behavioral change affects both old- and new-protocol clients.
2003-05-13 20:39:50 +02:00
/* If column is a domain, send the base type and typmod instead */
Fix a bunch of problems with domains by making them use special input functions that apply the necessary domain constraint checks immediately. This fixes cases where domain constraints went unchecked for statement parameters, PL function local variables and results, etc. We can also eliminate existing special cases for domains in places that had gotten it right, eg COPY. Also, allow domains over domains (base of a domain is another domain type). This almost worked before, but was disallowed because the original patch hadn't gotten it quite right.
2006-04-06 00:11:58 +02:00
atttypid = getBaseTypeAndTypmod(atttypid, &atttypmod);
In RowDescription messages, report columns of domain datatypes as having the type OID and typmod of the underlying base type. Per discussions a few weeks ago with Andreas Pflug and others. Note that this behavioral change affects both old- and new-protocol clients.
2003-05-13 20:39:50 +02:00
pq_sendint(&buf, (int) atttypid, sizeof(atttypid));
pq_sendint(&buf, attrs[i]->attlen, sizeof(attrs[i]->attlen));
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
/* typmod appears in protocol 2.0 and up */
if (proto >= 2)
In RowDescription messages, report columns of domain datatypes as having the type OID and typmod of the underlying base type. Per discussions a few weeks ago with Andreas Pflug and others. Note that this behavioral change affects both old- and new-protocol clients.
2003-05-13 20:39:50 +02:00
pq_sendint(&buf, atttypmod, sizeof(atttypmod));
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
/* format info appears in protocol 3.0 and up */
if (proto >= 3)
{
if (formats)
pq_sendint(&buf, formats[i], 2);
else
pq_sendint(&buf, 0, 2);
}
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
}
pq_endmessage(&buf);
}
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/*
* Get the lookup info that printtup() needs
*/
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
static void
Another pgindent run. Sorry folks.
1999-05-26 00:43:53 +02:00
printtup_prepare_info(DR_printtup *myState, TupleDesc typeinfo, int numAttrs)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
int16 *formats = myState->portal->formats;
pgindent run over code.
1999-05-25 18:15:34 +02:00
int i;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
/* get rid of any old data */
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
if (myState->myinfo)
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
pfree(myState->myinfo);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
myState->myinfo = NULL;
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
myState->attrinfo = typeinfo;
myState->nattrs = numAttrs;
if (numAttrs <= 0)
return;
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
myState->myinfo = (PrinttupAttrInfo *)
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
palloc0(numAttrs * sizeof(PrinttupAttrInfo));
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
for (i = 0; i < numAttrs; i++)
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
PrinttupAttrInfo *thisState = myState->myinfo + i;
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
int16 format = (formats ? formats[i] : 0);
pgindent run over code.
1999-05-25 18:15:34 +02:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
thisState->format = format;
if (format == 0)
{
getTypeOutputInfo(typeinfo->attrs[i]->atttypid,
&thisState->typoutput,
&thisState->typisvarlena);
Turns out it's easy to cache the fmgr function lookup result, too ... another little bit of speed for SELECT.
1999-01-27 02:11:43 +01:00
fmgr_info(thisState->typoutput, &thisState->finfo);
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
}
else if (format == 1)
{
getTypeBinaryOutputInfo(typeinfo->attrs[i]->atttypid,
&thisState->typsend,
&thisState->typisvarlena);
fmgr_info(thisState->typsend, &thisState->finfo);
}
else
Error message editing in backend/access.
2003-07-21 22:29:40 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unsupported format code: %d", format)));
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
}
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/* ----------------
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* printtup --- print a tuple in protocol 3.0
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
static void
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
printtup(TupleTableSlot *slot, DestReceiver *self)
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
DR_printtup *myState = (DR_printtup *) self;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext oldcontext;
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
StringInfoData buf;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
int i;
/* Set or update my derived attribute info, if needed */
if (myState->attrinfo != typeinfo || myState->nattrs != natts)
printtup_prepare_info(myState, typeinfo, natts);
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
/* Make sure the tuple is fully deconstructed */
slot_getallattrs(slot);
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Switch into per-row context so we can recover memory below */
oldcontext = MemoryContextSwitchTo(myState->tmpcontext);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
/*
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
* Prepare a DataRow message (note buffer is in per-row context)
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
*/
pq_beginmessage(&buf, 'D');
pq_sendint(&buf, natts, 2);
/*
* send the attributes of this tuple
*/
for (i = 0; i < natts; ++i)
{
PrinttupAttrInfo *thisState = myState->myinfo + i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr = slot->tts_values[i];
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
if (slot->tts_isnull[i])
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
{
pq_sendint(&buf, -1, 4);
continue;
}
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/*
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
* Here we catch undefined bytes in datums that are returned to the
Cooperate with the Valgrind instrumentation framework. Valgrind "client requests" in aset.c and mcxt.c teach Valgrind and its Memcheck tool about the PostgreSQL allocator. This makes Valgrind roughly as sensitive to memory errors involving palloc chunks as it is to memory errors involving malloc chunks. Further client requests in PageAddItem() and printtup() verify that all bits being added to a buffer page or furnished to an output function are predictably-defined. Those tests catch failures of C-language functions to fully initialize the bits of a Datum, which in turn stymie optimizations that rely on _equalConst(). Define the USE_VALGRIND symbol in pg_config_manual.h to enable these additions. An included "suppression file" silences nominal errors we don't plan to fix. Reviewed in earlier versions by Peter Geoghegan and Korry Douglas.
2013-06-27 02:00:08 +02:00
* client without hitting disk; see comments at the related check in
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
* PageAddItem(). This test is most useful for uncompressed,
* non-external datums, but we're quite likely to see such here when
* testing new C functions.
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
*/
if (thisState->typisvarlena)
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
VALGRIND_CHECK_MEM_IS_DEFINED(DatumGetPointer(attr),
VARSIZE_ANY(attr));
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
if (thisState->format == 0)
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
{
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/* Text output */
char *outputstr;
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputstr = OutputFunctionCall(&thisState->finfo, attr);
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
pq_sendcountedtext(&buf, outputstr, strlen(outputstr), false);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
}
else
{
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/* Binary output */
bytea *outputbytes;
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputbytes = SendFunctionCall(&thisState->finfo, attr);
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
pq_sendint(&buf, VARSIZE(outputbytes) - VARHDRSZ, 4);
pq_sendbytes(&buf, VARDATA(outputbytes),
VARSIZE(outputbytes) - VARHDRSZ);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
}
}
pq_endmessage(&buf);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Return to caller's context, and flush row's temporary memory */
MemoryContextSwitchTo(oldcontext);
MemoryContextReset(myState->tmpcontext);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
}
/* ----------------
* printtup_20 --- print a tuple in protocol 2.0
* ----------------
*/
static void
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
printtup_20(TupleTableSlot *slot, DestReceiver *self)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
pgindent run over code.
1999-05-25 18:15:34 +02:00
DR_printtup *myState = (DR_printtup *) self;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext oldcontext;
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
StringInfoData buf;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
j,
My first cut at libpq revision didn't handle MULTIBYTE correctly, but I think it's OK now...
1999-04-25 21:27:47 +02:00
k;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* Set or update my derived attribute info, if needed */
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
if (myState->attrinfo != typeinfo || myState->nattrs != natts)
printtup_prepare_info(myState, typeinfo, natts);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
/* Make sure the tuple is fully deconstructed */
slot_getallattrs(slot);
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Switch into per-row context so we can recover memory below */
oldcontext = MemoryContextSwitchTo(myState->tmpcontext);
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* tell the frontend to expect new tuple data (in ASCII style)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
pq_beginmessage(&buf, 'D');
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send a bitmap of which attributes are not null
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
j = 0;
k = 1 << 7;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Recent changes got the sense of the notnull bit backwards in the 2.0 protocol output routines. Mea culpa :-(. Per report from Kris Jurka.
2005-04-23 19:45:35 +02:00
if (!slot->tts_isnull[i])
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
j |= k; /* set bit if not null */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
k >>= 1;
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k == 0) /* end of byte? */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
pq_sendint(&buf, j, 1);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
j = 0;
k = 1 << 7;
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k != (1 << 7)) /* flush last partial byte */
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
pq_sendint(&buf, j, 1);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send the attributes of this tuple
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
PrinttupAttrInfo *thisState = myState->myinfo + i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr = slot->tts_values[i];
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
char *outputstr;
pgindent run over code.
1999-05-25 18:15:34 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
if (slot->tts_isnull[i])
From: Massimo Dal Zotto <dz@cs.unitn.it> After some playing with gdb I found that in printtup() there is a non null attribute with typeinfo->attrs[i]->atttypid = 0 (invalid oid). Unfortunately attibutes with invalid type are neither printed nor marked as null, and this explains why psql doesn't get all the expected data. So I made this patch to printtup():
1998-08-30 21:30:38 +02:00
continue;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
Assert(thisState->format == 0);
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputstr = OutputFunctionCall(&thisState->finfo, attr);
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
pq_sendcountedtext(&buf, outputstr, strlen(outputstr), true);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
pq_endmessage(&buf);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Return to caller's context, and flush row's temporary memory */
MemoryContextSwitchTo(oldcontext);
MemoryContextReset(myState->tmpcontext);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
* printtup_shutdown
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* ----------------
*/
static void
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
printtup_shutdown(DestReceiver *self)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
DR_printtup *myState = (DR_printtup *) self;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
if (myState->myinfo)
pfree(myState->myinfo);
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
myState->myinfo = NULL;
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
myState->attrinfo = NULL;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
if (myState->tmpcontext)
MemoryContextDelete(myState->tmpcontext);
myState->tmpcontext = NULL;
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
}
/* ----------------
* printtup_destroy
* ----------------
*/
static void
printtup_destroy(DestReceiver *self)
{
pfree(self);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/* ----------------
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* printatt
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
static void
printatt(unsigned attributeId,
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
Form_pg_attribute attributeP,
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
char *value)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Pass atttypmod to the frontend.
1998-05-14 19:18:14 +02:00
printf("\t%2d: %s%s%s%s\t(typeid = %u, len = %d, typmod = %d, byval = %c)\n",
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
attributeId,
New NameStr macro to convert Name to Str. No need for var.data anymore. Fewer calls to nameout. Better use of RelationGetRelationName.
1999-11-08 00:08:36 +01:00
NameStr(attributeP->attname),
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
value != NULL ? " = \"" : "",
value != NULL ? value : "",
value != NULL ? "\"" : "",
(unsigned int) (attributeP->atttypid),
attributeP->attlen,
Pass atttypmod to the frontend.
1998-05-14 19:18:14 +02:00
attributeP->atttypmod,
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
attributeP->attbyval ? 't' : 'f');
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/* ----------------
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
* debugStartup - prepare to print tuples for an interactive backend
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
* ----------------
*/
void
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
debugStartup(DestReceiver *self, int operation, TupleDesc typeinfo)
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
{
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
int natts = typeinfo->natts;
Form_pg_attribute *attinfo = typeinfo->attrs;
int i;
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
/*
* show the return type of the tuples
*/
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
for (i = 0; i < natts; ++i)
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
printatt((unsigned) i + 1, attinfo[i], NULL);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
printf("\t----\n");
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
}
/* ----------------
* debugtup - print one tuple for an interactive backend
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
void
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
debugtup(TupleTableSlot *slot, DestReceiver *self)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
pgindent run before 6.3 release, with Thomas' requested changes.
1998-02-26 05:46:47 +01:00
int i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr;
heapattr functions now return a Datum, not char *.
1997-09-12 06:09:08 +02:00
char *value;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
bool isnull;
Change CREATE TYPE to require datatype output and send functions to have only one argument. (Per recent discussion, the option to accept multiple arguments is pretty useless for user-defined types, and would be a likely source of security holes if it was used.) Simplify call sites of output/send functions to not bother passing more than one argument.
2005-05-01 20:56:19 +02:00
Oid typoutput;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
bool typisvarlena;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
attr = slot_getattr(slot, i + 1, &isnull);
Replace typtoout() and gettypelem() with a single routine, so that fetching an attribute value needs only one SearchSysCacheTuple call instead of two redundant searches. This speeds up a large SELECT by about ten percent, and probably will help GROUP BY and SELECT DISTINCT too.
1999-01-24 06:40:49 +01:00
if (isnull)
continue;
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
getTypeOutputInfo(typeinfo->attrs[i]->atttypid,
Change CREATE TYPE to require datatype output and send functions to have only one argument. (Per recent discussion, the option to accept multiple arguments is pretty useless for user-defined types, and would be a likely source of security holes if it was used.) Simplify call sites of output/send functions to not bother passing more than one argument.
2005-05-01 20:56:19 +02:00
&typoutput, &typisvarlena);
pgindent run.
2003-08-04 02:43:34 +02:00
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
value = OidOutputFunctionCall(typoutput, attr);
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
printatt((unsigned) i + 1, typeinfo->attrs[i], value);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
printf("\t----\n");
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/* ----------------
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* printtup_internal_20 --- print a binary tuple in protocol 2.0
*
* We use a different message type, i.e. 'B' instead of 'D' to
* indicate a tuple in internal (binary) form.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
* This is largely same as printtup_20, except we use binary formatting.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
static void
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
printtup_internal_20(TupleTableSlot *slot, DestReceiver *self)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
DR_printtup *myState = (DR_printtup *) self;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext oldcontext;
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
StringInfoData buf;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
j,
k;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
/* Set or update my derived attribute info, if needed */
if (myState->attrinfo != typeinfo || myState->nattrs != natts)
printtup_prepare_info(myState, typeinfo, natts);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
/* Make sure the tuple is fully deconstructed */
slot_getallattrs(slot);
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Switch into per-row context so we can recover memory below */
oldcontext = MemoryContextSwitchTo(myState->tmpcontext);
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* tell the frontend to expect new tuple data (in binary style)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
pq_beginmessage(&buf, 'B');
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send a bitmap of which attributes are not null
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
j = 0;
k = 1 << 7;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Recent changes got the sense of the notnull bit backwards in the 2.0 protocol output routines. Mea culpa :-(. Per report from Kris Jurka.
2005-04-23 19:45:35 +02:00
if (!slot->tts_isnull[i])
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
j |= k; /* set bit if not null */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
k >>= 1;
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k == 0) /* end of byte? */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
pq_sendint(&buf, j, 1);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
j = 0;
k = 1 << 7;
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k != (1 << 7)) /* flush last partial byte */
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
pq_sendint(&buf, j, 1);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send the attributes of this tuple
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
PrinttupAttrInfo *thisState = myState->myinfo + i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr = slot->tts_values[i];
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
bytea *outputbytes;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
if (slot->tts_isnull[i])
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
continue;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
Assert(thisState->format == 1);
Included patches should fix following problems in the muti-byte enabled PostgreSQL 6.4. o binary cursor does not work o pg_dumpall produces incorrect create database statemnt Tatsuo Ishii t-ishii@sra.co.jp
1998-12-12 23:04:09 +01:00
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputbytes = SendFunctionCall(&thisState->finfo, attr);
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
pq_sendint(&buf, VARSIZE(outputbytes) - VARHDRSZ, 4);
pq_sendbytes(&buf, VARDATA(outputbytes),
VARSIZE(outputbytes) - VARHDRSZ);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
pq_endmessage(&buf);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Return to caller's context, and flush row's temporary memory */
MemoryContextSwitchTo(oldcontext);
MemoryContextReset(myState->tmpcontext);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Reference in New Issue Copy Permalink