postgresql/src/backend
Tom Lane 2f48ede080 Avoid using a cursor in plpgsql's RETURN QUERY statement.
plpgsql has always executed the query given in a RETURN QUERY command
by opening it as a cursor and then fetching a few rows at a time,
which it turns around and dumps into the function's result tuplestore.
The point of this was to keep from blowing out memory with an oversized
SPITupleTable result (note that while a tuplestore can spill tuples
to disk, SPITupleTable cannot).  However, it's rather inefficient, both
because of extra data copying and because of executor entry/exit
overhead.  In recent versions, a new performance problem has emerged:
use of a cursor prevents use of a parallel plan for the executed query.

We can improve matters by skipping use of a cursor and having the
executor push result tuples directly into the function's result
tuplestore.  However, a moderate amount of new infrastructure is needed
to make that idea work:

* We can use the existing tstoreReceiver.c DestReceiver code to funnel
executor output to the tuplestore, but it has to be extended to support
plpgsql's requirement for possibly applying a tuple conversion map.

* SPI needs to be extended to allow use of a caller-supplied
DestReceiver instead of its usual receiver that puts tuples into
a SPITupleTable.  Two new API calls are needed to handle both the
RETURN QUERY and RETURN QUERY EXECUTE cases.

I also felt that I didn't want these new API calls to use the legacy
method of specifying query parameter values with "char" null flags
(the old ' '/'n' convention); rather they should accept ParamListInfo
objects containing the parameter type and value info.  This required
a bit of additional new infrastructure since we didn't yet have any
parse analysis callback that would interpret $N parameter symbols
according to type data supplied in a ParamListInfo.  There seems to be
no harm in letting makeParamList install that callback by default,
rather than leaving a new ParamListInfo's parserSetup hook as NULL.
(Indeed, as of HEAD, I couldn't find anyplace that was using the
parserSetup field at all; plpgsql was using parserSetupArg for its
own purposes, but parserSetup seemed to be write-only.)

We can actually get plpgsql out of the business of using legacy null
flags altogether, and using ParamListInfo instead of its ad-hoc
PreparedParamsData structure; but this requires inventing one more
SPI API call that can replace SPI_cursor_open_with_args.  That seems
worth doing, though.

SPI_execute_with_args and SPI_cursor_open_with_args are now unused
anywhere in the core PG distribution.  Perhaps someday we could
deprecate/remove them.  But cleaning up the crufty bits of the SPI
API is a task for a different patch.

Per bug #16040 from Jeremy Smith.  This is unfortunately too invasive to
consider back-patching.  Patch by me; thanks to Hamid Akhtar for review.

Discussion: https://postgr.es/m/16040-eaacad11fecfb198@postgresql.org
2020-06-12 12:14:32 -04:00
..
access Improve comments for [Heap]CheckForSerializableConflictOut(). 2020-06-12 10:55:38 +12:00
bootstrap Skip WAL for new relfilenodes, under wal_level=minimal. 2020-04-04 12:25:34 -07:00
catalog Make more use of RELKIND_HAS_STORAGE() 2020-06-12 09:10:26 +02:00
commands Avoid using a cursor in plpgsql's RETURN QUERY statement. 2020-06-12 12:14:32 -04:00
executor Avoid using a cursor in plpgsql's RETURN QUERY statement. 2020-06-12 12:14:32 -04:00
foreign Update copyrights for 2020 2020-01-01 12:21:45 -05:00
jit pgindent run prior to branching v13. 2020-06-07 16:57:08 -04:00
lib Move src/backend/utils/hash/hashfn.c to src/common 2020-02-27 09:25:41 +05:30
libpq Fix comment in be-secure-openssl.c 2020-06-04 13:02:59 +09:00
main Add PostgreSQL home page to --help output 2020-02-28 13:12:21 +01:00
nodes Avoid using a cursor in plpgsql's RETURN QUERY statement. 2020-06-12 12:14:32 -04:00
optimizer Rework HashAgg GUCs. 2020-06-11 12:57:43 -07:00
parser Refactor DROP LANGUAGE grammar 2020-06-11 11:18:15 +02:00
partitioning Fix two typos in a comment 2020-05-22 17:39:16 -04:00
po Translation updates 2020-05-18 12:49:30 +02:00
port Spelling adjustments 2020-06-07 15:06:51 +02:00
postmaster Make more use of RELKIND_HAS_STORAGE() 2020-06-12 09:10:26 +02:00
regex Dial back -Wimplicit-fallthrough to level 3 2020-05-13 15:31:14 -04:00
replication Fix typos and some format mistakes in comments 2020-06-12 21:05:10 +09:00
rewrite Add missing invocations to object access hooks 2020-05-23 14:03:04 +09:00
snowball Update snowball 2020-06-08 08:07:15 +02:00
statistics Run pgindent with new pg_bsd_indent version 2.1.1. 2020-05-16 11:54:51 -04:00
storage Improve comments for [Heap]CheckForSerializableConflictOut(). 2020-06-12 10:55:38 +12:00
tcop Avoid using a cursor in plpgsql's RETURN QUERY statement. 2020-06-12 12:14:32 -04:00
tsearch Further cleanup of ts_headline code. 2020-04-09 15:38:43 -04:00
utils Fix typos and some format mistakes in comments 2020-06-12 21:05:10 +09:00
.gitignore Add .gitignore entries for AIX-specific intermediate build artifacts. 2015-07-08 20:44:22 -04:00
common.mk Remove PARTIAL_LINKING build mode. 2018-03-30 17:33:04 -07:00
Makefile Update copyrights for 2020 2020-01-01 12:21:45 -05:00
nls.mk Add missing gettext triggers 2020-04-28 13:35:40 +02:00