rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/include/utils/tuplestore.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

92 lines
3.3 KiB
C
Raw Normal View History

Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
/*-------------------------------------------------------------------------
*
* tuplestore.h
* Generalized routines for temporary tuple storage.
*
* This module handles temporary storage of tuples for purposes such
* as Materialize nodes, hashjoin batch files, etc. It is essentially
* a dumbed-down version of tuplesort.c; it does no sorting of tuples
Revise tuplestore and nodeMaterial so that we don't have to read the entire contents of the subplan into the tuplestore before we can return any tuples. Instead, the tuplestore holds what we've already read, and we fetch additional rows from the subplan as needed. Random access to the previously-read rows works with the tuplestore, and doesn't affect the state of the partially-read subplan. This is a step towards fixing the problems with cursors over complex queries --- we don't want to stick in Materialize nodes if they'll prevent quick startup for a cursor.
2003-03-09 03:19:13 +01:00
* but can only store and regurgitate a sequence of tuples. However,
* because no sort is required, it is allowed to start reading the sequence
* before it has all been written. This is particularly useful for cursors,
* because it allows random access within the already-scanned portion of
* a query without having to process the underlying scan to completion.
Improve tuplestore.c to support multiple concurrent read positions. This facility replaces the former mark/restore support but is otherwise upward-compatible with previous uses. It's expected to be needed for single evaluation of CTEs and also for window functions, so I'm committing it separately instead of waiting for either one of those patches to be finished. Per discussion with Greg Stark and Hitoshi Harada. Note: I removed nodeFunctionscan's mark/restore support, instead of bothering to update it for this change, because it was dead code anyway.
2008-10-01 21:51:50 +02:00
* Also, it is possible to support multiple independent read pointers.
*
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
* A temporary file is used to handle the data if it exceeds the
* space limit specified by the caller.
*
Create infrastructure for 'MinimalTuple' representation of in-memory tuples with less header overhead than a regular HeapTuple, per my recent proposal. Teach TupleTableSlot code how to deal with these. As proof of concept, change tuplestore.c to store MinimalTuples instead of HeapTuples. Future patches will expand the concept to other places where it is useful.
2006-06-27 04:51:40 +02:00
* Beginning in Postgres 8.2, what is stored is just MinimalTuples;
* callers cannot expect valid system columns in regurgitated tuples.
* Also, we have changed the API to return tuples in TupleTableSlots,
* so that there is a check to prevent attempted access to system columns.
*
Update copyright for 2022 Backpatch-through: 10
2022-01-08 01:04:57 +01:00
* Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
* Portions Copyright (c) 1994, Regents of the University of California
*
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/include/utils/tuplestore.h
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
*
*-------------------------------------------------------------------------
*/
#ifndef TUPLESTORE_H
#define TUPLESTORE_H
Create infrastructure for 'MinimalTuple' representation of in-memory tuples with less header overhead than a regular HeapTuple, per my recent proposal. Teach TupleTableSlot code how to deal with these. As proof of concept, change tuplestore.c to store MinimalTuples instead of HeapTuples. Future patches will expand the concept to other places where it is useful.
2006-06-27 04:51:40 +02:00
#include "executor/tuptable.h"
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
/* Tuplestorestate is an opaque type whose details are not known outside
* tuplestore.c.
*/
typedef struct Tuplestorestate Tuplestorestate;
/*
Create infrastructure for 'MinimalTuple' representation of in-memory tuples with less header overhead than a regular HeapTuple, per my recent proposal. Teach TupleTableSlot code how to deal with these. As proof of concept, change tuplestore.c to store MinimalTuples instead of HeapTuples. Future patches will expand the concept to other places where it is useful.
2006-06-27 04:51:40 +02:00
* Currently we only need to store MinimalTuples, but it would be easy
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
* to support the same behavior for IndexTuples and/or bare Datums.
*/
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
Code review for holdable-cursors patch. Fix error recovery, memory context sloppiness, some other things. Includes Neil's mopup patch of 22-Apr.
2003-04-29 05:21:30 +02:00
bool interXact,
This patch implements holdable cursors, following the proposal (materialization into a tuple store) discussed on pgsql-hackers earlier. I've updated the documentation and the regression tests. Notes on the implementation: - I needed to change the tuple store API slightly -- it assumes that it won't be used to hold data across transaction boundaries, so the temp files that it uses for on-disk storage are automatically reclaimed at end-of-transaction. I added a flag to tuplestore_begin_heap() to control this behavior. Is changing the tuple store API in this fashion OK? - in order to store executor results in a tuple store, I added a new CommandDest. This works well for the most part, with one exception: the current DestFunction API doesn't provide enough information to allow the Executor to store results into an arbitrary tuple store (where the particular tuple store to use is chosen by the call site of ExecutorRun). To workaround this, I've temporarily hacked up a solution that works, but is not ideal: since the receiveTuple DestFunction is passed the portal name, we can use that to lookup the Portal data structure for the cursor and then use that to get at the tuple store the Portal is using. This unnecessarily ties the Portal code with the tupleReceiver code, but it works... The proper fix for this is probably to change the DestFunction API -- Tom suggested passing the full QueryDesc to the receiveTuple function. In that case, callers of ExecutorRun could "subclass" QueryDesc to add any additional fields that their particular CommandDest needed to get access to. This approach would work, but I'd like to think about it for a little bit longer before deciding which route to go. In the mean time, the code works fine, so I don't think a fix is urgent. - (semi-related) I added a NO SCROLL keyword to DECLARE CURSOR, and adjusted the behavior of SCROLL in accordance with the discussion on -hackers. - (unrelated) Cleaned up some SGML markup in sql.sgml, copy.sgml Neil Conway
2003-03-27 17:51:29 +01:00
int maxKBytes);
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
Teach tuplestore.c to throw away data before the "mark" point when the caller is using mark/restore but not rewind or backward-scan capability. Insert a materialize plan node between a mergejoin and its inner child if the inner child is a sort that is expected to spill to disk. The materialize shields the sort from the need to do mark/restore and thereby allows it to perform its final merge pass on-the-fly; while the materialize itself is normally cheap since it won't spill to disk unless the number of tuples with equal key values exceeds work_mem. Greg Stark, with some kibitzing from Tom Lane.
2007-05-21 19:57:35 +02:00
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
Create infrastructure for 'MinimalTuple' representation of in-memory tuples with less header overhead than a regular HeapTuple, per my recent proposal. Teach TupleTableSlot code how to deal with these. As proof of concept, change tuplestore.c to store MinimalTuples instead of HeapTuples. Future patches will expand the concept to other places where it is useful.
2006-06-27 04:51:40 +02:00
extern void tuplestore_puttupleslot(Tuplestorestate *state,
TupleTableSlot *slot);
extern void tuplestore_puttuple(Tuplestorestate *state, HeapTuple tuple);
Add a new tuplestore API function, tuplestore_putvalues(). This is identical to tuplestore_puttuple(), except it operates on arrays of Datums + nulls rather than a fully-formed HeapTuple. In several places that use the tuplestore API, this means we can avoid creating a HeapTuple altogether, saving a copy.
2008-03-25 20:26:54 +01:00
extern void tuplestore_putvalues(Tuplestorestate *state, TupleDesc tdesc,
Datum *values, bool *isnull);
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
Remove all traces of tuplestore_donestoring() in the C code This routine is a no-op since dd04e95 from 2003, with a macro kept around for compatibility purposes. This has led to the same code patterns being copy-pasted around for no effect, sometimes in confusing ways like in pg_logical_slot_get_changes_guts() from logical.c where the code was actually incorrect. This issue has been discussed on two different threads recently, so rather than living with this legacy, remove any uses of this routine in the C code to simplify things. The compatibility macro is kept to avoid breaking any out-of-core modules that depend on it. Reported-by: Tatsuhito Kasahara, Justin Pryzby Author: Tatsuhito Kasahara Discussion: https://postgr.es/m/20211217200419.GQ17618@telsasoft.com Discussion: https://postgr.es/m/CAP0=ZVJeeYfAeRfmzqAF2Lumdiv4S4FewyBnZd4DPTrsSQKJKw@mail.gmail.com
2022-02-17 01:52:02 +01:00
/* Backwards compatibility macro */
tuplestore_donestoring() isn't needed anymore, but provide a no-op macro definition so as not to create compatibility problems.
2003-03-09 04:34:10 +01:00
#define tuplestore_donestoring(state) ((void) 0)
Improve tuplestore.c to support multiple concurrent read positions. This facility replaces the former mark/restore support but is otherwise upward-compatible with previous uses. It's expected to be needed for single evaluation of CTEs and also for window functions, so I'm committing it separately instead of waiting for either one of those patches to be finished. Per discussion with Greg Stark and Hitoshi Harada. Note: I removed nodeFunctionscan's mark/restore support, instead of bothering to update it for this change, because it was dead code anyway.
2008-10-01 21:51:50 +02:00
extern int tuplestore_alloc_read_pointer(Tuplestorestate *state, int eflags);
extern void tuplestore_select_read_pointer(Tuplestorestate *state, int ptr);
extern void tuplestore_copy_read_pointer(Tuplestorestate *state,
int srcptr, int destptr);
Make a couple of small changes to the tuplestore API, for the benefit of the upcoming window-functions patch. First, tuplestore_trim is now an exported function that must be explicitly invoked by callers at appropriate times, rather than something that tuplestore tries to do behind the scenes. Second, a read pointer that is marked as allowing backward scan no longer prevents truncation. This means that a read pointer marked as having BACKWARD but not REWIND capability can only safely read backwards as far as the oldest other read pointer. (The expected use pattern for this involves having another read pointer that serves as the truncation fencepost.)
2008-12-27 18:39:00 +01:00
extern void tuplestore_trim(Tuplestorestate *state);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
extern bool tuplestore_in_memory(Tuplestorestate *state);
Create infrastructure for 'MinimalTuple' representation of in-memory tuples with less header overhead than a regular HeapTuple, per my recent proposal. Teach TupleTableSlot code how to deal with these. As proof of concept, change tuplestore.c to store MinimalTuples instead of HeapTuples. Future patches will expand the concept to other places where it is useful.
2006-06-27 04:51:40 +02:00
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
Fix possible failures when a tuplestore switches from in-memory to on-disk mode while callers hold pointers to in-memory tuples. I reported this for the case of nodeWindowAgg's primary scan tuple, but inspection of the code shows that all of the calls in nodeWindowAgg and nodeCtescan are at risk. For the moment, fix it with a rather brute-force approach of copying whenever one of the at-risk callers requests a tuple. Later we might think of some sort of reference-count approach to reduce tuple copying.
2009-03-27 19:30:21 +01:00
bool copy, TupleTableSlot *slot);
Improve some O(N^2) behavior in window function evaluation. Repositioning the tuplestore seek pointer in window_gettupleslot() turns out to be a very significant expense when the window frame is sizable and the frame end can move. To fix, introduce a tuplestore function for skipping an arbitrary number of tuples in one call, parallel to the one we introduced for tuplesort objects in commit 8d65da1f. This reduces the cost of window_gettupleslot() to O(1) if the tuplestore has not spilled to disk. As in the previous commit, I didn't try to do any real optimization of tuplestore_skiptuples for the case where the tuplestore has spilled to disk. There is probably no practical way to get the cost to less than O(N) anyway, but perhaps someone can think of something later. Also fix PersistHoldablePortal() to make use of this API now that we have it. Based on a suggestion by Dean Rasheed, though this turns out not to look much like his patch.
2014-04-13 19:59:17 +02:00
Create infrastructure for 'MinimalTuple' representation of in-memory tuples with less header overhead than a regular HeapTuple, per my recent proposal. Teach TupleTableSlot code how to deal with these. As proof of concept, change tuplestore.c to store MinimalTuples instead of HeapTuples. Future patches will expand the concept to other places where it is useful.
2006-06-27 04:51:40 +02:00
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
Improve some O(N^2) behavior in window function evaluation. Repositioning the tuplestore seek pointer in window_gettupleslot() turns out to be a very significant expense when the window frame is sizable and the frame end can move. To fix, introduce a tuplestore function for skipping an arbitrary number of tuples in one call, parallel to the one we introduced for tuplesort objects in commit 8d65da1f. This reduces the cost of window_gettupleslot() to O(1) if the tuplestore has not spilled to disk. As in the previous commit, I didn't try to do any real optimization of tuplestore_skiptuples for the case where the tuplestore has spilled to disk. There is probably no practical way to get the cost to less than O(N) anyway, but perhaps someone can think of something later. Also fix PersistHoldablePortal() to make use of this API now that we have it. Based on a suggestion by Dean Rasheed, though this turns out not to look much like his patch.
2014-04-13 19:59:17 +02:00
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
Add infrastructure to support EphemeralNamedRelation references. A QueryEnvironment concept is added, which allows new types of objects to be passed into queries from parsing on through execution. At this point, the only thing implemented is a collection of EphemeralNamedRelation objects -- relations which can be referenced by name in queries, but do not exist in the catalogs. The only type of ENR implemented is NamedTuplestore, but provision is made to add more types fairly easily. An ENR can carry its own TupleDesc or reference a relation in the catalogs by relid. Although these features can be used without SPI, convenience functions are added to SPI so that ENRs can easily be used by code run through SPI. The initial use of all this is going to be transition tables in AFTER triggers, but that will be added to each PL as a separate commit. An incidental effect of this patch is to produce a more informative error message if an attempt is made to modify the contents of a CTE from a referencing DML statement. No tests previously covered that possibility, so one is added. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others
2017-04-01 06:17:18 +02:00
extern int64 tuplestore_tuple_count(Tuplestorestate *state);
Revise tuplestore and nodeMaterial so that we don't have to read the entire contents of the subplan into the tuplestore before we can return any tuples. Instead, the tuplestore holds what we've already read, and we fetch additional rows from the subplan as needed. Random access to the previously-read rows works with the tuplestore, and doesn't affect the state of the partially-read subplan. This is a step towards fixing the problems with cursors over complex queries --- we don't want to stick in Materialize nodes if they'll prevent quick startup for a cursor.
2003-03-09 03:19:13 +01:00
extern bool tuplestore_ateof(Tuplestorestate *state);
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
extern void tuplestore_rescan(Tuplestorestate *state);
Improve tuplestore.c to support multiple concurrent read positions. This facility replaces the former mark/restore support but is otherwise upward-compatible with previous uses. It's expected to be needed for single evaluation of CTEs and also for window functions, so I'm committing it separately instead of waiting for either one of those patches to be finished. Per discussion with Greg Stark and Hitoshi Harada. Note: I removed nodeFunctionscan's mark/restore support, instead of bothering to update it for this change, because it was dead code anyway.
2008-10-01 21:51:50 +02:00
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
extern void tuplestore_clear(Tuplestorestate *state);
Improve tuplestore.c to support multiple concurrent read positions. This facility replaces the former mark/restore support but is otherwise upward-compatible with previous uses. It's expected to be needed for single evaluation of CTEs and also for window functions, so I'm committing it separately instead of waiting for either one of those patches to be finished. Per discussion with Greg Stark and Hitoshi Harada. Note: I removed nodeFunctionscan's mark/restore support, instead of bothering to update it for this change, because it was dead code anyway.
2008-10-01 21:51:50 +02:00
extern void tuplestore_end(Tuplestorestate *state);
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
Reimplement nodeMaterial to use a temporary BufFile (or even memory, if the materialized tupleset is small enough) instead of a temporary relation. This was something I was thinking of doing anyway for performance, and Jan says he needs it for TOAST because he doesn't want to cope with toasting noname relations. With this change, the 'noname table' support in heap.c is dead code, and I have accordingly removed it. Also clean up 'noname' plan handling in planner --- nonames are either sort or materialize plans, and it seems less confusing to handle them separately under those names.
2000-06-19 00:44:35 +02:00
#endif /* TUPLESTORE_H */