rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/executor/nodeBitmapIndexscan.c

327 lines
9.2 KiB
C
Raw Normal View History

Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*-------------------------------------------------------------------------
*
* nodeBitmapIndexscan.c
* Routines to support bitmapped index scans of relations
*
* Portions Copyright (c) 1996-2005, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
*
* IDENTIFICATION
Tweak indexscan machinery to avoid taking an AccessShareLock on an index if we already have a stronger lock due to the index's table being the update target table of the query. Same optimization I applied earlier at the table level. There doesn't seem to be much interest in the more radical idea of not locking indexes at all, so do what we can ...
2005-12-03 06:51:03 +01:00
* $PostgreSQL: pgsql/src/backend/executor/nodeBitmapIndexscan.c,v 1.14 2005/12/03 05:51:01 tgl Exp $
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*
*-------------------------------------------------------------------------
*/
/*
* INTERFACE ROUTINES
* MultiExecBitmapIndexScan scans a relation using index.
* ExecInitBitmapIndexScan creates and initializes state info.
* ExecBitmapIndexReScan prepares to rescan the plan.
* ExecEndBitmapIndexScan releases all storage.
*/
#include "postgres.h"
#include "access/genam.h"
#include "executor/execdebug.h"
#include "executor/instrument.h"
#include "executor/nodeBitmapIndexscan.h"
Remove support for OR'd indexscans internal to a single IndexScan plan node, as this behavior is now better done as a bitmap OR indexscan. This allows considerable simplification in nodeIndexscan.c itself as well as several planner modules concerned with indexscan plan generation. Also we can improve the sharing of code between regular and bitmap indexscans, since they are now working with nigh-identical Plan nodes.
2005-04-25 03:30:14 +02:00
#include "executor/nodeIndexscan.h"
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
#include "miscadmin.h"
For some reason access/tupmacs.h has been #including utils/memutils.h, which is neither needed by nor related to that header. Remove the bogus inclusion and instead include the header in those C files that actually need it. Also fix unnecessary inclusions and bad inclusion order in tsearch2 files.
2005-05-06 19:24:55 +02:00
#include "utils/memutils.h"
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/* ----------------------------------------------------------------
* MultiExecBitmapIndexScan(node)
* ----------------------------------------------------------------
*/
Node *
MultiExecBitmapIndexScan(BitmapIndexScanState *node)
{
#define MAX_TIDS 1024
TIDBitmap *tbm;
IndexScanDesc scandesc;
ItemPointerData tids[MAX_TIDS];
int32 ntids;
double nTuples = 0;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
bool doscan;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/* must provide our own instrumentation support */
if (node->ss.ps.instrument)
InstrStartNode(node->ss.ps.instrument);
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
/*
* extract necessary information from index scan node
*/
scandesc = node->biss_ScanDesc;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we have runtime keys and they've not already been set up, do it now.
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
* Array keys are also treated as runtime keys; note that if ExecReScan
* returns with biss_RuntimeKeysReady still false, then there is an
* empty array key so we should do nothing.
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (!node->biss_RuntimeKeysReady &&
(node->biss_NumRuntimeKeys != 0 || node->biss_NumArrayKeys != 0))
{
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
ExecReScan((PlanState *) node, NULL);
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
doscan = node->biss_RuntimeKeysReady;
}
else
doscan = true;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
Minor performance improvement: avoid unnecessary creation/unioning of bitmaps for multiple indexscans. Instead just let each indexscan add TIDs directly into the BitmapOr node's result bitmap.
2005-04-20 17:48:36 +02:00
* Prepare the result bitmap. Normally we just create a new one to pass
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* back; however, our parent node is allowed to store a pre-made one into
* node->biss_result, in which case we just OR our tuple IDs into the
* existing bitmap. (This saves needing explicit UNION steps.)
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
Minor performance improvement: avoid unnecessary creation/unioning of bitmaps for multiple indexscans. Instead just let each indexscan add TIDs directly into the BitmapOr node's result bitmap.
2005-04-20 17:48:36 +02:00
if (node->biss_result)
{
tbm = node->biss_result;
node->biss_result = NULL; /* reset for next time */
}
else
{
/* XXX should we use less than work_mem for this? */
tbm = tbm_create(work_mem * 1024L);
}
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
* Get TIDs from index and insert into bitmap
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
while (doscan)
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
bool more = index_getmulti(scandesc, tids, MAX_TIDS, &ntids);
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
if (ntids > 0)
{
tbm_add_tuples(tbm, tids, ntids);
nTuples += ntids;
}
CHECK_FOR_INTERRUPTS();
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (!more)
{
doscan = ExecIndexAdvanceArrayKeys(node->biss_ArrayKeys,
node->biss_NumArrayKeys);
if (doscan) /* reset index scan */
index_rescan(node->biss_ScanDesc, node->biss_ScanKeys);
}
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
}
/* must provide our own instrumentation support */
if (node->ss.ps.instrument)
InstrStopNodeMulti(node->ss.ps.instrument, nTuples);
return (Node *) tbm;
}
/* ----------------------------------------------------------------
* ExecBitmapIndexReScan(node)
*
* Recalculates the value of the scan keys whose value depends on
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
* information known at runtime and rescans the indexed relation.
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
* ----------------------------------------------------------------
*/
void
ExecBitmapIndexReScan(BitmapIndexScanState *node, ExprContext *exprCtxt)
{
ExprContext *econtext;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
econtext = node->biss_RuntimeContext; /* context for runtime keys */
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
if (econtext)
{
/*
* If we are being passed an outer tuple, save it for runtime key
Actually, nodeBitmapIndexscan.c doesn't need to create a standard ExprContext at all, since it never evaluates any qual or tlist expressions.
2005-04-24 19:32:46 +02:00
* calc.
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
if (exprCtxt != NULL)
econtext->ecxt_outertuple = exprCtxt->ecxt_outertuple;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Reset the runtime-key context so we don't leak memory as each outer
* tuple is scanned. Note this assumes that we will recalculate *all*
* runtime keys on each call.
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
ResetExprContext(econtext);
}
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we are doing runtime key calculations (ie, the index keys depend on
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
* data from an outer scan), compute the new key values.
*
* Array keys are also treated as runtime keys; note that if we
* return with biss_RuntimeKeysReady still false, then there is an
* empty array key so no index scan is needed.
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (node->biss_NumRuntimeKeys != 0)
Remove support for OR'd indexscans internal to a single IndexScan plan node, as this behavior is now better done as a bitmap OR indexscan. This allows considerable simplification in nodeIndexscan.c itself as well as several planner modules concerned with indexscan plan generation. Also we can improve the sharing of code between regular and bitmap indexscans, since they are now working with nigh-identical Plan nodes.
2005-04-25 03:30:14 +02:00
ExecIndexEvalRuntimeKeys(econtext,
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
node->biss_RuntimeKeys,
node->biss_NumRuntimeKeys);
if (node->biss_NumArrayKeys != 0)
node->biss_RuntimeKeysReady =
ExecIndexEvalArrayKeys(econtext,
node->biss_ArrayKeys,
node->biss_NumArrayKeys);
else
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
node->biss_RuntimeKeysReady = true;
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
/* reset index scan */
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (node->biss_RuntimeKeysReady)
index_rescan(node->biss_ScanDesc, node->biss_ScanKeys);
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
}
/* ----------------------------------------------------------------
* ExecEndBitmapIndexScan
* ----------------------------------------------------------------
*/
void
ExecEndBitmapIndexScan(BitmapIndexScanState *node)
{
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
Relation indexRelationDesc;
IndexScanDesc indexScanDesc;
/*
* extract information from the node
*/
indexRelationDesc = node->biss_RelationDesc;
indexScanDesc = node->biss_ScanDesc;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
Actually, nodeBitmapIndexscan.c doesn't need to create a standard ExprContext at all, since it never evaluates any qual or tlist expressions.
2005-04-24 19:32:46 +02:00
* Free the exprcontext ... now dead code, see ExecFreeExprContext
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
Remove explicit FreeExprContext calls during plan node shutdown. The ExprContexts will be freed anyway when FreeExecutorState() is reached, and letting that routine do the work is more efficient because it will automatically free the ExprContexts in reverse creation order. The existing coding was effectively freeing them in exactly the worst possible order, resulting in O(N^2) behavior inside list_delete_ptr, which becomes highly visible in cases with a few thousand plan nodes. ExecFreeExprContext is now effectively a no-op and could be removed, but I left it in place in case we ever want to put it back to use.
2005-04-23 23:32:34 +02:00
#ifdef NOT_USED
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
if (node->biss_RuntimeContext)
FreeExprContext(node->biss_RuntimeContext);
Remove explicit FreeExprContext calls during plan node shutdown. The ExprContexts will be freed anyway when FreeExecutorState() is reached, and letting that routine do the work is more efficient because it will automatically free the ExprContexts in reverse creation order. The existing coding was effectively freeing them in exactly the worst possible order, resulting in O(N^2) behavior inside list_delete_ptr, which becomes highly visible in cases with a few thousand plan nodes. ExecFreeExprContext is now effectively a no-op and could be removed, but I left it in place in case we ever want to put it back to use.
2005-04-23 23:32:34 +02:00
#endif
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
/*
* close the index relation
*/
index_endscan(indexScanDesc);
index_close(indexRelationDesc);
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
}
/* ----------------------------------------------------------------
* ExecInitBitmapIndexScan
*
Adjust nodeBitmapIndexscan.c to not keep the index open across calls, but just to open and close it during MultiExecBitmapIndexScan. This avoids acquiring duplicate resources (eg, multiple locks on the same relation) in a tree with many bitmap scans. Also, don't bother to lock the parent heap at all here, since we must be underneath a BitmapHeapScan node that will be holding a suitable lock.
2005-04-24 20:16:38 +02:00
* Initializes the index scan's state information.
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
* ----------------------------------------------------------------
*/
BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate)
{
BitmapIndexScanState *indexstate;
Tweak indexscan machinery to avoid taking an AccessShareLock on an index if we already have a stronger lock due to the index's table being the update target table of the query. Same optimization I applied earlier at the table level. There doesn't seem to be much interest in the more radical idea of not locking indexes at all, so do what we can ...
2005-12-03 06:51:03 +01:00
bool relistarget;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
* create state structure
*/
indexstate = makeNode(BitmapIndexScanState);
indexstate->ss.ps.plan = (Plan *) node;
indexstate->ss.ps.state = estate;
Minor performance improvement: avoid unnecessary creation/unioning of bitmaps for multiple indexscans. Instead just let each indexscan add TIDs directly into the BitmapOr node's result bitmap.
2005-04-20 17:48:36 +02:00
/* normally we don't make the result bitmap till runtime */
indexstate->biss_result = NULL;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
* Miscellaneous initialization
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* We do not need a standard exprcontext for this node, though we may
* decide below to create a runtime-key exprcontext
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
/*
* initialize child expressions
*
* We don't need to initialize targetlist or qual since neither are used.
*
Remove support for OR'd indexscans internal to a single IndexScan plan node, as this behavior is now better done as a bitmap OR indexscan. This allows considerable simplification in nodeIndexscan.c itself as well as several planner modules concerned with indexscan plan generation. Also we can improve the sharing of code between regular and bitmap indexscans, since they are now working with nigh-identical Plan nodes.
2005-04-25 03:30:14 +02:00
* Note: we don't initialize all of the indexqual expression, only the
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
* sub-parts corresponding to runtime keys (see below).
*/
#define BITMAPINDEXSCAN_NSLOTS 0
/*
* Initialize index-specific scan state
*/
indexstate->biss_RuntimeKeysReady = false;
CXT1_printf("ExecInitBitmapIndexScan: context is %d\n", CurrentMemoryContext);
/*
* build the index scan keys from the index qualification
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
ExecIndexBuildScanKeys((PlanState *) indexstate,
node->indexqual,
node->indexstrategy,
node->indexsubtype,
&indexstate->biss_ScanKeys,
&indexstate->biss_NumScanKeys,
&indexstate->biss_RuntimeKeys,
&indexstate->biss_NumRuntimeKeys,
&indexstate->biss_ArrayKeys,
&indexstate->biss_NumArrayKeys);
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
* If we have runtime keys or array keys, we need an ExprContext to
* evaluate them. We could just create a "standard" plan node exprcontext,
* but to keep the code looking similar to nodeIndexscan.c, it seems
* better to stick with the approach of using a separate ExprContext.
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (indexstate->biss_NumRuntimeKeys != 0 ||
indexstate->biss_NumArrayKeys != 0)
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
{
ExprContext *stdecontext = indexstate->ss.ps.ps_ExprContext;
ExecAssignExprContext(estate, &indexstate->ss.ps);
indexstate->biss_RuntimeContext = indexstate->ss.ps.ps_ExprContext;
indexstate->ss.ps.ps_ExprContext = stdecontext;
}
else
{
indexstate->biss_RuntimeContext = NULL;
}
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
/*
* We do not open or lock the base relation here. We assume that an
Adjust scan plan nodes to avoid getting an extra AccessShareLock on a relation if it's already been locked by execMain.c as either a result relation or a FOR UPDATE/SHARE relation. This avoids an extra trip to the shared lock manager state. Per my suggestion yesterday.
2005-12-02 21:03:42 +01:00
* ancestor BitmapHeapScan node is holding AccessShareLock (or better)
* on the heap relation throughout the execution of the plan tree.
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
*/
Adjust nodeBitmapIndexscan.c to not keep the index open across calls, but just to open and close it during MultiExecBitmapIndexScan. This avoids acquiring duplicate resources (eg, multiple locks on the same relation) in a tree with many bitmap scans. Also, don't bother to lock the parent heap at all here, since we must be underneath a BitmapHeapScan node that will be holding a suitable lock.
2005-04-24 20:16:38 +02:00
indexstate->ss.ss_currentRelation = NULL;
indexstate->ss.ss_currentScanDesc = NULL;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
/*
Tweak indexscan machinery to avoid taking an AccessShareLock on an index if we already have a stronger lock due to the index's table being the update target table of the query. Same optimization I applied earlier at the table level. There doesn't seem to be much interest in the more radical idea of not locking indexes at all, so do what we can ...
2005-12-03 06:51:03 +01:00
* Open the index relation and initialize relation and scan descriptors.
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Note we acquire no locks here; the index machinery does its own locks
Tweak indexscan machinery to avoid taking an AccessShareLock on an index if we already have a stronger lock due to the index's table being the update target table of the query. Same optimization I applied earlier at the table level. There doesn't seem to be much interest in the more radical idea of not locking indexes at all, so do what we can ...
2005-12-03 06:51:03 +01:00
* and unlocks. (We rely on having a lock on the parent table to
* ensure the index won't go away!) Furthermore, if the parent table
* is one of the target relations of the query, then InitPlan already
* opened and write-locked the index, so we can tell the index machinery
* not to bother getting an extra lock.
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
*/
indexstate->biss_RelationDesc = index_open(node->indexid);
Tweak indexscan machinery to avoid taking an AccessShareLock on an index if we already have a stronger lock due to the index's table being the update target table of the query. Same optimization I applied earlier at the table level. There doesn't seem to be much interest in the more radical idea of not locking indexes at all, so do what we can ...
2005-12-03 06:51:03 +01:00
relistarget = ExecRelationIsTargetRelation(estate, node->scan.scanrelid);
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
indexstate->biss_ScanDesc =
index_beginscan_multi(indexstate->biss_RelationDesc,
Tweak indexscan machinery to avoid taking an AccessShareLock on an index if we already have a stronger lock due to the index's table being the update target table of the query. Same optimization I applied earlier at the table level. There doesn't seem to be much interest in the more radical idea of not locking indexes at all, so do what we can ...
2005-12-03 06:51:03 +01:00
!relistarget,
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
estate->es_snapshot,
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
indexstate->biss_NumScanKeys,
indexstate->biss_ScanKeys);
Adjust nodeBitmapIndexscan to keep the target index opened from plan startup to end, rather than re-opening it in each MultiExecBitmapIndexScan call. I had foolishly thought that opening/closing wouldn't be much more expensive than a rescan call, but that was sheer brain fade. This seems to fix about half of the performance lossage reported by Sergey Koposov. I'm still not sure where the other half went.
2005-05-05 05:37:23 +02:00
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
2005-04-20 00:35:18 +02:00
/*
* all done.
*/
return indexstate;
}
int
ExecCountSlotsBitmapIndexScan(BitmapIndexScan *node)
{
return ExecCountSlotsNode(outerPlan((Plan *) node)) +
ExecCountSlotsNode(innerPlan((Plan *) node)) + BITMAPINDEXSCAN_NSLOTS;
}