tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
|
*
|
|
|
|
|
* tableam.h
|
|
|
|
|
* POSTGRES table access method definitions.
|
|
|
|
|
*
|
|
|
|
|
*
|
2019-03-11 03:46:14 +01:00
|
|
|
* Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
|
|
|
*
|
|
|
|
|
* src/include/access/tableam.h
|
|
|
|
|
*
|
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
#ifndef TABLEAM_H
|
|
|
|
|
#define TABLEAM_H
|
|
|
|
|
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
#include "access/relscan.h"
|
|
|
|
|
#include "access/sdir.h"
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
#include "utils/guc.h"
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
#include "utils/rel.h"
|
|
|
|
|
#include "utils/snapshot.h"
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
#define DEFAULT_TABLE_ACCESS_METHOD "heap"
|
|
|
|
|
|
|
|
|
|
extern char *default_table_access_method;
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
extern bool synchronize_seqscans;
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
|
|
|
|
|
|
tableam: Add tuple_{insert, delete, update, lock} and use.
This adds new, required, table AM callbacks for insert/delete/update
and lock_tuple. To be able to reasonably use those, the EvalPlanQual
mechanism had to be adapted, moving more logic into the AM.
Previously both delete/update/lock call-sites and the EPQ mechanism had
to have awareness of the specific tuple format to be able to fetch the
latest version of a tuple. Obviously that needs to be abstracted
away. To do so, move the logic that find the latest row version into
the AM. lock_tuple has a new flag argument,
TUPLE_LOCK_FLAG_FIND_LAST_VERSION, that forces it to lock the last
version, rather than the current one. It'd have been possible to do
so via a separate callback as well, but finding the last version
usually also necessitates locking the newest version, making it
sensible to combine the two. This replaces the previous use of
EvalPlanQualFetch(). Additionally HeapTupleUpdated, which previously
signaled either a concurrent update or delete, is now split into two,
to avoid callers needing AM specific knowledge to differentiate.
The move of finding the latest row version into tuple_lock means that
encountering a row concurrently moved into another partition will now
raise an error about "tuple to be locked" rather than "tuple to be
updated/deleted" - which is accurate, as that always happens when
locking rows. While possible slightly less helpful for users, it seems
like an acceptable trade-off.
As part of this commit HTSU_Result has been renamed to TM_Result, and
its members been expanded to differentiated between updating and
deleting. HeapUpdateFailureData has been renamed to TM_FailureData.
The interface to speculative insertion is changed so nodeModifyTable.c
does not have to set the speculative token itself anymore. Instead
there's a version of tuple_insert, tuple_insert_speculative, that
performs the speculative insertion (without requiring a flag to signal
that fact), and the speculative insertion is either made permanent
with table_complete_speculative(succeeded = true) or aborted with
succeeded = false).
Note that multi_insert is not yet routed through tableam, nor is
COPY. Changing multi_insert requires changes to copy.c that are large
enough to better be done separately.
Similarly, although simpler, CREATE TABLE AS and CREATE MATERIALIZED
VIEW are also only going to be adjusted in a later commit.
Author: Andres Freund and Haribabu Kommi
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20190313003903.nwvrxi7rw3ywhdel@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-24 03:55:57 +01:00
|
|
|
struct BulkInsertStateData;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Result codes for table_{update,delete,lock}_tuple, and for visibility
|
|
|
|
|
* routines inside table AMs.
|
|
|
|
|
*/
|
|
|
|
|
typedef enum TM_Result
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Signals that the action succeeded (i.e. update/delete performed, lock
|
|
|
|
|
* was acquired)
|
|
|
|
|
*/
|
|
|
|
|
TM_Ok,
|
|
|
|
|
|
|
|
|
|
/* The affected tuple wasn't visible to the relevant snapshot */
|
|
|
|
|
TM_Invisible,
|
|
|
|
|
|
|
|
|
|
/* The affected tuple was already modified by the calling backend */
|
|
|
|
|
TM_SelfModified,
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The affected tuple was updated by another transaction. This includes
|
|
|
|
|
* the case where tuple was moved to another partition.
|
|
|
|
|
*/
|
|
|
|
|
TM_Updated,
|
|
|
|
|
|
|
|
|
|
/* The affected tuple was deleted by another transaction */
|
|
|
|
|
TM_Deleted,
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The affected tuple is currently being modified by another session. This
|
|
|
|
|
* will only be returned if (update/delete/lock)_tuple are instructed not
|
|
|
|
|
* to wait.
|
|
|
|
|
*/
|
|
|
|
|
TM_BeingModified,
|
|
|
|
|
|
|
|
|
|
/* lock couldn't be acquired, action skipped. Only used by lock_tuple */
|
|
|
|
|
TM_WouldBlock
|
|
|
|
|
} TM_Result;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* When table_update, table_delete, or table_lock_tuple fail because the target
|
|
|
|
|
* tuple is already outdated, they fill in this struct to provide information
|
|
|
|
|
* to the caller about what happened.
|
|
|
|
|
* ctid is the target's ctid link: it is the same as the target's TID if the
|
|
|
|
|
* target was deleted, or the location of the replacement tuple if the target
|
|
|
|
|
* was updated.
|
|
|
|
|
* xmax is the outdating transaction's XID. If the caller wants to visit the
|
|
|
|
|
* replacement tuple, it must check that this matches before believing the
|
|
|
|
|
* replacement is really a match.
|
|
|
|
|
* cmax is the outdating command's CID, but only when the failure code is
|
|
|
|
|
* TM_SelfModified (i.e., something in the current transaction outdated the
|
|
|
|
|
* tuple); otherwise cmax is zero. (We make this restriction because
|
|
|
|
|
* HeapTupleHeaderGetCmax doesn't work for tuples outdated in other
|
|
|
|
|
* transactions.)
|
|
|
|
|
*/
|
|
|
|
|
typedef struct TM_FailureData
|
|
|
|
|
{
|
|
|
|
|
ItemPointerData ctid;
|
|
|
|
|
TransactionId xmax;
|
|
|
|
|
CommandId cmax;
|
|
|
|
|
bool traversed;
|
|
|
|
|
} TM_FailureData;
|
|
|
|
|
|
|
|
|
|
/* "options" flag bits for table_insert */
|
|
|
|
|
#define TABLE_INSERT_SKIP_WAL 0x0001
|
|
|
|
|
#define TABLE_INSERT_SKIP_FSM 0x0002
|
|
|
|
|
#define TABLE_INSERT_FROZEN 0x0004
|
|
|
|
|
#define TABLE_INSERT_NO_LOGICAL 0x0008
|
|
|
|
|
|
|
|
|
|
/* flag bits fortable_lock_tuple */
|
|
|
|
|
/* Follow tuples whose update is in progress if lock modes don't conflict */
|
|
|
|
|
#define TUPLE_LOCK_FLAG_LOCK_UPDATE_IN_PROGRESS (1 << 0)
|
|
|
|
|
/* Follow update chain and lock lastest version of tuple */
|
|
|
|
|
#define TUPLE_LOCK_FLAG_FIND_LAST_VERSION (1 << 1)
|
|
|
|
|
|
|
|
|
|
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
/*
|
|
|
|
|
* API struct for a table AM. Note this must be allocated in a
|
|
|
|
|
* server-lifetime manner, typically as a static const struct, which then gets
|
|
|
|
|
* returned by FormData_pg_am.amhandler.
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
*
|
|
|
|
|
* I most cases it's not appropriate to directly call the callbacks directly,
|
|
|
|
|
* instead use the table_* wrapper functions.
|
|
|
|
|
*
|
|
|
|
|
* GetTableAmRoutine() asserts that required callbacks are filled in, remember
|
|
|
|
|
* to update when adding a callback.
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
*/
|
|
|
|
|
typedef struct TableAmRoutine
|
|
|
|
|
{
|
|
|
|
|
/* this must be set to T_TableAmRoutine */
|
|
|
|
|
NodeTag type;
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ------------------------------------------------------------------------
|
|
|
|
|
* Slot related callbacks.
|
|
|
|
|
* ------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Return slot implementation suitable for storing a tuple of this AM.
|
|
|
|
|
*/
|
|
|
|
|
const TupleTableSlotOps *(*slot_callbacks) (Relation rel);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ------------------------------------------------------------------------
|
|
|
|
|
* Table scan callbacks.
|
|
|
|
|
* ------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Start a scan of `rel`. The callback has to return a TableScanDesc,
|
|
|
|
|
* which will typically be embedded in a larger, AM specific, struct.
|
|
|
|
|
*
|
|
|
|
|
* If nkeys != 0, the results need to be filtered by those scan keys.
|
|
|
|
|
*
|
|
|
|
|
* pscan, if not NULL, will have already been initialized with
|
|
|
|
|
* parallelscan_initialize(), and has to be for the same relation. Will
|
|
|
|
|
* only be set coming from table_beginscan_parallel().
|
|
|
|
|
*
|
|
|
|
|
* allow_{strat, sync, pagemode} specify whether a scan strategy,
|
|
|
|
|
* synchronized scans, or page mode may be used (although not every AM
|
|
|
|
|
* will support those).
|
|
|
|
|
*
|
|
|
|
|
* is_{bitmapscan, samplescan} specify whether the scan is inteded to
|
|
|
|
|
* support those types of scans.
|
|
|
|
|
*
|
|
|
|
|
* if temp_snap is true, the snapshot will need to be deallocated at
|
|
|
|
|
* scan_end.
|
|
|
|
|
*/
|
|
|
|
|
TableScanDesc (*scan_begin) (Relation rel,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
int nkeys, struct ScanKeyData *key,
|
|
|
|
|
ParallelTableScanDesc pscan,
|
|
|
|
|
bool allow_strat,
|
|
|
|
|
bool allow_sync,
|
|
|
|
|
bool allow_pagemode,
|
|
|
|
|
bool is_bitmapscan,
|
|
|
|
|
bool is_samplescan,
|
|
|
|
|
bool temp_snap);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Release resources and deallocate scan. If TableScanDesc.temp_snap,
|
|
|
|
|
* TableScanDesc.rs_snapshot needs to be unregistered.
|
|
|
|
|
*/
|
|
|
|
|
void (*scan_end) (TableScanDesc scan);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Restart relation scan. If set_params is set to true, allow{strat,
|
|
|
|
|
* sync, pagemode} (see scan_begin) changes should be taken into account.
|
|
|
|
|
*/
|
|
|
|
|
void (*scan_rescan) (TableScanDesc scan, struct ScanKeyData *key, bool set_params,
|
|
|
|
|
bool allow_strat, bool allow_sync, bool allow_pagemode);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Return next tuple from `scan`, store in slot.
|
|
|
|
|
*/
|
|
|
|
|
bool (*scan_getnextslot) (TableScanDesc scan,
|
|
|
|
|
ScanDirection direction, TupleTableSlot *slot);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ------------------------------------------------------------------------
|
|
|
|
|
* Parallel table scan related functions.
|
|
|
|
|
* ------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Estimate the size of shared memory needed for a parallel scan of this
|
|
|
|
|
* relation. The snapshot does not need to be accounted for.
|
|
|
|
|
*/
|
|
|
|
|
Size (*parallelscan_estimate) (Relation rel);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Initialize ParallelTableScanDesc for a parallel scan of this relation.
|
|
|
|
|
* pscan will be sized according to parallelscan_estimate() for the same
|
|
|
|
|
* relation.
|
|
|
|
|
*/
|
|
|
|
|
Size (*parallelscan_initialize) (Relation rel, ParallelTableScanDesc pscan);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Reinitilize `pscan` for a new scan. `rel` will be the same relation as
|
|
|
|
|
* when `pscan` was initialized by parallelscan_initialize.
|
|
|
|
|
*/
|
|
|
|
|
void (*parallelscan_reinitialize) (Relation rel, ParallelTableScanDesc pscan);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ------------------------------------------------------------------------
|
|
|
|
|
* Index Scan Callbacks
|
|
|
|
|
* ------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Prepare to fetch tuples from the relation, as needed when fetching
|
|
|
|
|
* tuples for an index scan. The callback has to return a
|
|
|
|
|
* IndexFetchTableData, which the AM will typically embed in a larger
|
|
|
|
|
* structure with additional information.
|
|
|
|
|
*
|
|
|
|
|
* Tuples for an index scan can then be fetched via index_fetch_tuple.
|
|
|
|
|
*/
|
|
|
|
|
struct IndexFetchTableData *(*index_fetch_begin) (Relation rel);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Reset index fetch. Typically this will release cross index fetch
|
|
|
|
|
* resources held in IndexFetchTableData.
|
|
|
|
|
*/
|
|
|
|
|
void (*index_fetch_reset) (struct IndexFetchTableData *data);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Release resources and deallocate index fetch.
|
|
|
|
|
*/
|
|
|
|
|
void (*index_fetch_end) (struct IndexFetchTableData *data);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Fetch tuple at `tid` into `slot`, after doing a visibility test
|
|
|
|
|
* according to `snapshot`. If a tuple was found and passed the visibility
|
|
|
|
|
* test, return true, false otherwise.
|
|
|
|
|
*
|
|
|
|
|
* Note that AMs that do not necessarily update indexes when indexed
|
|
|
|
|
* columns do not change, need to return the current/correct version of a
|
|
|
|
|
* tuple as appropriate, even if the tid points to an older version of the
|
|
|
|
|
* tuple.
|
|
|
|
|
*
|
|
|
|
|
* *call_again is false on the first call to index_fetch_tuple for a tid.
|
|
|
|
|
* If there potentially is another tuple matching the tid, *call_again
|
|
|
|
|
* needs be set to true by index_fetch_tuple, signalling to the caller
|
|
|
|
|
* that index_fetch_tuple should be called again for the same tid.
|
|
|
|
|
*
|
2019-03-26 00:52:55 +01:00
|
|
|
* *all_dead, if all_dead is not NULL, should be set to true if by
|
|
|
|
|
* index_fetch_tuple iff it is guaranteed that no backend needs to see
|
|
|
|
|
* that tuple. Index AMs can use that do avoid returning that tid in
|
|
|
|
|
* future searches.
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
*/
|
|
|
|
|
bool (*index_fetch_tuple) (struct IndexFetchTableData *scan,
|
|
|
|
|
ItemPointer tid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
TupleTableSlot *slot,
|
|
|
|
|
bool *call_again, bool *all_dead);
|
|
|
|
|
|
|
|
|
|
/* ------------------------------------------------------------------------
|
|
|
|
|
* Callbacks for non-modifying operations on individual tuples
|
|
|
|
|
* ------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
2019-03-25 08:13:42 +01:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Fetch tuple at `tid` into `slot, after doing a visibility test
|
|
|
|
|
* according to `snapshot`. If a tuple was found and passed the visibility
|
|
|
|
|
* test, returns true, false otherwise.
|
|
|
|
|
*/
|
|
|
|
|
bool (*tuple_fetch_row_version) (Relation rel,
|
|
|
|
|
ItemPointer tid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
TupleTableSlot *slot);
|
|
|
|
|
|
2019-03-26 01:14:48 +01:00
|
|
|
/*
|
|
|
|
|
* Return the latest version of the tuple at `tid`, by updating `tid` to
|
|
|
|
|
* point at the newest version.
|
|
|
|
|
*/
|
|
|
|
|
void (*tuple_get_latest_tid) (Relation rel,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
ItemPointer tid);
|
|
|
|
|
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
/*
|
|
|
|
|
* Does the tuple in `slot` satisfy `snapshot`? The slot needs to be of
|
|
|
|
|
* the appropriate type for the AM.
|
|
|
|
|
*/
|
|
|
|
|
bool (*tuple_satisfies_snapshot) (Relation rel,
|
|
|
|
|
TupleTableSlot *slot,
|
|
|
|
|
Snapshot snapshot);
|
|
|
|
|
|
Compute XID horizon for page level index vacuum on primary.
Previously the xid horizon was only computed during WAL replay. That
had two major problems:
1) It relied on knowing what the table pointed to looks like. That was
easy enough before the introducing of tableam (we knew it had to be
heap, although some trickery around logging the heap relfilenodes
was required). But to properly handle table AMs we need
per-database catalog access to look up the AM handler, which
recovery doesn't allow.
2) Not knowing the xid horizon also makes it hard to support logical
decoding on standbys. When on a catalog table, we need to be able
to conflict with slots that have an xid horizon that's too old. But
computing the horizon by visiting the heap only works once
consistency is reached, but we always need to be able to detect
conflicts.
There's also a secondary problem, in that the current method performs
redundant work on every standby. But that's counterbalanced by
potentially computing the value when not necessary (either because
there's no standby, or because there's no connected backends).
Solve 1) and 2) by moving computation of the xid horizon to the
primary and by involving tableam in the computation of the horizon.
To address the potentially increased overhead, increase the efficiency
of the xid horizon computation for heap by sorting the tids, and
eliminating redundant buffer accesses. When prefetching is available,
additionally perform prefetching of buffers. As this is more of a
maintenance task, rather than something routinely done in every read
only query, we add an arbitrary 10 to the effective concurrency -
thereby using IO concurrency, when not globally enabled. That's
possibly not the perfect formula, but seems good enough for now.
Bumps WAL format, as latestRemovedXid is now part of the records, and
the heap's relfilenode isn't anymore.
Author: Andres Freund, Amit Khandekar, Robert Haas
Reviewed-By: Robert Haas
Discussion:
https://postgr.es/m/20181212204154.nsxf3gzqv3gesl32@alap3.anarazel.de
https://postgr.es/m/20181214014235.dal5ogljs3bmlq44@alap3.anarazel.de
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
2019-03-26 22:41:46 +01:00
|
|
|
/* see table_compute_xid_horizon_for_tuples() */
|
|
|
|
|
TransactionId (*compute_xid_horizon_for_tuples) (Relation rel,
|
|
|
|
|
ItemPointerData *items,
|
|
|
|
|
int nitems);
|
|
|
|
|
|
|
|
|
|
|
tableam: Add tuple_{insert, delete, update, lock} and use.
This adds new, required, table AM callbacks for insert/delete/update
and lock_tuple. To be able to reasonably use those, the EvalPlanQual
mechanism had to be adapted, moving more logic into the AM.
Previously both delete/update/lock call-sites and the EPQ mechanism had
to have awareness of the specific tuple format to be able to fetch the
latest version of a tuple. Obviously that needs to be abstracted
away. To do so, move the logic that find the latest row version into
the AM. lock_tuple has a new flag argument,
TUPLE_LOCK_FLAG_FIND_LAST_VERSION, that forces it to lock the last
version, rather than the current one. It'd have been possible to do
so via a separate callback as well, but finding the last version
usually also necessitates locking the newest version, making it
sensible to combine the two. This replaces the previous use of
EvalPlanQualFetch(). Additionally HeapTupleUpdated, which previously
signaled either a concurrent update or delete, is now split into two,
to avoid callers needing AM specific knowledge to differentiate.
The move of finding the latest row version into tuple_lock means that
encountering a row concurrently moved into another partition will now
raise an error about "tuple to be locked" rather than "tuple to be
updated/deleted" - which is accurate, as that always happens when
locking rows. While possible slightly less helpful for users, it seems
like an acceptable trade-off.
As part of this commit HTSU_Result has been renamed to TM_Result, and
its members been expanded to differentiated between updating and
deleting. HeapUpdateFailureData has been renamed to TM_FailureData.
The interface to speculative insertion is changed so nodeModifyTable.c
does not have to set the speculative token itself anymore. Instead
there's a version of tuple_insert, tuple_insert_speculative, that
performs the speculative insertion (without requiring a flag to signal
that fact), and the speculative insertion is either made permanent
with table_complete_speculative(succeeded = true) or aborted with
succeeded = false).
Note that multi_insert is not yet routed through tableam, nor is
COPY. Changing multi_insert requires changes to copy.c that are large
enough to better be done separately.
Similarly, although simpler, CREATE TABLE AS and CREATE MATERIALIZED
VIEW are also only going to be adjusted in a later commit.
Author: Andres Freund and Haribabu Kommi
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20190313003903.nwvrxi7rw3ywhdel@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-24 03:55:57 +01:00
|
|
|
/* ------------------------------------------------------------------------
|
|
|
|
|
* Manipulations of physical tuples.
|
|
|
|
|
* ------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/* see table_insert() for reference about parameters */
|
|
|
|
|
void (*tuple_insert) (Relation rel, TupleTableSlot *slot, CommandId cid,
|
|
|
|
|
int options, struct BulkInsertStateData *bistate);
|
|
|
|
|
|
|
|
|
|
/* see table_insert() for reference about parameters */
|
|
|
|
|
void (*tuple_insert_speculative) (Relation rel,
|
|
|
|
|
TupleTableSlot *slot,
|
|
|
|
|
CommandId cid,
|
|
|
|
|
int options,
|
|
|
|
|
struct BulkInsertStateData *bistate,
|
|
|
|
|
uint32 specToken);
|
|
|
|
|
|
|
|
|
|
/* see table_insert() for reference about parameters */
|
|
|
|
|
void (*tuple_complete_speculative) (Relation rel,
|
|
|
|
|
TupleTableSlot *slot,
|
|
|
|
|
uint32 specToken,
|
|
|
|
|
bool succeeded);
|
|
|
|
|
|
|
|
|
|
/* see table_insert() for reference about parameters */
|
|
|
|
|
TM_Result (*tuple_delete) (Relation rel,
|
|
|
|
|
ItemPointer tid,
|
|
|
|
|
CommandId cid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
Snapshot crosscheck,
|
|
|
|
|
bool wait,
|
|
|
|
|
TM_FailureData *tmfd,
|
|
|
|
|
bool changingPart);
|
|
|
|
|
|
|
|
|
|
/* see table_insert() for reference about parameters */
|
|
|
|
|
TM_Result (*tuple_update) (Relation rel,
|
|
|
|
|
ItemPointer otid,
|
|
|
|
|
TupleTableSlot *slot,
|
|
|
|
|
CommandId cid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
Snapshot crosscheck,
|
|
|
|
|
bool wait,
|
|
|
|
|
TM_FailureData *tmfd,
|
|
|
|
|
LockTupleMode *lockmode,
|
|
|
|
|
bool *update_indexes);
|
|
|
|
|
|
|
|
|
|
/* see table_insert() for reference about parameters */
|
|
|
|
|
TM_Result (*tuple_lock) (Relation rel,
|
|
|
|
|
ItemPointer tid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
TupleTableSlot *slot,
|
|
|
|
|
CommandId cid,
|
|
|
|
|
LockTupleMode mode,
|
|
|
|
|
LockWaitPolicy wait_policy,
|
|
|
|
|
uint8 flags,
|
|
|
|
|
TM_FailureData *tmfd);
|
|
|
|
|
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
} TableAmRoutine;
|
|
|
|
|
|
|
|
|
|
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
/* ----------------------------------------------------------------------------
|
|
|
|
|
* Slot functions.
|
|
|
|
|
* ----------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Returns slot callbacks suitable for holding tuples of the appropriate type
|
|
|
|
|
* for the relation. Works for tables, views, foreign tables and partitioned
|
|
|
|
|
* tables.
|
|
|
|
|
*/
|
|
|
|
|
extern const TupleTableSlotOps *table_slot_callbacks(Relation rel);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Returns slot using the callbacks returned by table_slot_callbacks(), and
|
|
|
|
|
* registers it on *reglist.
|
|
|
|
|
*/
|
|
|
|
|
extern TupleTableSlot *table_slot_create(Relation rel, List **reglist);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ----------------------------------------------------------------------------
|
|
|
|
|
* Table scan functions.
|
|
|
|
|
* ----------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Start a scan of `rel`. Returned tuples pass a visibility test of
|
|
|
|
|
* `snapshot`, and if nkeys != 0, the results are filtered by those scan keys.
|
|
|
|
|
*/
|
|
|
|
|
static inline TableScanDesc
|
|
|
|
|
table_beginscan(Relation rel, Snapshot snapshot,
|
|
|
|
|
int nkeys, struct ScanKeyData *key)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL,
|
|
|
|
|
true, true, true, false, false, false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Like table_beginscan(), but for scanning catalog. It'll automatically use a
|
|
|
|
|
* snapshot appropriate for scanning catalog relations.
|
|
|
|
|
*/
|
|
|
|
|
extern TableScanDesc table_beginscan_catalog(Relation rel, int nkeys,
|
|
|
|
|
struct ScanKeyData *key);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Like table_beginscan(), but table_beginscan_strat() offers an extended API
|
|
|
|
|
* that lets the caller control whether a nondefault buffer access strategy
|
|
|
|
|
* can be used, and whether syncscan can be chosen (possibly resulting in the
|
|
|
|
|
* scan not starting from block zero). Both of these default to true with
|
|
|
|
|
* plain table_beginscan.
|
|
|
|
|
*/
|
|
|
|
|
static inline TableScanDesc
|
|
|
|
|
table_beginscan_strat(Relation rel, Snapshot snapshot,
|
|
|
|
|
int nkeys, struct ScanKeyData *key,
|
|
|
|
|
bool allow_strat, bool allow_sync)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL,
|
|
|
|
|
allow_strat, allow_sync, true,
|
|
|
|
|
false, false, false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* table_beginscan_bm is an alternative entry point for setting up a
|
|
|
|
|
* TableScanDesc for a bitmap heap scan. Although that scan technology is
|
|
|
|
|
* really quite unlike a standard seqscan, there is just enough commonality to
|
|
|
|
|
* make it worth using the same data structure.
|
|
|
|
|
*/
|
|
|
|
|
static inline TableScanDesc
|
|
|
|
|
table_beginscan_bm(Relation rel, Snapshot snapshot,
|
|
|
|
|
int nkeys, struct ScanKeyData *key)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL,
|
|
|
|
|
false, false, true, true, false, false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* table_beginscan_sampling is an alternative entry point for setting up a
|
|
|
|
|
* TableScanDesc for a TABLESAMPLE scan. As with bitmap scans, it's worth
|
|
|
|
|
* using the same data structure although the behavior is rather different.
|
|
|
|
|
* In addition to the options offered by table_beginscan_strat, this call
|
|
|
|
|
* also allows control of whether page-mode visibility checking is used.
|
|
|
|
|
*/
|
|
|
|
|
static inline TableScanDesc
|
|
|
|
|
table_beginscan_sampling(Relation rel, Snapshot snapshot,
|
|
|
|
|
int nkeys, struct ScanKeyData *key,
|
|
|
|
|
bool allow_strat, bool allow_sync, bool allow_pagemode)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL,
|
|
|
|
|
allow_strat, allow_sync, allow_pagemode,
|
|
|
|
|
false, true, false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* table_beginscan_analyze is an alternative entry point for setting up a
|
|
|
|
|
* TableScanDesc for an ANALYZE scan. As with bitmap scans, it's worth using
|
|
|
|
|
* the same data structure although the behavior is rather different.
|
|
|
|
|
*/
|
|
|
|
|
static inline TableScanDesc
|
|
|
|
|
table_beginscan_analyze(Relation rel)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->scan_begin(rel, NULL, 0, NULL, NULL,
|
|
|
|
|
true, false, true,
|
|
|
|
|
false, true, false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* End relation scan.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_endscan(TableScanDesc scan)
|
|
|
|
|
{
|
|
|
|
|
scan->rs_rd->rd_tableam->scan_end(scan);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Restart a relation scan.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_rescan(TableScanDesc scan,
|
|
|
|
|
struct ScanKeyData *key)
|
|
|
|
|
{
|
|
|
|
|
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Restart a relation scan after changing params.
|
|
|
|
|
*
|
|
|
|
|
* This call allows changing the buffer strategy, syncscan, and pagemode
|
|
|
|
|
* options before starting a fresh scan. Note that although the actual use of
|
|
|
|
|
* syncscan might change (effectively, enabling or disabling reporting), the
|
|
|
|
|
* previously selected startblock will be kept.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
|
|
|
|
|
bool allow_strat, bool allow_sync, bool allow_pagemode)
|
|
|
|
|
{
|
|
|
|
|
scan->rs_rd->rd_tableam->scan_rescan(scan, key, true,
|
|
|
|
|
allow_strat, allow_sync,
|
|
|
|
|
allow_pagemode);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Update snapshot used by the scan.
|
|
|
|
|
*/
|
|
|
|
|
extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Return next tuple from `scan`, store in slot.
|
|
|
|
|
*/
|
|
|
|
|
static inline bool
|
|
|
|
|
table_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, TupleTableSlot *slot)
|
|
|
|
|
{
|
|
|
|
|
slot->tts_tableOid = RelationGetRelid(sscan->rs_rd);
|
|
|
|
|
return sscan->rs_rd->rd_tableam->scan_getnextslot(sscan, direction, slot);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ----------------------------------------------------------------------------
|
|
|
|
|
* Parallel table scan related functions.
|
|
|
|
|
* ----------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Estimate the size of shared memory needed for a parallel scan of this
|
|
|
|
|
* relation.
|
|
|
|
|
*/
|
|
|
|
|
extern Size table_parallelscan_estimate(Relation rel, Snapshot snapshot);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Initialize ParallelTableScanDesc for a parallel scan of this
|
|
|
|
|
* relation. `pscan` needs to be sized according to parallelscan_estimate()
|
|
|
|
|
* for the same relation. Call this just once in the leader process; then,
|
|
|
|
|
* individual workers attach via table_beginscan_parallel.
|
|
|
|
|
*/
|
|
|
|
|
extern void table_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan, Snapshot snapshot);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Begin a parallel scan. `pscan` needs to have been initialized with
|
|
|
|
|
* table_parallelscan_initialize(), for the same relation. The initialization
|
|
|
|
|
* does not need to have happened in this backend.
|
|
|
|
|
*
|
|
|
|
|
* Caller must hold a suitable lock on the correct relation.
|
|
|
|
|
*/
|
|
|
|
|
extern TableScanDesc table_beginscan_parallel(Relation rel, ParallelTableScanDesc pscan);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Restart a parallel scan. Call this in the leader process. Caller is
|
|
|
|
|
* responsible for making sure that all workers have finished the scan
|
|
|
|
|
* beforehand.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan)
|
|
|
|
|
{
|
2019-03-11 23:03:27 +01:00
|
|
|
rel->rd_tableam->parallelscan_reinitialize(rel, pscan);
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ----------------------------------------------------------------------------
|
|
|
|
|
* Index scan related functions.
|
|
|
|
|
* ----------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Prepare to fetch tuples from the relation, as needed when fetching tuples
|
|
|
|
|
* for an index scan.
|
|
|
|
|
*
|
|
|
|
|
* Tuples for an index scan can then be fetched via table_index_fetch_tuple().
|
|
|
|
|
*/
|
|
|
|
|
static inline IndexFetchTableData *
|
|
|
|
|
table_index_fetch_begin(Relation rel)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->index_fetch_begin(rel);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Reset index fetch. Typically this will release cross index fetch resources
|
|
|
|
|
* held in IndexFetchTableData.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_index_fetch_reset(struct IndexFetchTableData *scan)
|
|
|
|
|
{
|
|
|
|
|
scan->rel->rd_tableam->index_fetch_reset(scan);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Release resources and deallocate index fetch.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_index_fetch_end(struct IndexFetchTableData *scan)
|
|
|
|
|
{
|
|
|
|
|
scan->rel->rd_tableam->index_fetch_end(scan);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
2019-03-25 08:13:42 +01:00
|
|
|
* Fetches, as part of an index scan, tuple at `tid` into `slot`, after doing
|
|
|
|
|
* a visibility test according to `snapshot`. If a tuple was found and passed
|
|
|
|
|
* the visibility test, returns true, false otherwise.
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
*
|
|
|
|
|
* *call_again needs to be false on the first call to table_index_fetch_tuple() for
|
|
|
|
|
* a tid. If there potentially is another tuple matching the tid, *call_again
|
|
|
|
|
* will be set to true, signalling that table_index_fetch_tuple() should be called
|
|
|
|
|
* again for the same tid.
|
|
|
|
|
*
|
2019-03-26 00:52:55 +01:00
|
|
|
* *all_dead, if all_dead is not NULL, will be set to true by
|
|
|
|
|
* table_index_fetch_tuple() iff it is guaranteed that no backend needs to see
|
|
|
|
|
* that tuple. Index AMs can use that do avoid returning that tid in future
|
|
|
|
|
* searches.
|
2019-03-25 08:13:42 +01:00
|
|
|
*
|
|
|
|
|
* The difference between this function and table_fetch_row_version is that
|
|
|
|
|
* this function returns the currently visible version of a row if the AM
|
|
|
|
|
* supports storing multiple row versions reachable via a single index entry
|
|
|
|
|
* (like heap's HOT). Whereas table_fetch_row_version only evaluates the the
|
|
|
|
|
* tuple exactly at `tid`. Outside of index entry ->table tuple lookups,
|
|
|
|
|
* table_fetch_row_version is what's usually needed.
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
*/
|
|
|
|
|
static inline bool
|
|
|
|
|
table_index_fetch_tuple(struct IndexFetchTableData *scan,
|
|
|
|
|
ItemPointer tid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
TupleTableSlot *slot,
|
|
|
|
|
bool *call_again, bool *all_dead)
|
|
|
|
|
{
|
|
|
|
|
|
|
|
|
|
return scan->rel->rd_tableam->index_fetch_tuple(scan, tid, snapshot,
|
|
|
|
|
slot, call_again,
|
|
|
|
|
all_dead);
|
|
|
|
|
}
|
|
|
|
|
|
2019-03-26 00:52:55 +01:00
|
|
|
/*
|
|
|
|
|
* This is a convenience wrapper around table_index_fetch_tuple() which
|
|
|
|
|
* returns whether there are table tuple items corresponding to an index
|
|
|
|
|
* entry. This likely is only useful to verify if there's a conflict in a
|
|
|
|
|
* unique index.
|
|
|
|
|
*/
|
|
|
|
|
extern bool table_index_fetch_tuple_check(Relation rel,
|
|
|
|
|
ItemPointer tid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
bool *all_dead);
|
|
|
|
|
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
|
|
|
|
|
/* ------------------------------------------------------------------------
|
|
|
|
|
* Functions for non-modifying operations on individual tuples
|
|
|
|
|
* ------------------------------------------------------------------------
|
|
|
|
|
*/
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
|
2019-03-25 08:13:42 +01:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Fetch tuple at `tid` into `slot, after doing a visibility test according to
|
|
|
|
|
* `snapshot`. If a tuple was found and passed the visibility test, returns
|
|
|
|
|
* true, false otherwise.
|
|
|
|
|
*
|
|
|
|
|
* See table_index_fetch_tuple's comment about what the difference between
|
|
|
|
|
* these functions is. This function is the correct to use outside of
|
|
|
|
|
* index entry->table tuple lookups.
|
|
|
|
|
*/
|
|
|
|
|
static inline bool
|
|
|
|
|
table_fetch_row_version(Relation rel,
|
|
|
|
|
ItemPointer tid,
|
|
|
|
|
Snapshot snapshot,
|
|
|
|
|
TupleTableSlot *slot)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->tuple_fetch_row_version(rel, tid, snapshot, slot);
|
|
|
|
|
}
|
|
|
|
|
|
2019-03-26 01:14:48 +01:00
|
|
|
/*
|
|
|
|
|
* Return the latest version of the tuple at `tid`, by updating `tid` to
|
|
|
|
|
* point at the newest version.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_get_latest_tid(Relation rel, Snapshot snapshot, ItemPointer tid)
|
|
|
|
|
{
|
|
|
|
|
rel->rd_tableam->tuple_get_latest_tid(rel, snapshot, tid);
|
|
|
|
|
}
|
|
|
|
|
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
/*
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
* Return true iff tuple in slot satisfies the snapshot.
|
|
|
|
|
*
|
|
|
|
|
* This assumes the slot's tuple is valid, and of the appropriate type for the
|
|
|
|
|
* AM.
|
|
|
|
|
*
|
|
|
|
|
* Some AMs might modify the data underlying the tuple as a side-effect. If so
|
|
|
|
|
* they ought to mark the relevant buffer dirty.
|
|
|
|
|
*/
|
|
|
|
|
static inline bool
|
|
|
|
|
table_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, Snapshot snapshot)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->tuple_satisfies_snapshot(rel, slot, snapshot);
|
|
|
|
|
}
|
|
|
|
|
|
Compute XID horizon for page level index vacuum on primary.
Previously the xid horizon was only computed during WAL replay. That
had two major problems:
1) It relied on knowing what the table pointed to looks like. That was
easy enough before the introducing of tableam (we knew it had to be
heap, although some trickery around logging the heap relfilenodes
was required). But to properly handle table AMs we need
per-database catalog access to look up the AM handler, which
recovery doesn't allow.
2) Not knowing the xid horizon also makes it hard to support logical
decoding on standbys. When on a catalog table, we need to be able
to conflict with slots that have an xid horizon that's too old. But
computing the horizon by visiting the heap only works once
consistency is reached, but we always need to be able to detect
conflicts.
There's also a secondary problem, in that the current method performs
redundant work on every standby. But that's counterbalanced by
potentially computing the value when not necessary (either because
there's no standby, or because there's no connected backends).
Solve 1) and 2) by moving computation of the xid horizon to the
primary and by involving tableam in the computation of the horizon.
To address the potentially increased overhead, increase the efficiency
of the xid horizon computation for heap by sorting the tids, and
eliminating redundant buffer accesses. When prefetching is available,
additionally perform prefetching of buffers. As this is more of a
maintenance task, rather than something routinely done in every read
only query, we add an arbitrary 10 to the effective concurrency -
thereby using IO concurrency, when not globally enabled. That's
possibly not the perfect formula, but seems good enough for now.
Bumps WAL format, as latestRemovedXid is now part of the records, and
the heap's relfilenode isn't anymore.
Author: Andres Freund, Amit Khandekar, Robert Haas
Reviewed-By: Robert Haas
Discussion:
https://postgr.es/m/20181212204154.nsxf3gzqv3gesl32@alap3.anarazel.de
https://postgr.es/m/20181214014235.dal5ogljs3bmlq44@alap3.anarazel.de
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
2019-03-26 22:41:46 +01:00
|
|
|
/*
|
|
|
|
|
* Compute the newest xid among the tuples pointed to by items. This is used
|
|
|
|
|
* to compute what snapshots to conflict with when replaying WAL records for
|
|
|
|
|
* page-level index vacuums.
|
|
|
|
|
*/
|
|
|
|
|
static inline TransactionId
|
|
|
|
|
table_compute_xid_horizon_for_tuples(Relation rel,
|
|
|
|
|
ItemPointerData *items,
|
|
|
|
|
int nitems)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->compute_xid_horizon_for_tuples(rel, items, nitems);
|
|
|
|
|
}
|
|
|
|
|
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
|
tableam: Add tuple_{insert, delete, update, lock} and use.
This adds new, required, table AM callbacks for insert/delete/update
and lock_tuple. To be able to reasonably use those, the EvalPlanQual
mechanism had to be adapted, moving more logic into the AM.
Previously both delete/update/lock call-sites and the EPQ mechanism had
to have awareness of the specific tuple format to be able to fetch the
latest version of a tuple. Obviously that needs to be abstracted
away. To do so, move the logic that find the latest row version into
the AM. lock_tuple has a new flag argument,
TUPLE_LOCK_FLAG_FIND_LAST_VERSION, that forces it to lock the last
version, rather than the current one. It'd have been possible to do
so via a separate callback as well, but finding the last version
usually also necessitates locking the newest version, making it
sensible to combine the two. This replaces the previous use of
EvalPlanQualFetch(). Additionally HeapTupleUpdated, which previously
signaled either a concurrent update or delete, is now split into two,
to avoid callers needing AM specific knowledge to differentiate.
The move of finding the latest row version into tuple_lock means that
encountering a row concurrently moved into another partition will now
raise an error about "tuple to be locked" rather than "tuple to be
updated/deleted" - which is accurate, as that always happens when
locking rows. While possible slightly less helpful for users, it seems
like an acceptable trade-off.
As part of this commit HTSU_Result has been renamed to TM_Result, and
its members been expanded to differentiated between updating and
deleting. HeapUpdateFailureData has been renamed to TM_FailureData.
The interface to speculative insertion is changed so nodeModifyTable.c
does not have to set the speculative token itself anymore. Instead
there's a version of tuple_insert, tuple_insert_speculative, that
performs the speculative insertion (without requiring a flag to signal
that fact), and the speculative insertion is either made permanent
with table_complete_speculative(succeeded = true) or aborted with
succeeded = false).
Note that multi_insert is not yet routed through tableam, nor is
COPY. Changing multi_insert requires changes to copy.c that are large
enough to better be done separately.
Similarly, although simpler, CREATE TABLE AS and CREATE MATERIALIZED
VIEW are also only going to be adjusted in a later commit.
Author: Andres Freund and Haribabu Kommi
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20190313003903.nwvrxi7rw3ywhdel@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-24 03:55:57 +01:00
|
|
|
/* ----------------------------------------------------------------------------
|
|
|
|
|
* Functions for manipulations of physical tuples.
|
|
|
|
|
* ----------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Insert a tuple from a slot into table AM routine.
|
|
|
|
|
*
|
|
|
|
|
* The options bitmask allows to specify options that allow to change the
|
|
|
|
|
* behaviour of the AM. Several options might be ignored by AMs not supporting
|
|
|
|
|
* them.
|
|
|
|
|
*
|
|
|
|
|
* If the TABLE_INSERT_SKIP_WAL option is specified, the new tuple will not
|
|
|
|
|
* necessarily logged to WAL, even for a non-temp relation. It is the AMs
|
|
|
|
|
* choice whether this optimization is supported.
|
|
|
|
|
*
|
|
|
|
|
* If the TABLE_INSERT_SKIP_FSM option is specified, AMs are free to not reuse
|
|
|
|
|
* free space in the relation. This can save some cycles when we know the
|
|
|
|
|
* relation is new and doesn't contain useful amounts of free space. It's
|
|
|
|
|
* commonly passed directly to RelationGetBufferForTuple, see for more info.
|
|
|
|
|
*
|
|
|
|
|
* TABLE_INSERT_FROZEN should only be specified for inserts into
|
|
|
|
|
* relfilenodes created during the current subtransaction and when
|
|
|
|
|
* there are no prior snapshots or pre-existing portals open.
|
|
|
|
|
* This causes rows to be frozen, which is an MVCC violation and
|
|
|
|
|
* requires explicit options chosen by user.
|
|
|
|
|
*
|
|
|
|
|
* TABLE_INSERT_NO_LOGICAL force-disables the emitting of logical decoding
|
|
|
|
|
* information for the tuple. This should solely be used during table rewrites
|
|
|
|
|
* where RelationIsLogicallyLogged(relation) is not yet accurate for the new
|
|
|
|
|
* relation.
|
|
|
|
|
*
|
|
|
|
|
* Note that most of these options will be applied when inserting into the
|
|
|
|
|
* heap's TOAST table, too, if the tuple requires any out-of-line data
|
|
|
|
|
*
|
|
|
|
|
*
|
|
|
|
|
* The BulkInsertState object (if any; bistate can be NULL for default
|
|
|
|
|
* behavior) is also just passed through to RelationGetBufferForTuple.
|
|
|
|
|
*
|
|
|
|
|
* On return the slot's tts_tid and tts_tableOid are updated to reflect the
|
|
|
|
|
* insertion. But note that any toasting of fields within the slot is NOT
|
|
|
|
|
* reflected in the slots contents.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_insert(Relation rel, TupleTableSlot *slot, CommandId cid,
|
|
|
|
|
int options, struct BulkInsertStateData *bistate)
|
|
|
|
|
{
|
|
|
|
|
rel->rd_tableam->tuple_insert(rel, slot, cid, options,
|
|
|
|
|
bistate);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Perform a "speculative insertion". These can be backed out afterwards
|
|
|
|
|
* without aborting the whole transaction. Other sessions can wait for the
|
|
|
|
|
* speculative insertion to be confirmed, turning it into a regular tuple, or
|
|
|
|
|
* aborted, as if it never existed. Speculatively inserted tuples behave as
|
|
|
|
|
* "value locks" of short duration, used to implement INSERT .. ON CONFLICT.
|
|
|
|
|
*
|
|
|
|
|
* A transaction having performed a speculative insertion has to either abort,
|
|
|
|
|
* or finish the speculative insertion with
|
|
|
|
|
* table_complete_speculative(succeeded = ...).
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_insert_speculative(Relation rel, TupleTableSlot *slot, CommandId cid,
|
|
|
|
|
int options, struct BulkInsertStateData *bistate, uint32 specToken)
|
|
|
|
|
{
|
|
|
|
|
rel->rd_tableam->tuple_insert_speculative(rel, slot, cid, options,
|
|
|
|
|
bistate, specToken);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Complete "speculative insertion" started in the same transaction. If
|
|
|
|
|
* succeeded is true, the tuple is fully inserted, if false, it's removed.
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
table_complete_speculative(Relation rel, TupleTableSlot *slot, uint32 specToken,
|
|
|
|
|
bool succeeded)
|
|
|
|
|
{
|
2019-03-24 05:09:39 +01:00
|
|
|
rel->rd_tableam->tuple_complete_speculative(rel, slot, specToken,
|
|
|
|
|
succeeded);
|
tableam: Add tuple_{insert, delete, update, lock} and use.
This adds new, required, table AM callbacks for insert/delete/update
and lock_tuple. To be able to reasonably use those, the EvalPlanQual
mechanism had to be adapted, moving more logic into the AM.
Previously both delete/update/lock call-sites and the EPQ mechanism had
to have awareness of the specific tuple format to be able to fetch the
latest version of a tuple. Obviously that needs to be abstracted
away. To do so, move the logic that find the latest row version into
the AM. lock_tuple has a new flag argument,
TUPLE_LOCK_FLAG_FIND_LAST_VERSION, that forces it to lock the last
version, rather than the current one. It'd have been possible to do
so via a separate callback as well, but finding the last version
usually also necessitates locking the newest version, making it
sensible to combine the two. This replaces the previous use of
EvalPlanQualFetch(). Additionally HeapTupleUpdated, which previously
signaled either a concurrent update or delete, is now split into two,
to avoid callers needing AM specific knowledge to differentiate.
The move of finding the latest row version into tuple_lock means that
encountering a row concurrently moved into another partition will now
raise an error about "tuple to be locked" rather than "tuple to be
updated/deleted" - which is accurate, as that always happens when
locking rows. While possible slightly less helpful for users, it seems
like an acceptable trade-off.
As part of this commit HTSU_Result has been renamed to TM_Result, and
its members been expanded to differentiated between updating and
deleting. HeapUpdateFailureData has been renamed to TM_FailureData.
The interface to speculative insertion is changed so nodeModifyTable.c
does not have to set the speculative token itself anymore. Instead
there's a version of tuple_insert, tuple_insert_speculative, that
performs the speculative insertion (without requiring a flag to signal
that fact), and the speculative insertion is either made permanent
with table_complete_speculative(succeeded = true) or aborted with
succeeded = false).
Note that multi_insert is not yet routed through tableam, nor is
COPY. Changing multi_insert requires changes to copy.c that are large
enough to better be done separately.
Similarly, although simpler, CREATE TABLE AS and CREATE MATERIALIZED
VIEW are also only going to be adjusted in a later commit.
Author: Andres Freund and Haribabu Kommi
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20190313003903.nwvrxi7rw3ywhdel@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-24 03:55:57 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Delete a tuple.
|
|
|
|
|
*
|
|
|
|
|
* NB: do not call this directly unless prepared to deal with
|
|
|
|
|
* concurrent-update conditions. Use simple_table_delete instead.
|
|
|
|
|
*
|
|
|
|
|
* Input parameters:
|
|
|
|
|
* relation - table to be modified (caller must hold suitable lock)
|
|
|
|
|
* tid - TID of tuple to be deleted
|
|
|
|
|
* cid - delete command ID (used for visibility test, and stored into
|
|
|
|
|
* cmax if successful)
|
|
|
|
|
* crosscheck - if not InvalidSnapshot, also check tuple against this
|
|
|
|
|
* wait - true if should wait for any conflicting update to commit/abort
|
|
|
|
|
* Output parameters:
|
|
|
|
|
* tmfd - filled in failure cases (see below)
|
|
|
|
|
* changingPart - true iff the tuple is being moved to another partition
|
|
|
|
|
* table due to an update of the partition key. Otherwise, false.
|
|
|
|
|
*
|
|
|
|
|
* Normal, successful return value is TM_Ok, which
|
|
|
|
|
* actually means we did delete it. Failure return codes are
|
|
|
|
|
* TM_SelfModified, TM_Updated, or TM_BeingModified
|
|
|
|
|
* (the last only possible if wait == false).
|
|
|
|
|
*
|
|
|
|
|
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
|
|
|
|
|
* t_xmax, and, if possible, and, if possible, t_cmax. See comments for
|
|
|
|
|
* struct TM_FailureData for additional info.
|
|
|
|
|
*/
|
|
|
|
|
static inline TM_Result
|
|
|
|
|
table_delete(Relation rel, ItemPointer tid, CommandId cid,
|
|
|
|
|
Snapshot snapshot, Snapshot crosscheck, bool wait,
|
|
|
|
|
TM_FailureData *tmfd, bool changingPart)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->tuple_delete(rel, tid, cid,
|
|
|
|
|
snapshot, crosscheck,
|
|
|
|
|
wait, tmfd, changingPart);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Update a tuple.
|
|
|
|
|
*
|
|
|
|
|
* NB: do not call this directly unless you are prepared to deal with
|
|
|
|
|
* concurrent-update conditions. Use simple_table_update instead.
|
|
|
|
|
*
|
|
|
|
|
* Input parameters:
|
|
|
|
|
* relation - table to be modified (caller must hold suitable lock)
|
|
|
|
|
* otid - TID of old tuple to be replaced
|
|
|
|
|
* newtup - newly constructed tuple data to store
|
|
|
|
|
* cid - update command ID (used for visibility test, and stored into
|
|
|
|
|
* cmax/cmin if successful)
|
|
|
|
|
* crosscheck - if not InvalidSnapshot, also check old tuple against this
|
|
|
|
|
* wait - true if should wait for any conflicting update to commit/abort
|
|
|
|
|
* Output parameters:
|
|
|
|
|
* tmfd - filled in failure cases (see below)
|
|
|
|
|
* lockmode - filled with lock mode acquired on tuple
|
|
|
|
|
* update_indexes - in success cases this is set to true if new index entries
|
|
|
|
|
* are required for this tuple
|
|
|
|
|
*
|
|
|
|
|
* Normal, successful return value is TM_Ok, which
|
|
|
|
|
* actually means we *did* update it. Failure return codes are
|
|
|
|
|
* TM_SelfModified, TM_Updated, or TM_BeingModified
|
|
|
|
|
* (the last only possible if wait == false).
|
|
|
|
|
*
|
|
|
|
|
* On success, the header fields of *newtup are updated to match the new
|
|
|
|
|
* stored tuple; in particular, newtup->t_self is set to the TID where the
|
|
|
|
|
* new tuple was inserted, and its HEAP_ONLY_TUPLE flag is set iff a HOT
|
|
|
|
|
* update was done. However, any TOAST changes in the new tuple's
|
|
|
|
|
* data are not reflected into *newtup.
|
|
|
|
|
*
|
|
|
|
|
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
|
|
|
|
|
* t_xmax, and, if possible, t_cmax. See comments for struct TM_FailureData
|
|
|
|
|
* for additional info.
|
|
|
|
|
*/
|
|
|
|
|
static inline TM_Result
|
|
|
|
|
table_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
|
|
|
|
|
CommandId cid, Snapshot snapshot, Snapshot crosscheck, bool wait,
|
|
|
|
|
TM_FailureData *tmfd, LockTupleMode *lockmode,
|
|
|
|
|
bool *update_indexes)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->tuple_update(rel, otid, slot,
|
|
|
|
|
cid, snapshot, crosscheck,
|
|
|
|
|
wait, tmfd,
|
|
|
|
|
lockmode, update_indexes);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Lock a tuple in the specified mode.
|
|
|
|
|
*
|
|
|
|
|
* Input parameters:
|
|
|
|
|
* relation: relation containing tuple (caller must hold suitable lock)
|
|
|
|
|
* tid: TID of tuple to lock
|
|
|
|
|
* snapshot: snapshot to use for visibility determinations
|
|
|
|
|
* cid: current command ID (used for visibility test, and stored into
|
|
|
|
|
* tuple's cmax if lock is successful)
|
|
|
|
|
* mode: lock mode desired
|
|
|
|
|
* wait_policy: what to do if tuple lock is not available
|
|
|
|
|
* flags:
|
|
|
|
|
* If TUPLE_LOCK_FLAG_LOCK_UPDATE_IN_PROGRESS, follow the update chain to
|
|
|
|
|
* also lock descendant tuples if lock modes don't conflict.
|
|
|
|
|
* If TUPLE_LOCK_FLAG_FIND_LAST_VERSION, update chain and lock lastest
|
|
|
|
|
* version.
|
|
|
|
|
*
|
|
|
|
|
* Output parameters:
|
|
|
|
|
* *slot: contains the target tuple
|
|
|
|
|
* *tmfd: filled in failure cases (see below)
|
|
|
|
|
*
|
|
|
|
|
* Function result may be:
|
|
|
|
|
* TM_Ok: lock was successfully acquired
|
|
|
|
|
* TM_Invisible: lock failed because tuple was never visible to us
|
|
|
|
|
* TM_SelfModified: lock failed because tuple updated by self
|
|
|
|
|
* TM_Updated: lock failed because tuple updated by other xact
|
|
|
|
|
* TM_Deleted: lock failed because tuple deleted by other xact
|
|
|
|
|
* TM_WouldBlock: lock couldn't be acquired and wait_policy is skip
|
|
|
|
|
*
|
|
|
|
|
* In the failure cases other than TM_Invisible, the routine fills *tmfd with
|
|
|
|
|
* the tuple's t_ctid, t_xmax, and, if possible, t_cmax. See comments for
|
|
|
|
|
* struct TM_FailureData for additional info.
|
|
|
|
|
*/
|
|
|
|
|
static inline TM_Result
|
|
|
|
|
table_lock_tuple(Relation rel, ItemPointer tid, Snapshot snapshot,
|
|
|
|
|
TupleTableSlot *slot, CommandId cid, LockTupleMode mode,
|
|
|
|
|
LockWaitPolicy wait_policy, uint8 flags,
|
|
|
|
|
TM_FailureData *tmfd)
|
|
|
|
|
{
|
|
|
|
|
return rel->rd_tableam->tuple_lock(rel, tid, snapshot, slot,
|
|
|
|
|
cid, mode, wait_policy,
|
|
|
|
|
flags, tmfd);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ----------------------------------------------------------------------------
|
|
|
|
|
* Functions to make modifications a bit simpler.
|
|
|
|
|
* ----------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
extern void simple_table_insert(Relation rel, TupleTableSlot *slot);
|
|
|
|
|
extern void simple_table_delete(Relation rel, ItemPointer tid,
|
|
|
|
|
Snapshot snapshot);
|
|
|
|
|
extern void simple_table_update(Relation rel, ItemPointer otid,
|
|
|
|
|
TupleTableSlot *slot, Snapshot snapshot,
|
|
|
|
|
bool *update_indexes);
|
|
|
|
|
|
|
|
|
|
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
/* ----------------------------------------------------------------------------
|
|
|
|
|
* Helper functions to implement parallel scans for block oriented AMs.
|
|
|
|
|
* ----------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
extern Size table_block_parallelscan_estimate(Relation rel);
|
|
|
|
|
extern Size table_block_parallelscan_initialize(Relation rel,
|
|
|
|
|
ParallelTableScanDesc pscan);
|
|
|
|
|
extern void table_block_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan);
|
|
|
|
|
extern BlockNumber table_block_parallelscan_nextpage(Relation rel, ParallelBlockTableScanDesc pbscan);
|
|
|
|
|
extern void table_block_parallelscan_startblock_init(Relation rel, ParallelBlockTableScanDesc pbscan);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ----------------------------------------------------------------------------
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
* Functions in tableamapi.c
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
* ----------------------------------------------------------------------------
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
*/
|
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several
new abstractions are needed. Specifically:
1) Heap scans need to be generalized into table scans. Do this by
introducing TableScanDesc, which will be the "base class" for
individual AMs. This contains the AM independent fields from
HeapScanDesc.
The previous heap_{beginscan,rescan,endscan} et al. have been
replaced with a table_ version.
There's no direct replacement for heap_getnext(), as that returned
a HeapTuple, which is undesirable for a other AMs. Instead there's
table_scan_getnextslot(). But note that heap_getnext() lives on,
it's still used widely to access catalog tables.
This is achieved by new scan_begin, scan_end, scan_rescan,
scan_getnextslot callbacks.
2) The portion of parallel scans that's shared between backends need
to be able to do so without the user doing per-AM work. To achieve
that new parallelscan_{estimate, initialize, reinitialize}
callbacks are introduced, which operate on a new
ParallelTableScanDesc, which again can be subclassed by AMs.
As it is likely that several AMs are going to be block oriented,
block oriented callbacks that can be shared between such AMs are
provided and used by heap. table_block_parallelscan_{estimate,
intiialize, reinitialize} as callbacks, and
table_block_parallelscan_{nextpage, init} for use in AMs. These
operate on a ParallelBlockTableScanDesc.
3) Index scans need to be able to access tables to return a tuple, and
there needs to be state across individual accesses to the heap to
store state like buffers. That's now handled by introducing a
sort-of-scan IndexFetchTable, which again is intended to be
subclassed by individual AMs (for heap IndexFetchHeap).
The relevant callbacks for an AM are index_fetch_{end, begin,
reset} to create the necessary state, and index_fetch_tuple to
retrieve an indexed tuple. Note that index_fetch_tuple
implementations need to be smarter than just blindly fetching the
tuples for AMs that have optimizations similar to heap's HOT - the
currently alive tuple in the update chain needs to be fetched if
appropriate.
Similar to table_scan_getnextslot(), it's undesirable to continue
to return HeapTuples. Thus index_fetch_heap (might want to rename
that later) now accepts a slot as an argument. Core code doesn't
have a lot of call sites performing index scans without going
through the systable_* API (in contrast to loads of heap_getnext
calls and working directly with HeapTuples).
Index scans now store the result of a search in
IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the
target is not generally a HeapTuple anymore that seems cleaner.
To be able to sensible adapt code to use the above, two further
callbacks have been introduced:
a) slot_callbacks returns a TupleTableSlotOps* suitable for creating
slots capable of holding a tuple of the AMs
type. table_slot_callbacks() and table_slot_create() are based
upon that, but have additional logic to deal with views, foreign
tables, etc.
While this change could have been done separately, nearly all the
call sites that needed to be adapted for the rest of this commit
also would have been needed to be adapted for
table_slot_callbacks(), making separation not worthwhile.
b) tuple_satisfies_snapshot checks whether the tuple in a slot is
currently visible according to a snapshot. That's required as a few
places now don't have a buffer + HeapTuple around, but a
slot (which in heap's case internally has that information).
Additionally a few infrastructure changes were needed:
I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now
internally uses a slot to keep track of tuples. While
systable_getnext() still returns HeapTuples, and will so for the
foreseeable future, the index API (see 1) above) now only deals with
slots.
The remainder, and largest part, of this commit is then adjusting all
scans in postgres to use the new APIs.
Author: Andres Freund, Haribabu Kommi, Alvaro Herrera
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
2019-03-11 20:46:41 +01:00
|
|
|
|
tableam: introduce table AM infrastructure.
This introduces the concept of table access methods, i.e. CREATE
ACCESS METHOD ... TYPE TABLE and
CREATE TABLE ... USING (storage-engine).
No table access functionality is delegated to table AMs as of this
commit, that'll be done in following commits.
Subsequent commits will incrementally abstract table access
functionality to be routed through table access methods. That change
is too large to be reviewed & committed at once, so it'll be done
incrementally.
Docs will be updated at the end, as adding them incrementally would
likely make them less coherent, and definitely is a lot more work,
without a lot of benefit.
Table access methods are specified similar to index access methods,
i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with
callbacks. In contrast to index AMs that struct needs to live as long
as a backend, typically that's achieved by just returning a pointer to
a constant struct.
Psql's \d+ now displays a table's access method. That can be disabled
with HIDE_TABLEAM=true, which is mainly useful so regression tests can
be run against different AMs. It's quite possible that this behaviour
still needs to be fine tuned.
For now it's not allowed to set a table AM for a partitioned table, as
we've not resolved how partitions would inherit that. Disallowing
allows us to introduce, if we decide that's the way forward, such a
behaviour without a compatibility break.
Catversion bumped, to add the heap table AM and references to it.
Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de
https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
|
|
|
extern const TableAmRoutine *GetTableAmRoutine(Oid amhandler);
|
|
|
|
|
extern const TableAmRoutine *GetTableAmRoutineByAmId(Oid amoid);
|
|
|
|
|
extern const TableAmRoutine *GetHeapamTableAmRoutine(void);
|
|
|
|
|
extern bool check_default_table_access_method(char **newval, void **extra,
|
|
|
|
|
GucSource source);
|
|
|
|
|
|
|
|
|
|
#endif /* TABLEAM_H */
|
