rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/include/utils/catcache.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

229 lines
8.3 KiB
C
Raw Normal View History

More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* catcache.h
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
* Low-level catalog cache definitions.
*
Clean up syscache so that recursive invocation is safe, and remove error message about recursive use of a syscache. Also remove most of the specialized indexscan routines in indexing.c --- it turns out that catcache.c is perfectly able to perform the indexscan for itself, in fact has already looked up all the information needed to do so! This should be faster as well as needing far less boilerplate code.
2000-11-10 01:33:12 +01:00
* NOTE: every catalog cache must have a corresponding unique index on
* the system table that it caches --- ie, the index must match the keys
* used to do lookups in this cache. All cache fetches are done with
* indexscans (under normal conditions). The index should be unique to
* guarantee that there can only be one matching row for a key combination.
*
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
*
Update copyright for 2023 Backpatch-through: 11
2023-01-02 21:00:37 +01:00
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
*
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/include/utils/catcache.h
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
*
*-------------------------------------------------------------------------
*/
#ifndef CATCACHE_H
#define CATCACHE_H
Change #include's to use <> and "" as appropriate.
1999-07-16 01:04:24 +02:00
#include "access/htup.h"
Add operator strategy and comparison-value datatype fields to ScanKey. Remove the 'strategy map' code, which was a large amount of mechanism that no longer had any use except reverse-mapping from procedure OID to strategy number. Passing the strategy number to the index AM in the first place is simpler and faster. This is a preliminary step in planned support for cross-datatype index operations. I'm committing it now since the ScanKeyEntryInitialize() API change touches quite a lot of files, and I want to commit those changes before the tree drifts under me.
2003-11-09 22:30:38 +01:00
#include "access/skey.h"
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
#include "lib/ilist.h"
Improve our #include situation by moving pointer types away from the corresponding struct definitions. This allows other headers to avoid including certain highly-loaded headers such as rel.h and relscan.h, instead using just relcache.h, heapam.h or genam.h, which are more lightweight and thus cause less unnecessary dependencies.
2008-06-19 02:46:06 +02:00
#include "utils/relcache.h"
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
Clean out another pocket of functions called via nonspecific function pointers, namely the catcache tuple fetch routines. Also get rid of the unused and possibly confusing 'size' field in struct cachedesc. Since it doesn't allow for variable-length fields, anyone who actually trusted it would likely be making a mistake...
2000-06-17 06:56:39 +02:00
/*
Clean up syscache so that recursive invocation is safe, and remove error message about recursive use of a syscache. Also remove most of the specialized indexscan routines in indexing.c --- it turns out that catcache.c is perfectly able to perform the indexscan for itself, in fact has already looked up all the information needed to do so! This should be faster as well as needing far less boilerplate code.
2000-11-10 01:33:12 +01:00
* struct catctup: individual tuple in the cache.
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
* struct catclist: list of tuples matching a partial key.
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
* struct catcache: information for managing a cache.
Do some restructuring to improve performance of the catcaches. Teach CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed) how to distinguish tuples it should flush from those it needn't; this means a relcache flush event now only removes the catcache entries it ought to, rather than zapping the caches completely as it used to. Testing with the regression tests indicates that this considerably improves the lifespan of catcache entries. Also, rearrange catcache data structures so that the limit on number of cached tuples applies globally across all the catcaches, rather than being per-catcache. It was a little silly to have the same size limit on both, say, pg_attribute caches and pg_am caches (there being only four possible rows in the latter...). Doing LRU removal across all the caches instead of locally in each one should reduce cache reload traffic in the more heavily used caches and improve the efficiency of cache memory use.
2001-06-18 05:35:07 +02:00
* struct catcacheheader: information for managing all the caches.
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
*/
Wrap calls to SearchSysCache and related functions using macros. The purpose of this change is to eliminate the need for every caller of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists, GetSysCacheOid, and SearchSysCacheList to know the maximum number of allowable keys for a syscache entry (currently 4). This will make it far easier to increase the maximum number of keys in a future release should we choose to do so, and it makes the code shorter, too. Design and review by Tom Lane.
2010-02-14 19:42:19 +01:00
#define CATCACHE_MAXKEYS 4
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
/* function computing a datum's hash */
typedef uint32 (*CCHashFN) (Datum datum);
/* function computing equality of two datums */
typedef bool (*CCFastEqualFN) (Datum a, Datum b);
Do some restructuring to improve performance of the catcaches. Teach CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed) how to distinguish tuples it should flush from those it needn't; this means a relcache flush event now only removes the catcache entries it ought to, rather than zapping the caches completely as it used to. Testing with the regression tests indicates that this considerably improves the lifespan of catcache entries. Also, rearrange catcache data structures so that the limit on number of cached tuples applies globally across all the catcaches, rather than being per-catcache. It was a little silly to have the same size limit on both, say, pg_attribute caches and pg_am caches (there being only four possible rows in the latter...). Doing LRU removal across all the caches instead of locally in each one should reduce cache reload traffic in the more heavily used caches and improve the efficiency of cache memory use.
2001-06-18 05:35:07 +02:00
typedef struct catcache
{
int id; /* cache identifier --- see syscache.h */
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
int cc_nbuckets; /* # of hash buckets in this cache */
TupleDesc cc_tupdesc; /* tuple descriptor (copied from reldesc) */
dlist_head *cc_bucket; /* hash buckets */
CCHashFN cc_hashfunc[CATCACHE_MAXKEYS]; /* hash function for each key */
CCFastEqualFN cc_fastequal[CATCACHE_MAXKEYS]; /* fast equal function for
* each key */
int cc_keyno[CATCACHE_MAXKEYS]; /* AttrNumber of each key */
dlist_head cc_lists; /* list of CatCList structs */
int cc_ntup; /* # of tuples currently in this cache */
int cc_nkeys; /* # of keys (1..CATCACHE_MAXKEYS) */
pg_class has a relnamespace column. You can create and access tables in schemas other than the system namespace; however, there's no search path yet, and not all operations work yet on tables outside the system namespace.
2002-03-26 20:17:02 +01:00
const char *cc_relname; /* name of relation the tuples come from */
Catcaches can now store negative entries as well as positive ones, to speed up repetitive failed searches; per pghackers discussion in late January. inval.c logic substantially simplified, since we can now treat inserts and deletes alike as far as inval events are concerned. Some repair work needed in heap_create_with_catalog, which turns out to have been doing CommandCounterIncrement at a point where the new relation has non-self-consistent catalog entries. With the new inval code, that resulted in assert failures during a relcache entry rebuild.
2002-03-03 18:47:56 +01:00
Oid cc_reloid; /* OID of relation the tuples come from */
Completion of project to use fixed OIDs for all system catalogs and indexes. Replace all heap_openr and index_openr calls by heap_open and index_open. Remove runtime lookups of catalog OID numbers in various places. Remove relcache's support for looking up system catalogs by name. Bulky but mostly very boring patch ...
2005-04-14 22:03:27 +02:00
Oid cc_indexoid; /* OID of index matching cache keys */
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
bool cc_relisshared; /* is relation shared across databases? */
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
slist_node cc_next; /* list link */
Wrap calls to SearchSysCache and related functions using macros. The purpose of this change is to eliminate the need for every caller of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists, GetSysCacheOid, and SearchSysCacheList to know the maximum number of allowable keys for a syscache entry (currently 4). This will make it far easier to increase the maximum number of keys in a future release should we choose to do so, and it makes the code shorter, too. Design and review by Tom Lane.
2010-02-14 19:42:19 +01:00
ScanKeyData cc_skey[CATCACHE_MAXKEYS]; /* precomputed key info for heap
* scans */
Fix field order in struct catcache. Somebody failed to grasp the point of having the #ifdef CATCACHE_STATS fields at the end of the struct. Put that back the way it should be, and add a comment making it more explicit why it should be that way.
2017-01-13 00:59:46 +01:00
/*
* Keep these at the end, so that compiling catcache.c with CATCACHE_STATS
* doesn't break ABI for other modules
*/
A bunch of changes aimed at reducing backend startup time... Improve 'pg_internal.init' relcache entry preload mechanism so that it is safe to use for all system catalogs, and arrange to preload a realistic set of system-catalog entries instead of only the three nailed-in-cache indexes that were formerly loaded this way. Fix mechanism for deleting out-of-date pg_internal.init files: this must be synchronized with transaction commit, not just done at random times within transactions. Drive it off relcache invalidation mechanism so that no special-case tests are needed. Cache additional information in relcache entries for indexes (their pg_index tuples and index-operator OIDs) to eliminate repeated lookups. Also cache index opclass info at the per-opclass level to avoid repeated lookups during relcache load. Generalize 'systable scan' utilities originally developed by Hiroshi, move them into genam.c, use in a number of places where there was formerly ugly code for choosing either heap or index scan. In particular this allows simplification of the logic that prevents infinite recursion between syscache and relcache during startup: we can easily switch to heapscans in relcache.c when and where needed to avoid recursion, so IndexScanOK becomes simpler and does not need any expensive initialization. Eliminate useless opening of a heapscan data structure while doing an indexscan (this saves an mdnblocks call and thus at least one kernel call).
2002-02-19 21:11:20 +01:00
#ifdef CATCACHE_STATS
long cc_searches; /* total # searches against this cache */
long cc_hits; /* # of matches against existing entry */
Catcaches can now store negative entries as well as positive ones, to speed up repetitive failed searches; per pghackers discussion in late January. inval.c logic substantially simplified, since we can now treat inserts and deletes alike as far as inval events are concerned. Some repair work needed in heap_create_with_catalog, which turns out to have been doing CommandCounterIncrement at a point where the new relation has non-self-consistent catalog entries. With the new inval code, that resulted in assert failures during a relcache entry rebuild.
2002-03-03 18:47:56 +01:00
long cc_neg_hits; /* # of matches against negative entry */
A bunch of changes aimed at reducing backend startup time... Improve 'pg_internal.init' relcache entry preload mechanism so that it is safe to use for all system catalogs, and arrange to preload a realistic set of system-catalog entries instead of only the three nailed-in-cache indexes that were formerly loaded this way. Fix mechanism for deleting out-of-date pg_internal.init files: this must be synchronized with transaction commit, not just done at random times within transactions. Drive it off relcache invalidation mechanism so that no special-case tests are needed. Cache additional information in relcache entries for indexes (their pg_index tuples and index-operator OIDs) to eliminate repeated lookups. Also cache index opclass info at the per-opclass level to avoid repeated lookups during relcache load. Generalize 'systable scan' utilities originally developed by Hiroshi, move them into genam.c, use in a number of places where there was formerly ugly code for choosing either heap or index scan. In particular this allows simplification of the logic that prevents infinite recursion between syscache and relcache during startup: we can easily switch to heapscans in relcache.c when and where needed to avoid recursion, so IndexScanOK becomes simpler and does not need any expensive initialization. Eliminate useless opening of a heapscan data structure while doing an indexscan (this saves an mdnblocks call and thus at least one kernel call).
2002-02-19 21:11:20 +01:00
long cc_newloads; /* # of successful loads of new entry */
pgindent run.
2002-09-04 22:31:48 +02:00
Catcaches can now store negative entries as well as positive ones, to speed up repetitive failed searches; per pghackers discussion in late January. inval.c logic substantially simplified, since we can now treat inserts and deletes alike as far as inval events are concerned. Some repair work needed in heap_create_with_catalog, which turns out to have been doing CommandCounterIncrement at a point where the new relation has non-self-consistent catalog entries. With the new inval code, that resulted in assert failures during a relcache entry rebuild.
2002-03-03 18:47:56 +01:00
/*
* cc_searches - (cc_hits + cc_neg_hits + cc_newloads) is number of failed
* searches, each of which will result in loading a negative entry
*/
long cc_invals; /* # of entries invalidated from cache */
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
long cc_lsearches; /* total # list-searches */
long cc_lhits; /* # of matches against existing lists */
A bunch of changes aimed at reducing backend startup time... Improve 'pg_internal.init' relcache entry preload mechanism so that it is safe to use for all system catalogs, and arrange to preload a realistic set of system-catalog entries instead of only the three nailed-in-cache indexes that were formerly loaded this way. Fix mechanism for deleting out-of-date pg_internal.init files: this must be synchronized with transaction commit, not just done at random times within transactions. Drive it off relcache invalidation mechanism so that no special-case tests are needed. Cache additional information in relcache entries for indexes (their pg_index tuples and index-operator OIDs) to eliminate repeated lookups. Also cache index opclass info at the per-opclass level to avoid repeated lookups during relcache load. Generalize 'systable scan' utilities originally developed by Hiroshi, move them into genam.c, use in a number of places where there was formerly ugly code for choosing either heap or index scan. In particular this allows simplification of the logic that prevents infinite recursion between syscache and relcache during startup: we can easily switch to heapscans in relcache.c when and where needed to avoid recursion, so IndexScanOK becomes simpler and does not need any expensive initialization. Eliminate useless opening of a heapscan data structure while doing an indexscan (this saves an mdnblocks call and thus at least one kernel call).
2002-02-19 21:11:20 +01:00
#endif
Make catalog cache hash tables resizeable. If the hash table backing a catalog cache becomes too full (fillfactor > 2), enlarge it. A new buckets array, double the size of the old, is allocated, and all entries in the old hash are moved to the right bucket in the new hash. This has two benefits. First, cache lookups don't get so expensive when there are lots of entries in a cache, like if you access hundreds of thousands of tables. Second, we can make the (initial) sizes of the caches much smaller, which saves memory. This patch dials down the initial sizes of the catcaches. The new sizes are chosen so that a backend that only runs a few basic queries still won't need to enlarge any of them.
2013-09-05 18:47:56 +02:00
} CatCache;
Do some restructuring to improve performance of the catcaches. Teach CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed) how to distinguish tuples it should flush from those it needn't; this means a relcache flush event now only removes the catcache entries it ought to, rather than zapping the caches completely as it used to. Testing with the regression tests indicates that this considerably improves the lifespan of catcache entries. Also, rearrange catcache data structures so that the limit on number of cached tuples applies globally across all the catcaches, rather than being per-catcache. It was a little silly to have the same size limit on both, say, pg_attribute caches and pg_am caches (there being only four possible rows in the latter...). Doing LRU removal across all the caches instead of locally in each one should reduce cache reload traffic in the more heavily used caches and improve the efficiency of cache memory use.
2001-06-18 05:35:07 +02:00
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
typedef struct catctup
{
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
int ct_magic; /* for identifying CatCTup entries */
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
#define CT_MAGIC 0x57261502
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
uint32 hash_value; /* hash value for this tuple's keys */
/*
* Lookup keys for the entry. By-reference datums point into the tuple for
* positive cache entries, and are separately allocated for negative ones.
*/
Datum keys[CATCACHE_MAXKEYS];
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Fix for failure to clean SysCache entry when a relation is deleted in the same transaction that created it.
1999-06-04 04:19:47 +02:00
/*
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
* Each tuple in a cache is a member of a dlist that stores the elements
* of its hash bucket. We keep each dlist in LRU order to speed repeated
Remove the limit on the number of entries allowed in catcaches, and remove the infrastructure needed to enforce the limit, ie, the global LRU list of cache entries. On small-to-middling databases this wins because maintaining the LRU list is a waste of time. On large databases this wins because it's better to keep more cache entries (we assume such users can afford to use some more per-backend memory than was contemplated in the Berkeley-era catcache design). This provides a noticeable improvement in the speed of psql \d on a 10000-table database, though it doesn't make it instantaneous. While at it, use per-catcache settings for the number of hash buckets per catcache, rather than the former one-size-fits-all value. It's a bit silly to be using the same number of hash buckets for, eg, pg_am and pg_attribute. The specific values I used might need some tuning, but they seem to be in the right ballpark based on CATCACHE_STATS results from the standard regression tests.
2006-06-15 04:08:09 +02:00
* lookups.
Tweak catalog cache management algorithms to reduce cost of SearchCatCacheList and ReleaseCatCacheList. Previously, we incremented and decremented the refcounts of list member tuples along with the list itself, but that's unnecessary, and very expensive when the list is big. It's cheaper to change only the list refcount. When we are considering deleting a cache entry, we have to check not only its own refcount but its parent list's ... but it's easy to arrange the code so that this check is not made in any commonly-used paths, so the cost is really nil. The bigger gain though is to refrain from DLMoveToFront'ing each individual member tuple each time the list is referenced. To keep some semblance of fair space management, lists are just marked as used or not since the last cache cleanout search, and we do a MoveToFront pass only when about to run a cleanout. In combination, these changes reduce the costs of SearchCatCacheList and ReleaseCatCacheList from about 4.5% of pgbench runtime to under 1%, according to my gprof results.
2005-08-14 00:18:07 +02:00
*/
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_node cache_elem; /* list member of per-bucket list */
Tweak catalog cache management algorithms to reduce cost of SearchCatCacheList and ReleaseCatCacheList. Previously, we incremented and decremented the refcounts of list member tuples along with the list itself, but that's unnecessary, and very expensive when the list is big. It's cheaper to change only the list refcount. When we are considering deleting a cache entry, we have to check not only its own refcount but its parent list's ... but it's easy to arrange the code so that this check is not made in any commonly-used paths, so the cost is really nil. The bigger gain though is to refrain from DLMoveToFront'ing each individual member tuple each time the list is referenced. To keep some semblance of fair space management, lists are just marked as used or not since the last cache cleanout search, and we do a MoveToFront pass only when about to run a cleanout. In combination, these changes reduce the costs of SearchCatCacheList and ReleaseCatCacheList from about 4.5% of pgbench runtime to under 1%, according to my gprof results.
2005-08-14 00:18:07 +02:00
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
/*
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
* A tuple marked "dead" must not be returned by subsequent searches.
* However, it won't be physically deleted from the cache until its
Tweak catalog cache management algorithms to reduce cost of SearchCatCacheList and ReleaseCatCacheList. Previously, we incremented and decremented the refcounts of list member tuples along with the list itself, but that's unnecessary, and very expensive when the list is big. It's cheaper to change only the list refcount. When we are considering deleting a cache entry, we have to check not only its own refcount but its parent list's ... but it's easy to arrange the code so that this check is not made in any commonly-used paths, so the cost is really nil. The bigger gain though is to refrain from DLMoveToFront'ing each individual member tuple each time the list is referenced. To keep some semblance of fair space management, lists are just marked as used or not since the last cache cleanout search, and we do a MoveToFront pass only when about to run a cleanout. In combination, these changes reduce the costs of SearchCatCacheList and ReleaseCatCacheList from about 4.5% of pgbench runtime to under 1%, according to my gprof results.
2005-08-14 00:18:07 +02:00
* refcount goes to zero. (If it's a member of a CatCList, the list's
* refcount must go to zero, too; also, remember to mark the list dead at
* the same time the tuple is marked.)
Catcaches can now store negative entries as well as positive ones, to speed up repetitive failed searches; per pghackers discussion in late January. inval.c logic substantially simplified, since we can now treat inserts and deletes alike as far as inval events are concerned. Some repair work needed in heap_create_with_catalog, which turns out to have been doing CommandCounterIncrement at a point where the new relation has non-self-consistent catalog entries. With the new inval code, that resulted in assert failures during a relcache entry rebuild.
2002-03-03 18:47:56 +01:00
*
* A negative cache entry is an assertion that there is no tuple matching
* a particular key. This is just as useful as a normal entry so far as
* avoiding catalog searches is concerned. Management of positive and
* negative entries is identical.
Fix for failure to clean SysCache entry when a relation is deleted in the same transaction that created it.
1999-06-04 04:19:47 +02:00
*/
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
int refcount; /* number of active references */
bool dead; /* dead but not yet removed? */
Catcaches can now store negative entries as well as positive ones, to speed up repetitive failed searches; per pghackers discussion in late January. inval.c logic substantially simplified, since we can now treat inserts and deletes alike as far as inval events are concerned. Some repair work needed in heap_create_with_catalog, which turns out to have been doing CommandCounterIncrement at a point where the new relation has non-self-consistent catalog entries. With the new inval code, that resulted in assert failures during a relcache entry rebuild.
2002-03-03 18:47:56 +01:00
bool negative; /* negative cache entry? */
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
HeapTupleData tuple; /* tuple management header */
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
/*
* The tuple may also be a member of at most one CatCList. (If a single
* catcache is list-searched with varying numbers of keys, we may have to
* make multiple entries for the same tuple because of this restriction.
* Currently, that's not expected to be common, so we accept the potential
* inefficiency.)
*/
struct catclist *c_list; /* containing CatCList, or NULL if none */
CatCache *my_cache; /* link to owning catcache */
/* properly aligned tuple data follows, unless a negative entry */
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
} CatCTup;
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
/*
* A CatCList describes the result of a partial search, ie, a search using
* only the first K key columns of an N-key cache. We store the keys used
* into the keys attribute to represent the stored key set. The CatCList
* object contains links to cache entries for all the table rows satisfying
* the partial key. (Note: none of these will be negative cache entries.)
*
* A CatCList is only a member of a per-cache list; we do not currently
* divide them into hash buckets.
*
* A list marked "dead" must not be returned by subsequent searches.
* However, it won't be physically deleted from the cache until its
* refcount goes to zero. (A list should be marked dead if any of its
* member entries are dead.)
*
* If "ordered" is true then the member tuples appear in the order of the
* cache's underlying index. This will be true in normal operation, but
* might not be true during bootstrap or recovery operations. (namespace.c
* is able to save some cycles when it is true.)
*/
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
typedef struct catclist
{
int cl_magic; /* for identifying CatCList entries */
#define CL_MAGIC 0x52765103
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
uint32 hash_value; /* hash value for lookup keys */
dlist_node cache_elem; /* list member of per-catcache list */
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
/*
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
* Lookup keys for the entry, with the first nkeys elements being valid.
* All by-reference are separately allocated.
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
*/
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
Datum keys[CATCACHE_MAXKEYS];
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
int refcount; /* number of active references */
bool dead; /* dead but not yet removed? */
bool ordered; /* members listed in index order? */
short nkeys; /* number of lookup keys specified */
int n_members; /* number of member tuples */
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
CatCache *my_cache; /* link to owning catcache */
Use FLEXIBLE_ARRAY_MEMBER in a bunch more places. Replace some bogus "x[1]" declarations with "x[FLEXIBLE_ARRAY_MEMBER]". Aside from being more self-documenting, this should help prevent bogus warnings from static code analyzers and perhaps compiler misoptimizations. This patch is just a down payment on eliminating the whole problem, but it gets rid of a lot of easy-to-fix cases. Note that the main problem with doing this is that one must no longer rely on computing sizeof(the containing struct), since the result would be compiler-dependent. Instead use offsetof(struct, lastfield). Autoconf also warns against spelling that offsetof(struct, lastfield[0]). Michael Paquier, review and additional fixes by me.
2015-02-20 06:11:42 +01:00
CatCTup *members[FLEXIBLE_ARRAY_MEMBER]; /* members */
} CatCList;
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
Do some restructuring to improve performance of the catcaches. Teach CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed) how to distinguish tuples it should flush from those it needn't; this means a relcache flush event now only removes the catcache entries it ought to, rather than zapping the caches completely as it used to. Testing with the regression tests indicates that this considerably improves the lifespan of catcache entries. Also, rearrange catcache data structures so that the limit on number of cached tuples applies globally across all the catcaches, rather than being per-catcache. It was a little silly to have the same size limit on both, say, pg_attribute caches and pg_am caches (there being only four possible rows in the latter...). Doing LRU removal across all the caches instead of locally in each one should reduce cache reload traffic in the more heavily used caches and improve the efficiency of cache memory use.
2001-06-18 05:35:07 +02:00
typedef struct catcacheheader
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
slist_head ch_caches; /* head of list of CatCache structs */
Do some restructuring to improve performance of the catcaches. Teach CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed) how to distinguish tuples it should flush from those it needn't; this means a relcache flush event now only removes the catcache entries it ought to, rather than zapping the caches completely as it used to. Testing with the regression tests indicates that this considerably improves the lifespan of catcache entries. Also, rearrange catcache data structures so that the limit on number of cached tuples applies globally across all the catcaches, rather than being per-catcache. It was a little silly to have the same size limit on both, say, pg_attribute caches and pg_am caches (there being only four possible rows in the latter...). Doing LRU removal across all the caches instead of locally in each one should reduce cache reload traffic in the more heavily used caches and improve the efficiency of cache memory use.
2001-06-18 05:35:07 +02:00
int ch_ntup; /* # of tuples in all caches */
} CatCacheHeader;
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
/* this extern duplicates utils/memutils.h... */
Rename DLLIMPORT macro to PGDLLIMPORT to avoid conflict with third party includes (like tcl) that define DLLIMPORT.
2007-07-25 14:22:54 +02:00
extern PGDLLIMPORT MemoryContext CacheMemoryContext;
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
extern void CreateCacheMemoryContext(void);
Clean up syscache so that recursive invocation is safe, and remove error message about recursive use of a syscache. Also remove most of the specialized indexscan routines in indexing.c --- it turns out that catcache.c is perfectly able to perform the indexscan for itself, in fact has already looked up all the information needed to do so! This should be faster as well as needing far less boilerplate code.
2000-11-10 01:33:12 +01:00
Completion of project to use fixed OIDs for all system catalogs and indexes. Replace all heap_openr and index_openr calls by heap_open and index_open. Remove runtime lookups of catalog OID numbers in various places. Remove relcache's support for looking up system catalogs by name. Bulky but mostly very boring patch ...
2005-04-14 22:03:27 +02:00
extern CatCache *InitCatCache(int id, Oid reloid, Oid indexoid,
Remove the limit on the number of entries allowed in catcaches, and remove the infrastructure needed to enforce the limit, ie, the global LRU list of cache entries. On small-to-middling databases this wins because maintaining the LRU list is a waste of time. On large databases this wins because it's better to keep more cache entries (we assume such users can afford to use some more per-backend memory than was contemplated in the Berkeley-era catcache design). This provides a noticeable improvement in the speed of psql \d on a 10000-table database, though it doesn't make it instantaneous. While at it, use per-catcache settings for the number of hash buckets per catcache, rather than the former one-size-fits-all value. It's a bit silly to be using the same number of hash buckets for, eg, pg_am and pg_attribute. The specific values I used might need some tuning, but they seem to be in the right ballpark based on CATCACHE_STATS results from the standard regression tests.
2006-06-15 04:08:09 +02:00
int nkeys, const int *key,
int nbuckets);
Fix SysCacheGetAttr() to handle the case where the specified syscache has not been initialized yet. This can happen because there are code paths that call SysCacheGetAttr() on a tuple originally fetched from a different syscache (hopefully on the same catalog) than the one specified in the call. It doesn't seem useful or robust to try to prevent that from happening, so just improve the function to cope instead. Per bug#2678 from Jeff Trout. The specific example shown by Jeff is new in 8.1, but to be on the safe side I'm backpatching 8.0 as well. We could patch 7.x similarly but I think that's probably overkill, given the lack of evidence of old bugs of this ilk.
2006-10-06 20:23:35 +02:00
extern void InitCatCachePhase2(CatCache *cache, bool touch_index);
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
extern HeapTuple SearchCatCache(CatCache *cache,
Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
2017-10-13 22:16:50 +02:00
Datum v1, Datum v2, Datum v3, Datum v4);
extern HeapTuple SearchCatCache1(CatCache *cache,
Datum v1);
extern HeapTuple SearchCatCache2(CatCache *cache,
Datum v1, Datum v2);
extern HeapTuple SearchCatCache3(CatCache *cache,
Datum v1, Datum v2, Datum v3);
extern HeapTuple SearchCatCache4(CatCache *cache,
Datum v1, Datum v2, Datum v3, Datum v4);
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
extern void ReleaseCatCache(HeapTuple tuple);
Clean up syscache so that recursive invocation is safe, and remove error message about recursive use of a syscache. Also remove most of the specialized indexscan routines in indexing.c --- it turns out that catcache.c is perfectly able to perform the indexscan for itself, in fact has already looked up all the information needed to do so! This should be faster as well as needing far less boilerplate code.
2000-11-10 01:33:12 +01:00
Expose an API for calculating catcache hash values. Now that cache invalidation callbacks get only a hash value, and not a tuple TID (per commits 632ae6829f7abda34e15082c91d9dfb3fc0f298b and b5282aa893e565b7844f8237462cb843438cdd5e), the only way they can restrict what they invalidate is to know what the hash values mean. setrefs.c was doing this via a hard-wired assumption but that seems pretty grotty, and it'll only get worse as more cases come up. So let's expose a calculation function that takes the same parameters as SearchSysCache. Per complaint from Marko Kreen.
2012-03-07 20:51:13 +01:00
extern uint32 GetCatCacheHashValue(CatCache *cache,
Datum v1, Datum v2,
Datum v3, Datum v4);
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
extern CatCList *SearchCatCacheList(CatCache *cache, int nkeys,
Datum v1, Datum v2,
Save a few bytes by removing useless last argument to SearchCatCacheList. There's never any value in giving a fully specified cache key to SearchCatCacheList: you might as well call SearchCatCache instead, since there could be only one match. So the maximum useful number of key arguments is one less than the supported number of key columns. We might as well remove the useless extra argument and save some few bytes per call site, as well as a cycle or so per call. I believe the reason it was coded like this is that originally, callers had to write out all the dummy arguments in each call, and so it seemed less confusing if SearchCatCache and SearchCatCacheList took the same number of key arguments. But since commit e26c539e9, callers only write their live arguments explicitly, making that a non-factor; and there's surely been enough time for third-party modules to adapt to that coding style. So this is only an ABI break not an API break for callers. Per discussion with Oliver Ford, this might also make it less confusing how to use SearchCatCacheList correctly. Discussion: https://postgr.es/m/27788.1517069693@sss.pgh.pa.us
2018-01-29 21:13:07 +01:00
Datum v3);
Implement partial-key searching of syscaches, per recent suggestion to pghackers. Use this to do searching for ambiguous functions --- it will get more uses soon.
2002-04-06 08:59:25 +02:00
extern void ReleaseCatCacheList(CatCList *list);
Do some restructuring to improve performance of the catcaches. Teach CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed) how to distinguish tuples it should flush from those it needn't; this means a relcache flush event now only removes the catcache entries it ought to, rather than zapping the caches completely as it used to. Testing with the regression tests indicates that this considerably improves the lifespan of catcache entries. Also, rearrange catcache data structures so that the limit on number of cached tuples applies globally across all the catcaches, rather than being per-catcache. It was a little silly to have the same size limit on both, say, pg_attribute caches and pg_am caches (there being only four possible rows in the latter...). Doing LRU removal across all the caches instead of locally in each one should reduce cache reload traffic in the more heavily used caches and improve the efficiency of cache memory use.
2001-06-18 05:35:07 +02:00
extern void ResetCatalogCaches(void);
Create a "relation mapping" infrastructure to support changing the relfilenodes of shared or nailed system catalogs. This has two key benefits: * The new CLUSTER-based VACUUM FULL can be applied safely to all catalogs. * We no longer have to use an unsafe reindex-in-place approach for reindexing shared catalogs. CLUSTER on nailed catalogs now works too, although I left it disabled on shared catalogs because the resulting pg_index.indisclustered update would only be visible in one database. Since reindexing shared system catalogs is now fully transactional and crash-safe, the former special cases in REINDEX behavior have been removed; shared catalogs are treated the same as non-shared. This commit does not do anything about the recently-discussed problem of deadlocks between VACUUM FULL/CLUSTER on a system catalog and other concurrent queries; will address that in a separate patch. As a stopgap, parallel_schedule has been tweaked to run vacuum.sql by itself, to avoid such failures during the regression tests.
2010-02-07 21:48:13 +01:00
extern void CatalogCacheFlushCatalog(Oid catId);
Avoid searching for the target catcache in CatalogCacheIdInvalidate. A test case provided by Mathieu Fenniak shows that the initial search for the target catcache in CatalogCacheIdInvalidate consumes a very significant amount of overhead in cases where cache invalidation is triggered but has little useful work to do. There is no good reason for that search to exist at all, as the index array maintained by syscache.c allows direct lookup of the catcache from its ID. We just need a frontend function in syscache.c, matching the division of labor for most other cache-accessing operations. While there's more that can be done in this area, this patch alone reduces the runtime of Mathieu's example by 2X. We can hope that it offers some useful benefit in other cases too, although usually cache invalidation overhead is not such a striking fraction of the total runtime. Back-patch to 9.4 where logical decoding was introduced. It might be worth going further back, but presently the only case we know of where cache invalidation is really a significant burden is in logical decoding. Also, older branches have fewer catcaches, reducing the possible benefit. (Note: although this nominally changes catcache's API, we have always documented CatalogCacheIdInvalidate as a private function, so I would have little sympathy for an external module calling it directly. So backpatching should be fine.) Discussion: https://postgr.es/m/CAHoiPjzea6N0zuCi=+f9v_j94nfsy6y8SU7-=bp4=7qw6_i=Rg@mail.gmail.com
2017-05-13 00:17:29 +02:00
extern void CatCacheInvalidate(CatCache *cache, uint32 hashValue);
Rename and document some invalidation routines to make it clearer that they don't themselves flush any cache entries, only add to to-do lists that will be processed later.
2001-01-05 23:54:37 +01:00
extern void PrepareToInvalidateCacheTuple(Relation relation,
Clean up some longstanding problems in shared-cache invalidation. SI messages now include the relevant database OID, so that operations in one database do not cause useless cache flushes in backends attached to other databases. Declare SI messages properly using a union, to eliminate the former assumption that Oid is the same size as int or Index. Rewrite the nearly-unreadable code in inval.c, and document it better. Arrange for catcache flushes at end of command/transaction to happen before relcache flushes do --- this avoids loading a new tuple into the catcache while setting up new relcache entry, only to have it be flushed again immediately.
2001-06-19 21:42:16 +02:00
HeapTuple tuple,
Revise sinval code to remove no-longer-used tuple TID from inval messages. This requires adjusting the API for syscache callback functions: they now get a hash value, not a TID, to identify the target tuple. Most of them weren't paying any attention to that argument anyway, but plancache did require a small amount of fixing. Also, improve performance a trifle by avoiding sending duplicate inval messages when a heap_update isn't changing the catcache lookup columns.
2011-08-17 01:27:46 +02:00
HeapTuple newtuple,
void (*function) (int, uint32, Oid));
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
More cleanups of the include files - centralizing to simplify the -I's required to compile
1996-08-28 03:59:28 +02:00
#endif /* CATCACHE_H */