rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/include/access/hash.h

373 lines
14 KiB
C
Raw Normal View History

Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* hash.h
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* header file for postgres hash access method implementation
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*
*
Update copyright for 2016 Backpatch certain files through 9.1
2016-01-02 19:33:40 +01:00
* Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/include/access/hash.h
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*
* NOTES
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* modeled after Margo Seltzer's hash implementation for unix.
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*
*-------------------------------------------------------------------------
*/
#ifndef HASH_H
#define HASH_H
Restructure index access method API to hide most of it at the C level. This patch reduces pg_am to just two columns, a name and a handler function. All the data formerly obtained from pg_am is now provided in a C struct returned by the handler function. This is similar to the designs we've adopted for FDWs and tablesample methods. There are multiple advantages. For one, the index AM's support functions are now simple C functions, making them faster to call and much less error-prone, since the C compiler can now check function signatures. For another, this will make it far more practical to define index access methods in installable extensions. A disadvantage is that SQL-level code can no longer see attributes of index AMs; in particular, some of the crosschecks in the opr_sanity regression test are no longer possible from SQL. We've addressed that by adding a facility for the index AM to perform such checks instead. (Much more could be done in that line, but for now we're content if the amvalidate functions more or less replace what opr_sanity used to do.) We might also want to expose some sort of reporting functionality, but this patch doesn't do that. Alexander Korotkov, reviewed by Petr Jelínek, and rather heavily editorialized on by me.
2016-01-18 01:36:59 +01:00
#include "access/amapi.h"
Change #include's to use <> and "" as appropriate.
1999-07-16 01:04:24 +02:00
#include "access/itup.h"
More cleanup
1999-07-16 19:07:40 +02:00
#include "access/sdir.h"
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 16:56:26 +01:00
#include "access/xlogreader.h"
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
#include "fmgr.h"
Move the backup-block logic from XLogInsert to a new file, xloginsert.c. xlog.c is huge, this makes it a little bit smaller, which is nice. Functions related to putting together the WAL record are in xloginsert.c, and the lower level stuff for managing WAL buffers and such are in xlog.c. Also move the definition of XLogRecord to a separate header file. This causes churn in the #includes of all the files that write WAL records, and redo routines, but it avoids pulling in xlog.h into most places. Reviewed by Michael Paquier, Alvaro Herrera, Andres Freund and Amit Kapila.
2014-11-06 12:52:08 +01:00
#include "lib/stringinfo.h"
Allow more include files to be compiled in their own by adding missing include dependencies. Modify pgcompinclude to skip a common fcinfo error.
2011-08-27 17:05:33 +02:00
#include "storage/bufmgr.h"
Don't include low level locking code from frontend code. Some frontend code like e.g. pg_xlogdump or pg_resetxlog, has to use backend headers. Unfortunately until now that code includes most of the locking code. It's generally not nice to expose such low level details, but de6fd1c898 made that a hard problem. We fall back to defining 'inline' away if the compiler doesn't support it - that can cause linker errors like on buildfarm animal pademelon if a inline function references backend only code. To fix that problem separate definitions from lock.h that are required from frontend code into lockdefs.h and use it in the relevant places. I've only removed the minimal amount of necessary definitions for now - it might turn out that we want more for other reasons. To avoid such details being exposed again put some checks against being included from frontend code into atomics.h, lock.h, lwlock.h and s_lock.h. It's otherwise fairly easy to indirectly include these headers. Discussion: 20150806070902.GE12214@awork2.anarazel.de
2015-08-07 15:10:56 +02:00
#include "storage/lockdefs.h"
Improve our #include situation by moving pointer types away from the corresponding struct definitions. This allows other headers to avoid including certain highly-loaded headers such as rel.h and relscan.h, instead using just relcache.h, heapam.h or genam.h, which are more lightweight and thus cause less unnecessary dependencies.
2008-06-19 02:46:06 +02:00
#include "utils/relcache.h"
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
* Mapping from hash bucket number to physical block number of bucket's
Rewrite hashbulkdelete() to make it amenable to new bucket locking scheme. A pleasant side effect is that it is *much* faster when deleting a large fraction of the indexed tuples, because of elimination of redundant hash_step activity induced by hash_adjscans. Various other continuing code cleanup.
2003-09-02 04:18:38 +02:00
* starting page. Beware of multiple evaluations of argument!
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
typedef uint32 Bucket;
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Rewrite hashbulkdelete() to make it amenable to new bucket locking scheme. A pleasant side effect is that it is *much* faster when deleting a large fraction of the indexed tuples, because of elimination of redundant hash_step activity induced by hash_adjscans. Various other continuing code cleanup.
2003-09-02 04:18:38 +02:00
#define BUCKET_TO_BLKNO(metap,B) \
((BlockNumber) ((B) + ((B) ? (metap)->hashm_spares[_hash_log2((B)+1)-1] : 0)) + 1)
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
* Special space for hash index pages.
*
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
* hasho_flag tells us which type of page we're looking at. For
* example, knowing overflow pages from bucket pages is necessary
* information when you're deleting tuples from a page. If all the
* tuples are deleted from an overflow page, the overflow is made
* available to other buckets by calling _hash_freeovflpage(). If all
* the tuples are deleted from a bucket page, no additional action is
* necessary.
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#define LH_UNUSED_PAGE (0)
#define LH_OVERFLOW_PAGE (1 << 0)
#define LH_BUCKET_PAGE (1 << 1)
#define LH_BITMAP_PAGE (1 << 2)
#define LH_META_PAGE (1 << 3)
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
typedef struct HashPageOpaqueData
{
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
BlockNumber hasho_prevblkno; /* previous ovfl (or bucket) blkno */
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
BlockNumber hasho_nextblkno; /* next ovfl blkno */
Bucket hasho_bucket; /* bucket number this pg belongs to */
uint16 hasho_flag; /* page type code, see above */
Minor tweaking of index special-space definitions so that the various index types can be reliably distinguished by examining the special space on an index page. Per my earlier proposal, plus the realization that there's no need for btree's vacuum cycle ID to cycle through every possible 16-bit value. Restricting its range a little costs nearly nothing and eliminates the possibility of collisions. Memo to self: remember to make bitmap indexes play along with this scheme, assuming that patch ever gets accepted.
2007-04-10 00:04:08 +02:00
uint16 hasho_page_id; /* for identification of hash indexes */
Used modified version of indent that understands over 100 typedefs.
1997-09-08 23:56:23 +02:00
} HashPageOpaqueData;
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
typedef HashPageOpaqueData *HashPageOpaque;
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Minor tweaking of index special-space definitions so that the various index types can be reliably distinguished by examining the special space on an index page. Per my earlier proposal, plus the realization that there's no need for btree's vacuum cycle ID to cycle through every possible 16-bit value. Restricting its range a little costs nearly nothing and eliminates the possibility of collisions. Memo to self: remember to make bitmap indexes play along with this scheme, assuming that patch ever gets accepted.
2007-04-10 00:04:08 +02:00
/*
* The page ID is for the convenience of pg_filedump and similar utilities,
* which otherwise would have a hard time telling pages of different index
* types apart. It should be the last 2 bytes on the page. This is more or
* less "free" due to alignment considerations.
*/
#define HASHO_PAGE_ID 0xFF80
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
* HashScanOpaqueData is private state for a hash index scan.
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
typedef struct HashScanOpaqueData
{
Change hash indexes to store only the hash code rather than the whole indexed value. This means that hash index lookups are always lossy and have to be rechecked when the heap is visited; however, the gain in index compactness outweighs this when the indexed values are wide. Also, we only need to perform datatype comparisons when the hash codes match exactly, rather than for every entry in the hash bucket; so it could also win for datatypes that have expensive comparison functions. A small additional win is gained by keeping hash index pages sorted by hash code and using binary search to reduce the number of index tuples we have to look at. Xiao Meng This commit also incorporates Zdenek Kotala's patch to isolate hash metapages and hash bitmaps a bit better from the page header datastructures.
2008-09-15 20:43:41 +02:00
/* Hash value of the scan key, ie, the hash key we seek */
uint32 hashso_sk_hash;
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* By definition, a hash scan should be examining only one bucket. We
* record the bucket number here as soon as it is known.
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
*/
Bucket hashso_bucket;
bool hashso_bucket_valid;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
/*
* If we have a share lock on the bucket, we record it here. When
* hashso_bucket_blkno is zero, we have no such lock.
*/
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
BlockNumber hashso_bucket_blkno;
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
/*
Remove useless mark/restore support in hash index AM, per discussion. (I'm leaving GiST/GIN cleanup to Teodor.)
2008-10-18 01:50:57 +02:00
* We also want to remember which buffer we're currently examining in the
* scan. We keep the buffer pinned (but not locked) across hashgettuple
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* calls, in order to avoid doing a ReadBuffer() for every tuple in the
* index.
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
*/
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
Buffer hashso_curbuf;
Refactor the index AM API slightly: move currentItemData and currentMarkData from IndexScanDesc to the opaque structs for the AMs that need this information (currently gist and hash). Patch from Heikki Linnakangas, fixes by Neil Conway.
2007-01-20 19:43:35 +01:00
Dept of second thoughts: after studying index_getnext() a bit more I realize that it can scribble on scan->xs_ctup.t_self while following HOT chains, so we can't rely on that to stay valid between hashgettuple() calls. Introduce a private variable in HashScanOpaque, instead.
2009-11-01 23:30:54 +01:00
/* Current position of the scan, as an index TID */
Refactor the index AM API slightly: move currentItemData and currentMarkData from IndexScanDesc to the opaque structs for the AMs that need this information (currently gist and hash). Patch from Heikki Linnakangas, fixes by Neil Conway.
2007-01-20 19:43:35 +01:00
ItemPointerData hashso_curpos;
Dept of second thoughts: after studying index_getnext() a bit more I realize that it can scribble on scan->xs_ctup.t_self while following HOT chains, so we can't rely on that to stay valid between hashgettuple() calls. Introduce a private variable in HashScanOpaque, instead.
2009-11-01 23:30:54 +01:00
/* Current position of the scan, as a heap TID */
ItemPointerData hashso_heappos;
Used modified version of indent that understands over 100 typedefs.
1997-09-08 23:56:23 +02:00
} HashScanOpaqueData;
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
typedef HashScanOpaqueData *HashScanOpaque;
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
* Definitions for metapage.
*/
#define HASH_METAPAGE 0 /* metapage is always block 0 */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#define HASH_MAGIC 0x6440640
Change hash indexes to store only the hash code rather than the whole indexed value. This means that hash index lookups are always lossy and have to be rechecked when the heap is visited; however, the gain in index compactness outweighs this when the indexed values are wide. Also, we only need to perform datatype comparisons when the hash codes match exactly, rather than for every entry in the hash bucket; so it could also win for datatypes that have expensive comparison functions. A small additional win is gained by keeping hash index pages sorted by hash code and using binary search to reduce the number of index tuples we have to look at. Xiao Meng This commit also incorporates Zdenek Kotala's patch to isolate hash metapages and hash bitmaps a bit better from the page header datastructures.
2008-09-15 20:43:41 +02:00
#define HASH_VERSION 2 /* 2 signifies only hash key value is stored */
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
* Spares[] holds the number of overflow pages currently allocated at or
* before a certain splitpoint. For example, if spares[3] = 7 then there are
* 7 ovflpages before splitpoint 3 (compare BUCKET_TO_BLKNO macro). The
* value in spares[ovflpoint] increases as overflow pages are added at the
* end of the index. Once ovflpoint increases (ie, we have actually allocated
* the bucket pages belonging to that splitpoint) the number of spares at the
* prior splitpoint cannot change anymore.
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
* ovflpages that have been recycled for reuse can be found by looking at
* bitmaps that are stored within ovflpages dedicated for the purpose.
* The blknos of these bitmap pages are kept in bitmaps[]; nmaps is the
* number of currently existing bitmaps.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
* The limitation on the size of spares[] comes from the fact that there's
* no point in having more than 2^32 buckets with only uint32 hashcodes.
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
* There is no particular upper limit on the size of mapp[], other than
* needing to fit into the metapage. (With 8K block size, 128 bitmaps
* limit us to 64 Gb of overflow space...)
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
#define HASH_MAX_SPLITPOINTS 32
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
#define HASH_MAX_BITMAPS 128
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
typedef struct HashMetaPageData
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
uint32 hashm_magic; /* magic no. for hash tables */
uint32 hashm_version; /* version ID */
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
double hashm_ntuples; /* number of tuples stored in the table */
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
uint16 hashm_ffactor; /* target fill factor (tuples/bucket) */
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
uint16 hashm_bsize; /* index page size (bytes) */
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
uint16 hashm_bmsize; /* bitmap array size (bytes) - must be a power
* of 2 */
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
uint16 hashm_bmshift; /* log2(bitmap array size in BITS) */
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
uint32 hashm_maxbucket; /* ID of maximum bucket in use */
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
uint32 hashm_highmask; /* mask to modulo into entire table */
uint32 hashm_lowmask; /* mask to modulo into lower half of table */
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
uint32 hashm_ovflpoint;/* splitpoint from which ovflpgs being
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
* allocated */
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
uint32 hashm_firstfree; /* lowest-number free ovflpage (bit#) */
uint32 hashm_nmaps; /* number of bitmap pages */
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
RegProcedure hashm_procid; /* hash procedure id from pg_proc */
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
uint32 hashm_spares[HASH_MAX_SPLITPOINTS]; /* spare pages before
* each splitpoint */
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
BlockNumber hashm_mapp[HASH_MAX_BITMAPS]; /* blknos of ovfl bitmaps */
Used modified version of indent that understands over 100 typedefs.
1997-09-08 23:56:23 +02:00
} HashMetaPageData;
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
typedef HashMetaPageData *HashMetaPage;
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
/*
* Maximum size of a hash index item (it's okay to have only one per page)
*/
#define HashMaxItemSize(page) \
Clean up the use of some page-header-access macros: principally, use SizeOfPageHeaderData instead of sizeof(PageHeaderData) in places where that makes the code clearer, and avoid casting between Page and PageHeader where possible. Zdenek Kotala, with some additional cleanup by Heikki Linnakangas. I did not apply the parts of the proposed patch that would have resulted in slightly changing the on-disk format of hash indexes; it seems to me that's not a win as long as there's any chance of having in-place upgrade for 8.4.
2008-07-13 22:45:47 +02:00
MAXALIGN_DOWN(PageGetPageSize(page) - \
SizeOfPageHeaderData - \
sizeof(ItemIdData) - \
MAXALIGN(sizeof(HashPageOpaqueData)))
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
Tweak fillfactor code as per my recent proposal. Fix nbtsort.c so that it can handle small fillfactors for ordinary-sized index entries without failing on large ones; fix nbtinsert.c to distinguish leaf and nonleaf pages; change the minimum fillfactor to 10% for all index types.
2006-07-11 23:05:57 +02:00
#define HASH_MIN_FILLFACTOR 10
Code review for FILLFACTOR patch. Change WITH grammar as per earlier discussion (including making def_arg allow reserved words), add missed opt_definition for UNIQUE case. Put the reloptions support code in a less random place (I chose to make a new file access/common/reloptions.c). Eliminate header inclusion creep. Make the index options functions safely user-callable (seems like client apps might like to be able to test validity of options before trying to make an index). Reduce overhead for normal case with no options by allowing rd_options to be NULL. Fix some unmaintainably klugy code, including getting rid of Natts_pg_class_fixed at long last. Some stylistic cleanup too, and pay attention to keeping comments in sync with code. Documentation still needs work, though I did fix the omissions in catalogs.sgml and indexam.sgml.
2006-07-04 00:45:41 +02:00
#define HASH_DEFAULT_FILLFACTOR 75
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
* Constants
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#define BYTE_TO_BIT 3 /* 2^3 bits/byte */
#define ALL_SET ((uint32) ~0)
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Bitmap pages do not contain tuples. They do contain the standard
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
* page headers and trailers; however, everything in between is a
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
* giant bit array. The number of bits that fit on a page obviously
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
* depends on the page size and the header/trailer overhead. We require
* the number of bits per page to be a power of 2.
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#define BMPGSZ_BYTE(metap) ((metap)->hashm_bmsize)
#define BMPGSZ_BIT(metap) ((metap)->hashm_bmsize << BYTE_TO_BIT)
Several fixes for hash indexes that involve changing the on-disk index layout; therefore, this change forces REINDEX of hash indexes (though not a full initdb). Widen hashm_ntuples to double so that hash space management doesn't get confused by more than 4G entries; enlarge the allowed number of free-space-bitmap pages; replace the useless bshift field with a useful bmshift field; eliminate 4 bytes of wasted space in the per-page special area.
2003-09-02 20:13:32 +02:00
#define BMPG_SHIFT(metap) ((metap)->hashm_bmshift)
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
#define BMPG_MASK(metap) (BMPGSZ_BIT(metap) - 1)
Change hash indexes to store only the hash code rather than the whole indexed value. This means that hash index lookups are always lossy and have to be rechecked when the heap is visited; however, the gain in index compactness outweighs this when the indexed values are wide. Also, we only need to perform datatype comparisons when the hash codes match exactly, rather than for every entry in the hash bucket; so it could also win for datatypes that have expensive comparison functions. A small additional win is gained by keeping hash index pages sorted by hash code and using binary search to reduce the number of index tuples we have to look at. Xiao Meng This commit also incorporates Zdenek Kotala's patch to isolate hash metapages and hash bitmaps a bit better from the page header datastructures.
2008-09-15 20:43:41 +02:00
#define HashPageGetBitmap(page) \
((uint32 *) PageGetContents(page))
#define HashGetMaxBitmapSize(page) \
(PageGetPageSize((Page) page) - \
(MAXALIGN(SizeOfPageHeaderData) + MAXALIGN(sizeof(HashPageOpaqueData))))
#define HashPageGetMeta(page) \
((HashMetaPage) PageGetContents(page))
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
* The number of bits in an ovflpage bitmap word.
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
#define BITS_PER_MAP 32 /* Number of bits in uint32 */
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
/* Given the address of the beginning of a bit map, clear/set the nth bit */
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
#define CLRBIT(A, N) ((A)[(N)/BITS_PER_MAP] &= ~(1<<((N)%BITS_PER_MAP)))
#define SETBIT(A, N) ((A)[(N)/BITS_PER_MAP] |= (1<<((N)%BITS_PER_MAP)))
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#define ISSET(A, N) ((A)[(N)/BITS_PER_MAP] & (1<<((N)%BITS_PER_MAP)))
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
* page-level and high-level locking modes (see README)
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
#define HASH_READ BUFFER_LOCK_SHARE
#define HASH_WRITE BUFFER_LOCK_EXCLUSIVE
#define HASH_NOLOCK (-1)
#define HASH_SHARE ShareLock
#define HASH_EXCLUSIVE ExclusiveLock
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* Strategy number. There's only one valid strategy for hashing: equality.
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#define HTEqualStrategyNumber 1
#define HTMaxStrategyNumber 1
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* When a new operator class is declared, we require that the user supply
* us with an amproc procudure for hashing a key of the new type.
* Since we only have one such proc in amproc, it's number 1.
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#define HASHPROC 1
Prevent to use magic constants Use macroses for definition amstrategies/amsupport fields instead of hardcoded values. Author: Nikolay Shaplov with addition for contrib/bloom
2016-04-28 15:39:25 +02:00
#define HASHNProcs 1
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/* public routines */
Restructure index access method API to hide most of it at the C level. This patch reduces pg_am to just two columns, a name and a handler function. All the data formerly obtained from pg_am is now provided in a C struct returned by the handler function. This is similar to the designs we've adopted for FDWs and tablesample methods. There are multiple advantages. For one, the index AM's support functions are now simple C functions, making them faster to call and much less error-prone, since the C compiler can now check function signatures. For another, this will make it far more practical to define index access methods in installable extensions. A disadvantage is that SQL-level code can no longer see attributes of index AMs; in particular, some of the crosschecks in the opr_sanity regression test are no longer possible from SQL. We've addressed that by adding a facility for the index AM to perform such checks instead. (Much more could be done in that line, but for now we're content if the amvalidate functions more or less replace what opr_sanity used to do.) We might also want to expose some sort of reporting functionality, but this patch doesn't do that. Alexander Korotkov, reviewed by Petr Jelínek, and rather heavily editorialized on by me.
2016-01-18 01:36:59 +01:00
extern Datum hashhandler(PG_FUNCTION_ARGS);
extern IndexBuildResult *hashbuild(Relation heap, Relation index,
struct IndexInfo *indexInfo);
extern void hashbuildempty(Relation index);
extern bool hashinsert(Relation rel, Datum *values, bool *isnull,
ItemPointer ht_ctid, Relation heapRel,
IndexUniqueCheck checkUnique);
extern bool hashgettuple(IndexScanDesc scan, ScanDirection dir);
extern int64 hashgetbitmap(IndexScanDesc scan, TIDBitmap *tbm);
extern IndexScanDesc hashbeginscan(Relation rel, int nkeys, int norderbys);
extern void hashrescan(IndexScanDesc scan, ScanKey scankey, int nscankeys,
ScanKey orderbys, int norderbys);
extern void hashendscan(IndexScanDesc scan);
extern IndexBulkDeleteResult *hashbulkdelete(IndexVacuumInfo *info,
IndexBulkDeleteResult *stats,
IndexBulkDeleteCallback callback,
void *callback_state);
extern IndexBulkDeleteResult *hashvacuumcleanup(IndexVacuumInfo *info,
IndexBulkDeleteResult *stats);
extern bytea *hashoptions(Datum reloptions, bool validate);
extern bool hashvalidate(Oid opclassoid);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
/*
* Datatype-specific hash functions in hashfunc.c.
*
Revise hash join and hash aggregation code to use the same datatype- specific hash functions used by hash indexes, rather than the old not-datatype-aware ComputeHashFunc routine. This makes it safe to do hash joining on several datatypes that previously couldn't use hashing. The sets of datatypes that are hash indexable and hash joinable are now exactly the same, whereas before each had some that weren't in the other.
2003-06-23 00:04:55 +02:00
* These support both hash indexes and hash joins.
*
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
* NOTE: some of these are also used by catcache operations, without
Code review for improved-hashing patch. Fix some portability issues (char != unsigned char, Datum != uint32); make use of new hash code in dynahash hash tables and hash joins.
2002-03-09 18:35:37 +01:00
* any direct connection to hash indexes. Also, the common hash_any
Revise hash join and hash aggregation code to use the same datatype- specific hash functions used by hash indexes, rather than the old not-datatype-aware ComputeHashFunc routine. This makes it safe to do hash joining on several datatypes that previously couldn't use hashing. The sets of datatypes that are hash indexable and hash joinable are now exactly the same, whereas before each had some that weren't in the other.
2003-06-23 00:04:55 +02:00
* routine is also used by dynahash tables.
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
*/
Clean up bogosities in pg_opclass, pg_amop, pg_amproc. There are amproc entries now for int8 and network hash indexes. int24_ops and int42_ops are gone. pg_opclass no longer contains multiple entries claiming to be the default opclass for the same datatype. opr_sanity regress test extended to catch errors like these in the future.
2000-06-19 05:55:01 +02:00
extern Datum hashchar(PG_FUNCTION_ARGS);
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
extern Datum hashint2(PG_FUNCTION_ARGS);
extern Datum hashint4(PG_FUNCTION_ARGS);
extern Datum hashint8(PG_FUNCTION_ARGS);
Clean up bogosities in pg_opclass, pg_amop, pg_amproc. There are amproc entries now for int8 and network hash indexes. int24_ops and int42_ops are gone. pg_opclass no longer contains multiple entries claiming to be the default opclass for the same datatype. opr_sanity regress test extended to catch errors like these in the future.
2000-06-19 05:55:01 +02:00
extern Datum hashoid(PG_FUNCTION_ARGS);
Support enum data types. Along the way, use macros for the values of pg_type.typtype whereever practical. Tom Dunstan, with some kibitzing from Tom Lane.
2007-04-02 05:49:42 +02:00
extern Datum hashenum(PG_FUNCTION_ARGS);
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
extern Datum hashfloat4(PG_FUNCTION_ARGS);
extern Datum hashfloat8(PG_FUNCTION_ARGS);
extern Datum hashoidvector(PG_FUNCTION_ARGS);
extern Datum hashint2vector(PG_FUNCTION_ARGS);
extern Datum hashname(PG_FUNCTION_ARGS);
Revise hash join and hash aggregation code to use the same datatype- specific hash functions used by hash indexes, rather than the old not-datatype-aware ComputeHashFunc routine. This makes it safe to do hash joining on several datatypes that previously couldn't use hashing. The sets of datatypes that are hash indexable and hash joinable are now exactly the same, whereas before each had some that weren't in the other.
2003-06-23 00:04:55 +02:00
extern Datum hashtext(PG_FUNCTION_ARGS);
Clean up bogosities in pg_opclass, pg_amop, pg_amproc. There are amproc entries now for int8 and network hash indexes. int24_ops and int42_ops are gone. pg_opclass no longer contains multiple entries claiming to be the default opclass for the same datatype. opr_sanity regress test extended to catch errors like these in the future.
2000-06-19 05:55:01 +02:00
extern Datum hashvarlena(PG_FUNCTION_ARGS);
Code review for improved-hashing patch. Fix some portability issues (char != unsigned char, Datum != uint32); make use of new hash code in dynahash hash tables and hash joins.
2002-03-09 18:35:37 +01:00
extern Datum hash_any(register const unsigned char *k, register int keylen);
Fix several hash functions that were taking chintzy shortcuts instead of delivering a well-randomized hash value. I got religion on this after observing that performance of multi-batch hash join degrades terribly if the higher-order bits of hash values aren't random, as indeed was true for say hashes of small integer values. It's now expected and documented that hash functions should use hash_any or some comparable method to ensure that all bits of their output are about equally random. initdb forced because this change invalidates existing hash indexes. For the same reason, this isn't back-patchable; the hash join performance problem will get a band-aid fix in the back branches.
2007-06-01 17:33:19 +02:00
extern Datum hash_uint32(uint32 k);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/* private routines */
/* hashinsert.c */
Remove the no-longer-useful HashItem/HashItemData level of structure. Same motivation as for BTItem.
2006-01-26 00:26:11 +01:00
extern void _hash_doinsert(Relation rel, IndexTuple itup);
Fix two serious bugs introduced into hash indexes by the 8.4 patch that made hash indexes keep entries sorted by hash value. First, the original plans for concurrency assumed that insertions would happen only at the end of a page, which is no longer true; this could cause scans to transiently fail to find index entries in the presence of concurrent insertions. We can compensate by teaching scans to re-find their position after re-acquiring read locks. Second, neither the bucket split nor the bucket compaction logic had been fixed to preserve hashvalue ordering, so application of either of those processes could lead to permanent corruption of an index, in the sense that searches might fail to find entries that are present. This patch fixes the split and compaction logic to preserve hashvalue ordering, but it cannot do anything about pre-existing corruption. We will need to recommend reindexing all hash indexes in the 8.4.2 release notes. To buy back the performance loss hereby induced in split and compaction, fix them to use PageIndexMultiDelete instead of retail PageIndexDelete operations. We might later want to do something with qsort'ing the page contents rather than doing a binary search for each insertion, but that seemed more invasive than I cared to risk in a back-patch. Per bug #5157 from Jeff Janes and subsequent investigation.
2009-11-01 22:25:25 +01:00
extern OffsetNumber _hash_pgaddtup(Relation rel, Buffer buf,
Size itemsize, IndexTuple itup);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/* hashovfl.c */
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
extern Buffer _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf);
Make large sequential scans and VACUUMs work in a limited-size "ring" of buffers, rather than blowing out the whole shared-buffer arena. Aside from avoiding cache spoliation, this fixes the problem that VACUUM formerly tended to cause a WAL flush for every page it modified, because we had it hacked to use only a single buffer. Those flushes will now occur only once per ring-ful. The exact ring size, and the threshold for seqscans to switch into the ring usage pattern, remain under debate; but the infrastructure seems done. The key bit of infrastructure is a new optional BufferAccessStrategy object that can be passed to ReadBuffer operations; this replaces the former StrategyHintVacuum API. This patch also changes the buffer usage-count methodology a bit: we now advance usage_count when first pinning a buffer, rather than when last unpinning it. To preserve the behavior that a buffer's lifetime starts to decrease when it's released, the clock sweep code is modified to not decrement usage_count of pinned buffers. Work not done in this commit: teach GiST and GIN indexes to use the vacuum BufferAccessStrategy for vacuum-driven fetches. Original patch by Simon, reworked by Heikki and again by Tom.
2007-05-30 22:12:03 +02:00
extern BlockNumber _hash_freeovflpage(Relation rel, Buffer ovflbuf,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
BufferAccessStrategy bstrategy);
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
extern void _hash_initbitmap(Relation rel, HashMetaPage metap,
Support unlogged tables. The contents of an unlogged table are WAL-logged; thus, they are not available on standby servers and are truncated whenever the database system enters recovery. Indexes on unlogged tables are also unlogged. Unlogged GiST indexes are not currently supported.
2010-12-29 12:48:53 +01:00
BlockNumber blkno, ForkNumber forkNum);
Rewrite hashbulkdelete() to make it amenable to new bucket locking scheme. A pleasant side effect is that it is *much* faster when deleting a large fraction of the indexed tuples, because of elimination of redundant hash_step activity induced by hash_adjscans. Various other continuing code cleanup.
2003-09-02 04:18:38 +02:00
extern void _hash_squeezebucket(Relation rel,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Bucket bucket, BlockNumber bucket_blkno,
BufferAccessStrategy bstrategy);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/* hashpage.c */
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
extern void _hash_getlock(Relation rel, BlockNumber whichlock, int access);
extern bool _hash_try_getlock(Relation rel, BlockNumber whichlock, int access);
extern void _hash_droplock(Relation rel, BlockNumber whichlock, int access);
Tweak hash index AM to use the new ReadOrZeroBuffer bufmgr API when fetching pages it intends to zero immediately. Just to show there is some use for that function besides WAL recovery :-). Along the way, fold _hash_checkpage and _hash_pageinit calls into _hash_getbuf and friends, instead of expecting callers to do that separately.
2007-05-03 18:45:58 +02:00
extern Buffer _hash_getbuf(Relation rel, BlockNumber blkno,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
int access, int flags);
Tweak hash index AM to use the new ReadOrZeroBuffer bufmgr API when fetching pages it intends to zero immediately. Just to show there is some use for that function besides WAL recovery :-). Along the way, fold _hash_checkpage and _hash_pageinit calls into _hash_getbuf and friends, instead of expecting callers to do that separately.
2007-05-03 18:45:58 +02:00
extern Buffer _hash_getinitbuf(Relation rel, BlockNumber blkno);
Support unlogged tables. The contents of an unlogged table are WAL-logged; thus, they are not available on standby servers and are truncated whenever the database system enters recovery. Indexes on unlogged tables are also unlogged. Unlogged GiST indexes are not currently supported.
2010-12-29 12:48:53 +01:00
extern Buffer _hash_getnewbuf(Relation rel, BlockNumber blkno,
ForkNumber forkNum);
Make large sequential scans and VACUUMs work in a limited-size "ring" of buffers, rather than blowing out the whole shared-buffer arena. Aside from avoiding cache spoliation, this fixes the problem that VACUUM formerly tended to cause a WAL flush for every page it modified, because we had it hacked to use only a single buffer. Those flushes will now occur only once per ring-ful. The exact ring size, and the threshold for seqscans to switch into the ring usage pattern, remain under debate; but the infrastructure seems done. The key bit of infrastructure is a new optional BufferAccessStrategy object that can be passed to ReadBuffer operations; this replaces the former StrategyHintVacuum API. This patch also changes the buffer usage-count methodology a bit: we now advance usage_count when first pinning a buffer, rather than when last unpinning it. To preserve the behavior that a buffer's lifetime starts to decrease when it's released, the clock sweep code is modified to not decrement usage_count of pinned buffers. Work not done in this commit: teach GiST and GIN indexes to use the vacuum BufferAccessStrategy for vacuum-driven fetches. Original patch by Simon, reworked by Heikki and again by Tom.
2007-05-30 22:12:03 +02:00
extern Buffer _hash_getbuf_with_strategy(Relation rel, BlockNumber blkno,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
int access, int flags,
BufferAccessStrategy bstrategy);
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
extern void _hash_relbuf(Relation rel, Buffer buf);
extern void _hash_dropbuf(Relation rel, Buffer buf);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
extern void _hash_wrtbuf(Relation rel, Buffer buf);
Preliminary cleanup for hash index code (doesn't attack the locking problem yet). Fix a couple of bugs that would only appear if multiple bitmap pages are used, including a buffer reference leak and incorrect computation of bit indexes. Get rid of 'overflow address' concept, which accomplished nothing except obfuscating the code and creating a risk of failure due to limited range of offset field. Rename some misleadingly-named fields and routines, and improve documentation.
2003-09-01 22:26:34 +02:00
extern void _hash_chgbufaccess(Relation rel, Buffer buf, int from_access,
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
int to_access);
Support unlogged tables. The contents of an unlogged table are WAL-logged; thus, they are not available on standby servers and are truncated whenever the database system enters recovery. Indexes on unlogged tables are also unlogged. Unlogged GiST indexes are not currently supported.
2010-12-29 12:48:53 +01:00
extern uint32 _hash_metapinit(Relation rel, double num_tuples,
ForkNumber forkNum);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
extern void _hash_pageinit(Page page, Size size);
extern void _hash_expandtable(Relation rel, Buffer metabuf);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/* hashscan.c */
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
extern void _hash_regscan(IndexScanDesc scan);
extern void _hash_dropscan(IndexScanDesc scan);
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
extern bool _hash_has_active_scan(Relation rel, Bucket bucket);
Invent ResourceOwner mechanism as per my recent proposal, and use it to keep track of portal-related resources separately from transaction-related resources. This allows cursors to work in a somewhat sane fashion with nested transactions. For now, cursor behavior is non-subtransactional, that is a cursor's state does not roll back if you abort a subtransaction that fetched from the cursor. We might want to change that later.
2004-07-17 05:32:14 +02:00
extern void ReleaseResources_hash(void);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/* hashsearch.c */
Restructure indexscan API (index_beginscan, index_getnext) per yesterday's proposal to pghackers. Also remove unnecessary parameters to heap_beginscan, heap_rescan. I modified pg_proc.h to reflect the new numbers of parameters for the AM interface routines, but did not force an initdb because nothing actually looks at those fields.
2002-05-21 01:51:44 +02:00
extern bool _hash_next(IndexScanDesc scan, ScanDirection dir);
extern bool _hash_first(IndexScanDesc scan, ScanDirection dir);
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
extern bool _hash_step(IndexScanDesc scan, Buffer *bufP, ScanDirection dir);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
When creating a large hash index, pre-sort the index entries by estimated bucket number, so as to ensure locality of access to the index during the insertion step. Without this, building an index significantly larger than available RAM takes a very long time because of thrashing. On the other hand, sorting is just useless overhead when the index does fit in RAM. We choose to sort when the initial index size exceeds effective_cache_size. This is a revised version of work by Tom Raney and Shreya Bhargava.
2008-03-17 00:15:08 +01:00
/* hashsort.c */
typedef struct HSpool HSpool; /* opaque struct in hashsort.c */
Provide database object names as separate fields in error messages. This patch addresses the problem that applications currently have to extract object names from possibly-localized textual error messages, if they want to know for example which index caused a UNIQUE_VIOLATION failure. It adds new error message fields to the wire protocol, which can carry the name of a table, table column, data type, or constraint associated with the error. (Since the protocol spec has always instructed clients to ignore unrecognized field types, this should not create any compatibility problem.) Support for providing these new fields has been added to just a limited set of error reports (mainly, those in the "integrity constraint violation" SQLSTATE class), but we will doubtless add them to more calls in future. Pavel Stehule, reviewed and extensively revised by Peter Geoghegan, with additional hacking by Tom Lane.
2013-01-29 23:06:26 +01:00
extern HSpool *_h_spoolinit(Relation heap, Relation index, uint32 num_buckets);
When creating a large hash index, pre-sort the index entries by estimated bucket number, so as to ensure locality of access to the index during the insertion step. Without this, building an index significantly larger than available RAM takes a very long time because of thrashing. On the other hand, sorting is just useless overhead when the index does fit in RAM. We choose to sort when the initial index size exceeds effective_cache_size. This is a revised version of work by Tom Raney and Shreya Bhargava.
2008-03-17 00:15:08 +01:00
extern void _h_spooldestroy(HSpool *hspool);
Avoid copying index tuples when building an index. The previous code, perhaps out of concern for avoid memory leaks, formed the tuple in one memory context and then copied it to another memory context. However, this doesn't appear to be necessary, since index_form_tuple and the functions it calls take precautions against leaking memory. In my testing, building the tuple directly inside the sort context shaves several percent off the index build time. Rearrange things so we do that. Patch by me. Review by Amit Kapila, Tom Lane, Andres Freund.
2014-07-01 16:34:42 +02:00
extern void _h_spool(HSpool *hspool, ItemPointer self,
pgindent run for 9.5
2015-05-24 03:35:49 +02:00
Datum *values, bool *isnull);
When creating a large hash index, pre-sort the index entries by estimated bucket number, so as to ensure locality of access to the index during the insertion step. Without this, building an index significantly larger than available RAM takes a very long time because of thrashing. On the other hand, sorting is just useless overhead when the index does fit in RAM. We choose to sort when the initial index size exceeds effective_cache_size. This is a revised version of work by Tom Raney and Shreya Bhargava.
2008-03-17 00:15:08 +01:00
extern void _h_indexbuild(HSpool *hspool);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
/* hashutil.c */
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
extern bool _hash_checkqual(IndexScanDesc scan, IndexTuple itup);
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
extern uint32 _hash_datum2hashkey(Relation rel, Datum key);
Add support for cross-type hashing in hash index searches and hash joins. Hashing for aggregation purposes still needs work, so it's not time to mark any cross-type operators as hashable for general use, but these cases work if the operators are so marked by hand in the system catalogs.
2007-01-30 02:33:36 +01:00
extern uint32 _hash_datum2hashkey_type(Relation rel, Datum key, Oid keytype);
Reimplement hash index locking algorithms, per my recent proposal to pghackers. This fixes the problem recently reported by Markus KrÌutner (hash bucket split corrupts the state of scans being done concurrently), and I believe it also fixes all the known problems with deadlocks in hash index operations. Hash indexes are still not really ready for prime time (since they aren't WAL-logged), but this is a step forward.
2003-09-05 00:06:27 +02:00
extern Bucket _hash_hashkey2bucket(uint32 hashkey, uint32 maxbucket,
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
uint32 highmask, uint32 lowmask);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
extern uint32 _hash_log2(uint32 num);
Add defenses to btree and hash index AMs to do simple sanity checks on every index page they read; in particular to catch the case of an all-zero page, which PageHeaderIsValid allows to pass. It turns out hash already had this idea, but it was just Assert()ing things rather than doing a straight error check, and the Asserts were partially redundant with PageHeaderIsValid anyway. Per recent failure example from Jim Nasby. (gist still needs the same treatment.)
2005-11-06 20:29:01 +01:00
extern void _hash_checkpage(Relation rel, Buffer buf, int flags);
Change hash indexes to store only the hash code rather than the whole indexed value. This means that hash index lookups are always lossy and have to be rechecked when the heap is visited; however, the gain in index compactness outweighs this when the indexed values are wide. Also, we only need to perform datatype comparisons when the hash codes match exactly, rather than for every entry in the hash bucket; so it could also win for datatypes that have expensive comparison functions. A small additional win is gained by keeping hash index pages sorted by hash code and using binary search to reduce the number of index tuples we have to look at. Xiao Meng This commit also incorporates Zdenek Kotala's patch to isolate hash metapages and hash bitmaps a bit better from the page header datastructures.
2008-09-15 20:43:41 +02:00
extern uint32 _hash_get_indextuple_hashkey(IndexTuple itup);
extern IndexTuple _hash_form_tuple(Relation index,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Datum *values, bool *isnull);
Change hash indexes to store only the hash code rather than the whole indexed value. This means that hash index lookups are always lossy and have to be rechecked when the heap is visited; however, the gain in index compactness outweighs this when the indexed values are wide. Also, we only need to perform datatype comparisons when the hash codes match exactly, rather than for every entry in the hash bucket; so it could also win for datatypes that have expensive comparison functions. A small additional win is gained by keeping hash index pages sorted by hash code and using binary search to reduce the number of index tuples we have to look at. Xiao Meng This commit also incorporates Zdenek Kotala's patch to isolate hash metapages and hash bitmaps a bit better from the page header datastructures.
2008-09-15 20:43:41 +02:00
extern OffsetNumber _hash_binsearch(Page page, uint32 hash_value);
extern OffsetNumber _hash_binsearch_last(Page page, uint32 hash_value);
Moved from backend/access to include/access
1996-08-27 23:50:29 +02:00
Put external declarations into header files.
2000-11-21 22:16:06 +01:00
/* hash.c */
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 16:56:26 +01:00
extern void hash_redo(XLogReaderState *record);
extern void hash_desc(StringInfo buf, XLogReaderState *record);
Add rmgr callback to name xlog record types for display purposes. This is primarily useful for the upcoming pg_xlogdump --stats feature, but also allows to remove some duplicated code in the rmgr_desc routines. Due to the separation and harmonization, the output of dipsplayed records changes somewhat. But since this isn't enduser oriented content that's ok. It's potentially desirable to further change pg_xlogdump's display of records. It previously wasn't possible to show the record type separately from the description forcing it to be in the last column. But that's better done in a separate commit. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Álvaro Herrera, Andres Freund, and Heikki Linnakangas Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 15:17:12 +02:00
extern const char *hash_identify(uint8 info);
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
New pgindent run with fixes suggested by Tom. Patch manually reviewed, initdb/regression tests pass.
2001-11-05 18:46:40 +01:00
#endif /* HASH_H */