rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/commands/vacuum.c

3571 lines
104 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* vacuum.c
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
* The postgres vacuum cleaner.
*
* This file includes the "full" version of VACUUM, as well as control code
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* used by all three of full VACUUM, lazy VACUUM, and ANALYZE. See
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
* vacuumlazy.c and analyze.c for the rest of the code for the latter two.
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Update copyright for 2006. Update scripts.
2006-03-05 16:59:11 +01:00
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*
* IDENTIFICATION
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
* $PostgreSQL: pgsql/src/backend/commands/vacuum.c,v 1.333 2006/07/10 16:20:50 alvherre Exp $
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*-------------------------------------------------------------------------
*/
Avoid repeated detoasting (and possible memory leaks) when processing a toasted datum in VACUUM ANALYZE.
2000-12-02 20:38:34 +01:00
#include "postgres.h"
Put back #include <sys/time.h> in files that seem to need it on Linux.
2004-05-21 18:08:47 +02:00
#include <sys/time.h>
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
#include <unistd.h>
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
#include "access/clog.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "access/genam.h"
#include "access/heapam.h"
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
#include "access/multixact.h"
Nested transactions. There is still much left to do, especially on the performance front, but with feature freeze upon us I think it's time to drive a stake in the ground and say that this will be in 7.5. Alvaro Herrera, with some help from Tom Lane.
2004-07-01 02:52:04 +02:00
#include "access/subtrans.h"
Avoid repeated detoasting (and possible memory leaks) when processing a toasted datum in VACUUM ANALYZE.
2000-12-02 20:38:34 +01:00
#include "access/xlog.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "catalog/catalog.h"
Create a syscache for pg_database-indexed-by-oid, and make use of it in various places that were previously doing ad hoc pg_database searches. This may speed up database-related privilege checks a little bit, but the main motivation is to eliminate the performance reason for having ReverifyMyDatabase do such a lot of stuff (viz, avoiding repeat scans of pg_database during backend startup). The locking reason for having that routine is about to go away, and it'd be good to have the option to break it up.
2006-05-04 00:45:26 +02:00
#include "catalog/indexing.h"
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
#include "catalog/namespace.h"
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
#include "catalog/pg_database.h"
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
#include "catalog/pg_index.h"
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
#include "commands/dbcommands.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "commands/vacuum.h"
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
#include "executor/executor.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "miscadmin.h"
Modify pgstats code to reduce performance penalties from oversized stats data files: avoid creating stats hashtable entries for tables that aren't being touched except by vacuum/analyze, ensure that entries for dropped tables are removed promptly, and tweak the data layout to avoid storing useless struct padding. Also improve the performance of pgstat_vacuum_tabstat(), and make sure that autovacuum invokes it exactly once per autovac cycle rather than multiple times or not at all. This should cure recent complaints about 8.1 showing much higher stats I/O volume than was seen in 8.0. It'd still be a good idea to revisit the design with an eye to not re-writing the entire stats dataset every half second ... but that would be too much to backpatch, I fear.
2006-01-18 21:35:06 +01:00
#include "postmaster/autovacuum.h"
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
#include "storage/freespace.h"
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
#include "storage/pmsignal.h"
Split the shared-memory array of PGPROC pointers out of the sinval communication structure, and make it its own module with its own lock. This should reduce contention at least a little, and it definitely makes the code seem cleaner. Per my recent proposal.
2005-05-19 23:35:48 +02:00
#include "storage/procarray.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "storage/smgr.h"
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
#include "tcop/pquery.h"
Add permissions check: now one must be the Postgres superuser or the table owner in order to vacuum a table. This is mainly to prevent denial-of-service attacks via repeated vacuums. Allow VACUUM to gather statistics about system relations, except for pg_statistic itself --- not clear that it's worth the trouble to make that case work cleanly. Cope with possible tuple size overflow in pg_statistic tuples; I'm surprised we never realized that could happen. Hold a couple of locks a little longer to try to prevent deadlocks between concurrent VACUUMs. There still seem to be some problems in that last area though :-(
1999-11-29 05:43:15 +01:00
#include "utils/acl.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "utils/builtins.h"
Clean up a number of autovacuum loose ends. Make the stats collector track shared relations in a separate hashtable, so that operations done from different databases are counted correctly. Add proper support for anti-XID-wraparound vacuuming, even in databases that are never connected to and so have no stats entries. Miscellaneous other bug fixes. Alvaro Herrera, some additional fixes by Tom Lane.
2005-07-29 21:30:09 +02:00
#include "utils/flatfiles.h"
First round of changes for new fmgr interface. fmgr itself and the key call sites are changed, but most called functions are still oldstyle. An exception is that the PL managers are updated (so, for example, NULL handling now behaves as expected in plperl and plpgsql functions). NOTE initdb is forced due to added column in pg_proc.
2000-05-28 19:56:29 +02:00
#include "utils/fmgroids.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "utils/inval.h"
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
#include "utils/lsyscache.h"
For some reason access/tupmacs.h has been #including utils/memutils.h, which is neither needed by nor related to that header. Remove the bogus inclusion and instead include the header in those C files that actually need it. Also fix unnecessary inclusions and bad inclusion order in tsearch2 files.
2005-05-06 19:24:55 +02:00
#include "utils/memutils.h"
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
#include "utils/pg_rusage.h"
Arrange for VACUUM to delete the init file that relcache.c uses to save a little bit of backend startup time. This way, the first backend started after a VACUUM will rebuild the init file with up-to-date statistics for the critical system indexes.
1999-05-01 21:09:46 +02:00
#include "utils/relcache.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "utils/syscache.h"
Statistical system views (yet without the config stuff, but it's hard to keep such massive changes in sync with the tree so I need to get it in and work from there now). Jan
2001-06-22 21:16:24 +02:00
#include "pgstat.h"
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/*
* VacPage structures keep track of each page on which we find useful
* amounts of free space.
*/
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
typedef struct VacPageData
{
BlockNumber blkno; /* BlockNumber of this Page */
Size free; /* FreeSpace on this Page */
uint16 offsets_used; /* Number of OffNums used by vacuum */
uint16 offsets_free; /* Number of OffNums free or to be free */
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
OffsetNumber offsets[1]; /* Array of free OffNums */
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
} VacPageData;
typedef VacPageData *VacPage;
typedef struct VacPageListData
{
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
BlockNumber empty_end_pages; /* Number of "empty" end-pages */
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
int num_pages; /* Number of pages in pagedesc */
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
int num_allocated_pages; /* Number of allocated pages in
* pagedesc */
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
VacPage *pagedesc; /* Descriptions of pages */
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
} VacPageListData;
typedef VacPageListData *VacPageList;
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/*
* The "vtlinks" array keeps information about each recently-updated tuple
* ("recent" meaning its XMAX is too new to let us recycle the tuple).
* We store the tuple's own TID as well as its t_ctid (its link to the next
* newer tuple version). Searching in this array allows us to follow update
* chains backwards from newer to older tuples. When we move a member of an
* update chain, we must move *all* the live members of the chain, so that we
* can maintain their t_ctid link relationships (we must not just overwrite
* t_ctid in an existing tuple).
*
* Note: because t_ctid links can be stale (this would only occur if a prior
* VACUUM crashed partway through), it is possible that new_tid points to an
* empty slot or unrelated tuple. We have to check the linkage as we follow
* it, just as is done in EvalPlanQual.
*/
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
typedef struct VTupleLinkData
{
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
ItemPointerData new_tid; /* t_ctid of an updated tuple */
ItemPointerData this_tid; /* t_self of the tuple */
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
} VTupleLinkData;
typedef VTupleLinkData *VTupleLink;
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/*
* We use an array of VTupleMoveData to plan a chain tuple move fully
* before we do it.
*/
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
typedef struct VTupleMoveData
{
ItemPointerData tid; /* tuple ID */
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
VacPage vacpage; /* where to move it to */
bool cleanVpd; /* clean vacpage before using? */
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
} VTupleMoveData;
typedef VTupleMoveData *VTupleMove;
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/*
* VRelStats contains the data acquired by scan_heap for use later
*/
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
typedef struct VRelStats
{
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* miscellaneous statistics */
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
BlockNumber rel_pages;
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
double rel_tuples;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Size min_tlen;
Size max_tlen;
bool hasindex;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
TransactionId minxid; /* Minimum Xid present anywhere on table */
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* vtlinks array for tuple chain following - sorted by new_tid */
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
int num_vtlinks;
VTupleLink vtlinks;
} VRelStats;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*----------------------------------------------------------------------
* ExecContext:
*
* As these variables always appear together, we put them into one struct
* and pull initialization and cleanup into separate routines.
* ExecContext is used by repair_frag() and move_xxx_tuple(). More
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* accurately: It is *used* only in move_xxx_tuple(), but because this
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
* routine is called many times, we initialize the struct just once in
* repair_frag() and pass it on to move_xxx_tuple().
*/
typedef struct ExecContextData
{
ResultRelInfo *resultRelInfo;
EState *estate;
TupleTableSlot *slot;
} ExecContextData;
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
typedef ExecContextData *ExecContext;
static void
ExecContext_Init(ExecContext ec, Relation rel)
{
TupleDesc tupdesc = RelationGetDescr(rel);
/*
* We need a ResultRelInfo and an EState so we can use the regular
* executor's index-entry-making machinery.
*/
ec->estate = CreateExecutorState();
ec->resultRelInfo = makeNode(ResultRelInfo);
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
ec->resultRelInfo->ri_RangeTableIndex = 1; /* dummy */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
ec->resultRelInfo->ri_RelationDesc = rel;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
ec->resultRelInfo->ri_TrigDesc = NULL; /* we don't fire triggers */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
ExecOpenIndices(ec->resultRelInfo);
ec->estate->es_result_relations = ec->resultRelInfo;
ec->estate->es_num_result_relations = 1;
ec->estate->es_result_relation_info = ec->resultRelInfo;
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
/* Set up a tuple slot too */
ec->slot = MakeSingleTupleTableSlot(tupdesc);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
}
static void
ExecContext_Finish(ExecContext ec)
{
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
ExecDropSingleTupleTableSlot(ec->slot);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
ExecCloseIndices(ec->resultRelInfo);
FreeExecutorState(ec->estate);
}
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*
* End of ExecContext Implementation
*----------------------------------------------------------------------
*/
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
static MemoryContext vac_context = NULL;
Analyze data only if specified and specified attrs only.
1997-04-23 08:28:48 +02:00
pgindent run.
2002-09-04 22:31:48 +02:00
static int elevel = -1;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/* non-export function prototypes */
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
static List *get_rel_oids(List *relids, const RangeVar *vacrel,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
const char *stmttype);
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
static void vac_update_dbminxid(Oid dbid,
TransactionId *minxid,
TransactionId *vacuumxid);
static void vac_truncate_clog(TransactionId myminxid, TransactionId myvacxid);
static void vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
static void full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
static void scan_heap(VRelStats *vacrelstats, Relation onerel,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
VacPageList vacuum_pages, VacPageList fraged_pages,
TransactionId FreezeLimit, TransactionId OldestXmin);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
static void repair_frag(VRelStats *vacrelstats, Relation onerel,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
VacPageList vacuum_pages, VacPageList fraged_pages,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
int nindexes, Relation *Irel, TransactionId OldestXmin);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
static void move_chain_tuple(Relation rel,
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Buffer old_buf, Page old_page, HeapTuple old_tup,
Buffer dst_buf, Page dst_page, VacPage dst_vacpage,
ExecContext ec, ItemPointer ctid, bool cleanVpd);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
static void move_plain_tuple(Relation rel,
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Buffer old_buf, Page old_page, HeapTuple old_tup,
Buffer dst_buf, Page dst_page, VacPage dst_vacpage,
ExecContext ec);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
static void update_hint_bits(Relation rel, VacPageList fraged_pages,
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
int num_fraged_pages, BlockNumber last_move_dest_block,
int num_moved);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
static void vacuum_heap(VRelStats *vacrelstats, Relation onerel,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
VacPageList vacpagelist);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
static void vacuum_page(Relation onerel, Buffer buffer, VacPage vacpage);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
static void vacuum_index(VacPageList vacpagelist, Relation indrel,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
double num_tuples, int keep_tuples);
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
static void scan_index(Relation indrel, double num_tuples);
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
static bool tid_reaped(ItemPointer itemptr, void *state);
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
static void vac_update_fsm(Relation onerel, VacPageList fraged_pages,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
BlockNumber rel_pages);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
static VacPage copy_vac_page(VacPage vacpage);
More vacuum cleanup
2000-05-29 19:06:15 +02:00
static void vpage_insert(VacPageList vacpagelist, VacPage vpnew);
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
static void *vac_bsearch(const void *key, const void *base,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
size_t nelem, size_t size,
int (*compar) (const void *, const void *));
More vacuum renaming.
2000-05-29 03:46:00 +02:00
static int vac_cmp_blk(const void *left, const void *right);
static int vac_cmp_offno(const void *left, const void *right);
static int vac_cmp_vtlinks(const void *left, const void *right);
More vacuum cleanup
2000-05-29 19:06:15 +02:00
static bool enough_space(VacPage vacpage, Size len);
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
static Size PageGetFreeSpaceWithFillFactor(Relation relation, Page page);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
/****************************************************************************
* *
* Code common to all flavors of VACUUM and ANALYZE *
* *
****************************************************************************
*/
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
* Primary entry point for VACUUM and ANALYZE commands.
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
*
* relids is normally NIL; if it is not, then it provides the list of
* relation OIDs to be processed, and vacstmt->relation is ignored.
* (The non-NIL case is currently only used by autovacuum.)
*
* It is the caller's responsibility that both vacstmt and relids
* (if given) be allocated in a memory context that won't disappear
* at transaction commit. In fact this context must be QueryContext
* to avoid complaints from PreventTransactionChain.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
void
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
vacuum(VacuumStmt *vacstmt, List *relids)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
const char *stmttype = vacstmt->vacuum ? "VACUUM" : "ANALYZE";
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
volatile MemoryContext anl_context = NULL;
volatile bool all_rels,
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
in_outer_xact,
use_own_xacts;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
List *relations;
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
if (vacstmt->verbose)
elevel = INFO;
else
Make debug_ GUC varables output DEBUG1 rather than LOG, and mention in docs that CLIENT/LOG_MIN_MESSAGES now controls debug_* output location. Doc changes included.
2003-05-27 19:49:47 +02:00
elevel = DEBUG2;
Remove pg_vlock locking from VACUUM, allowing multiple VACUUMs to run in parallel --- and, not incidentally, removing a common reason for needing manual cleanup by the DB admin after a crash. Remove initial global delete of pg_statistics rows in VACUUM ANALYZE; this was not only bad for performance of other backends that had to run without stats for a while, but it was fundamentally broken because it was done outside any transaction. Surprising we didn't see more consequences of that. Detect attempt to run VACUUM inside a transaction block. Check for query cancel request before starting vacuum of each table. Clean up vacuum's private portal storage if vacuum is aborted.
1999-11-28 03:10:01 +01:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We cannot run VACUUM inside a user transaction block; if we were inside
* a transaction, then our commit- and start-transaction-command calls
* would not have the intended effect! Furthermore, the forced commit that
* occurs before truncating the relation's file would have the effect of
* committing the rest of the user's transaction too, which would
* certainly not be the desired behavior. (This only applies to VACUUM
* FULL, though. We could in theory run lazy VACUUM inside a transaction
* block, but we choose to disallow that case because we'd rather commit
* as soon as possible after finishing the vacuum. This is mainly so that
* we can let go the AccessExclusiveLock that we may be holding.)
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
*
* ANALYZE (without VACUUM) can run either way.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Fix places that were using IsTransactionBlock() as an (inadequate) check that they'd get to commit immediately on finishing. There's now a centralized routine PreventTransactionChain() that implements the necessary tests.
2002-10-22 00:06:20 +02:00
if (vacstmt->vacuum)
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
{
Fix places that were using IsTransactionBlock() as an (inadequate) check that they'd get to commit immediately on finishing. There's now a centralized routine PreventTransactionChain() that implements the necessary tests.
2002-10-22 00:06:20 +02:00
PreventTransactionChain((void *) vacstmt, stmttype);
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
in_outer_xact = false;
}
else
in_outer_xact = IsInTransactionChain((void *) vacstmt);
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
/*
* Disallow the combination VACUUM FULL FREEZE; although it would mostly
* work, VACUUM FULL's ability to move tuples around means that it is
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* injecting its own XID into tuple visibility checks. We'd have to
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
* guarantee that every moved tuple is properly marked XMIN_COMMITTED or
* XMIN_INVALID before the end of the operation. There are corner cases
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* where this does not happen, and getting rid of them all seems hard (not
* to mention fragile to maintain). On the whole it's not worth it
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
* compared to telling people to use two operations. See pgsql-hackers
* discussion of 27-Nov-2004, and comments below for update_hint_bits().
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Note: this is enforced here, and not in the grammar, since (a) we can
* give a better error message, and (b) we might want to allow it again
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
* someday.
*/
if (vacstmt->vacuum && vacstmt->full && vacstmt->freeze)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("VACUUM FULL FREEZE is not supported"),
errhint("Use VACUUM FULL, then VACUUM FREEZE.")));
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
Modify pgstats code to reduce performance penalties from oversized stats data files: avoid creating stats hashtable entries for tables that aren't being touched except by vacuum/analyze, ensure that entries for dropped tables are removed promptly, and tweak the data layout to avoid storing useless struct padding. Also improve the performance of pgstat_vacuum_tabstat(), and make sure that autovacuum invokes it exactly once per autovac cycle rather than multiple times or not at all. This should cure recent complaints about 8.1 showing much higher stats I/O volume than was seen in 8.0. It'd still be a good idea to revisit the design with an eye to not re-writing the entire stats dataset every half second ... but that would be too much to backpatch, I fear.
2006-01-18 21:35:06 +01:00
* Send info about dead objects to the statistics collector, unless
* we are in autovacuum --- autovacuum.c does this for itself.
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
*/
Modify pgstats code to reduce performance penalties from oversized stats data files: avoid creating stats hashtable entries for tables that aren't being touched except by vacuum/analyze, ensure that entries for dropped tables are removed promptly, and tweak the data layout to avoid storing useless struct padding. Also improve the performance of pgstat_vacuum_tabstat(), and make sure that autovacuum invokes it exactly once per autovac cycle rather than multiple times or not at all. This should cure recent complaints about 8.1 showing much higher stats I/O volume than was seen in 8.0. It'd still be a good idea to revisit the design with an eye to not re-writing the entire stats dataset every half second ... but that would be too much to backpatch, I fear.
2006-01-18 21:35:06 +01:00
if (vacstmt->vacuum && !IsAutoVacuumProcess())
Fix up gcc warnings, improve comments.
2002-06-15 23:52:31 +02:00
pgstat_vacuum_tabstat();
Statistical system views (yet without the config stuff, but it's hard to keep such massive changes in sync with the tree so I need to get it in and work from there now). Jan
2001-06-22 21:16:24 +02:00
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
/*
* Create special memory context for cross-transaction storage.
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Since it is a child of PortalContext, it will go away eventually even
* if we suffer an error; there's no need for special abort cleanup logic.
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
*/
Portal and memory management infrastructure for extended query protocol. Both plannable queries and utility commands are now always executed within Portals, which have been revamped so that they can handle the load (they used to be good only for single SELECT queries). Restructure code to push command-completion-tag selection logic out of postgres.c, so that it won't have to be duplicated between simple and extended queries. initdb forced due to addition of a field to Query nodes.
2003-05-02 22:54:36 +02:00
vac_context = AllocSetContextCreate(PortalContext,
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
"Vacuum",
ALLOCSET_DEFAULT_MINSIZE,
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
Repair access-to-already-freed-memory error recently introduced into VACUUM.
2000-03-09 00:41:00 +01:00
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
/* Remember whether we are processing everything in the DB */
all_rels = (relids == NIL && vacstmt->relation == NULL);
ARGH! Vacuum must not advance pg_database.datvacuumxid nor truncate CLOG unless it's processed *all* tables in the database. Vacuums run by unprivileged users don't count. (Beats head against nearest convenient wall...)
2002-09-27 22:57:08 +02:00
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Build list of relations to process, unless caller gave us one. (If we
* build one, we put it in vac_context for safekeeping.)
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
*/
relations = get_rel_oids(relids, vacstmt->relation, stmttype);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
/*
* Decide whether we need to start/commit our own transactions.
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* For VACUUM (with or without ANALYZE): always do so, so that we can
* release locks as soon as possible. (We could possibly use the outer
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* transaction for a one-table VACUUM, but handling TOAST tables would be
* problematic.)
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
*
* For ANALYZE (no VACUUM): if inside a transaction block, we cannot
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* start/commit our own transactions. Also, there's no need to do so if
* only processing one relation. For multiple relations when not within a
* transaction block, use own transactions so we can release locks sooner.
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
*/
if (vacstmt->vacuum)
use_own_xacts = true;
else
{
Assert(vacstmt->analyze);
if (in_outer_xact)
use_own_xacts = false;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
else if (list_length(relations) > 1)
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
use_own_xacts = true;
else
use_own_xacts = false;
}
Repair access-to-already-freed-memory error recently introduced into VACUUM.
2000-03-09 00:41:00 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we are running ANALYZE without per-table transactions, we'll need a
* memory context with table lifetime.
Fix up gcc warnings, improve comments.
2002-06-15 23:52:31 +02:00
*/
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
if (!use_own_xacts)
anl_context = AllocSetContextCreate(PortalContext,
"Analyze",
ALLOCSET_DEFAULT_MINSIZE,
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
Fix up gcc warnings, improve comments.
2002-06-15 23:52:31 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* vacuum_rel expects to be entered with no transaction active; it will
* start and commit its own transaction. But we are called by an SQL
* command, and so we are executing inside a transaction already. We
* commit the transaction started in PostgresMain() here, and start
* another one before exiting to match the commit waiting for us back in
* PostgresMain().
Repair access-to-already-freed-memory error recently introduced into VACUUM.
2000-03-09 00:41:00 +01:00
*/
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
if (use_own_xacts)
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
{
/* matches the StartTransaction in PostgresMain() */
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
CommitTransactionCommand();
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
}
Repair access-to-already-freed-memory error recently introduced into VACUUM.
2000-03-09 00:41:00 +01:00
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
/* Turn vacuum cost accounting on or off */
PG_TRY();
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
ListCell *cur;
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
Rename vacuum_cost_naptime to vacuum_cost_delay, with agreement from Jan.
2004-08-06 06:15:09 +02:00
VacuumCostActive = (VacuumCostDelay > 0);
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
VacuumCostBalance = 0;
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Loop to process each selected relation.
*/
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
foreach(cur, relations)
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
{
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
Oid relid = lfirst_oid(cur);
if (vacstmt->vacuum)
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
vacuum_rel(relid, vacstmt, RELKIND_RELATION);
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
if (vacstmt->analyze)
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
{
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
MemoryContext old_context = NULL;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If using separate xacts, start one for analyze. Otherwise,
* we can use the outer transaction, but we still need to call
* analyze_rel in a memory context that will be cleaned up on
* return (else we leak memory while processing multiple
* tables).
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
*/
if (use_own_xacts)
{
StartTransactionCommand();
Redesign query-snapshot timing so that volatile functions in READ COMMITTED mode see a fresh snapshot for each command in the function, rather than using the latest interactive command's snapshot. Also, suppress fresh snapshots as well as CommandCounterIncrement inside STABLE and IMMUTABLE functions, instead using the snapshot taken for the most closely nested regular query. (This behavior is only sane for read-only functions, so the patch also enforces that such functions contain only SELECT commands.) As per my proposal of 6-Sep-2004; I note that I floated essentially the same proposal on 19-Jun-2002, but that discussion tailed off without any action. Since 8.0 seems like the right place to be taking possibly nontrivial backwards compatibility hits, let's get it done now.
2004-09-13 22:10:13 +02:00
/* functions in indexes may want a snapshot set */
ActiveSnapshot = CopySnapshot(GetTransactionSnapshot());
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
}
else
old_context = MemoryContextSwitchTo(anl_context);
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Tell the buffer replacement strategy that vacuum is causing
* the IO
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
*/
StrategyHintVacuum(true);
analyze_rel(relid, vacstmt);
StrategyHintVacuum(false);
if (use_own_xacts)
CommitTransactionCommand();
else
{
MemoryContextSwitchTo(old_context);
MemoryContextResetAndDeleteChildren(anl_context);
}
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
}
}
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
PG_CATCH();
{
/* Make sure cost accounting is turned off after error */
VacuumCostActive = false;
PG_RE_THROW();
}
PG_END_TRY();
/* Turn off vacuum cost accounting */
VacuumCostActive = false;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Finish up processing.
*/
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
if (use_own_xacts)
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
{
Fix up gcc warnings, improve comments.
2002-06-15 23:52:31 +02:00
/* here, we are not in a transaction */
Arrange for VACUUM to delete the init file that relcache.c uses to save a little bit of backend startup time. This way, the first backend started after a VACUUM will rebuild the init file with up-to-date statistics for the critical system indexes.
1999-05-01 21:09:46 +02:00
AUTOCOMMIT mode is now an available backend GUC variable; setting it to false provides more SQL-spec-compliant behavior than we had before. I am not sure that setting it false is actually a good idea yet; there is a lot of client-side code that will probably be broken by turning autocommit off. But it's a start. Loosely based on a patch by David Van Wie.
2002-08-31 00:18:07 +02:00
/*
pgindent run.
2002-09-04 22:31:48 +02:00
* This matches the CommitTransaction waiting for us in
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
* PostgresMain().
AUTOCOMMIT mode is now an available backend GUC variable; setting it to false provides more SQL-spec-compliant behavior than we had before. I am not sure that setting it false is actually a good idea yet; there is a lot of client-side code that will probably be broken by turning autocommit off. But it's a start. Loosely based on a patch by David Van Wie.
2002-08-31 00:18:07 +02:00
*/
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
StartTransactionCommand();
Fix another case in which autovacuum would fail while analyzing expressional indexes. Per report from Brian Hirt.
2006-01-04 20:16:24 +01:00
/*
* Re-establish the transaction snapshot. This is wasted effort
* when we are called as a normal utility command, because the
* new transaction will be dropped immediately by PostgresMain();
* but it's necessary if we are called from autovacuum because
* autovacuum might continue on to do an ANALYZE-only call.
*/
ActiveSnapshot = CopySnapshot(GetTransactionSnapshot());
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
}
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
For multi-table ANALYZE, use per-table transactions when possible (ie, when not inside a transaction block), so that we can avoid holding locks longer than necessary. Per trouble report from Philip Warner.
2004-05-23 01:14:38 +02:00
if (vacstmt->vacuum)
{
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
TransactionId minxid,
vacuumxid;
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If it was a database-wide VACUUM, print FSM usage statistics (we
* don't make you be superuser to see these).
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
*/
Clean up a number of autovacuum loose ends. Make the stats collector track shared relations in a separate hashtable, so that operations done from different databases are counted correctly. Add proper support for anti-XID-wraparound vacuuming, even in databases that are never connected to and so have no stats entries. Miscellaneous other bug fixes. Alvaro Herrera, some additional fixes by Tom Lane.
2005-07-29 21:30:09 +02:00
if (all_rels)
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
PrintFreeSpaceMapStatistics(elevel);
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/* Update pg_database.datminxid and datvacuumxid */
vac_update_dbminxid(MyDatabaseId, &minxid, &vacuumxid);
/* Try to truncate pg_clog. */
vac_truncate_clog(minxid, vacuumxid);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
}
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
/*
* Clean up working storage --- note we must do this after
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* StartTransactionCommand, else we might be trying to delete the active
* context!
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
*/
MemoryContextDelete(vac_context);
vac_context = NULL;
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
Fix up gcc warnings, improve comments.
2002-06-15 23:52:31 +02:00
if (anl_context)
Allow ANALYZE to run in a transaction.
2002-06-13 21:52:02 +02:00
MemoryContextDelete(anl_context);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
* Build a list of Oids for each relation to be processed
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
*
* The list is built in vac_context so that it will survive across our
* per-relation transactions.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
static List *
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
get_rel_oids(List *relids, const RangeVar *vacrel, const char *stmttype)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Code cleanup, mostly in the smgr: - Update comment in IsReservedName() to the present day - Improve some variable & function names in commands/vacuum.c. I was planning to rewrite this to avoid lappend(), but since I still intend to do the list rewrite, there's no need for that. - Update some smgr comments which seemed to imply that we still forced all dirty pages to disk at commit-time. - Replace some #ifdef DIAGNOSTIC code with assertions. - Make the distinction between OS-level file descriptors and virtual file descriptors a little clearer in a few comments - Other minor comment improvements in the smgr code
2004-01-06 19:07:32 +01:00
List *oid_list = NIL;
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
MemoryContext oldcontext;
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
/* List supplied by VACUUM's caller? */
if (relids)
return relids;
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
if (vacrel)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Code cleanup, mostly in the smgr: - Update comment in IsReservedName() to the present day - Improve some variable & function names in commands/vacuum.c. I was planning to rewrite this to avoid lappend(), but since I still intend to do the list rewrite, there's no need for that. - Update some smgr comments which seemed to imply that we still forced all dirty pages to disk at commit-time. - Replace some #ifdef DIAGNOSTIC code with assertions. - Make the distinction between OS-level file descriptors and virtual file descriptors a little clearer in a few comments - Other minor comment improvements in the smgr code
2004-01-06 19:07:32 +01:00
/* Process a specific relation */
pgindent run.
2002-09-04 22:31:48 +02:00
Oid relid;
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
relid = RangeVarGetRelid(vacrel, false);
/* Make a relation list entry for this guy */
oldcontext = MemoryContextSwitchTo(vac_context);
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
oid_list = lappend_oid(oid_list, relid);
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
MemoryContextSwitchTo(oldcontext);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
{
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
/* Process all plain relations listed in pg_class */
Relation pgclass;
HeapScanDesc scan;
HeapTuple tuple;
ScanKeyData key;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Cross-data-type comparisons are now indexable by btrees, pursuant to my pghackers proposal of 8-Nov. All the existing cross-type comparison operators (int2/int4/int8 and float4/float8) have appropriate support. The original proposal of storing the right-hand-side datatype as part of the primary key for pg_amop and pg_amproc got modified a bit in the event; it is easier to store zero as the 'default' case and only store a nonzero when the operator is actually cross-type. Along the way, remove the long-since-defunct bigbox_ops operator class.
2003-11-12 22:15:59 +01:00
ScanKeyInit(&key,
Anum_pg_class_relkind,
BTEqualStrategyNumber, F_CHAREQ,
CharGetDatum(RELKIND_RELATION));
Added restriction to tables vacuum'able. Added NOTICE messages for bad table names.
1996-10-03 22:11:41 +02:00
Completion of project to use fixed OIDs for all system catalogs and indexes. Replace all heap_openr and index_openr calls by heap_open and index_open. Remove runtime lookups of catalog OID numbers in various places. Remove relcache's support for looking up system catalogs by name. Bulky but mostly very boring patch ...
2005-04-14 22:03:27 +02:00
pgclass = heap_open(RelationRelationId, AccessShareLock);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure indexscan API (index_beginscan, index_getnext) per yesterday's proposal to pghackers. Also remove unnecessary parameters to heap_beginscan, heap_rescan. I modified pg_proc.h to reflect the new numbers of parameters for the AM interface routines, but did not force an initdb because nothing actually looks at those fields.
2002-05-21 01:51:44 +02:00
scan = heap_beginscan(pgclass, SnapshotNow, 1, &key);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Restructure indexscan API (index_beginscan, index_getnext) per yesterday's proposal to pghackers. Also remove unnecessary parameters to heap_beginscan, heap_rescan. I modified pg_proc.h to reflect the new numbers of parameters for the AM interface routines, but did not force an initdb because nothing actually looks at those fields.
2002-05-21 01:51:44 +02:00
while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
/* Make a relation list entry for this guy */
oldcontext = MemoryContextSwitchTo(vac_context);
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
oid_list = lappend_oid(oid_list, HeapTupleGetOid(tuple));
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
MemoryContextSwitchTo(oldcontext);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
heap_endscan(scan);
heap_close(pgclass, AccessShareLock);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Code cleanup, mostly in the smgr: - Update comment in IsReservedName() to the present day - Improve some variable & function names in commands/vacuum.c. I was planning to rewrite this to avoid lappend(), but since I still intend to do the list rewrite, there's no need for that. - Update some smgr comments which seemed to imply that we still forced all dirty pages to disk at commit-time. - Replace some #ifdef DIAGNOSTIC code with assertions. - Make the distinction between OS-level file descriptors and virtual file descriptors a little clearer in a few comments - Other minor comment improvements in the smgr code
2004-01-06 19:07:32 +01:00
return oid_list;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
/*
* vacuum_set_xid_limits() -- compute oldest-Xmin and freeze cutoff points
*/
void
vacuum_set_xid_limits(VacuumStmt *vacstmt, bool sharedRel,
TransactionId *oldestXmin,
TransactionId *freezeLimit)
{
TransactionId limit;
*oldestXmin = GetOldestXmin(sharedRel);
Assert(TransactionIdIsNormal(*oldestXmin));
if (vacstmt->freeze)
{
/* FREEZE option: use oldest Xmin as freeze cutoff too */
limit = *oldestXmin;
}
else
{
/*
* Normal case: freeze cutoff is well in the past, to wit, about
* halfway to the wrap horizon
*/
limit = GetCurrentTransactionId() - (MaxTransactionId >> 2);
}
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Be careful not to generate a "permanent" XID
*/
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
if (!TransactionIdIsNormal(limit))
limit = FirstNormalTransactionId;
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Ensure sane relationship of limits
*/
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
if (TransactionIdFollows(limit, *oldestXmin))
{
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
(errmsg("oldest xmin is far in the past"),
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
errhint("Close open transactions soon to avoid wraparound problems.")));
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
limit = *oldestXmin;
}
*freezeLimit = limit;
}
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
* vac_update_relstats() -- update statistics for one relation
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
* Update the whole-relation statistics that are kept in its pg_class
* row. There are additional stats that will be updated if we are
* doing ANALYZE, but we always update these stats. This routine works
* for both index and heap relation entries in pg_class.
*
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
* We violate transaction semantics here by overwriting the rel's
* existing pg_class tuple with the new values. This is reasonably
* safe since the new values are correct whether or not this transaction
* commits. The reason for this is that if we updated these tuples in
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
* the usual way, vacuuming pg_class itself wouldn't work very well ---
* by the time we got done with a vacuum cycle, most of the tuples in
* pg_class would've been obsoleted. Of course, this only works for
* fixed-size never-null columns, but these are.
*
* This routine is shared by full VACUUM, lazy VACUUM, and stand-alone
* ANALYZE.
*/
void
vac_update_relstats(Oid relid, BlockNumber num_pages, double num_tuples,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
bool hasindex, TransactionId minxid,
TransactionId vacuumxid)
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
{
Relation rd;
HeapTuple ctup;
Form_pg_class pgcform;
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
bool dirty;
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Completion of project to use fixed OIDs for all system catalogs and indexes. Replace all heap_openr and index_openr calls by heap_open and index_open. Remove runtime lookups of catalog OID numbers in various places. Remove relcache's support for looking up system catalogs by name. Bulky but mostly very boring patch ...
2005-04-14 22:03:27 +02:00
rd = heap_open(RelationRelationId, RowExclusiveLock);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
/* Fetch a copy of the tuple to scribble on */
ctup = SearchSysCacheCopy(RELOID,
ObjectIdGetDatum(relid),
0, 0, 0);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
if (!HeapTupleIsValid(ctup))
elog(ERROR, "pg_class entry for relid %u vanished during vacuuming",
relid);
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
pgcform = (Form_pg_class) GETSTRUCT(ctup);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
/* Apply required updates, if any, to copied tuple */
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
dirty = false;
if (pgcform->relpages != (int32) num_pages)
{
pgcform->relpages = (int32) num_pages;
dirty = true;
}
if (pgcform->reltuples != (float4) num_tuples)
{
pgcform->reltuples = (float4) num_tuples;
dirty = true;
}
if (pgcform->relhasindex != hasindex)
{
pgcform->relhasindex = hasindex;
dirty = true;
}
Make OIDs optional, per discussions in pghackers. WITH OIDS is still the default, but OIDS are removed from many system catalogs that don't need them. Some interesting side effects: TOAST pointers are 20 bytes not 32 now; pg_description has a three-column key instead of one. Bugs fixed in passing: BINARY cursors work again; pg_class.relhaspkey has some usefulness; pg_dump dumps comments on indexes, rules, and triggers in a valid order. initdb forced.
2001-08-10 20:57:42 +02:00
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* If we have discovered that there are no indexes, then there's no
* primary key either. This could be done more thoroughly...
Make OIDs optional, per discussions in pghackers. WITH OIDS is still the default, but OIDS are removed from many system catalogs that don't need them. Some interesting side effects: TOAST pointers are 20 bytes not 32 now; pg_description has a three-column key instead of one. Bugs fixed in passing: BINARY cursors work again; pg_class.relhaspkey has some usefulness; pg_dump dumps comments on indexes, rules, and triggers in a valid order. initdb forced.
2001-08-10 20:57:42 +02:00
*/
if (!hasindex)
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
{
if (pgcform->relhaspkey)
{
pgcform->relhaspkey = false;
dirty = true;
}
}
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
if (TransactionIdIsValid(minxid) && pgcform->relminxid != minxid)
{
pgcform->relminxid = minxid;
dirty = true;
}
if (TransactionIdIsValid(vacuumxid) && pgcform->relvacuumxid != vacuumxid)
{
pgcform->relvacuumxid = vacuumxid;
dirty = true;
}
Add a bit more locking to vac_update_relstats and vac_update_dbstats to make them comparable to what UpdateStats does in the same situation. I'm not certain two instances of vac_update_relstats could run in parallel for the same relation, but parallel invocations of vac_update_dbstats do seem possible.
2003-10-03 01:19:44 +02:00
A bunch of changes aimed at reducing backend startup time... Improve 'pg_internal.init' relcache entry preload mechanism so that it is safe to use for all system catalogs, and arrange to preload a realistic set of system-catalog entries instead of only the three nailed-in-cache indexes that were formerly loaded this way. Fix mechanism for deleting out-of-date pg_internal.init files: this must be synchronized with transaction commit, not just done at random times within transactions. Drive it off relcache invalidation mechanism so that no special-case tests are needed. Cache additional information in relcache entries for indexes (their pg_index tuples and index-operator OIDs) to eliminate repeated lookups. Also cache index opclass info at the per-opclass level to avoid repeated lookups during relcache load. Generalize 'systable scan' utilities originally developed by Hiroshi, move them into genam.c, use in a number of places where there was formerly ugly code for choosing either heap or index scan. In particular this allows simplification of the logic that prevents infinite recursion between syscache and relcache during startup: we can easily switch to heapscans in relcache.c when and where needed to avoid recursion, so IndexScanOK becomes simpler and does not need any expensive initialization. Eliminate useless opening of a heapscan data structure while doing an indexscan (this saves an mdnblocks call and thus at least one kernel call).
2002-02-19 21:11:20 +01:00
/*
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
* If anything changed, write out the tuple
A bunch of changes aimed at reducing backend startup time... Improve 'pg_internal.init' relcache entry preload mechanism so that it is safe to use for all system catalogs, and arrange to preload a realistic set of system-catalog entries instead of only the three nailed-in-cache indexes that were formerly loaded this way. Fix mechanism for deleting out-of-date pg_internal.init files: this must be synchronized with transaction commit, not just done at random times within transactions. Drive it off relcache invalidation mechanism so that no special-case tests are needed. Cache additional information in relcache entries for indexes (their pg_index tuples and index-operator OIDs) to eliminate repeated lookups. Also cache index opclass info at the per-opclass level to avoid repeated lookups during relcache load. Generalize 'systable scan' utilities originally developed by Hiroshi, move them into genam.c, use in a number of places where there was formerly ugly code for choosing either heap or index scan. In particular this allows simplification of the logic that prevents infinite recursion between syscache and relcache during startup: we can easily switch to heapscans in relcache.c when and where needed to avoid recursion, so IndexScanOK becomes simpler and does not need any expensive initialization. Eliminate useless opening of a heapscan data structure while doing an indexscan (this saves an mdnblocks call and thus at least one kernel call).
2002-02-19 21:11:20 +01:00
*/
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
if (dirty)
heap_inplace_update(rd, ctup);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
heap_close(rd, RowExclusiveLock);
}
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
/*
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
* vac_update_dbminxid() -- update the minimum Xid present in one database
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
*
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
* Update pg_database's datminxid and datvacuumxid, and the flat-file copy
* of it. datminxid is updated to the minimum of all relminxid found in
* pg_class. datvacuumxid is updated to the minimum of all relvacuumxid
* found in pg_class. The values are also returned in minxid and
* vacuumxid, respectively.
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
*
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
* We violate transaction semantics here by overwriting the database's
* existing pg_database tuple with the new values. This is reasonably
* safe since the new values are correct whether or not this transaction
* commits. As with vac_update_relstats, this avoids leaving dead tuples
* behind after a VACUUM.
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
*
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
* This routine is shared by full and lazy VACUUM.
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
*/
static void
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
vac_update_dbminxid(Oid dbid, TransactionId *minxid, TransactionId *vacuumxid)
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
{
HeapTuple tuple;
Form_pg_database dbform;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
Relation relation;
SysScanDesc scan;
HeapTuple classTup;
TransactionId newMinXid = InvalidTransactionId;
TransactionId newVacXid = InvalidTransactionId;
bool dirty = false;
/*
* We must seqscan pg_class to find the minimum Xid, because there
* is no index that can help us here.
*/
relation = heap_open(RelationRelationId, AccessShareLock);
scan = systable_beginscan(relation, InvalidOid, false,
SnapshotNow, 0, NULL);
while ((classTup = systable_getnext(scan)) != NULL)
{
Form_pg_class classForm;
classForm = (Form_pg_class) GETSTRUCT(classTup);
/*
* Only consider heap and TOAST tables (anything else should have
* InvalidTransactionId in both fields anyway.)
*/
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_TOASTVALUE)
continue;
Assert(TransactionIdIsNormal(classForm->relminxid));
Assert(TransactionIdIsNormal(classForm->relvacuumxid));
/*
* Compute the minimum relminxid in all the tables in the database.
*/
if ((!TransactionIdIsValid(newMinXid) ||
TransactionIdPrecedes(classForm->relminxid, newMinXid)))
newMinXid = classForm->relminxid;
/* ditto, for relvacuumxid */
if ((!TransactionIdIsValid(newVacXid) ||
TransactionIdPrecedes(classForm->relvacuumxid, newVacXid)))
newVacXid = classForm->relvacuumxid;
}
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/* we're done with pg_class */
systable_endscan(scan);
heap_close(relation, AccessShareLock);
Assert(TransactionIdIsNormal(newMinXid));
Assert(TransactionIdIsNormal(newVacXid));
/* Now fetch the pg_database tuple we need to update. */
Completion of project to use fixed OIDs for all system catalogs and indexes. Replace all heap_openr and index_openr calls by heap_open and index_open. Remove runtime lookups of catalog OID numbers in various places. Remove relcache's support for looking up system catalogs by name. Bulky but mostly very boring patch ...
2005-04-14 22:03:27 +02:00
relation = heap_open(DatabaseRelationId, RowExclusiveLock);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Clean up code associated with updating pg_class statistics columns (relpages/reltuples). To do this, create formal support in heapam.c for "overwrite" tuple updates (including xlog replay capability) and use that instead of the ad-hoc overwrites we'd been using in VACUUM and CREATE INDEX. Take the responsibility for updating stats during CREATE INDEX out of the individual index AMs, and do it where it belongs, in catalog/index.c. Aside from being more modular, this avoids having to update the same tuple twice in some paths through CREATE INDEX. It's probably not measurably faster, but for sure it's a lot cleaner than before.
2006-05-11 01:18:39 +02:00
/* Fetch a copy of the tuple to scribble on */
tuple = SearchSysCacheCopy(DATABASEOID,
ObjectIdGetDatum(dbid),
0, 0, 0);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
if (!HeapTupleIsValid(tuple))
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
elog(ERROR, "could not find tuple for database %u", dbid);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
dbform = (Form_pg_database) GETSTRUCT(tuple);
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
if (TransactionIdPrecedes(dbform->datminxid, newMinXid))
{
dbform->datminxid = newMinXid;
dirty = true;
}
if (TransactionIdPrecedes(dbform->datvacuumxid, newVacXid))
{
dbform->datvacuumxid = newVacXid;
dirty = true;
}
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
if (dirty)
heap_inplace_update(relation, tuple);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
heap_freetuple(tuple);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
heap_close(relation, RowExclusiveLock);
Clean up a number of autovacuum loose ends. Make the stats collector track shared relations in a separate hashtable, so that operations done from different databases are counted correctly. Add proper support for anti-XID-wraparound vacuuming, even in databases that are never connected to and so have no stats entries. Miscellaneous other bug fixes. Alvaro Herrera, some additional fixes by Tom Lane.
2005-07-29 21:30:09 +02:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/* set return values */
*minxid = newMinXid;
*vacuumxid = newVacXid;
Clean up a number of autovacuum loose ends. Make the stats collector track shared relations in a separate hashtable, so that operations done from different databases are counted correctly. Add proper support for anti-XID-wraparound vacuuming, even in databases that are never connected to and so have no stats entries. Miscellaneous other bug fixes. Alvaro Herrera, some additional fixes by Tom Lane.
2005-07-29 21:30:09 +02:00
/* Mark the flat-file copy of pg_database for update at commit */
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
database_file_update_needed();
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
}
/*
* vac_truncate_clog() -- attempt to truncate the commit log
*
* Scan pg_database to determine the system-wide oldest datvacuumxid,
* and use it to truncate the transaction commit log (pg_clog).
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
* Also update the XID wrap limit point maintained by varsup.c.
*
* We also generate a warning if the system-wide oldest datfrozenxid
* seems to be in danger of wrapping around. This is a long-in-advance
* warning; if we start getting uncomfortably close, GetNewTransactionId
* will generate more-annoying warnings, and ultimately refuse to issue
* any more new XIDs.
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
*
* The passed XIDs are simply the ones I just wrote into my pg_database
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* entry. They're used to initialize the "min" calculations.
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
*
* This routine is shared by full and lazy VACUUM. Note that it is only
* applied after a database-wide VACUUM operation.
*/
static void
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
vac_truncate_clog(TransactionId myminxid, TransactionId myvacxid)
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
{
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
TransactionId myXID = GetCurrentTransactionId();
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
TransactionId minXID;
TransactionId vacuumXID;
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Relation relation;
HeapScanDesc scan;
HeapTuple tuple;
int32 age;
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
NameData oldest_datname;
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
bool vacuumAlreadyWrapped = false;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
bool minAlreadyWrapped = false;
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/* Initialize the minimum values. */
minXID = myminxid;
vacuumXID = myvacxid;
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
namestrcpy(&oldest_datname, get_database_name(MyDatabaseId));
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
/*
* Note: the "already wrapped" cases should now be impossible due to the
* defenses in GetNewTransactionId, but we keep them anyway.
*/
Completion of project to use fixed OIDs for all system catalogs and indexes. Replace all heap_openr and index_openr calls by heap_open and index_open. Remove runtime lookups of catalog OID numbers in various places. Remove relcache's support for looking up system catalogs by name. Bulky but mostly very boring patch ...
2005-04-14 22:03:27 +02:00
relation = heap_open(DatabaseRelationId, AccessShareLock);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Restructure indexscan API (index_beginscan, index_getnext) per yesterday's proposal to pghackers. Also remove unnecessary parameters to heap_beginscan, heap_rescan. I modified pg_proc.h to reflect the new numbers of parameters for the AM interface routines, but did not force an initdb because nothing actually looks at those fields.
2002-05-21 01:51:44 +02:00
scan = heap_beginscan(relation, SnapshotNow, 0, NULL);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Restructure indexscan API (index_beginscan, index_getnext) per yesterday's proposal to pghackers. Also remove unnecessary parameters to heap_beginscan, heap_rescan. I modified pg_proc.h to reflect the new numbers of parameters for the AM interface routines, but did not force an initdb because nothing actually looks at those fields.
2002-05-21 01:51:44 +02:00
while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
{
Form_pg_database dbform = (Form_pg_database) GETSTRUCT(tuple);
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
Assert(TransactionIdIsNormal(dbform->datvacuumxid));
Assert(TransactionIdIsNormal(dbform->datminxid));
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
if (TransactionIdPrecedes(myXID, dbform->datvacuumxid))
vacuumAlreadyWrapped = true;
else if (TransactionIdPrecedes(dbform->datvacuumxid, vacuumXID))
vacuumXID = dbform->datvacuumxid;
if (TransactionIdPrecedes(myXID, dbform->datminxid))
minAlreadyWrapped = true;
else if (TransactionIdPrecedes(dbform->datminxid, minXID))
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
{
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
minXID = dbform->datminxid;
namecpy(&oldest_datname, &dbform->datname);
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
}
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
}
heap_endscan(scan);
heap_close(relation, AccessShareLock);
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Do not truncate CLOG if we seem to have suffered wraparound already;
* the computed minimum XID might be bogus.
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
*/
if (vacuumAlreadyWrapped)
{
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
(errmsg("some databases have not been vacuumed in over 2 billion transactions"),
errdetail("You may have already suffered transaction-wraparound data loss.")));
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
return;
}
Rearrange pg_subtrans handling as per recent discussion. pg_subtrans updates are no longer WAL-logged nor even fsync'd; we do not need to, since after a crash no old pg_subtrans data is needed again. We truncate pg_subtrans to RecentGlobalXmin at each checkpoint. slru.c's API is refactored a little bit to separate out the necessary decisions.
2004-08-24 01:22:45 +02:00
/* Truncate CLOG to the oldest vacuumxid */
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
TruncateCLOG(vacuumXID);
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Do not update varsup.c if we seem to have suffered wraparound already;
* the computed XID might be bogus.
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
*/
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
if (minAlreadyWrapped)
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
{
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
(errmsg("some databases have not been vacuumed in over 1 billion transactions"),
errhint("Better vacuum them soon, or you may have a wraparound failure.")));
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
return;
Fix CLOG truncation code to not do the Wrong Thing when there are already wrapped-around databases. The unvacuumed databases might be fine, or they might not, but things will definitely not be fine if we remove the wrong CLOG segments. Per trouble report from Gary Wolfe, 1-Apr-2002.
2002-04-02 07:11:55 +02:00
}
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
/* Update the wrap limit for GetNewTransactionId */
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
SetTransactionIdLimit(minXID, &oldest_datname);
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
/* Give warning about impending wraparound problems */
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
age = (int32) (myXID - minXID);
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
if (age > (int32) ((MaxTransactionId >> 3) * 3))
ereport(WARNING,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("database \"%s\" must be vacuumed within %u transactions",
NameStr(oldest_datname),
(MaxTransactionId >> 1) - age),
errhint("To avoid a database shutdown, execute a full-database VACUUM in \"%s\".",
NameStr(oldest_datname))));
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/*
* Have the postmaster start an autovacuum iteration. If the user has
* autovacuum configured, this is not needed; otherwise, we need to make
* sure we have some mechanism to cope with transaction Id wraparound.
* Ideally this would only be needed for template databases, because all
* other databases should be kept nicely pruned by regular vacuuming.
*
* XXX -- the test we use here is fairly arbitrary. Note that in the
* autovacuum database-wide code, a template database is always processed
* with VACUUM FREEZE, so we can be sure that it will be truly frozen so
* it won't be need to be processed here again soon.
*
* FIXME -- here we could get into a kind of loop if the database being
* chosen is not actually a template database, because we'll not freeze
* it, so its age may not really decrease if there are any live
* non-freezable tuples. Consider forcing a vacuum freeze if autovacuum
* is invoked by a backend. On the other hand, forcing a vacuum freeze
* on a user database may not a be a very polite thing to do.
*/
if (!AutoVacuumingActive() && age > (int32) ((MaxTransactionId >> 3) * 3))
SendPostmasterSignal(PMSIGNAL_START_AUTOVAC);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
}
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
/****************************************************************************
* *
* Code common to both flavors of VACUUM *
* *
****************************************************************************
*/
/*
* vacuum_rel() -- vacuum one heap relation
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* Doing one heap at a time incurs extra overhead, since we need to
* check that the heap exists again just before we vacuum it. The
* reason that we do this is so that vacuuming can be spread across
* many small transactions. Otherwise, two-phase locking would require
* us to lock the entire database during one pass of the vacuum cleaner.
Revise lock manager to support "session level" locks as well as "transaction level" locks. A session lock is not released at transaction commit (but it is released on transaction abort, to ensure recovery after an elog(ERROR)). In VACUUM, use a session lock to protect the master table while vacuuming a TOAST table, so that the TOAST table can be done in an independent transaction. I also took this opportunity to do some cleanup and renaming in the lock code. The previously noted bug in ProcLockWakeup, that it couldn't wake up any waiters beyond the first non-wakeable waiter, is now fixed. Also found a previously unknown bug of the same kind (failure to scan all members of a lock queue in some cases) in DeadLockCheck. This might have led to failure to detect a deadlock condition, resulting in indefinite waits, but it's difficult to characterize the conditions required to trigger a failure.
2000-12-22 01:51:54 +01:00
*
* At entry and exit, we are not inside a transaction.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
static void
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
LOCKMODE lmode;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
Relation onerel;
Revise lock manager to support "session level" locks as well as "transaction level" locks. A session lock is not released at transaction commit (but it is released on transaction abort, to ensure recovery after an elog(ERROR)). In VACUUM, use a session lock to protect the master table while vacuuming a TOAST table, so that the TOAST table can be done in an independent transaction. I also took this opportunity to do some cleanup and renaming in the lock code. The previously noted bug in ProcLockWakeup, that it couldn't wake up any waiters beyond the first non-wakeable waiter, is now fixed. Also found a previously unknown bug of the same kind (failure to scan all members of a lock queue in some cases) in DeadLockCheck. This might have led to failure to detect a deadlock condition, resulting in indefinite waits, but it's difficult to characterize the conditions required to trigger a failure.
2000-12-22 01:51:54 +01:00
LockRelId onerelid;
Changed TOAST relations to have relkind RELKIND_TOASTVALUE. Special handling of TOAST relations during VACUUM. TOAST relations are vacuumed while the lock on the master table is still active. The ANALYZE flag doesn't propagate to their vacuuming because the toaster access routines allways use index access ignoring stats, so why compute them at all. Protection of TOAST relations against normal INSERT/UPDATE/DELETE while offering SELECT for debugging purposes. Jan
2000-07-05 18:17:43 +02:00
Oid toast_relid;
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
Revise lock manager to support "session level" locks as well as "transaction level" locks. A session lock is not released at transaction commit (but it is released on transaction abort, to ensure recovery after an elog(ERROR)). In VACUUM, use a session lock to protect the master table while vacuuming a TOAST table, so that the TOAST table can be done in an independent transaction. I also took this opportunity to do some cleanup and renaming in the lock code. The previously noted bug in ProcLockWakeup, that it couldn't wake up any waiters beyond the first non-wakeable waiter, is now fixed. Also found a previously unknown bug of the same kind (failure to scan all members of a lock queue in some cases) in DeadLockCheck. This might have led to failure to detect a deadlock condition, resulting in indefinite waits, but it's difficult to characterize the conditions required to trigger a failure.
2000-12-22 01:51:54 +01:00
/* Begin a transaction for vacuuming this relation */
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
StartTransactionCommand();
Redesign query-snapshot timing so that volatile functions in READ COMMITTED mode see a fresh snapshot for each command in the function, rather than using the latest interactive command's snapshot. Also, suppress fresh snapshots as well as CommandCounterIncrement inside STABLE and IMMUTABLE functions, instead using the snapshot taken for the most closely nested regular query. (This behavior is only sane for read-only functions, so the patch also enforces that such functions contain only SELECT commands.) As per my proposal of 6-Sep-2004; I note that I floated essentially the same proposal on 19-Jun-2002, but that discussion tailed off without any action. Since 8.0 seems like the right place to be taking possibly nontrivial backwards compatibility hits, let's get it done now.
2004-09-13 22:10:13 +02:00
/* functions in indexes may want a snapshot set */
ActiveSnapshot = CopySnapshot(GetTransactionSnapshot());
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
2nd try for the ARC strategy. I added a couple more Assertions while tracking down the exact cause of the former bug. All 93 regression tests pass now. Jan
2003-11-13 15:57:15 +01:00
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* Tell the cache replacement strategy that vacuum is causing all
* following IO
2nd try for the ARC strategy. I added a couple more Assertions while tracking down the exact cause of the former bug. All 93 regression tests pass now. Jan
2003-11-13 15:57:15 +01:00
*/
StrategyHintVacuum(true);
Remove pg_vlock locking from VACUUM, allowing multiple VACUUMs to run in parallel --- and, not incidentally, removing a common reason for needing manual cleanup by the DB admin after a crash. Remove initial global delete of pg_statistics rows in VACUUM ANALYZE; this was not only bad for performance of other backends that had to run without stats for a while, but it was fundamentally broken because it was done outside any transaction. Surprising we didn't see more consequences of that. Detect attempt to run VACUUM inside a transaction block. Check for query cancel request before starting vacuum of each table. Clean up vacuum's private portal storage if vacuum is aborted.
1999-11-28 03:10:01 +01:00
/*
Cleanup vacuum names.
2000-01-20 21:01:25 +01:00
* Check for user-requested abort. Note we want this to be inside a
Change made to elog: o Change all current CVS messages of NOTICE to WARNING. We were going to do this just before 7.3 beta but it has to be done now, as you will see below. o Change current INFO messages that should be controlled by client_min_messages to NOTICE. o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc. to always go to the client. o Remove INFO from the client_min_messages options and add NOTICE. Seems we do need three non-ERROR elog levels to handle the various behaviors we need for these messages. Regression passed.
2002-03-06 07:10:59 +01:00
* transaction, so xact.c doesn't issue useless WARNING.
Remove pg_vlock locking from VACUUM, allowing multiple VACUUMs to run in parallel --- and, not incidentally, removing a common reason for needing manual cleanup by the DB admin after a crash. Remove initial global delete of pg_statistics rows in VACUUM ANALYZE; this was not only bad for performance of other backends that had to run without stats for a while, but it was fundamentally broken because it was done outside any transaction. Surprising we didn't see more consequences of that. Detect attempt to run VACUUM inside a transaction block. Check for query cancel request before starting vacuum of each table. Clean up vacuum's private portal storage if vacuum is aborted.
1999-11-28 03:10:01 +01:00
*/
Restructure backend SIGINT/SIGTERM handling so that 'die' interrupts are treated more like 'cancel' interrupts: the signal handler sets a flag that is examined at well-defined spots, rather than trying to cope with an interrupt that might happen anywhere. See pghackers discussion of 1/12/01.
2001-01-14 06:08:17 +01:00
CHECK_FOR_INTERRUPTS();
Remove pg_vlock locking from VACUUM, allowing multiple VACUUMs to run in parallel --- and, not incidentally, removing a common reason for needing manual cleanup by the DB admin after a crash. Remove initial global delete of pg_statistics rows in VACUUM ANALYZE; this was not only bad for performance of other backends that had to run without stats for a while, but it was fundamentally broken because it was done outside any transaction. Surprising we didn't see more consequences of that. Detect attempt to run VACUUM inside a transaction block. Check for query cancel request before starting vacuum of each table. Clean up vacuum's private portal storage if vacuum is aborted.
1999-11-28 03:10:01 +01:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Race condition -- if the pg_class tuple has gone away since the last
* time we saw it, we don't need to vacuum it.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
if (!SearchSysCacheExists(RELOID,
ObjectIdGetDatum(relid),
0, 0, 0))
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
2nd try for the ARC strategy. I added a couple more Assertions while tracking down the exact cause of the former bug. All 93 regression tests pass now. Jan
2003-11-13 15:57:15 +01:00
StrategyHintVacuum(false);
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
CommitTransactionCommand();
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
return;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Mega-commit to make heap_open/heap_openr/heap_close take an additional argument specifying the kind of lock to acquire/release (or 'NoLock' to do no lock processing). Ensure that all relations are locked with some appropriate lock level before being examined --- this ensures that relevant shared-inval messages have been processed and should prevent problems caused by concurrent VACUUM. Fix several bugs having to do with mismatched increment/decrement of relation ref count and mismatched heap_open/close (which amounts to the same thing). A bogus ref count on a relation doesn't matter much *unless* a SI Inval message happens to arrive at the wrong time, which is probably why we got away with this sloppiness for so long. Repair missing grab of AccessExclusiveLock in DROP TABLE, ALTER/RENAME TABLE, etc, as noted by Hiroshi. Recommend 'make clean all' after pulling this update; I modified the Relation struct layout slightly. Will post further discussion to pghackers list shortly.
1999-09-18 21:08:25 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Determine the type of lock we want --- hard exclusive lock for a FULL
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* vacuum, but just ShareUpdateExclusiveLock for concurrent vacuum. Either
* way, we can be sure that no other backend is vacuuming the same table.
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
*/
lmode = vacstmt->full ? AccessExclusiveLock : ShareUpdateExclusiveLock;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Open the class, get an appropriate lock on it, and check permissions.
Add permissions check: now one must be the Postgres superuser or the table owner in order to vacuum a table. This is mainly to prevent denial-of-service attacks via repeated vacuums. Allow VACUUM to gather statistics about system relations, except for pg_statistic itself --- not clear that it's worth the trouble to make that case work cleanly. Cope with possible tuple size overflow in pg_statistic tuples; I'm surprised we never realized that could happen. Hold a couple of locks a little longer to try to prevent deadlocks between concurrent VACUUMs. There still seem to be some problems in that last area though :-(
1999-11-29 05:43:15 +01:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* We allow the user to vacuum a table if he is superuser, the table
* owner, or the database owner (but in the latter case, only if it's not
* a shared relation). pg_class_ownercheck includes the superuser case.
Allow a non-superuser database owner to vacuum all tables in his database, including system catalogs (but not the shared catalogs, since they don't really belong to his database). This is per recent mailing list discussion. Clean up some other code that also checks for database ownerness by introducing a test function is_dbadmin().
2001-06-13 23:44:41 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Note we choose to treat permissions failure as a WARNING and keep
* trying to vacuum the rest of the DB --- is this appropriate?
Mega-commit to make heap_open/heap_openr/heap_close take an additional argument specifying the kind of lock to acquire/release (or 'NoLock' to do no lock processing). Ensure that all relations are locked with some appropriate lock level before being examined --- this ensures that relevant shared-inval messages have been processed and should prevent problems caused by concurrent VACUUM. Fix several bugs having to do with mismatched increment/decrement of relation ref count and mismatched heap_open/close (which amounts to the same thing). A bogus ref count on a relation doesn't matter much *unless* a SI Inval message happens to arrive at the wrong time, which is probably why we got away with this sloppiness for so long. Repair missing grab of AccessExclusiveLock in DROP TABLE, ALTER/RENAME TABLE, etc, as noted by Hiroshi. Recommend 'make clean all' after pulling this update; I modified the Relation struct layout slightly. Will post further discussion to pghackers list shortly.
1999-09-18 21:08:25 +02:00
*/
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
onerel = relation_open(relid, lmode);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Change the aclchk.c routines to uniformly use OIDs to identify the objects to be privilege-checked. Some change in their APIs would be necessary no matter what in the schema environment, and simply getting rid of the name-based interface entirely seems like the best way.
2002-03-22 00:27:25 +01:00
if (!(pg_class_ownercheck(RelationGetRelid(onerel), GetUserId()) ||
First batch of object rename commands.
2003-06-27 16:45:32 +02:00
(pg_database_ownercheck(MyDatabaseId, GetUserId()) && !onerel->rd_rel->relisshared)))
Add permissions check: now one must be the Postgres superuser or the table owner in order to vacuum a table. This is mainly to prevent denial-of-service attacks via repeated vacuums. Allow VACUUM to gather statistics about system relations, except for pg_statistic itself --- not clear that it's worth the trouble to make that case work cleanly. Cope with possible tuple size overflow in pg_statistic tuples; I'm surprised we never realized that could happen. Hold a couple of locks a little longer to try to prevent deadlocks between concurrent VACUUMs. There still seem to be some problems in that last area though :-(
1999-11-29 05:43:15 +01:00
{
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
(errmsg("skipping \"%s\" --- only table or database owner can vacuum it",
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
RelationGetRelationName(onerel))));
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
relation_close(onerel, lmode);
2nd try for the ARC strategy. I added a couple more Assertions while tracking down the exact cause of the former bug. All 93 regression tests pass now. Jan
2003-11-13 15:57:15 +01:00
StrategyHintVacuum(false);
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
CommitTransactionCommand();
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
return;
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
}
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Check that it's a plain table; we used to do this in get_rel_oids() but
* seems safer to check after we've locked the relation.
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
*/
if (onerel->rd_rel->relkind != expected_relkind)
{
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
(errmsg("skipping \"%s\" --- cannot vacuum indexes, views, or special system tables",
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
RelationGetRelationName(onerel))));
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
relation_close(onerel, lmode);
2nd try for the ARC strategy. I added a couple more Assertions while tracking down the exact cause of the former bug. All 93 regression tests pass now. Jan
2003-11-13 15:57:15 +01:00
StrategyHintVacuum(false);
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
CommitTransactionCommand();
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
return;
Add permissions check: now one must be the Postgres superuser or the table owner in order to vacuum a table. This is mainly to prevent denial-of-service attacks via repeated vacuums. Allow VACUUM to gather statistics about system relations, except for pg_statistic itself --- not clear that it's worth the trouble to make that case work cleanly. Cope with possible tuple size overflow in pg_statistic tuples; I'm surprised we never realized that could happen. Hold a couple of locks a little longer to try to prevent deadlocks between concurrent VACUUMs. There still seem to be some problems in that last area though :-(
1999-11-29 05:43:15 +01:00
}
Disallow VACUUM, ANALYZE, TRUNCATE on temp tables belonging to other backends. Given that temp tables now store data locally in the local buffer manager, these things are not going to work safely.
2002-09-23 22:43:41 +02:00
/*
* Silently ignore tables that are temp tables of other backends ---
* trying to vacuum these will lead to great unhappiness, since their
* contents are probably not up-to-date on disk. (We don't throw a
* warning here; it would just lead to chatter during a database-wide
* VACUUM.)
*/
if (isOtherTempNamespace(RelationGetNamespace(onerel)))
{
relation_close(onerel, lmode);
2nd try for the ARC strategy. I added a couple more Assertions while tracking down the exact cause of the former bug. All 93 regression tests pass now. Jan
2003-11-13 15:57:15 +01:00
StrategyHintVacuum(false);
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
CommitTransactionCommand();
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
return; /* assume no long-lived data in temp tables */
Disallow VACUUM, ANALYZE, TRUNCATE on temp tables belonging to other backends. Given that temp tables now store data locally in the local buffer manager, these things are not going to work safely.
2002-09-23 22:43:41 +02:00
}
Make sure that FlushRelationBuffers() is invoked by all paths through vacuum.c. This is needed to make the world safe for pg_upgrade.
2000-09-19 21:30:03 +02:00
/*
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
* Get a session-level lock too. This will protect our access to the
* relation across multiple transactions, so that we can vacuum the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* relation's TOAST table (if any) secure in the knowledge that no one is
* deleting the parent relation.
Revise lock manager to support "session level" locks as well as "transaction level" locks. A session lock is not released at transaction commit (but it is released on transaction abort, to ensure recovery after an elog(ERROR)). In VACUUM, use a session lock to protect the master table while vacuuming a TOAST table, so that the TOAST table can be done in an independent transaction. I also took this opportunity to do some cleanup and renaming in the lock code. The previously noted bug in ProcLockWakeup, that it couldn't wake up any waiters beyond the first non-wakeable waiter, is now fixed. Also found a previously unknown bug of the same kind (failure to scan all members of a lock queue in some cases) in DeadLockCheck. This might have led to failure to detect a deadlock condition, resulting in indefinite waits, but it's difficult to characterize the conditions required to trigger a failure.
2000-12-22 01:51:54 +01:00
*
* NOTE: this cannot block, even if someone else is waiting for access,
* because the lock manager knows that both lock requests are from the
* same process.
*/
onerelid = onerel->rd_lockInfo.lockRelId;
Simplify shared-memory lock data structures as per recent discussion: it is sufficient to track whether a backend holds a lock or not, and store information about transaction vs. session locks only in the inside-the-backend LocalLockTable. Since there can now be but one PROCLOCK per lock per backend, LockCountMyLocks() is no longer needed, thus eliminating some O(N^2) behavior when a backend holds many locks. Also simplify the LockAcquire/LockRelease API by passing just a 'sessionLock' boolean instead of a transaction ID. The previous API was designed with the idea that per-transaction lock holding would be important for subtransactions, but now that we have subtransactions we know that this is unwanted. While at it, add an 'isTempObject' parameter to LockAcquire to indicate whether the lock is being taken on a temp table. This is not used just yet, but will be needed shortly for two-phase commit.
2005-06-15 00:15:33 +02:00
LockRelationForSession(&onerelid, onerel->rd_istemp, lmode);
Revise lock manager to support "session level" locks as well as "transaction level" locks. A session lock is not released at transaction commit (but it is released on transaction abort, to ensure recovery after an elog(ERROR)). In VACUUM, use a session lock to protect the master table while vacuuming a TOAST table, so that the TOAST table can be done in an independent transaction. I also took this opportunity to do some cleanup and renaming in the lock code. The previously noted bug in ProcLockWakeup, that it couldn't wake up any waiters beyond the first non-wakeable waiter, is now fixed. Also found a previously unknown bug of the same kind (failure to scan all members of a lock queue in some cases) in DeadLockCheck. This might have led to failure to detect a deadlock condition, resulting in indefinite waits, but it's difficult to characterize the conditions required to trigger a failure.
2000-12-22 01:51:54 +01:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Remember the relation's TOAST relation for later
*/
Make sure that FlushRelationBuffers() is invoked by all paths through vacuum.c. This is needed to make the world safe for pg_upgrade.
2000-09-19 21:30:03 +02:00
toast_relid = onerel->rd_rel->reltoastrelid;
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
if (vacstmt->full)
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
full_vacuum_rel(onerel, vacstmt);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
else
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
lazy_vacuum_rel(onerel, vacstmt);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
/* all done with this class, but hold lock until commit */
Make VACUUM handle schema-qualified relation names properly.
2002-04-02 03:03:07 +02:00
relation_close(onerel, NoLock);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Complete the transaction and free all temporary memory used.
*/
2nd try for the ARC strategy. I added a couple more Assertions while tracking down the exact cause of the former bug. All 93 regression tests pass now. Jan
2003-11-13 15:57:15 +01:00
StrategyHintVacuum(false);
Backend support for autocommit removed, per recent discussions. The only remnant of this failed experiment is that the server will take SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit logic in libpq.
2003-05-14 05:26:03 +02:00
CommitTransactionCommand();
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
/*
* If the relation has a secondary toast rel, vacuum that too while we
* still hold the session lock on the master table. Note however that
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* "analyze" will not get done on the toast table. This is good, because
* the toaster always uses hardcoded index access and statistics are
* totally unimportant for toast relations.
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
*/
if (toast_relid != InvalidOid)
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
vacuum_rel(toast_relid, vacstmt, RELKIND_TOASTVALUE);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Now release the session-level lock on the master table.
*/
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
UnlockRelationForSession(&onerelid, lmode);
ARGH! Vacuum must not advance pg_database.datvacuumxid nor truncate CLOG unless it's processed *all* tables in the database. Vacuums run by unprivileged users don't count. (Beats head against nearest convenient wall...)
2002-09-27 22:57:08 +02:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
return;
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
}
/****************************************************************************
* *
* Code for VACUUM FULL (only) *
* *
****************************************************************************
*/
/*
* full_vacuum_rel() -- perform FULL VACUUM for one heap relation
*
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
* This routine vacuums a single heap, cleans out its indexes, and
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
* updates its num_pages and num_tuples statistics.
*
* At entry, we have already established a transaction and opened
* and locked the relation.
*/
static void
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
{
VacPageListData vacuum_pages; /* List of pages to vacuum and/or
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
* clean indexes */
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
VacPageListData fraged_pages; /* List of pages with space enough for
* re-using */
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Relation *Irel;
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
int nindexes,
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
i;
VRelStats *vacrelstats;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
TransactionId FreezeLimit,
OldestXmin;
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
vacuum_set_xid_limits(vacstmt, onerel->rd_rel->relisshared,
&OldestXmin, &FreezeLimit);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Set up statistics-gathering machinery.
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
vacrelstats = (VRelStats *) palloc(sizeof(VRelStats));
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
vacrelstats->rel_pages = 0;
vacrelstats->rel_tuples = 0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
vacrelstats->hasindex = false;
Cleanup vacuum names.
2000-01-20 21:01:25 +01:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/*
* Set initial minimum Xid, which will be updated if a smaller Xid is found
* in the relation by scan_heap.
*
* We use RecentXmin here (the minimum Xid that belongs to a transaction
* that is still open according to our snapshot), because it is the
* earliest transaction that could insert new tuples in the table after our
* VACUUM is done.
*/
vacrelstats->minxid = RecentXmin;
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
/* scan the heap */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacuum_pages.num_pages = fraged_pages.num_pages = 0;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
scan_heap(vacrelstats, onerel, &vacuum_pages, &fraged_pages, FreezeLimit,
OldestXmin);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
/* Now open all indexes of the relation */
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
vac_open_indexes(onerel, AccessExclusiveLock, &nindexes, &Irel);
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
if (nindexes > 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
vacrelstats->hasindex = true;
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/* Clean/scan index relation(s) */
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
if (Irel != NULL)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (vacuum_pages.num_pages > 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
for (i = 0; i < nindexes; i++)
Make sure that FlushRelationBuffers() is invoked by all paths through vacuum.c. This is needed to make the world safe for pg_upgrade.
2000-09-19 21:30:03 +02:00
vacuum_index(&vacuum_pages, Irel[i],
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
vacrelstats->rel_tuples, 0);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
{
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
/* just scan indexes to update statistic */
for (i = 0; i < nindexes; i++)
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
scan_index(Irel[i], vacrelstats->rel_tuples);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
}
Make sure that FlushRelationBuffers() is invoked by all paths through vacuum.c. This is needed to make the world safe for pg_upgrade.
2000-09-19 21:30:03 +02:00
if (fraged_pages.num_pages > 0)
{
/* Try to shrink heap */
repair_frag(vacrelstats, onerel, &vacuum_pages, &fraged_pages,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
nindexes, Irel, OldestXmin);
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
vac_close_indexes(nindexes, Irel, NoLock);
Make sure that FlushRelationBuffers() is invoked by all paths through vacuum.c. This is needed to make the world safe for pg_upgrade.
2000-09-19 21:30:03 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
else
{
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
vac_close_indexes(nindexes, Irel, NoLock);
Make sure that FlushRelationBuffers() is invoked by all paths through vacuum.c. This is needed to make the world safe for pg_upgrade.
2000-09-19 21:30:03 +02:00
if (vacuum_pages.num_pages > 0)
{
/* Clean pages from vacuum_pages list */
More vacuum renaming.
2000-05-29 03:46:00 +02:00
vacuum_heap(vacrelstats, onerel, &vacuum_pages);
Make sure that FlushRelationBuffers() is invoked by all paths through vacuum.c. This is needed to make the world safe for pg_upgrade.
2000-09-19 21:30:03 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
/* update shared free space map with final free space info */
vac_update_fsm(onerel, &fraged_pages, vacrelstats->rel_pages);
Allow vacuum to perform analyze with shared lock. Update cvs manual.
2000-05-29 17:44:55 +02:00
/* update statistics in pg_class */
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
vac_update_relstats(RelationGetRelid(onerel), vacrelstats->rel_pages,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
vacrelstats->rel_tuples, vacrelstats->hasindex,
vacrelstats->minxid, OldestXmin);
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
/* report results to the stats collector, too */
Clean up a number of autovacuum loose ends. Make the stats collector track shared relations in a separate hashtable, so that operations done from different databases are counted correctly. Add proper support for anti-XID-wraparound vacuuming, even in databases that are never connected to and so have no stats entries. Miscellaneous other bug fixes. Alvaro Herrera, some additional fixes by Tom Lane.
2005-07-29 21:30:09 +02:00
pgstat_report_vacuum(RelationGetRelid(onerel), onerel->rd_rel->relisshared,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
vacstmt->analyze, vacrelstats->rel_tuples);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
More vacuum renaming.
2000-05-29 03:46:00 +02:00
* scan_heap() -- scan an open heap relation
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
* This routine sets commit status bits, constructs vacuum_pages (list
* of pages we need to compact free space on and/or clean indexes of
* deleted tuples), constructs fraged_pages (list of pages with free
* space that tuples could be moved into), and calculates statistics
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
* on the number of live tuples in the heap.
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
*
* It also updates the minimum Xid found anywhere on the table in
* vacrelstats->minxid, for later storing it in pg_class.relminxid.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static void
More vacuum renaming.
2000-05-29 03:46:00 +02:00
scan_heap(VRelStats *vacrelstats, Relation onerel,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
VacPageList vacuum_pages, VacPageList fraged_pages,
TransactionId FreezeLimit, TransactionId OldestXmin)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
BlockNumber nblocks,
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
blkno;
char *relname;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
VacPage vacpage;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
BlockNumber empty_pages,
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
empty_end_pages;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
double num_tuples,
tups_vacuumed,
nkeep,
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
nunused;
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
double free_space,
usable_free_space;
Fix to prevent too large tuple from being created.
1999-07-03 02:33:04 +02:00
Size min_tlen = MaxTupleSize;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
Size max_tlen = 0;
bool do_shrinking = true;
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
VTupleLink vtlinks = (VTupleLink) palloc(100 * sizeof(VTupleLinkData));
int num_vtlinks = 0;
int free_vtlinks = 100;
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
PGRUsage ru0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
pg_rusage_init(&ru0);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
New NameStr macro to convert Name to Str. No need for var.data anymore. Fewer calls to nameout. Better use of RelationGetRelationName.
1999-11-08 00:08:36 +01:00
relname = RelationGetRelationName(onerel);
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(elevel,
(errmsg("vacuuming \"%s.%s\"",
get_namespace_name(RelationGetNamespace(onerel)),
relname)));
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
empty_pages = empty_end_pages = 0;
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
num_tuples = tups_vacuumed = nkeep = nunused = 0;
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
free_space = 0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
nblocks = RelationGetNumberOfBlocks(onerel);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/*
* We initially create each VacPage item in a maximal-sized workspace,
* then copy the workspace into a just-large-enough copy.
*/
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage = (VacPage) palloc(sizeof(VacPageData) + MaxOffsetNumber * sizeof(OffsetNumber));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
for (blkno = 0; blkno < nblocks; blkno++)
{
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
Page page,
tempPage = NULL;
bool do_reap,
do_frag;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Buffer buf;
OffsetNumber offnum,
maxoff;
bool pgchanged,
notup;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
Centralize implementation of delay code by creating a pg_usleep() subroutine in src/port/pgsleep.c. Remove platform dependencies from miscadmin.h and put them in port.h where they belong. Extend recent vacuum cost-based-delay patch to apply to VACUUM FULL, ANALYZE, and non-btree index vacuuming. By the way, where is the documentation for the cost-based-delay patch?
2004-02-10 04:42:45 +01:00
vacuum_delay_point();
Add CHECK_FOR_INTERRUPTS() in various strategic spots, per comments from Hiroshi.
2002-01-06 01:37:44 +01:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
buf = ReadBuffer(onerel, blkno);
page = BufferGetPage(buf);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
Repair possible failure to update hint bits back to disk, per http://archives.postgresql.org/pgsql-hackers/2004-10/msg00464.php. This fix is intended to be permanent: it moves the responsibility for calling SetBufferCommitInfoNeedsSave() into the tqual.c routines, eliminating the requirement for callers to test whether t_infomask changed. Also, tighten validity checking on buffer IDs in bufmgr.c --- several routines were paranoid about out-of-range shared buffer numbers but not about out-of-range local ones, which seems a tad pointless.
2004-10-16 00:40:29 +02:00
/*
Take exclusive buffer lock in scan_heap() to eliminate some corner cases in which invalid page data could be transiently written to disk by concurrent bgwriter activity. There doesn't seem any risk of loss of actual user data, but an empty page could possibly be left corrupt if a crash occurs before the correct data gets written out. Pointed out by Alvaro Herrera.
2005-09-22 19:32:58 +02:00
* Since we are holding exclusive lock on the relation, no other
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* backend can be accessing the page; however it is possible that the
* background writer will try to write the page if it's already marked
* dirty. To ensure that invalid data doesn't get written to disk, we
* must take exclusive buffer lock wherever we potentially modify
* pages.
Repair possible failure to update hint bits back to disk, per http://archives.postgresql.org/pgsql-hackers/2004-10/msg00464.php. This fix is intended to be permanent: it moves the responsibility for calling SetBufferCommitInfoNeedsSave() into the tqual.c routines, eliminating the requirement for callers to test whether t_infomask changed. Also, tighten validity checking on buffer IDs in bufmgr.c --- several routines were paranoid about out-of-range shared buffer numbers but not about out-of-range local ones, which seems a tad pointless.
2004-10-16 00:40:29 +02:00
*/
Take exclusive buffer lock in scan_heap() to eliminate some corner cases in which invalid page data could be transiently written to disk by concurrent bgwriter activity. There doesn't seem any risk of loss of actual user data, but an empty page could possibly be left corrupt if a crash occurs before the correct data gets written out. Pointed out by Alvaro Herrera.
2005-09-22 19:32:58 +02:00
LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
Repair possible failure to update hint bits back to disk, per http://archives.postgresql.org/pgsql-hackers/2004-10/msg00464.php. This fix is intended to be permanent: it moves the responsibility for calling SetBufferCommitInfoNeedsSave() into the tqual.c routines, eliminating the requirement for callers to test whether t_infomask changed. Also, tighten validity checking on buffer IDs in bufmgr.c --- several routines were paranoid about out-of-range shared buffer numbers but not about out-of-range local ones, which seems a tad pointless.
2004-10-16 00:40:29 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->blkno = blkno;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
vacpage->offsets_used = 0;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets_free = 0;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (PageIsNew(page))
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
VacPage vacpagecopy;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("relation \"%s\" page %u is uninitialized --- fixing",
relname, blkno)));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
PageInit(page, BufferGetPageSize(buf), 0);
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
MarkBufferDirty(buf);
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
vacpage->free = PageGetFreeSpaceWithFillFactor(onerel, page);
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
free_space += vacpage->free;
empty_pages++;
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
empty_end_pages++;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
vacpagecopy = copy_vac_page(vacpage);
vpage_insert(vacuum_pages, vacpagecopy);
vpage_insert(fraged_pages, vacpagecopy);
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
UnlockReleaseBuffer(buf);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
continue;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (PageIsEmpty(page))
Fix deleted tuples re-incarnation possible when vacuum transaction start time equal to tuple->t_tmax. Privent shrinking if there are tuples modifyed by running transactions (it concerns system relations only, currently).
1997-06-03 03:29:26 +02:00
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
VacPage vacpagecopy;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
vacpage->free = PageGetFreeSpaceWithFillFactor(onerel, page);
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
free_space += vacpage->free;
Vacuum fix. Was modifying cache.
1998-08-19 21:59:49 +02:00
empty_pages++;
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
empty_end_pages++;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
vacpagecopy = copy_vac_page(vacpage);
vpage_insert(vacuum_pages, vacpagecopy);
vpage_insert(fraged_pages, vacpagecopy);
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
UnlockReleaseBuffer(buf);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
continue;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pgchanged = false;
notup = true;
maxoff = PageGetMaxOffsetNumber(page);
for (offnum = FirstOffsetNumber;
offnum <= maxoff;
offnum = OffsetNumberNext(offnum))
Fix deleted tuples re-incarnation possible when vacuum transaction start time equal to tuple->t_tmax. Privent shrinking if there are tuples modifyed by running transactions (it concerns system relations only, currently).
1997-06-03 03:29:26 +02:00
{
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
ItemId itemid = PageGetItemId(page, offnum);
bool tupgone = false;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
HeapTupleData tuple;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
* Collect un-used items too - it's possible to have indexes
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* pointing here after crash.
*/
if (!ItemIdIsUsed(itemid))
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets[vacpage->offsets_free++] = offnum;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
nunused += 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
continue;
}
New HeapTuple structure/interface.
1998-11-27 20:52:36 +01:00
tuple.t_data = (HeapTupleHeader) PageGetItem(page, itemid);
tuple.t_len = ItemIdGetLength(itemid);
ItemPointerSet(&(tuple.t_self), blkno, offnum);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Repair possible failure to update hint bits back to disk, per http://archives.postgresql.org/pgsql-hackers/2004-10/msg00464.php. This fix is intended to be permanent: it moves the responsibility for calling SetBufferCommitInfoNeedsSave() into the tqual.c routines, eliminating the requirement for callers to test whether t_infomask changed. Also, tighten validity checking on buffer IDs in bufmgr.c --- several routines were paranoid about out-of-range shared buffer numbers but not about out-of-range local ones, which seems a tad pointless.
2004-10-16 00:40:29 +02:00
switch (HeapTupleSatisfiesVacuum(tuple.t_data, OldestXmin, buf))
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
case HEAPTUPLE_DEAD:
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
tupgone = true; /* we can delete the tuple */
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
break;
case HEAPTUPLE_LIVE:
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Tuple is good. Consider whether to replace its xmin
* value with FrozenTransactionId.
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
*/
This patch wraps all accesses to t_xmin, t_cmin, t_xmax, and t_cmax in HeapTupleHeaderData in setter and getter macros called HeapTupleHeaderGetXmin, HeapTupleHeaderSetXmin etc. It also introduces a "virtual" field xvac by defining HeapTupleHeaderGetXvac and HeapTupleHeaderSetXvac. Xvac is used by VACUUM, in fact it is stored in t_cmin. Manfred Koizar
2002-06-15 21:54:24 +02:00
if (TransactionIdIsNormal(HeapTupleHeaderGetXmin(tuple.t_data)) &&
TransactionIdPrecedes(HeapTupleHeaderGetXmin(tuple.t_data),
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
FreezeLimit))
{
This patch wraps all accesses to t_xmin, t_cmin, t_xmax, and t_cmax in HeapTupleHeaderData in setter and getter macros called HeapTupleHeaderGetXmin, HeapTupleHeaderSetXmin etc. It also introduces a "virtual" field xvac by defining HeapTupleHeaderGetXvac and HeapTupleHeaderSetXvac. Xvac is used by VACUUM, in fact it is stored in t_cmin. Manfred Koizar
2002-06-15 21:54:24 +02:00
HeapTupleHeaderSetXmin(tuple.t_data, FrozenTransactionId);
Fix comment, add Assert.
2001-09-04 21:12:05 +02:00
/* infomask should be okay already */
Assert(tuple.t_data->t_infomask & HEAP_XMIN_COMMITTED);
Transaction IDs wrap around, per my proposal of 13-Aug-01. More documentation to come, but the code is all here. initdb forced.
2001-08-26 18:56:03 +02:00
pgchanged = true;
}
Check only live tuples for OID validity, to avoid issues after ALTER TABLE. Per recent discussion.
2004-02-12 06:39:55 +01:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Other checks...
*/
Check only live tuples for OID validity, to avoid issues after ALTER TABLE. Per recent discussion.
2004-02-12 06:39:55 +01:00
if (onerel->rd_rel->relhasoids &&
!OidIsValid(HeapTupleGetOid(&tuple)))
elog(WARNING, "relation \"%s\" TID %u/%u: OID is invalid",
relname, blkno, offnum);
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
break;
case HEAPTUPLE_RECENTLY_DEAD:
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If tuple is recently deleted then we must not remove it
* from relation.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
nkeep += 1;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we do shrinking and this tuple is updated one then
* remember it to construct updated tuple dependencies.
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
*/
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
if (do_shrinking &&
!(ItemPointerEquals(&(tuple.t_self),
&(tuple.t_data->t_ctid))))
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
if (free_vtlinks == 0)
{
free_vtlinks = 1000;
pgindent run over code.
1999-05-25 18:15:34 +02:00
vtlinks = (VTupleLink) repalloc(vtlinks,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(free_vtlinks + num_vtlinks) *
sizeof(VTupleLinkData));
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
vtlinks[num_vtlinks].new_tid = tuple.t_data->t_ctid;
vtlinks[num_vtlinks].this_tid = tuple.t_self;
free_vtlinks--;
num_vtlinks++;
}
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
break;
case HEAPTUPLE_INSERT_IN_PROGRESS:
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* This should not happen, since we hold exclusive lock on
* the relation; shouldn't we raise an error? (Actually,
* it can happen in system catalogs, since we tend to
* release write lock before commit there.)
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
*/
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(NOTICE,
(errmsg("relation \"%s\" TID %u/%u: InsertTransactionInProgress %u --- can't shrink relation",
relname, blkno, offnum, HeapTupleHeaderGetXmin(tuple.t_data))));
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
do_shrinking = false;
break;
case HEAPTUPLE_DELETE_IN_PROGRESS:
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* This should not happen, since we hold exclusive lock on
* the relation; shouldn't we raise an error? (Actually,
* it can happen in system catalogs, since we tend to
* release write lock before commit there.)
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
*/
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(NOTICE,
(errmsg("relation \"%s\" TID %u/%u: DeleteTransactionInProgress %u --- can't shrink relation",
relname, blkno, offnum, HeapTupleHeaderGetXmax(tuple.t_data))));
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
do_shrinking = false;
break;
default:
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
break;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
if (tupgone)
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
ItemId lpp;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Patch VACUUM problem with moving chain of update tuples when source and destination of a tuple lie on the same page. (Previously fixed in REL7_0 branch, now apply to current.)
2000-10-22 21:49:43 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Here we are building a temporary copy of the page with dead
* tuples removed. Below we will apply
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
* PageRepairFragmentation to the copy, so that we can
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* determine how much space will be available after removal of
* dead tuples. But note we are NOT changing the real page
* yet...
Patch VACUUM problem with moving chain of update tuples when source and destination of a tuple lie on the same page. (Previously fixed in REL7_0 branch, now apply to current.)
2000-10-22 21:49:43 +02:00
*/
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
if (tempPage == NULL)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
Size pageSize;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pageSize = PageGetPageSize(page);
tempPage = (Page) palloc(pageSize);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
memcpy(tempPage, page, pageSize);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Patch VACUUM problem with moving chain of update tuples when source and destination of a tuple lie on the same page. (Previously fixed in REL7_0 branch, now apply to current.)
2000-10-22 21:49:43 +02:00
/* mark it unused on the temp page */
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
lpp = PageGetItemId(tempPage, offnum);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
lpp->lp_flags &= ~LP_USED;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets[vacpage->offsets_free++] = offnum;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
tups_vacuumed += 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
{
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
TransactionId min;
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
num_tuples += 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
notup = false;
New HeapTuple structure/interface.
1998-11-27 20:52:36 +01:00
if (tuple.t_len < min_tlen)
min_tlen = tuple.t_len;
if (tuple.t_len > max_tlen)
max_tlen = tuple.t_len;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/*
* If the tuple is alive, we consider it for the "minxid"
* calculations.
*/
min = vactuple_get_minxid(&tuple);
if (TransactionIdIsValid(min) &&
TransactionIdPrecedes(min, vacrelstats->minxid))
vacrelstats->minxid = min;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
} /* scan along page */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
if (tempPage != NULL)
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
{
/* Some tuples are removable; figure free space after removal */
1. WAL needs in zero-ed content of newly initialized page. 2. Log record for PageRepaireFragmentation now keeps array of !LP_USED offnums to redo cleanup properly.
2000-12-30 16:19:57 +01:00
PageRepairFragmentation(tempPage, NULL);
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
vacpage->free = PageGetFreeSpaceWithFillFactor(onerel, tempPage);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pfree(tempPage);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
do_reap = true;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
else
{
/* Just use current available space */
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
vacpage->free = PageGetFreeSpaceWithFillFactor(onerel, page);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/* Need to reap the page if it has ~LP_USED line pointers */
do_reap = (vacpage->offsets_free > 0);
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Fix longstanding error in VACUUM: sometimes would examine a buffer page after writing/unpinning it. An actual failure is unlikely, unless the system is tremendously short of buffers ... but a bug is a bug.
2001-06-29 18:34:30 +02:00
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
free_space += vacpage->free;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/*
* Add the page to fraged_pages if it has a useful amount of free
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* space. "Useful" means enough for a minimal-sized tuple. But we
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* don't know that accurately near the start of the relation, so add
* pages unconditionally if they have >= BLCKSZ/10 free space.
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
*/
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
do_frag = (vacpage->free >= min_tlen || vacpage->free >= BLCKSZ / 10);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
if (do_reap || do_frag)
Fix longstanding error in VACUUM: sometimes would examine a buffer page after writing/unpinning it. An actual failure is unlikely, unless the system is tremendously short of buffers ... but a bug is a bug.
2001-06-29 18:34:30 +02:00
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
VacPage vacpagecopy = copy_vac_page(vacpage);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
if (do_reap)
vpage_insert(vacuum_pages, vacpagecopy);
if (do_frag)
vpage_insert(fraged_pages, vacpagecopy);
Fix longstanding error in VACUUM: sometimes would examine a buffer page after writing/unpinning it. An actual failure is unlikely, unless the system is tremendously short of buffers ... but a bug is a bug.
2001-06-29 18:34:30 +02:00
}
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
/*
* Include the page in empty_end_pages if it will be empty after
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* vacuuming; this is to keep us from using it as a move destination.
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (notup)
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
{
empty_pages++;
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
empty_end_pages++;
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
else
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
empty_end_pages = 0;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
if (pgchanged)
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
MarkBufferDirty(buf);
UnlockReleaseBuffer(buf);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
More vacuum cleanup
2000-05-29 19:06:15 +02:00
pfree(vacpage);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/* save stats in the rel list for use later */
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
vacrelstats->rel_tuples = num_tuples;
vacrelstats->rel_pages = nblocks;
Vacuum fix. Was modifying cache.
1998-08-19 21:59:49 +02:00
if (num_tuples == 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
min_tlen = max_tlen = 0;
vacrelstats->min_tlen = min_tlen;
vacrelstats->max_tlen = max_tlen;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacuum_pages->empty_end_pages = empty_end_pages;
fraged_pages->empty_end_pages = empty_end_pages;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* Clear the fraged_pages list if we found we couldn't shrink. Else,
* remove any "empty" end-pages from the list, and compute usable free
* space = free space in remaining pages.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
if (do_shrinking)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
int i;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
Assert((BlockNumber) fraged_pages->num_pages >= empty_end_pages);
fraged_pages->num_pages -= empty_end_pages;
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
usable_free_space = 0;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
for (i = 0; i < fraged_pages->num_pages; i++)
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
usable_free_space += fraged_pages->pagedesc[i]->free;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
}
else
{
fraged_pages->num_pages = 0;
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
usable_free_space = 0;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/* don't bother to save vtlinks if we will not call repair_frag */
if (fraged_pages->num_pages > 0 && num_vtlinks > 0)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
qsort((char *) vtlinks, num_vtlinks, sizeof(VTupleLinkData),
More vacuum renaming.
2000-05-29 03:46:00 +02:00
vac_cmp_vtlinks);
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
vacrelstats->vtlinks = vtlinks;
vacrelstats->num_vtlinks = num_vtlinks;
}
else
{
vacrelstats->vtlinks = NULL;
vacrelstats->num_vtlinks = 0;
pfree(vtlinks);
}
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(elevel,
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
(errmsg("\"%s\": found %.0f removable, %.0f nonremovable row versions in %u pages",
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
RelationGetRelationName(onerel),
tups_vacuumed, num_tuples, nblocks),
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
errdetail("%.0f dead row versions cannot be removed yet.\n"
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
"Nonremovable row versions range from %lu to %lu bytes long.\n"
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
"There were %.0f unused item pointers.\n"
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
"Total free space (including removable row versions) is %.0f bytes.\n"
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
"%u pages are or will become empty, including %u at the end of the table.\n"
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
"%u pages containing %.0f free bytes are potential move destinations.\n"
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
"%s.",
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
nkeep,
(unsigned long) min_tlen, (unsigned long) max_tlen,
nunused,
free_space,
empty_pages, empty_end_pages,
fraged_pages->num_pages, usable_free_space,
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
pg_rusage_show(&ru0))));
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
/*
* vactuple_get_minxid
*
* Get the minimum relevant Xid for a tuple, not considering FrozenXid.
* Return InvalidXid if none (i.e., xmin=FrozenXid, xmax=InvalidXid).
* This is for the purpose of calculating pg_class.relminxid for a table
* we're vacuuming.
*/
TransactionId
vactuple_get_minxid(HeapTuple tuple)
{
TransactionId min = InvalidTransactionId;
/*
* Initialize calculations with Xmin. NB -- may be FrozenXid and
* we don't want that one.
*/
if (TransactionIdIsNormal(HeapTupleHeaderGetXmin(tuple->t_data)))
min = HeapTupleHeaderGetXmin(tuple->t_data);
/*
* If Xmax is not marked INVALID, we assume it's valid without making
* further checks on it --- it must be recently obsoleted or still running,
* else HeapTupleSatisfiesVacuum would have deemed it removable.
*/
if (!(tuple->t_data->t_infomask | HEAP_XMAX_INVALID))
{
TransactionId xmax = HeapTupleHeaderGetXmax(tuple->t_data);
/* If xmax is a plain Xid, consider it by itself */
if (!(tuple->t_data->t_infomask | HEAP_XMAX_IS_MULTI))
{
if (!TransactionIdIsValid(min) ||
(TransactionIdIsNormal(xmax) &&
TransactionIdPrecedes(xmax, min)))
min = xmax;
}
else
{
/* If it's a MultiXactId, consider each of its members */
TransactionId *members;
int nmembers,
membno;
nmembers = GetMultiXactIdMembers(xmax, &members);
for (membno = 0; membno < nmembers; membno++)
{
if (!TransactionIdIsValid(min) ||
TransactionIdPrecedes(members[membno], min))
min = members[membno];
}
}
}
return min;
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
More vacuum renaming.
2000-05-29 03:46:00 +02:00
* repair_frag() -- try to repair relation's fragmentation
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* This routine marks dead tuples as unused and tries re-use dead space
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
* by moving tuples (and inserting indexes if needed). It constructs
* Nvacpagelist list of free-ed pages (moved tuples) and clean indexes
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* for them after committing (in hack-manner - without losing locks
* and freeing memory!) current transaction. It truncates relation
* if some end-blocks are gone away.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static void
More vacuum renaming.
2000-05-29 03:46:00 +02:00
repair_frag(VRelStats *vacrelstats, Relation onerel,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
VacPageList vacuum_pages, VacPageList fraged_pages,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
int nindexes, Relation *Irel, TransactionId OldestXmin)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
TransactionId myXID = GetCurrentTransactionId();
Buffer dst_buffer = InvalidBuffer;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
BlockNumber nblocks,
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
blkno;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
BlockNumber last_move_dest_block = 0,
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
last_vacuum_block;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Page dst_page = NULL;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
ExecContextData ec;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
VacPageListData Nvacpagelist;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
VacPage dst_vacpage = NULL,
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
last_vacuum_page,
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage,
*curpage;
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
int i;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
int num_moved = 0,
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
num_fraged_pages,
vacuumed_pages;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
int keep_tuples = 0;
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
PGRUsage ru0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
pg_rusage_init(&ru0);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
ExecContext_Init(&ec, onerel);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
Nvacpagelist.num_pages = 0;
num_fraged_pages = fraged_pages->num_pages;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
Assert((BlockNumber) vacuum_pages->num_pages >= vacuum_pages->empty_end_pages);
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacuumed_pages = vacuum_pages->num_pages - vacuum_pages->empty_end_pages;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
if (vacuumed_pages > 0)
{
/* get last reaped page from vacuum_pages */
last_vacuum_page = vacuum_pages->pagedesc[vacuumed_pages - 1];
last_vacuum_block = last_vacuum_page->blkno;
}
else
{
last_vacuum_page = NULL;
last_vacuum_block = InvalidBlockNumber;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage = (VacPage) palloc(sizeof(VacPageData) + MaxOffsetNumber * sizeof(OffsetNumber));
vacpage->offsets_used = vacpage->offsets_free = 0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Scan pages backwards from the last nonempty page, trying to move tuples
* down to lower pages. Quit when we reach a page that we have moved any
* tuples onto, or the first page if we haven't moved anything, or when we
* find a page we cannot completely empty (this last condition is handled
* by "break" statements within the loop).
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* NB: this code depends on the vacuum_pages and fraged_pages lists being
* in order by blkno.
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
*/
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
nblocks = vacrelstats->rel_pages;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
for (blkno = nblocks - vacuum_pages->empty_end_pages - 1;
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
blkno > last_move_dest_block;
blkno--)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Buffer buf;
Page page;
OffsetNumber offnum,
maxoff;
bool isempty,
chain_tuple_moved;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Centralize implementation of delay code by creating a pg_usleep() subroutine in src/port/pgsleep.c. Remove platform dependencies from miscadmin.h and put them in port.h where they belong. Extend recent vacuum cost-based-delay patch to apply to VACUUM FULL, ANALYZE, and non-btree index vacuuming. By the way, where is the documentation for the cost-based-delay patch?
2004-02-10 04:42:45 +01:00
vacuum_delay_point();
Add CHECK_FOR_INTERRUPTS() in various strategic spots, per comments from Hiroshi.
2002-01-06 01:37:44 +01:00
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Forget fraged_pages pages at or after this one; they're no longer
* useful as move targets, since we only want to move down. Note that
* since we stop the outer loop at last_move_dest_block, pages removed
* here cannot have had anything moved onto them already.
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Also note that we don't change the stored fraged_pages list, only
* our local variable num_fraged_pages; so the forgotten pages are
* still available to be loaded into the free space map later.
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
*/
while (num_fraged_pages > 0 &&
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
fraged_pages->pagedesc[num_fraged_pages - 1]->blkno >= blkno)
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
{
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Assert(fraged_pages->pagedesc[num_fraged_pages - 1]->offsets_used == 0);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
--num_fraged_pages;
}
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Process this page of relation.
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
buf = ReadBuffer(onerel, blkno);
page = BufferGetPage(buf);
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets_free = 0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
isempty = PageIsEmpty(page);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/* Is the page in the vacuum_pages list? */
if (blkno == last_vacuum_block)
Added: dynamic re-moving page from list of pages usable for shrinking if its free-space < min_tuple_len.
1996-11-29 11:27:59 +01:00
{
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
if (last_vacuum_page->offsets_free > 0)
{
/* there are dead tuples on this page - clean them */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Assert(!isempty);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
vacuum_page(onerel, buf, last_vacuum_page);
LockBuffer(buf, BUFFER_LOCK_UNLOCK);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
Assert(isempty);
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
--vacuumed_pages;
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
if (vacuumed_pages > 0)
{
Cleanup vacuum names.
2000-01-20 21:01:25 +01:00
/* get prev reaped page from vacuum_pages */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
last_vacuum_page = vacuum_pages->pagedesc[vacuumed_pages - 1];
last_vacuum_block = last_vacuum_page->blkno;
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
}
else
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
last_vacuum_page = NULL;
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
last_vacuum_block = InvalidBlockNumber;
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (isempty)
{
ReleaseBuffer(buf);
continue;
}
}
else
Assert(!isempty);
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
chain_tuple_moved = false; /* no one chain-tuple was moved off
* this page, yet */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->blkno = blkno;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
maxoff = PageGetMaxOffsetNumber(page);
for (offnum = FirstOffsetNumber;
offnum <= maxoff;
offnum = OffsetNumberNext(offnum))
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Size tuple_len;
HeapTupleData tuple;
ItemId itemid = PageGetItemId(page, offnum);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (!ItemIdIsUsed(itemid))
continue;
New HeapTuple structure/interface.
1998-11-27 20:52:36 +01:00
tuple.t_data = (HeapTupleHeader) PageGetItem(page, itemid);
tuple_len = tuple.t_len = ItemIdGetLength(itemid);
ItemPointerSet(&(tuple.t_self), blkno, offnum);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Update comment on VACUUM FULL. Manfred Koizar
2005-02-15 04:50:07 +01:00
/* ---
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
* VACUUM FULL has an exclusive lock on the relation. So
* normally no other transaction can have pending INSERTs or
Update comment on VACUUM FULL. Manfred Koizar
2005-02-15 04:50:07 +01:00
* DELETEs in this relation. A tuple is either:
* (a) a tuple in a system catalog, inserted or deleted
* by a not yet committed transaction
* (b) known dead (XMIN_INVALID, or XMAX_COMMITTED and xmax
* is visible to all active transactions)
* (c) inserted by a committed xact (XMIN_COMMITTED)
* (d) moved by the currently running VACUUM.
* (e) deleted (XMAX_COMMITTED) but at least one active
* transaction does not see the deleting transaction
* In case (a) we wouldn't be in repair_frag() at all.
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
* In case (b) we cannot be here, because scan_heap() has
* already marked the item as unused, see continue above. Case
* (c) is what normally is to be expected. Case (d) is only
* possible, if a whole tuple chain has been moved while
* processing this or a higher numbered block.
Update comment on VACUUM FULL. Manfred Koizar
2005-02-15 04:50:07 +01:00
* ---
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
if (!(tuple.t_data->t_infomask & HEAP_XMIN_COMMITTED))
{
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
if (tuple.t_data->t_infomask & HEAP_MOVED_IN)
elog(ERROR, "HEAP_MOVED_IN was not expected");
if (!(tuple.t_data->t_infomask & HEAP_MOVED_OFF))
elog(ERROR, "HEAP_MOVED_OFF was expected");
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
* MOVED_OFF by another VACUUM would have caused the
* visibility check to set XMIN_COMMITTED or XMIN_INVALID.
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
if (HeapTupleHeaderGetXvac(tuple.t_data) != myXID)
elog(ERROR, "invalid XVAC in tuple header");
pgindent run over code.
1999-05-25 18:15:34 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If this (chain) tuple is moved by me already then I have to
* check is it in vacpage or not - i.e. is it moved while
* cleaning this page or some previous one.
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/* Can't we Assert(keep_tuples > 0) here? */
if (keep_tuples == 0)
continue;
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
if (chain_tuple_moved)
{
/* some chains were moved while cleaning this page */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Assert(vacpage->offsets_free > 0);
for (i = 0; i < vacpage->offsets_free; i++)
{
if (vacpage->offsets[i] == offnum)
break;
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
}
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
if (i >= vacpage->offsets_free) /* not found */
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets[vacpage->offsets_free++] = offnum;
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
keep_tuples--;
}
}
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
else
{
vacpage->offsets[vacpage->offsets_free++] = offnum;
keep_tuples--;
}
continue;
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If this tuple is in a chain of tuples created in updates by
* "recent" transactions then we have to move the whole chain of
* tuples to other places, so that we can write new t_ctid links
* that preserve the chain relationship.
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
*
* This test is complicated. Read it as "if tuple is a recently
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* created updated version, OR if it is an obsoleted version". (In
* the second half of the test, we needn't make any check on XMAX
* --- it must be recently obsoleted, else scan_heap would have
* deemed it removable.)
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* NOTE: this test is not 100% accurate: it is possible for a
* tuple to be an updated one with recent xmin, and yet not match
* any new_tid entry in the vtlinks list. Presumably there was
* once a parent tuple with xmax matching the xmin, but it's
* possible that that tuple has been removed --- for example, if
* it had xmin = xmax and wasn't itself an updated version, then
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* HeapTupleSatisfiesVacuum would deem it removable as soon as the
* xmin xact completes.
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* To be on the safe side, we abandon the repair_frag process if
* we cannot find the parent tuple in vtlinks. This may be overly
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* conservative; AFAICS it would be safe to move the chain.
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
*/
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
if (((tuple.t_data->t_infomask & HEAP_UPDATED) &&
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
!TransactionIdPrecedes(HeapTupleHeaderGetXmin(tuple.t_data),
OldestXmin)) ||
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
(!(tuple.t_data->t_infomask & (HEAP_XMAX_INVALID |
Implement sharable row-level locks, and use them for foreign key references to eliminate unnecessary deadlocks. This commit adds SELECT ... FOR SHARE paralleling SELECT ... FOR UPDATE. The implementation uses a new SLRU data structure (managed much like pg_subtrans) to represent multiple- transaction-ID sets. When more than one transaction is holding a shared lock on a particular row, we create a MultiXactId representing that set of transactions and store its ID in the row's XMAX. This scheme allows an effectively unlimited number of row locks, just as we did before, while not costing any extra overhead except when a shared lock actually has to be shared. Still TODO: use the regular lock manager to control the grant order when multiple backends are waiting for a row lock. Alvaro Herrera and Tom Lane.
2005-04-28 23:47:18 +02:00
HEAP_IS_LOCKED)) &&
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
!(ItemPointerEquals(&(tuple.t_self),
&(tuple.t_data->t_ctid)))))
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
Buffer Cbuf = buf;
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
bool freeCbuf = false;
bool chain_move_failed = false;
pgindent run over code.
1999-05-25 18:15:34 +02:00
ItemPointerData Ctid;
HeapTupleData tp = tuple;
Size tlen = tuple_len;
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
VTupleMove vtmove;
int num_vtmove;
int free_vtmove;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
VacPage to_vacpage = NULL;
pgindent run over code.
1999-05-25 18:15:34 +02:00
int to_item = 0;
int ti;
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (dst_buffer != InvalidBuffer)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
ReleaseBuffer(dst_buffer);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
dst_buffer = InvalidBuffer;
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
pgindent run over code.
1999-05-25 18:15:34 +02:00
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/* Quick exit if we have no vtlinks to search in */
if (vacrelstats->vtlinks == NULL)
{
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
elog(DEBUG2, "parent item in update-chain not found --- can't continue repair_frag");
pgindent run.
2002-09-04 22:31:48 +02:00
break; /* out of walk-along-page loop */
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
}
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If this tuple is in the begin/middle of the chain then we
* have to move to the end of chain. As with any t_ctid
* chase, we have to verify that each new tuple is really the
* descendant of the tuple we came from.
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
*/
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
while (!(tp.t_data->t_infomask & (HEAP_XMAX_INVALID |
Implement sharable row-level locks, and use them for foreign key references to eliminate unnecessary deadlocks. This commit adds SELECT ... FOR SHARE paralleling SELECT ... FOR UPDATE. The implementation uses a new SLRU data structure (managed much like pg_subtrans) to represent multiple- transaction-ID sets. When more than one transaction is holding a shared lock on a particular row, we create a MultiXactId representing that set of transactions and store its ID in the row's XMAX. This scheme allows an effectively unlimited number of row locks, just as we did before, while not costing any extra overhead except when a shared lock actually has to be shared. Still TODO: use the regular lock manager to control the grant order when multiple backends are waiting for a row lock. Alvaro Herrera and Tom Lane.
2005-04-28 23:47:18 +02:00
HEAP_IS_LOCKED)) &&
Create a new HeapTupleSatisfiesVacuum() routine in tqual.c that embodies the validity checking rules for VACUUM. Make some other rearrangements of the VACUUM code to allow more code to be shared between full and lazy VACUUM. Minor code cleanups and added comments for TransactionId manipulations.
2001-07-12 06:11:13 +02:00
!(ItemPointerEquals(&(tp.t_self),
&(tp.t_data->t_ctid))))
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
ItemPointerData nextTid;
TransactionId priorXmax;
Buffer nextBuf;
Page nextPage;
OffsetNumber nextOffnum;
ItemId nextItemid;
HeapTupleHeader nextTdata;
nextTid = tp.t_data->t_ctid;
priorXmax = HeapTupleHeaderGetXmax(tp.t_data);
/* assume block# is OK (see heap_fetch comments) */
nextBuf = ReadBuffer(onerel,
ItemPointerGetBlockNumber(&nextTid));
nextPage = BufferGetPage(nextBuf);
/* If bogus or unused slot, assume tp is end of chain */
nextOffnum = ItemPointerGetOffsetNumber(&nextTid);
if (nextOffnum < FirstOffsetNumber ||
nextOffnum > PageGetMaxOffsetNumber(nextPage))
1. xact.c: update comments about changing MyProc->xid and MyProc->xmin. 2. varsup.c:ReadNewTransactionId(): don't read nextXid from disk - this func doesn't allocate next xid, so ShmemVariableCache->nextXid may be used (but GetNewTransactionId() must be called first). 3. vacuum.c: change elog(ERROR, "Child item....") to elog(NOTICE) - this is not ERROR, proper handling is just not implemented, yet. 4. s_lock.c: increase S_MAX_BUSY by 2 times. 5. shmem.c:GetSnapshotData(): have to call ReadNewTransactionId() _after_ SpinAcquire(ShmemIndexLock).
1999-06-06 22:19:35 +02:00
{
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
ReleaseBuffer(nextBuf);
break;
}
nextItemid = PageGetItemId(nextPage, nextOffnum);
if (!ItemIdIsUsed(nextItemid))
{
ReleaseBuffer(nextBuf);
break;
1. xact.c: update comments about changing MyProc->xid and MyProc->xmin. 2. varsup.c:ReadNewTransactionId(): don't read nextXid from disk - this func doesn't allocate next xid, so ShmemVariableCache->nextXid may be used (but GetNewTransactionId() must be called first). 3. vacuum.c: change elog(ERROR, "Child item....") to elog(NOTICE) - this is not ERROR, proper handling is just not implemented, yet. 4. s_lock.c: increase S_MAX_BUSY by 2 times. 5. shmem.c:GetSnapshotData(): have to call ReadNewTransactionId() _after_ SpinAcquire(ShmemIndexLock).
1999-06-06 22:19:35 +02:00
}
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* if not matching XMIN, assume tp is end of chain */
nextTdata = (HeapTupleHeader) PageGetItem(nextPage,
nextItemid);
if (!TransactionIdEquals(HeapTupleHeaderGetXmin(nextTdata),
priorXmax))
{
ReleaseBuffer(nextBuf);
break;
}
/* OK, switch our attention to the next tuple in chain */
tp.t_data = nextTdata;
tp.t_self = nextTid;
tlen = tp.t_len = ItemIdGetLength(nextItemid);
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
if (freeCbuf)
ReleaseBuffer(Cbuf);
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
Cbuf = nextBuf;
freeCbuf = true;
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
}
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* Set up workspace for planning the chain move */
vtmove = (VTupleMove) palloc(100 * sizeof(VTupleMoveData));
num_vtmove = 0;
free_vtmove = 100;
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Now, walk backwards up the chain (towards older tuples) and
* check if all items in chain can be moved. We record all
* the moves that need to be made in the vtmove array.
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
*/
pgindent run over code.
1999-05-25 18:15:34 +02:00
for (;;)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
Buffer Pbuf;
Page Ppage;
ItemId Pitemid;
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
HeapTupleHeader PTdata;
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
VTupleLinkData vtld,
*vtlp;
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* Identify a target page to move this tuple to */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (to_vacpage == NULL ||
!enough_space(to_vacpage, tlen))
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
for (i = 0; i < num_fraged_pages; i++)
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (enough_space(fraged_pages->pagedesc[i], tlen))
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
break;
}
Update vacuum comments.
2000-01-19 23:23:00 +01:00
if (i == num_fraged_pages)
Cleanup vacuum names.
2000-01-20 21:01:25 +01:00
{
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/* can't move item anywhere */
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
chain_move_failed = true;
pgindent run.
2002-09-04 22:31:48 +02:00
break; /* out of check-all-items loop */
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
to_item = i;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
to_vacpage = fraged_pages->pagedesc[to_item];
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
More vacuum cleanup
2000-05-29 19:06:15 +02:00
to_vacpage->free -= MAXALIGN(tlen);
if (to_vacpage->offsets_used >= to_vacpage->offsets_free)
Fix miscalculation of remaining free space during tuple chain moving. Only affects machines where MAXALIGN > 4, and is a boundary-condition case even there, but still surprising that it's not been identified before. Also reduce tuple chain move give-up messages from WARNING to DEBUG1, since they are not unexpected conditions.
2002-10-31 20:25:29 +01:00
to_vacpage->free -= sizeof(ItemIdData);
More vacuum cleanup
2000-05-29 19:06:15 +02:00
(to_vacpage->offsets_used)++;
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* Add an entry to vtmove list */
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
if (free_vtmove == 0)
{
free_vtmove = 1000;
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
vtmove = (VTupleMove)
repalloc(vtmove,
(free_vtmove + num_vtmove) *
sizeof(VTupleMoveData));
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
vtmove[num_vtmove].tid = tp.t_self;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vtmove[num_vtmove].vacpage = to_vacpage;
if (to_vacpage->offsets_used == 1)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
vtmove[num_vtmove].cleanVpd = true;
else
vtmove[num_vtmove].cleanVpd = false;
free_vtmove--;
num_vtmove++;
pgindent run over code.
1999-05-25 18:15:34 +02:00
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* Done if at beginning of chain */
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (!(tp.t_data->t_infomask & HEAP_UPDATED) ||
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
TransactionIdPrecedes(HeapTupleHeaderGetXmin(tp.t_data),
OldestXmin))
break; /* out of check-all-items loop */
pgindent run over code.
1999-05-25 18:15:34 +02:00
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* Move to tuple with prior row version */
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
vtld.new_tid = tp.t_self;
vtlp = (VTupleLink)
vac_bsearch((void *) &vtld,
(void *) (vacrelstats->vtlinks),
vacrelstats->num_vtlinks,
sizeof(VTupleLinkData),
vac_cmp_vtlinks);
if (vtlp == NULL)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/* see discussion above */
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
elog(DEBUG2, "parent item in update-chain not found --- can't continue repair_frag");
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
chain_move_failed = true;
pgindent run.
2002-09-04 22:31:48 +02:00
break; /* out of check-all-items loop */
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
}
tp.t_self = vtlp->this_tid;
Pbuf = ReadBuffer(onerel,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
ItemPointerGetBlockNumber(&(tp.t_self)));
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
Ppage = BufferGetPage(Pbuf);
Pitemid = PageGetItemId(Ppage,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
ItemPointerGetOffsetNumber(&(tp.t_self)));
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/* this can't happen since we saw tuple earlier: */
if (!ItemIdIsUsed(Pitemid))
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
elog(ERROR, "parent itemid marked as unused");
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
PTdata = (HeapTupleHeader) PageGetItem(Ppage, Pitemid);
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/* ctid should not have changed since we saved it */
Assert(ItemPointerEquals(&(vtld.new_tid),
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
&(PTdata->t_ctid)));
Cleanup vacuum names.
2000-01-20 21:01:25 +01:00
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/*
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
* Read above about cases when !ItemIdIsUsed(nextItemid)
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* (child item is removed)... Due to the fact that at the
* moment we don't remove unuseful part of update-chain,
* it's possible to get non-matching parent row here. Like
* as in the case which caused this problem, we stop
* shrinking here. I could try to find real parent row but
* want not to do it because of real solution will be
* implemented anyway, later, and we are too close to 6.5
* release. - vadim 06/11/99
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
*/
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
if ((PTdata->t_infomask & HEAP_XMAX_IS_MULTI) ||
!(TransactionIdEquals(HeapTupleHeaderGetXmax(PTdata),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
HeapTupleHeaderGetXmin(tp.t_data))))
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
{
ReleaseBuffer(Pbuf);
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
elog(DEBUG2, "too old parent tuple found --- can't continue repair_frag");
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
chain_move_failed = true;
pgindent run.
2002-09-04 22:31:48 +02:00
break; /* out of check-all-items loop */
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
tp.t_data = PTdata;
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
tlen = tp.t_len = ItemIdGetLength(Pitemid);
if (freeCbuf)
ReleaseBuffer(Cbuf);
Cbuf = Pbuf;
freeCbuf = true;
pgindent run.
2002-09-04 22:31:48 +02:00
} /* end of check-all-items loop */
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
if (freeCbuf)
ReleaseBuffer(Cbuf);
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
freeCbuf = false;
if (chain_move_failed)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Undo changes to offsets_used state. We don't bother
* cleaning up the amount-free state, since we're not
* going to do any further tuple motion.
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
*/
for (i = 0; i < num_vtmove; i++)
{
Assert(vtmove[i].vacpage->offsets_used > 0);
(vtmove[i].vacpage->offsets_used)--;
}
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
pfree(vtmove);
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
break; /* out of walk-along-page loop */
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
}
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
* Okay, move the whole tuple chain in reverse order.
*
* Ctid tracks the new location of the previously-moved tuple.
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
*/
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
ItemPointerSetInvalid(&Ctid);
for (ti = 0; ti < num_vtmove; ti++)
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
VacPage destvacpage = vtmove[ti].vacpage;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Page Cpage;
ItemId Citemid;
Repair assert failure in tuple-chain-moving logic (introduced by yours truly, I'm afraid).
2000-04-06 02:29:51 +02:00
WAL misc
2000-10-24 11:56:23 +02:00
/* Get page to move from */
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
tuple.t_self = vtmove[ti].tid;
pgindent run over code.
1999-05-25 18:15:34 +02:00
Cbuf = ReadBuffer(onerel,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
ItemPointerGetBlockNumber(&(tuple.t_self)));
WAL misc
2000-10-24 11:56:23 +02:00
/* Get page to move to */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
dst_buffer = ReadBuffer(onerel, destvacpage->blkno);
WAL misc
2000-10-24 11:56:23 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
LockBuffer(dst_buffer, BUFFER_LOCK_EXCLUSIVE);
if (dst_buffer != Cbuf)
WAL misc
2000-10-24 11:56:23 +02:00
LockBuffer(Cbuf, BUFFER_LOCK_EXCLUSIVE);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
dst_page = BufferGetPage(dst_buffer);
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
Cpage = BufferGetPage(Cbuf);
WAL misc
2000-10-24 11:56:23 +02:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
Citemid = PageGetItemId(Cpage,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
ItemPointerGetOffsetNumber(&(tuple.t_self)));
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
tuple.t_data = (HeapTupleHeader) PageGetItem(Cpage, Citemid);
tuple_len = tuple.t_len = ItemIdGetLength(Citemid);
Patch VACUUM problem with moving chain of update tuples when source and destination of a tuple lie on the same page. (Previously fixed in REL7_0 branch, now apply to current.)
2000-10-22 21:49:43 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
move_chain_tuple(onerel, Cbuf, Cpage, &tuple,
dst_buffer, dst_page, destvacpage,
&ec, &Ctid, vtmove[ti].cleanVpd);
WAL misc
2000-10-24 11:56:23 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
num_moved++;
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
if (destvacpage->blkno > last_move_dest_block)
More vacuum cleanup
2000-05-29 19:06:15 +02:00
last_move_dest_block = destvacpage->blkno;
pgindent run over code.
1999-05-25 18:15:34 +02:00
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
/*
* Remember that we moved tuple from the current page
* (corresponding index tuple will be cleaned).
*/
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
if (Cbuf == buf)
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets[vacpage->offsets_free++] =
pgindent run over code.
1999-05-25 18:15:34 +02:00
ItemPointerGetOffsetNumber(&(tuple.t_self));
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
else
keep_tuples++;
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
ReleaseBuffer(dst_buffer);
ReleaseBuffer(Cbuf);
pgindent run.
2002-09-04 22:31:48 +02:00
} /* end of move-the-tuple-chain loop */
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
dst_buffer = InvalidBuffer;
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
pfree(vtmove);
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
chain_tuple_moved = true;
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/* advance to next tuple in walk-along-page loop */
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
continue;
pgindent run.
2002-09-04 22:31:48 +02:00
} /* end of is-tuple-in-chain test */
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/* try to find new page for this tuple */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (dst_buffer == InvalidBuffer ||
!enough_space(dst_vacpage, tuple_len))
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (dst_buffer != InvalidBuffer)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
ReleaseBuffer(dst_buffer);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
dst_buffer = InvalidBuffer;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
for (i = 0; i < num_fraged_pages; i++)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (enough_space(fraged_pages->pagedesc[i], tuple_len))
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
break;
}
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
if (i == num_fraged_pages)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
break; /* can't move item anywhere */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
dst_vacpage = fraged_pages->pagedesc[i];
dst_buffer = ReadBuffer(onerel, dst_vacpage->blkno);
LockBuffer(dst_buffer, BUFFER_LOCK_EXCLUSIVE);
dst_page = BufferGetPage(dst_buffer);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/* if this page was not used before - clean it */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (!PageIsEmpty(dst_page) && dst_vacpage->offsets_used == 0)
vacuum_page(onerel, dst_buffer, dst_vacpage);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
WAL misc
2000-10-24 11:56:23 +02:00
else
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
LockBuffer(dst_buffer, BUFFER_LOCK_EXCLUSIVE);
WAL misc
2000-10-24 11:56:23 +02:00
LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
move_plain_tuple(onerel, buf, page, &tuple,
dst_buffer, dst_page, dst_vacpage, &ec);
Fix for vacuum and cache use. Fix for BSDI 4.0.
1998-10-12 02:53:42 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
num_moved++;
if (dst_vacpage->blkno > last_move_dest_block)
last_move_dest_block = dst_vacpage->blkno;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
/*
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
* Remember that we moved tuple from the current page
* (corresponding index tuple will be cleaned).
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
*/
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets[vacpage->offsets_free++] = offnum;
Cleanup vacuum names.
2000-01-20 21:01:25 +01:00
} /* walk along page */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we broke out of the walk-along-page loop early (ie, still have
* offnum <= maxoff), then we failed to move some tuple off this page.
* No point in shrinking any more, so clean up and exit the per-page
* loop.
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
*/
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
if (offnum < maxoff && keep_tuples > 0)
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
OffsetNumber off;
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Fix vacpage state for any unvisited tuples remaining on page
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
*/
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
for (off = OffsetNumberNext(offnum);
pgindent run over code.
1999-05-25 18:15:34 +02:00
off <= maxoff;
off = OffsetNumberNext(off))
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
ItemId itemid = PageGetItemId(page, off);
HeapTupleHeader htup;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
if (!ItemIdIsUsed(itemid))
continue;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
htup = (HeapTupleHeader) PageGetItem(page, itemid);
if (htup->t_infomask & HEAP_XMIN_COMMITTED)
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
continue;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* See comments in the walk-along-page loop above about why
* only MOVED_OFF tuples should be found here.
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
*/
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
if (htup->t_infomask & HEAP_MOVED_IN)
elog(ERROR, "HEAP_MOVED_IN was not expected");
if (!(htup->t_infomask & HEAP_MOVED_OFF))
elog(ERROR, "HEAP_MOVED_OFF was expected");
if (HeapTupleHeaderGetXvac(htup) != myXID)
elog(ERROR, "invalid XVAC in tuple header");
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (chain_tuple_moved)
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
{
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
/* some chains were moved while cleaning this page */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Assert(vacpage->offsets_free > 0);
for (i = 0; i < vacpage->offsets_free; i++)
{
if (vacpage->offsets[i] == off)
break;
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
}
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
if (i >= vacpage->offsets_free) /* not found */
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets[vacpage->offsets_free++] = off;
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
Assert(keep_tuples > 0);
keep_tuples--;
}
}
This patch fixes a regression caused by my recent changes to heap tuple header. The fix is based on the thought that HEAP_MOVED_IN is not needed any more as soon as HEAP_XMIN_COMMITTED has been set. So in tqual.c and vacuum.c the HEAP_MOVED bits are cleared when HEAP_XMIN_COMMITTED is set. Vacuum robustness is enhanced by rearranging ifs, so that we have a chance to elog(ERROR, ...) before an assertion fails. A new regression test is included. Manfred Koizar
2002-07-20 06:57:13 +02:00
else
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
{
vacpage->offsets[vacpage->offsets_free++] = off;
Assert(keep_tuples > 0);
keep_tuples--;
}
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
}
}
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (vacpage->offsets_free > 0) /* some tuples were moved */
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
if (chain_tuple_moved) /* else - they are ordered */
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
qsort((char *) (vacpage->offsets), vacpage->offsets_free,
More vacuum renaming.
2000-05-29 03:46:00 +02:00
sizeof(OffsetNumber), vac_cmp_offno);
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
}
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
vpage_insert(&Nvacpagelist, copy_vac_page(vacpage));
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
ReleaseBuffer(buf);
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (offnum <= maxoff)
Fix tuple-chain-moving tests to handle marked-for-update tuples correctly (they are not part of a chain). When failing to find a parent tuple in an update chain, emit a warning and abandon repair_frag, but do not give an error as before. This should eliminate the infamous 'No one parent tuple was found' failure, which we now realize is not a can't-happen condition but a perfectly valid database state. Per recent pghackers discussion.
2002-08-13 22:14:24 +02:00
break; /* had to quit early, see above note */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
} /* walk along relation */
blkno++; /* new number of blocks */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (dst_buffer != InvalidBuffer)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
Assert(num_moved > 0);
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
ReleaseBuffer(dst_buffer);
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Vacuum cleanup.
1998-08-20 00:01:18 +02:00
if (num_moved > 0)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Revise lock manager to support "session level" locks as well as "transaction level" locks. A session lock is not released at transaction commit (but it is released on transaction abort, to ensure recovery after an elog(ERROR)). In VACUUM, use a session lock to protect the master table while vacuuming a TOAST table, so that the TOAST table can be done in an independent transaction. I also took this opportunity to do some cleanup and renaming in the lock code. The previously noted bug in ProcLockWakeup, that it couldn't wake up any waiters beyond the first non-wakeable waiter, is now fixed. Also found a previously unknown bug of the same kind (failure to scan all members of a lock queue in some cases) in DeadLockCheck. This might have led to failure to detect a deadlock condition, resulting in indefinite waits, but it's difficult to characterize the conditions required to trigger a failure.
2000-12-22 01:51:54 +01:00
* We have to commit our tuple movings before we truncate the
* relation. Ideally we should do Commit/StartTransactionCommand
* here, relying on the session-level table lock to protect our
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* exclusive access to the relation. However, that would require a
* lot of extra code to close and re-open the relation, indexes, etc.
* For now, a quick hack: record status of current transaction as
* committed, and continue.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
No more #ifdef XLOG.
2000-11-30 09:46:26 +01:00
RecordTransactionCommit();
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
* We are not going to move any more tuples across pages, but we still
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* need to apply vacuum_page to compact free space in the remaining pages
* in vacuum_pages list. Note that some of these pages may also be in the
* fraged_pages list, and may have had tuples moved onto them; if so, we
* already did vacuum_page and needn't do it again.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
for (i = 0, curpage = vacuum_pages->pagedesc;
i < vacuumed_pages;
i++, curpage++)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Centralize implementation of delay code by creating a pg_usleep() subroutine in src/port/pgsleep.c. Remove platform dependencies from miscadmin.h and put them in port.h where they belong. Extend recent vacuum cost-based-delay patch to apply to VACUUM FULL, ANALYZE, and non-btree index vacuuming. By the way, where is the documentation for the cost-based-delay patch?
2004-02-10 04:42:45 +01:00
vacuum_delay_point();
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
Assert((*curpage)->blkno < blkno);
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
if ((*curpage)->offsets_used == 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Buffer buf;
Page page;
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/* this page was not used as a move target, so must clean it */
buf = ReadBuffer(onerel, (*curpage)->blkno);
LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
page = BufferGetPage(buf);
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
if (!PageIsEmpty(page))
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
vacuum_page(onerel, buf, *curpage);
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
UnlockReleaseBuffer(buf);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
}
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* Now scan all the pages that we moved tuples onto and update tuple
* status bits. This is not really necessary, but will save time for
* future transactions examining these tuples.
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
update_hint_bits(onerel, fraged_pages, num_fraged_pages,
last_move_dest_block, num_moved);
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* It'd be cleaner to make this report at the bottom of this routine, but
* then the rusage would double-count the second pass of index vacuuming.
* So do it here and ignore the relatively small amount of processing that
* occurs below.
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
*/
ereport(elevel,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("\"%s\": moved %u row versions, truncated %u to %u pages",
RelationGetRelationName(onerel),
num_moved, nblocks, blkno),
errdetail("%s.",
pg_rusage_show(&ru0))));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Reflect the motion of system tuples to catalog cache here.
*/
Revise lock manager to support "session level" locks as well as "transaction level" locks. A session lock is not released at transaction commit (but it is released on transaction abort, to ensure recovery after an elog(ERROR)). In VACUUM, use a session lock to protect the master table while vacuuming a TOAST table, so that the TOAST table can be done in an independent transaction. I also took this opportunity to do some cleanup and renaming in the lock code. The previously noted bug in ProcLockWakeup, that it couldn't wake up any waiters beyond the first non-wakeable waiter, is now fixed. Also found a previously unknown bug of the same kind (failure to scan all members of a lock queue in some cases) in DeadLockCheck. This might have led to failure to detect a deadlock condition, resulting in indefinite waits, but it's difficult to characterize the conditions required to trigger a failure.
2000-12-22 01:51:54 +01:00
CommandCounterIncrement();
Cache invalidation for vacuum of system tables.
2000-12-08 07:43:44 +01:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (Nvacpagelist.num_pages > 0)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
/* vacuum indexes again if needed */
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
if (Irel != NULL)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
VacPage *vpleft,
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
*vpright,
vpsave;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
/* re-sort Nvacpagelist.pagedesc */
for (vpleft = Nvacpagelist.pagedesc,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
vpright = Nvacpagelist.pagedesc + Nvacpagelist.num_pages - 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
vpleft < vpright; vpleft++, vpright--)
{
vpsave = *vpleft;
*vpleft = *vpright;
*vpright = vpsave;
}
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* keep_tuples is the number of tuples that have been moved off a
* page during chain moves but not been scanned over subsequently.
* The tuple ids of these tuples are not recorded as free offsets
* for any VacPage, so they will not be cleared from the indexes.
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
Fix tuple chain moving bug found by "Hiroshi Inoue" <Inoue@tpf.co.jp>.
1999-05-23 11:10:24 +02:00
Assert(keep_tuples >= 0);
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
for (i = 0; i < nindexes; i++)
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacuum_index(&Nvacpagelist, Irel[i],
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
vacrelstats->rel_tuples, keep_tuples);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
/*
* Clean moved-off tuples from last page in Nvacpagelist list.
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* We need only do this in this one page, because higher-numbered
* pages are going to be truncated from the relation entirely. But see
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* comments for update_hint_bits().
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
*/
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
if (vacpage->blkno == (blkno - 1) &&
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage->offsets_free > 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Buffer buf;
Page page;
Clean up a couple of ad-hoc computations of the maximum number of tuples on a page, as suggested by ITAGAKI Takahiro. Also, change a few places that were using some other estimates of max-items-per-page to consistently use MaxOffsetNumber. This is conservatively large --- we could have used the new MaxHeapTuplesPerPage macro, or a similar one for index tuples --- but those places are simply declaring a fixed-size buffer and assuming it will work, rather than actively testing for overrun. It seems safer to size these buffers in a way that can't overflow even if the page is corrupt.
2005-09-02 21:02:20 +02:00
OffsetNumber unused[MaxOffsetNumber];
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
OffsetNumber offnum,
maxoff;
int uncnt;
int num_tuples = 0;
1. WAL needs in zero-ed content of newly initialized page. 2. Log record for PageRepaireFragmentation now keeps array of !LP_USED offnums to redo cleanup properly.
2000-12-30 16:19:57 +01:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
buf = ReadBuffer(onerel, vacpage->blkno);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
page = BufferGetPage(buf);
Fix unportable assumptions about alignment of local char[n] variables.
2001-03-26 01:23:59 +02:00
maxoff = PageGetMaxOffsetNumber(page);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
for (offnum = FirstOffsetNumber;
1. xact.c: update comments about changing MyProc->xid and MyProc->xmin. 2. varsup.c:ReadNewTransactionId(): don't read nextXid from disk - this func doesn't allocate next xid, so ShmemVariableCache->nextXid may be used (but GetNewTransactionId() must be called first). 3. vacuum.c: change elog(ERROR, "Child item....") to elog(NOTICE) - this is not ERROR, proper handling is just not implemented, yet. 4. s_lock.c: increase S_MAX_BUSY by 2 times. 5. shmem.c:GetSnapshotData(): have to call ReadNewTransactionId() _after_ SpinAcquire(ShmemIndexLock).
1999-06-06 22:19:35 +02:00
offnum <= maxoff;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
offnum = OffsetNumberNext(offnum))
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
ItemId itemid = PageGetItemId(page, offnum);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
HeapTupleHeader htup;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (!ItemIdIsUsed(itemid))
continue;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
htup = (HeapTupleHeader) PageGetItem(page, itemid);
if (htup->t_infomask & HEAP_XMIN_COMMITTED)
continue;
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* See comments in the walk-along-page loop above about why
* only MOVED_OFF tuples should be found here.
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
*/
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
if (htup->t_infomask & HEAP_MOVED_IN)
elog(ERROR, "HEAP_MOVED_IN was not expected");
if (!(htup->t_infomask & HEAP_MOVED_OFF))
elog(ERROR, "HEAP_MOVED_OFF was expected");
if (HeapTupleHeaderGetXvac(htup) != myXID)
elog(ERROR, "invalid XVAC in tuple header");
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
itemid->lp_flags &= ~LP_USED;
num_tuples++;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
More vacuum cleanup
2000-05-29 19:06:15 +02:00
Assert(vacpage->offsets_free == num_tuples);
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
Narrow scope of critical section, per discussion 1/19/01.
2001-01-24 00:32:45 +01:00
START_CRIT_SECTION();
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
1. WAL needs in zero-ed content of newly initialized page. 2. Log record for PageRepaireFragmentation now keeps array of !LP_USED offnums to redo cleanup properly.
2000-12-30 16:19:57 +01:00
uncnt = PageRepairFragmentation(page, unused);
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
MarkBufferDirty(buf);
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
/* XLOG stuff */
if (!onerel->rd_istemp)
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
{
1. WAL needs in zero-ed content of newly initialized page. 2. Log record for PageRepaireFragmentation now keeps array of !LP_USED offnums to redo cleanup properly.
2000-12-30 16:19:57 +01:00
XLogRecPtr recptr;
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Improve coding of log_heap_clean() and heap_xlog_clean().
2003-02-23 21:32:12 +01:00
recptr = log_heap_clean(onerel, buf, unused, uncnt);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
PageSetLSN(page, recptr);
Invent WAL timelines, as per recent discussion, to make point-in-time recovery more manageable. Also, undo recent change to add FILE_HEADER and WASTED_SPACE records to XLOG; instead make the XLOG page header variable-size with extra fields in the first page of an XLOG file. This should fix the boundary-case bugs observed by Mark Kirkwood. initdb forced due to change of XLOG representation.
2004-07-22 00:31:26 +02:00
PageSetTLI(page, ThisTimeLineID);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
}
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
else
{
pgindent run.
2002-09-04 22:31:48 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* No XLOG record, but still need to flag that XID exists on
* disk
pgindent run.
2002-09-04 22:31:48 +02:00
*/
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
MyXactMadeTempRelUpdate = true;
}
Add more critical-section calls: all code sections that hold spinlocks are now critical sections, so as to ensure die() won't interrupt us while we are munging shared-memory data structures. Avoid insecure intermediate states in some code that proc_exit will call, like palloc/pfree. Rename START/END_CRIT_CODE to START/END_CRIT_SECTION, since that seems to be what people tend to call them anyway, and make them be called with () like a function call, in hopes of not confusing pg_indent. I doubt that this is sufficient to make SIGTERM safe anywhere; there's just too much code that could get invoked during proc_exit().
2001-01-12 22:54:01 +01:00
END_CRIT_SECTION();
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
UnlockReleaseBuffer(buf);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Cleanup vacuum names.
2000-01-20 21:01:25 +01:00
/* now - free new list of reaped pages */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
curpage = Nvacpagelist.pagedesc;
for (i = 0; i < Nvacpagelist.num_pages; i++, curpage++)
pfree(*curpage);
pfree(Nvacpagelist.pagedesc);
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Remove unnecessary calls of FlushRelationBuffers: there is no need to write out data that we are about to tell the filesystem to drop. smgr_internal_unlink already had a DropRelFileNodeBuffers call to get rid of dead buffers without a write after it's no longer possible to roll back the deleting transaction. Adding a similar call in smgrtruncate simplifies callers and makes the overall division of labor clearer. This patch removes the former behavior that VACUUM would write all dirty buffers of a relation unconditionally.
2005-03-20 23:00:54 +01:00
/* Truncate relation, if needed */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (blkno < nblocks)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Get rid of rd_nblocks field in relcache entries. Turns out this was costing us lots more to maintain than it was worth. On shared tables it was of exactly zero benefit because we couldn't trust it to be up to date. On temp tables it sometimes saved an lseek, but not often enough to be worth getting excited about. And the real problem was that we forced an lseek on every relcache flush in order to update the field. So all in all it seems best to lose the complexity.
2004-05-08 21:09:25 +02:00
RelationTruncate(onerel, blkno);
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
vacrelstats->rel_pages = blkno; /* set new number of blocks */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
/* clean up */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
pfree(vacpage);
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
if (vacrelstats->vtlinks != NULL)
pfree(vacrelstats->vtlinks);
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
ExecContext_Finish(&ec);
}
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*
* move_chain_tuple() -- move one tuple that is part of a tuple chain
*
* This routine moves old_tup from old_page to dst_page.
* old_page and dst_page might be the same page.
* On entry old_buf and dst_buf are locked exclusively, both locks (or
* the single lock, if this is a intra-page-move) are released before
* exit.
*
* Yes, a routine with ten parameters is ugly, but it's still better
* than having these 120 lines of code in repair_frag() which is
* already too long and almost unreadable.
*/
static void
move_chain_tuple(Relation rel,
Buffer old_buf, Page old_page, HeapTuple old_tup,
Buffer dst_buf, Page dst_page, VacPage dst_vacpage,
ExecContext ec, ItemPointer ctid, bool cleanVpd)
{
TransactionId myXID = GetCurrentTransactionId();
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
HeapTupleData newtup;
OffsetNumber newoff;
ItemId newitemid;
Size tuple_len = old_tup->t_len;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/*
* make a modifiable copy of the source tuple.
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
heap_copytuple_with_tuple(old_tup, &newtup);
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* register invalidation of source tuple in catcaches.
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
CacheInvalidateHeapTuple(rel, old_tup);
/* NO EREPORT(ERROR) TILL CHANGES ARE LOGGED */
START_CRIT_SECTION();
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/*
* mark the source tuple MOVED_OFF.
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
old_tup->t_data->t_infomask &= ~(HEAP_XMIN_COMMITTED |
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
HEAP_XMIN_INVALID |
HEAP_MOVED_IN);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
old_tup->t_data->t_infomask |= HEAP_MOVED_OFF;
HeapTupleHeaderSetXvac(old_tup->t_data, myXID);
/*
* If this page was not used before - clean it.
*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* NOTE: a nasty bug used to lurk here. It is possible for the source and
* destination pages to be the same (since this tuple-chain member can be
* on a page lower than the one we're currently processing in the outer
* loop). If that's true, then after vacuum_page() the source tuple will
* have been moved, and tuple.t_data will be pointing at garbage.
* Therefore we must do everything that uses old_tup->t_data BEFORE this
* step!!
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* This path is different from the other callers of vacuum_page, because
* we have already incremented the vacpage's offsets_used field to account
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* for the tuple(s) we expect to move onto the page. Therefore
* vacuum_page's check for offsets_used == 0 is wrong. But since that's a
* good debugging check for all other callers, we work around it here
* rather than remove it.
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
if (!PageIsEmpty(dst_page) && cleanVpd)
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
int sv_offsets_used = dst_vacpage->offsets_used;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
dst_vacpage->offsets_used = 0;
vacuum_page(rel, dst_buf, dst_vacpage);
dst_vacpage->offsets_used = sv_offsets_used;
}
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Update the state of the copied tuple, and store it on the destination
* page.
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
newtup.t_data->t_infomask &= ~(HEAP_XMIN_COMMITTED |
HEAP_XMIN_INVALID |
HEAP_MOVED_OFF);
newtup.t_data->t_infomask |= HEAP_MOVED_IN;
HeapTupleHeaderSetXvac(newtup.t_data, myXID);
newoff = PageAddItem(dst_page, (Item) newtup.t_data, tuple_len,
InvalidOffsetNumber, LP_USED);
if (newoff == InvalidOffsetNumber)
elog(PANIC, "failed to add item with len = %lu to page %u while moving tuple chain",
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
(unsigned long) tuple_len, dst_vacpage->blkno);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
newitemid = PageGetItemId(dst_page, newoff);
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/* drop temporary copy, and point to the version on the dest page */
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
pfree(newtup.t_data);
newtup.t_data = (HeapTupleHeader) PageGetItem(dst_page, newitemid);
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
ItemPointerSet(&(newtup.t_self), dst_vacpage->blkno, newoff);
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Set new tuple's t_ctid pointing to itself if last tuple in chain, and
* to next tuple in chain otherwise. (Since we move the chain in reverse
* order, this is actually the previously processed tuple.)
Repair problems with VACUUM destroying t_ctid chains too soon, and with insufficient paranoia in code that follows t_ctid links. (We must do both because even with VACUUM doing it properly, the intermediate state with a dangling t_ctid link is visible concurrently during lazy VACUUM, and could be seen afterwards if either type of VACUUM crashes partway through.) Also try to improve documentation about what's going on. Patch is a bit bulky because passing the XMAX information around required changing the APIs of some low-level heapam.c routines, but it's not conceptually very complicated. Per trouble report from Teodor and subsequent analysis. This needs to be back-patched, but I'll do that after 8.1 beta is out.
2005-08-20 02:40:32 +02:00
*/
if (!ItemPointerIsValid(ctid))
newtup.t_data->t_ctid = newtup.t_self;
else
newtup.t_data->t_ctid = *ctid;
*ctid = newtup.t_self;
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
MarkBufferDirty(dst_buf);
if (dst_buf != old_buf)
MarkBufferDirty(old_buf);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/* XLOG stuff */
if (!rel->rd_istemp)
{
XLogRecPtr recptr = log_heap_move(rel, old_buf, old_tup->t_self,
dst_buf, &newtup);
if (old_buf != dst_buf)
{
PageSetLSN(old_page, recptr);
Invent WAL timelines, as per recent discussion, to make point-in-time recovery more manageable. Also, undo recent change to add FILE_HEADER and WASTED_SPACE records to XLOG; instead make the XLOG page header variable-size with extra fields in the first page of an XLOG file. This should fix the boundary-case bugs observed by Mark Kirkwood. initdb forced due to change of XLOG representation.
2004-07-22 00:31:26 +02:00
PageSetTLI(old_page, ThisTimeLineID);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
}
PageSetLSN(dst_page, recptr);
Invent WAL timelines, as per recent discussion, to make point-in-time recovery more manageable. Also, undo recent change to add FILE_HEADER and WASTED_SPACE records to XLOG; instead make the XLOG page header variable-size with extra fields in the first page of an XLOG file. This should fix the boundary-case bugs observed by Mark Kirkwood. initdb forced due to change of XLOG representation.
2004-07-22 00:31:26 +02:00
PageSetTLI(dst_page, ThisTimeLineID);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
}
else
{
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* No XLOG record, but still need to flag that XID exists on disk
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
MyXactMadeTempRelUpdate = true;
}
END_CRIT_SECTION();
LockBuffer(dst_buf, BUFFER_LOCK_UNLOCK);
if (dst_buf != old_buf)
LockBuffer(old_buf, BUFFER_LOCK_UNLOCK);
/* Create index entries for the moved tuple */
if (ec->resultRelInfo->ri_NumIndices > 0)
{
ExecStoreTuple(&newtup, ec->slot, InvalidBuffer, false);
ExecInsertIndexTuples(ec->slot, &(newtup.t_self), ec->estate, true);
Avoid memory leakage during VACUUM FULL when an index expression or index predicate uses temporary memory for evaluation. Per example from Jean-Gerard Pailloncy.
2004-12-23 23:42:15 +01:00
ResetPerTupleExprContext(ec->estate);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
}
}
/*
* move_plain_tuple() -- move one tuple that is not part of a chain
*
* This routine moves old_tup from old_page to dst_page.
* On entry old_buf and dst_buf are locked exclusively, both locks are
* released before exit.
*
* Yes, a routine with eight parameters is ugly, but it's still better
* than having these 90 lines of code in repair_frag() which is already
* too long and almost unreadable.
*/
static void
move_plain_tuple(Relation rel,
Buffer old_buf, Page old_page, HeapTuple old_tup,
Buffer dst_buf, Page dst_page, VacPage dst_vacpage,
ExecContext ec)
{
TransactionId myXID = GetCurrentTransactionId();
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
HeapTupleData newtup;
OffsetNumber newoff;
ItemId newitemid;
Size tuple_len = old_tup->t_len;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/* copy tuple */
heap_copytuple_with_tuple(old_tup, &newtup);
/*
* register invalidation of source tuple in catcaches.
*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* (Note: we do not need to register the copied tuple, because we are not
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* changing the tuple contents and so there cannot be any need to flush
* negative catcache entries.)
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
CacheInvalidateHeapTuple(rel, old_tup);
/* NO EREPORT(ERROR) TILL CHANGES ARE LOGGED */
START_CRIT_SECTION();
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Mark new tuple as MOVED_IN by me.
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
newtup.t_data->t_infomask &= ~(HEAP_XMIN_COMMITTED |
HEAP_XMIN_INVALID |
HEAP_MOVED_OFF);
newtup.t_data->t_infomask |= HEAP_MOVED_IN;
HeapTupleHeaderSetXvac(newtup.t_data, myXID);
/* add tuple to the page */
newoff = PageAddItem(dst_page, (Item) newtup.t_data, tuple_len,
InvalidOffsetNumber, LP_USED);
if (newoff == InvalidOffsetNumber)
elog(PANIC, "failed to add item with len = %lu to page %u (free space %lu, nusd %u, noff %u)",
(unsigned long) tuple_len,
dst_vacpage->blkno, (unsigned long) dst_vacpage->free,
dst_vacpage->offsets_used, dst_vacpage->offsets_free);
newitemid = PageGetItemId(dst_page, newoff);
pfree(newtup.t_data);
newtup.t_data = (HeapTupleHeader) PageGetItem(dst_page, newitemid);
ItemPointerSet(&(newtup.t_data->t_ctid), dst_vacpage->blkno, newoff);
newtup.t_self = newtup.t_data->t_ctid;
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* Mark old tuple as MOVED_OFF by me.
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
old_tup->t_data->t_infomask &= ~(HEAP_XMIN_COMMITTED |
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
HEAP_XMIN_INVALID |
HEAP_MOVED_IN);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
old_tup->t_data->t_infomask |= HEAP_MOVED_OFF;
HeapTupleHeaderSetXvac(old_tup->t_data, myXID);
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
MarkBufferDirty(dst_buf);
MarkBufferDirty(old_buf);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/* XLOG stuff */
if (!rel->rd_istemp)
{
XLogRecPtr recptr = log_heap_move(rel, old_buf, old_tup->t_self,
dst_buf, &newtup);
PageSetLSN(old_page, recptr);
Invent WAL timelines, as per recent discussion, to make point-in-time recovery more manageable. Also, undo recent change to add FILE_HEADER and WASTED_SPACE records to XLOG; instead make the XLOG page header variable-size with extra fields in the first page of an XLOG file. This should fix the boundary-case bugs observed by Mark Kirkwood. initdb forced due to change of XLOG representation.
2004-07-22 00:31:26 +02:00
PageSetTLI(old_page, ThisTimeLineID);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
PageSetLSN(dst_page, recptr);
Invent WAL timelines, as per recent discussion, to make point-in-time recovery more manageable. Also, undo recent change to add FILE_HEADER and WASTED_SPACE records to XLOG; instead make the XLOG page header variable-size with extra fields in the first page of an XLOG file. This should fix the boundary-case bugs observed by Mark Kirkwood. initdb forced due to change of XLOG representation.
2004-07-22 00:31:26 +02:00
PageSetTLI(dst_page, ThisTimeLineID);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
}
else
{
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
* No XLOG record, but still need to flag that XID exists on disk
*/
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
MyXactMadeTempRelUpdate = true;
}
END_CRIT_SECTION();
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
dst_vacpage->free = PageGetFreeSpaceWithFillFactor(rel, dst_page);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
LockBuffer(dst_buf, BUFFER_LOCK_UNLOCK);
LockBuffer(old_buf, BUFFER_LOCK_UNLOCK);
dst_vacpage->offsets_used++;
/* insert index' tuples if needed */
if (ec->resultRelInfo->ri_NumIndices > 0)
{
ExecStoreTuple(&newtup, ec->slot, InvalidBuffer, false);
ExecInsertIndexTuples(ec->slot, &(newtup.t_self), ec->estate, true);
Avoid memory leakage during VACUUM FULL when an index expression or index predicate uses temporary memory for evaluation. Per example from Jean-Gerard Pailloncy.
2004-12-23 23:42:15 +01:00
ResetPerTupleExprContext(ec->estate);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
}
}
/*
* update_hint_bits() -- update hint bits in destination pages
*
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
* Scan all the pages that we moved tuples onto and update tuple status bits.
* This is normally not really necessary, but it will save time for future
* transactions examining these tuples.
*
* This pass guarantees that all HEAP_MOVED_IN tuples are marked as
* XMIN_COMMITTED, so that future tqual tests won't need to check their XVAC.
*
* BUT NOTICE that this code fails to clear HEAP_MOVED_OFF tuples from
* pages that were move source pages but not move dest pages. The bulk
* of the move source pages will be physically truncated from the relation,
* and the last page remaining in the rel will be fixed separately in
* repair_frag(), so the only cases where a MOVED_OFF tuple won't get its
* hint bits updated are tuples that are moved as part of a chain and were
* on pages that were not either move destinations nor at the end of the rel.
* To completely ensure that no MOVED_OFF tuples remain unmarked, we'd have
* to remember and revisit those pages too.
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
* Because of this omission, VACUUM FULL FREEZE is not a safe combination;
* it's possible that the VACUUM's own XID remains exposed as something that
* tqual tests would need to check.
*
* For the non-freeze case, one wonders whether it wouldn't be better to skip
* this work entirely, and let the tuple status updates happen someplace
* that's not holding an exclusive lock on the relation.
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
static void
update_hint_bits(Relation rel, VacPageList fraged_pages, int num_fraged_pages,
BlockNumber last_move_dest_block, int num_moved)
{
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
TransactionId myXID = GetCurrentTransactionId();
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
int checked_moved = 0;
int i;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
VacPage *curpage;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
for (i = 0, curpage = fraged_pages->pagedesc;
i < num_fraged_pages;
i++, curpage++)
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Buffer buf;
Page page;
OffsetNumber max_offset;
OffsetNumber off;
int num_tuples = 0;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
vacuum_delay_point();
if ((*curpage)->blkno > last_move_dest_block)
break; /* no need to scan any further */
if ((*curpage)->offsets_used == 0)
continue; /* this page was never used as a move dest */
buf = ReadBuffer(rel, (*curpage)->blkno);
LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
page = BufferGetPage(buf);
max_offset = PageGetMaxOffsetNumber(page);
for (off = FirstOffsetNumber;
off <= max_offset;
off = OffsetNumberNext(off))
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
ItemId itemid = PageGetItemId(page, off);
HeapTupleHeader htup;
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (!ItemIdIsUsed(itemid))
continue;
htup = (HeapTupleHeader) PageGetItem(page, itemid);
if (htup->t_infomask & HEAP_XMIN_COMMITTED)
continue;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
/*
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
* Here we may see either MOVED_OFF or MOVED_IN tuples.
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
*/
Disallow the combination VACUUM FULL FREEZE for safety's sake, for the reasons I outlined in pghackers a few days ago. Also, undo someone's overly optimistic decision to reduce tuple state checks from if (...) elog() to Asserts. If I trusted this code more, I might think it was a good idea to disable these checks in production installations. But I don't.
2004-12-02 20:28:49 +01:00
if (!(htup->t_infomask & HEAP_MOVED))
elog(ERROR, "HEAP_MOVED_OFF/HEAP_MOVED_IN was expected");
if (HeapTupleHeaderGetXvac(htup) != myXID)
elog(ERROR, "invalid XVAC in tuple header");
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
if (htup->t_infomask & HEAP_MOVED_IN)
{
htup->t_infomask |= HEAP_XMIN_COMMITTED;
htup->t_infomask &= ~HEAP_MOVED;
num_tuples++;
}
else
htup->t_infomask |= HEAP_XMIN_INVALID;
}
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
MarkBufferDirty(buf);
UnlockReleaseBuffer(buf);
vacuum.c refactoring . rename variables . cur_buffer -> dst_buffer . ToPage -> dst_page . cur_page -> dst_vacpage . move variable declarations into block where variable is used . various Asserts instead of elog(ERROR, ...) . extract functionality from repair_frag() into new routines . move_chain_tuple() . move_plain_tuple() . update_hint_bits() . create type ExecContext . add comments Manfred Koizar
2004-06-08 15:59:36 +02:00
Assert((*curpage)->offsets_used == num_tuples);
checked_moved += num_tuples;
}
Assert(num_moved == checked_moved);
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
/*
More vacuum renaming.
2000-05-29 03:46:00 +02:00
* vacuum_heap() -- free dead tuples
Shrinking and other things.
1996-11-27 08:27:20 +01:00
*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* This routine marks dead tuples as unused and truncates relation
* if there are "empty" end-blocks.
Shrinking and other things.
1996-11-27 08:27:20 +01:00
*/
static void
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacuum_heap(VRelStats *vacrelstats, Relation onerel, VacPageList vacuum_pages)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
Buffer buf;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
VacPage *vacpage;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
BlockNumber relblocks;
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
int nblocks;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
nblocks = vacuum_pages->num_pages;
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
nblocks -= vacuum_pages->empty_end_pages; /* nothing to do with them */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
for (i = 0, vacpage = vacuum_pages->pagedesc; i < nblocks; i++, vacpage++)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Centralize implementation of delay code by creating a pg_usleep() subroutine in src/port/pgsleep.c. Remove platform dependencies from miscadmin.h and put them in port.h where they belong. Extend recent vacuum cost-based-delay patch to apply to VACUUM FULL, ANALYZE, and non-btree index vacuuming. By the way, where is the documentation for the cost-based-delay patch?
2004-02-10 04:42:45 +01:00
vacuum_delay_point();
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if ((*vacpage)->offsets_free > 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
buf = ReadBuffer(onerel, (*vacpage)->blkno);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
vacuum_page(onerel, buf, *vacpage);
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
UnlockReleaseBuffer(buf);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Remove unnecessary calls of FlushRelationBuffers: there is no need to write out data that we are about to tell the filesystem to drop. smgr_internal_unlink already had a DropRelFileNodeBuffers call to get rid of dead buffers without a write after it's no longer possible to roll back the deleting transaction. Adding a similar call in smgrtruncate simplifies callers and makes the overall division of labor clearer. This patch removes the former behavior that VACUUM would write all dirty buffers of a relation unconditionally.
2005-03-20 23:00:54 +01:00
/* Truncate relation if there are some empty end-pages */
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
Assert(vacrelstats->rel_pages >= vacuum_pages->empty_end_pages);
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (vacuum_pages->empty_end_pages > 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Remove unnecessary calls of FlushRelationBuffers: there is no need to write out data that we are about to tell the filesystem to drop. smgr_internal_unlink already had a DropRelFileNodeBuffers call to get rid of dead buffers without a write after it's no longer possible to roll back the deleting transaction. Adding a similar call in smgrtruncate simplifies callers and makes the overall division of labor clearer. This patch removes the former behavior that VACUUM would write all dirty buffers of a relation unconditionally.
2005-03-20 23:00:54 +01:00
relblocks = vacrelstats->rel_pages - vacuum_pages->empty_end_pages;
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(elevel,
(errmsg("\"%s\": truncated %u to %u pages",
RelationGetRelationName(onerel),
vacrelstats->rel_pages, relblocks)));
Get rid of rd_nblocks field in relcache entries. Turns out this was costing us lots more to maintain than it was worth. On shared tables it was of exactly zero benefit because we couldn't trust it to be up to date. On temp tables it sometimes saved an lseek, but not often enough to be worth getting excited about. And the real problem was that we forced an lseek on every relcache flush in order to update the field. So all in all it seems best to lose the complexity.
2004-05-08 21:09:25 +02:00
RelationTruncate(onerel, relblocks);
Remove unnecessary calls of FlushRelationBuffers: there is no need to write out data that we are about to tell the filesystem to drop. smgr_internal_unlink already had a DropRelFileNodeBuffers call to get rid of dead buffers without a write after it's no longer possible to roll back the deleting transaction. Adding a similar call in smgrtruncate simplifies callers and makes the overall division of labor clearer. This patch removes the former behavior that VACUUM would write all dirty buffers of a relation unconditionally.
2005-03-20 23:00:54 +01:00
vacrelstats->rel_pages = relblocks; /* set new number of blocks */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
/*
More vacuum renaming.
2000-05-29 03:46:00 +02:00
* vacuum_page() -- free dead tuples on a page
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
* and repair its fragmentation.
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
*
* Caller must hold pin and lock on buffer.
Shrinking and other things.
1996-11-27 08:27:20 +01:00
*/
static void
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
vacuum_page(Relation onerel, Buffer buffer, VacPage vacpage)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Clean up a couple of ad-hoc computations of the maximum number of tuples on a page, as suggested by ITAGAKI Takahiro. Also, change a few places that were using some other estimates of max-items-per-page to consistently use MaxOffsetNumber. This is conservatively large --- we could have used the new MaxHeapTuplesPerPage macro, or a similar one for index tuples --- but those places are simply declaring a fixed-size buffer and assuming it will work, rather than actively testing for overrun. It seems safer to size these buffers in a way that can't overflow even if the page is corrupt.
2005-09-02 21:02:20 +02:00
OffsetNumber unused[MaxOffsetNumber];
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
int uncnt;
Page page = BufferGetPage(buffer);
ItemId itemid;
int i;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
/* There shouldn't be any tuples moved onto the page yet! */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
Assert(vacpage->offsets_used == 0);
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
Add more critical-section calls: all code sections that hold spinlocks are now critical sections, so as to ensure die() won't interrupt us while we are munging shared-memory data structures. Avoid insecure intermediate states in some code that proc_exit will call, like palloc/pfree. Rename START/END_CRIT_CODE to START/END_CRIT_SECTION, since that seems to be what people tend to call them anyway, and make them be called with () like a function call, in hopes of not confusing pg_indent. I doubt that this is sufficient to make SIGTERM safe anywhere; there's just too much code that could get invoked during proc_exit().
2001-01-12 22:54:01 +01:00
START_CRIT_SECTION();
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
for (i = 0; i < vacpage->offsets_free; i++)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
itemid = PageGetItemId(page, vacpage->offsets[i]);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
itemid->lp_flags &= ~LP_USED;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
}
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
1. WAL needs in zero-ed content of newly initialized page. 2. Log record for PageRepaireFragmentation now keeps array of !LP_USED offnums to redo cleanup properly.
2000-12-30 16:19:57 +01:00
uncnt = PageRepairFragmentation(page, unused);
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
Clean up WAL/buffer interactions as per my recent proposal. Get rid of the misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
2006-04-01 01:32:07 +02:00
MarkBufferDirty(buffer);
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
/* XLOG stuff */
if (!onerel->rd_istemp)
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
{
1. WAL needs in zero-ed content of newly initialized page. 2. Log record for PageRepaireFragmentation now keeps array of !LP_USED offnums to redo cleanup properly.
2000-12-30 16:19:57 +01:00
XLogRecPtr recptr;
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Improve coding of log_heap_clean() and heap_xlog_clean().
2003-02-23 21:32:12 +01:00
recptr = log_heap_clean(onerel, buffer, unused, uncnt);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
PageSetLSN(page, recptr);
Invent WAL timelines, as per recent discussion, to make point-in-time recovery more manageable. Also, undo recent change to add FILE_HEADER and WASTED_SPACE records to XLOG; instead make the XLOG page header variable-size with extra fields in the first page of an XLOG file. This should fix the boundary-case bugs observed by Mark Kirkwood. initdb forced due to change of XLOG representation.
2004-07-22 00:31:26 +02:00
PageSetTLI(page, ThisTimeLineID);
New WAL version - CRC and data blocks backup.
2000-12-28 14:00:29 +01:00
}
Restructure local-buffer handling per recent pghackers discussion. The local buffer manager is no longer used for newly-created relations (unless they are TEMP); a new non-TEMP relation goes through the shared bufmgr and thus will participate normally in checkpoints. But TEMP relations use the local buffer manager throughout their lifespan. Also, operations in TEMP relations are not logged in WAL, thus improving performance. Since it's no longer necessary to fsync relations as they move out of the local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code is no longer needed and has been removed: there's no concept of a dirty relation anymore in md.c/fd.c, and we never fsync anything but WAL. Still TODO: improve local buffer management algorithms so that it would be reasonable to increase NLocBuffer.
2002-08-06 04:36:35 +02:00
else
{
/* No XLOG record, but still need to flag that XID exists on disk */
MyXactMadeTempRelUpdate = true;
}
Add more critical-section calls: all code sections that hold spinlocks are now critical sections, so as to ensure die() won't interrupt us while we are munging shared-memory data structures. Avoid insecure intermediate states in some code that proc_exit will call, like palloc/pfree. Rename START/END_CRIT_CODE to START/END_CRIT_SECTION, since that seems to be what people tend to call them anyway, and make them be called with () like a function call, in hopes of not confusing pg_indent. I doubt that this is sufficient to make SIGTERM safe anywhere; there's just too much code that could get invoked during proc_exit().
2001-01-12 22:54:01 +01:00
END_CRIT_SECTION();
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
/*
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
* scan_index() -- scan one index relation to update pg_class statistics.
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
*
* We use this when we have no deletions to do.
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
*/
static void
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
scan_index(Relation indrel, double num_tuples)
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
{
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
IndexBulkDeleteResult *stats;
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
IndexVacuumInfo ivinfo;
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
PGRUsage ru0;
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
pg_rusage_init(&ru0);
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
ivinfo.index = indrel;
ivinfo.vacuum_full = true;
ivinfo.message_level = elevel;
ivinfo.num_heap_tuples = num_tuples;
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
stats = index_vacuum_cleanup(&ivinfo, NULL);
More infrastructure for btree compaction project. Tree-traversal code now knows what to do upon hitting a dead page (in theory anyway, it's untested...). Add a post-VACUUM-cleanup entry point for index AMs, to provide a place for dead-page scavenging to happen. Also, fix oversight that broke btpo_prev links in temporary indexes. initdb forced due to additions in pg_am.
2003-02-22 01:45:05 +01:00
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
if (!stats)
return;
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
Revert based on Tom's recommendation: > Allow VACUUM to complete faster by avoiding scanning the indexes when no > rows were removed from the heap by the VACUUM.
2006-02-11 18:14:09 +01:00
/* now update statistics in pg_class */
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
vac_update_relstats(RelationGetRelid(indrel),
Revert based on Tom's recommendation: > Allow VACUUM to complete faster by avoiding scanning the indexes when no > rows were removed from the heap by the VACUUM.
2006-02-11 18:14:09 +01:00
stats->num_pages, stats->num_index_tuples,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
false, InvalidTransactionId, InvalidTransactionId);
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(elevel,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("index \"%s\" now contains %.0f row versions in %u pages",
RelationGetRelationName(indrel),
Revert based on Tom's recommendation: > Allow VACUUM to complete faster by avoiding scanning the indexes when no > rows were removed from the heap by the VACUUM.
2006-02-11 18:14:09 +01:00
stats->num_index_tuples,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
stats->num_pages),
errdetail("%u index pages have been deleted, %u are currently reusable.\n"
"%s.",
stats->pages_deleted, stats->pages_free,
pg_rusage_show(&ru0))));
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Check for tuple count mismatch. If the index is partial, then it's OK
* for it to have fewer tuples than the heap; else we got trouble.
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
*/
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
if (stats->num_index_tuples != num_tuples)
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
{
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
if (stats->num_index_tuples > num_tuples ||
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
!vac_is_partial_index(indrel))
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
(errmsg("index \"%s\" contains %.0f row versions, but table contains %.0f row versions",
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
RelationGetRelationName(indrel),
stats->num_index_tuples, num_tuples),
errhint("Rebuild the index with REINDEX.")));
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
}
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
pfree(stats);
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
New func _vc_scanoneind: scan one index relation to update statistic in pg_class if no one page was reapped by vacuum.
1997-01-29 03:59:03 +01:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
More vacuum renaming.
2000-05-29 03:46:00 +02:00
* vacuum_index() -- vacuum one index relation.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
More vacuum cleanup
2000-05-29 19:06:15 +02:00
* Vpl is the VacPageList of the heap we're currently vacuuming.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* It's locked. Indrel is an index relation on the vacuumed heap.
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
*
* We don't bother to set locks on the index relation here, since
* the parent table is exclusive-locked already.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* Finally, we arrange to update the index relation's statistics in
* pg_class.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static void
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
vacuum_index(VacPageList vacpagelist, Relation indrel,
Install infrastructure for shared-memory free space map. Doesn't actually do anything yet, but it has the necessary connections to initialization and so forth. Make some gestures towards allowing number of blocks in a relation to be BlockNumber, ie, unsigned int, rather than signed int. (I doubt I got all the places that are sloppy about it, yet.) On the way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC variable.
2001-06-28 01:31:40 +02:00
double num_tuples, int keep_tuples)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
IndexBulkDeleteResult *stats;
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
IndexVacuumInfo ivinfo;
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
PGRUsage ru0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Separate out the VacRUsage stuff as an independent module, in preparation for using it for other things besides VACUUM.
2005-10-04 00:52:26 +02:00
pg_rusage_init(&ru0);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
ivinfo.index = indrel;
ivinfo.vacuum_full = true;
ivinfo.message_level = elevel;
ivinfo.num_heap_tuples = num_tuples + keep_tuples;
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
/* Do bulk deletion */
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
stats = index_bulk_delete(&ivinfo, NULL, tid_reaped, (void *) vacpagelist);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
More infrastructure for btree compaction project. Tree-traversal code now knows what to do upon hitting a dead page (in theory anyway, it's untested...). Add a post-VACUUM-cleanup entry point for index AMs, to provide a place for dead-page scavenging to happen. Also, fix oversight that broke btpo_prev links in temporary indexes. initdb forced due to additions in pg_am.
2003-02-22 01:45:05 +01:00
/* Do post-VACUUM cleanup */
Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion. This formulation requires every AM to provide amvacuumcleanup, unlike before, but it's surely a whole lot cleaner. Also, add an 'amstorage' column to pg_am so that we can get rid of hardwired knowledge in DefineOpClass().
2006-05-03 00:25:10 +02:00
stats = index_vacuum_cleanup(&ivinfo, stats);
More infrastructure for btree compaction project. Tree-traversal code now knows what to do upon hitting a dead page (in theory anyway, it's untested...). Add a post-VACUUM-cleanup entry point for index AMs, to provide a place for dead-page scavenging to happen. Also, fix oversight that broke btpo_prev links in temporary indexes. initdb forced due to additions in pg_am.
2003-02-22 01:45:05 +01:00
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
if (!stats)
return;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/* now update statistics in pg_class */
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
vac_update_relstats(RelationGetRelid(indrel),
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
stats->num_pages, stats->num_index_tuples,
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
false, InvalidTransactionId, InvalidTransactionId);
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(elevel,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("index \"%s\" now contains %.0f row versions in %u pages",
RelationGetRelationName(indrel),
stats->num_index_tuples,
stats->num_pages),
errdetail("%.0f index row versions were removed.\n"
"%u index pages have been deleted, %u are currently reusable.\n"
"%s.",
stats->tuples_removed,
stats->pages_deleted, stats->pages_free,
pg_rusage_show(&ru0))));
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Check for tuple count mismatch. If the index is partial, then it's OK
* for it to have fewer tuples than the heap; else we got trouble.
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
*/
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
if (stats->num_index_tuples != num_tuples + keep_tuples)
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
{
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
if (stats->num_index_tuples > num_tuples + keep_tuples ||
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
!vac_is_partial_index(indrel))
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
ereport(WARNING,
More message editing, some suggested by Alvaro Herrera
2003-09-29 02:05:25 +02:00
(errmsg("index \"%s\" contains %.0f row versions, but table contains %.0f row versions",
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
RelationGetRelationName(indrel),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
stats->num_index_tuples, num_tuples + keep_tuples),
Another round of error message editing, covering backend/commands/.
2003-07-20 23:56:35 +02:00
errhint("Rebuild the index with REINDEX.")));
Remove direct calls of index_insert(), instead use ExecInsertIndexTuples(). This makes VACUUM work properly with partial indexes, and avoids memory leakage with functional indexes. Also, suppress complaint about fewer index tuples than heap tuples when the index is a partial index. From Martijn van Oosterhout.
2001-07-11 20:38:07 +02:00
}
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
pfree(stats);
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
/*
More vacuum renaming.
2000-05-29 03:46:00 +02:00
* tid_reaped() -- is a particular tid reaped?
Shrinking and other things.
1996-11-27 08:27:20 +01:00
*
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
* This has the right signature to be an IndexBulkDeleteCallback.
*
More vacuum cleanup
2000-05-29 19:06:15 +02:00
* vacpagelist->VacPage_array is sorted in right order.
Shrinking and other things.
1996-11-27 08:27:20 +01:00
*/
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
static bool
tid_reaped(ItemPointer itemptr, void *state)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
VacPageList vacpagelist = (VacPageList) state;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
OffsetNumber ioffno;
OffsetNumber *voff;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
VacPage vp,
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
*vpp;
More vacuum cleanup
2000-05-29 19:06:15 +02:00
VacPageData vacpage;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpage.blkno = ItemPointerGetBlockNumber(itemptr);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
ioffno = ItemPointerGetOffsetNumber(itemptr);
Shrinking and other things.
1996-11-27 08:27:20 +01:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vp = &vacpage;
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
vpp = (VacPage *) vac_bsearch((void *) &vp,
(void *) (vacpagelist->pagedesc),
vacpagelist->num_pages,
sizeof(VacPage),
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
vac_cmp_blk);
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
if (vpp == NULL)
return false;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
/* ok - we are on a partially or fully reaped page */
vp = *vpp;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (vp->offsets_free == 0)
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
{
/* this is EmptyPage, so claim all tuples on it are reaped!!! */
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
return true;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
voff = (OffsetNumber *) vac_bsearch((void *) &ioffno,
(void *) (vp->offsets),
vp->offsets_free,
sizeof(OffsetNumber),
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
vac_cmp_offno);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
if (voff == NULL)
return false;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
/* tid is reaped */
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
return true;
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Add attribute optimization statistics.
1997-02-07 17:24:12 +01:00
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
/*
* Update the shared Free Space Map with the info we now have about
* free space in the relation, discarding any old info the map may have.
*/
static void
vac_update_fsm(Relation onerel, VacPageList fraged_pages,
BlockNumber rel_pages)
{
int nPages = fraged_pages->num_pages;
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
VacPage *pagedesc = fraged_pages->pagedesc;
Size threshold;
Performance improvement for MultiRecordFreeSpace on large relations --- avoid O(N^2) behavior. Problem noted and fixed by Stephen Marshall <smarshall@wsicorp.com>, with some help from Tom Lane.
2002-09-20 21:56:01 +02:00
PageFreeSpaceInfo *pageSpaces;
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
int outPages;
int i;
/*
* We only report pages with free space at least equal to the average
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* request size --- this avoids cluttering FSM with uselessly-small bits
* of space. Although FSM would discard pages with little free space
* anyway, it's important to do this prefiltering because (a) it reduces
* the time spent holding the FSM lock in RecordRelationFreeSpace, and (b)
* FSM uses the number of pages reported as a statistic for guiding space
* management. If we didn't threshold our reports the same way
* vacuumlazy.c does, we'd be skewing that statistic.
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
*/
threshold = GetAvgFSMRequestSize(&onerel->rd_node);
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
Performance improvement for MultiRecordFreeSpace on large relations --- avoid O(N^2) behavior. Problem noted and fixed by Stephen Marshall <smarshall@wsicorp.com>, with some help from Tom Lane.
2002-09-20 21:56:01 +02:00
pageSpaces = (PageFreeSpaceInfo *)
Tweak palloc/repalloc to allow zero bytes to be requested, as per recent proposal. Eliminate several dozen now-unnecessary hacks to avoid palloc(0). (It's likely there are more that I didn't find.)
2004-06-05 21:48:09 +02:00
palloc(nPages * sizeof(PageFreeSpaceInfo));
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
outPages = 0;
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
for (i = 0; i < nPages; i++)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* fraged_pages may contain entries for pages that we later decided to
* truncate from the relation; don't enter them into the free space
* map!
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
*/
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
if (pagedesc[i]->blkno >= rel_pages)
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
break;
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
if (pagedesc[i]->free >= threshold)
{
pageSpaces[outPages].blkno = pagedesc[i]->blkno;
pageSpaces[outPages].avail = pagedesc[i]->free;
outPages++;
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
}
}
Reimplement free-space-map management as per recent discussions. Adjustable threshold is gone in favor of keeping track of total requested page storage and doling out proportional fractions to each relation (with a minimum amount per relation, and some quantization of the results to avoid thrashing with small changes in page counts). Provide special- case code for indexes so as not to waste space storing useless page free space counts. Restructure internal data storage to be a flat array instead of list-of-chunks; this may cost a little more work in data copying when reorganizing, but allows binary search to be used during lookup_fsm_page_entry().
2003-03-04 22:51:22 +01:00
RecordRelationFreeSpace(&onerel->rd_node, outPages, pageSpaces);
Performance improvement for MultiRecordFreeSpace on large relations --- avoid O(N^2) behavior. Problem noted and fixed by Stephen Marshall <smarshall@wsicorp.com>, with some help from Tom Lane.
2002-09-20 21:56:01 +02:00
pfree(pageSpaces);
First non-stub implementation of shared free space map. It's not super useful as yet, since its primary source of information is (full) VACUUM, which makes a concerted effort to get rid of free space before telling the map about it ... next stop is concurrent VACUUM ...
2001-07-02 22:50:46 +02:00
}
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/* Copy a VacPage structure */
static VacPage
copy_vac_page(VacPage vacpage)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
VacPage newvacpage;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
/* allocate a VacPageData entry */
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
newvacpage = (VacPage) palloc(sizeof(VacPageData) +
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
vacpage->offsets_free * sizeof(OffsetNumber));
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/* fill it in */
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (vacpage->offsets_free > 0)
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
memcpy(newvacpage->offsets, vacpage->offsets,
vacpage->offsets_free * sizeof(OffsetNumber));
More vacuum cleanup
2000-05-29 19:06:15 +02:00
newvacpage->blkno = vacpage->blkno;
newvacpage->free = vacpage->free;
newvacpage->offsets_used = vacpage->offsets_used;
newvacpage->offsets_free = vacpage->offsets_free;
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
return newvacpage;
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Fix VACUUM so that it can use pages as move targets even if they do not have any newly-dead tuples on them. This is a longstanding deficiency that prevents VACUUM from compacting a file as much as one would expect. Change requires fixing repair_frag to not assume that fraged_pages is a subset of vacuum_pages. Also make some further cleanups of places that assumed page numbers fit in int and tuple counts fit in uint32.
2001-06-29 22:14:27 +02:00
/*
* Add a VacPage pointer to a VacPageList.
*
* As a side effect of the way that scan_heap works,
* higher pages come after lower pages in the array
* (and highest tid on a page is last).
*/
More vacuum cleanup
2000-05-29 19:06:15 +02:00
static void
vpage_insert(VacPageList vacpagelist, VacPage vpnew)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Fix vacuum's memory consumption
1999-08-25 14:20:57 +02:00
#define PG_NPAGEDESC 1024
Shrinking and other things.
1996-11-27 08:27:20 +01:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
/* allocate a VacPage entry if needed */
if (vacpagelist->num_pages == 0)
Fix vacuum's memory consumption
1999-08-25 14:20:57 +02:00
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpagelist->pagedesc = (VacPage *) palloc(PG_NPAGEDESC * sizeof(VacPage));
vacpagelist->num_allocated_pages = PG_NPAGEDESC;
Fix vacuum's memory consumption
1999-08-25 14:20:57 +02:00
}
More vacuum cleanup
2000-05-29 19:06:15 +02:00
else if (vacpagelist->num_pages >= vacpagelist->num_allocated_pages)
Fix vacuum's memory consumption
1999-08-25 14:20:57 +02:00
{
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpagelist->num_allocated_pages *= 2;
vacpagelist->pagedesc = (VacPage *) repalloc(vacpagelist->pagedesc, vacpagelist->num_allocated_pages * sizeof(VacPage));
Fix vacuum's memory consumption
1999-08-25 14:20:57 +02:00
}
More vacuum cleanup
2000-05-29 19:06:15 +02:00
vacpagelist->pagedesc[vacpagelist->num_pages] = vpnew;
(vacpagelist->num_pages)++;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
/*
* vac_bsearch: just like standard C library routine bsearch(),
* except that we first test to see whether the target key is outside
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* the range of the table entries. This case is handled relatively slowly
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
* by the normal binary search algorithm (ie, no faster than any other key)
* but it occurs often enough in VACUUM to be worth optimizing.
*/
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
static void *
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
vac_bsearch(const void *key, const void *base,
size_t nelem, size_t size,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
int (*compar) (const void *, const void *))
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int res;
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
const void *last;
if (nelem == 0)
return NULL;
res = compar(key, base);
if (res < 0)
return NULL;
if (res == 0)
return (void *) base;
if (nelem > 1)
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
{
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
last = (const void *) ((const char *) base + (nelem - 1) * size);
res = compar(key, last);
if (res > 0)
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return NULL;
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
if (res == 0)
return (void *) last;
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
}
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
if (nelem <= 2)
return NULL; /* already checked 'em all */
return bsearch(key, base, nelem, size, compar);
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Replace poorly-coded vac_find_eq routine with call to standard bsearch library code. Tweak progress messages to include elapsed real time, not only CPU time.
2001-05-17 03:28:50 +02:00
/*
* Comparator routines for use with qsort() and bsearch().
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
static int
More vacuum renaming.
2000-05-29 03:46:00 +02:00
vac_cmp_blk(const void *left, const void *right)
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
BlockNumber lblk,
rblk;
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
lblk = (*((VacPage *) left))->blkno;
rblk = (*((VacPage *) right))->blkno;
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (lblk < rblk)
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return -1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (lblk == rblk)
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return 0;
return 1;
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
static int
More vacuum renaming.
2000-05-29 03:46:00 +02:00
vac_cmp_offno(const void *left, const void *right)
1. Reap deleted/unused tuples on page-base 2. Reap unused tuples too 3. Reap empty pages 4. Check if a page is initialized, initialize it if not and reap it 5. Binary search in list of reapped pages/tids to check is the heap' tid pointed by a index' tuple on this list (it's mu-u-uch faster)
1996-10-18 10:13:36 +02:00
{
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (*(OffsetNumber *) left < *(OffsetNumber *) right)
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return -1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (*(OffsetNumber *) left == *(OffsetNumber *) right)
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return 0;
return 1;
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
static int
More vacuum renaming.
2000-05-29 03:46:00 +02:00
vac_cmp_vtlinks(const void *left, const void *right)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (((VTupleLink) left)->new_tid.ip_blkid.bi_hi <
((VTupleLink) right)->new_tid.ip_blkid.bi_hi)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
return -1;
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (((VTupleLink) left)->new_tid.ip_blkid.bi_hi >
((VTupleLink) right)->new_tid.ip_blkid.bi_hi)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
return 1;
/* bi_hi-es are equal */
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (((VTupleLink) left)->new_tid.ip_blkid.bi_lo <
((VTupleLink) right)->new_tid.ip_blkid.bi_lo)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
return -1;
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (((VTupleLink) left)->new_tid.ip_blkid.bi_lo >
((VTupleLink) right)->new_tid.ip_blkid.bi_lo)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
return 1;
/* bi_lo-es are equal */
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (((VTupleLink) left)->new_tid.ip_posid <
((VTupleLink) right)->new_tid.ip_posid)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
return -1;
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (((VTupleLink) left)->new_tid.ip_posid >
((VTupleLink) right)->new_tid.ip_posid)
1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted.
1999-03-28 22:32:42 +02:00
return 1;
return 0;
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Fix performance problems with pg_index lookups (see, for example, discussion of 5/19/00). pg_index is now searched for indexes of a relation using an indexscan. Moreover, this is done once and cached in the relcache entry for the relation, in the form of a list of OIDs for the indexes. This list is used by the parser and executor to drive lookups in the pg_index syscache when they want to know the properties of the indexes. Net result: index information will be fully cached for repetitive operations such as inserts.
2000-06-17 23:49:04 +02:00
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
/*
* Open all the indexes of the given relation, obtaining the specified kind
* of lock on each. Return an array of Relation pointers for the indexes
* into *Irel, and the number of indexes into *nindexes.
*/
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
void
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
vac_open_indexes(Relation relation, LOCKMODE lockmode,
int *nindexes, Relation **Irel)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
List *indexoidlist;
ListCell *indexoidscan;
Fix performance problems with pg_index lookups (see, for example, discussion of 5/19/00). pg_index is now searched for indexes of a relation using an indexscan. Moreover, this is done once and cached in the relcache entry for the relation, in the form of a list of OIDs for the indexes. This list is used by the parser and executor to drive lookups in the pg_index syscache when they want to know the properties of the indexes. Net result: index information will be fully cached for repetitive operations such as inserts.
2000-06-17 23:49:04 +02:00
int i;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Fix performance problems with pg_index lookups (see, for example, discussion of 5/19/00). pg_index is now searched for indexes of a relation using an indexscan. Moreover, this is done once and cached in the relcache entry for the relation, in the form of a list of OIDs for the indexes. This list is used by the parser and executor to drive lookups in the pg_index syscache when they want to know the properties of the indexes. Net result: index information will be fully cached for repetitive operations such as inserts.
2000-06-17 23:49:04 +02:00
indexoidlist = RelationGetIndexList(relation);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
*nindexes = list_length(indexoidlist);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
if (*nindexes > 0)
*Irel = (Relation *) palloc(*nindexes * sizeof(Relation));
Fix performance problems with pg_index lookups (see, for example, discussion of 5/19/00). pg_index is now searched for indexes of a relation using an indexscan. Moreover, this is done once and cached in the relcache entry for the relation, in the form of a list of OIDs for the indexes. This list is used by the parser and executor to drive lookups in the pg_index syscache when they want to know the properties of the indexes. Net result: index information will be fully cached for repetitive operations such as inserts.
2000-06-17 23:49:04 +02:00
else
*Irel = NULL;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Fix performance problems with pg_index lookups (see, for example, discussion of 5/19/00). pg_index is now searched for indexes of a relation using an indexscan. Moreover, this is done once and cached in the relcache entry for the relation, in the form of a list of OIDs for the indexes. This list is used by the parser and executor to drive lookups in the pg_index syscache when they want to know the properties of the indexes. Net result: index information will be fully cached for repetitive operations such as inserts.
2000-06-17 23:49:04 +02:00
i = 0;
foreach(indexoidscan, indexoidlist)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
Oid indexoid = lfirst_oid(indexoidscan);
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
Relation ind;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
ind = index_open(indexoid);
(*Irel)[i++] = ind;
LockRelation(ind, lockmode);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
list_free(indexoidlist);
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Release the resources acquired by vac_open_indexes. Optionally release
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
* the locks (say NoLock to keep 'em).
*/
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
void
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
vac_close_indexes(int nindexes, Relation *Irel, LOCKMODE lockmode)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
if (Irel == NULL)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
return;
Shrinking and other things.
1996-11-27 08:27:20 +01:00
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
while (nindexes--)
Adjust index locking rules as per my proposal of earlier today. You now are supposed to take some kind of lock on an index whenever you are going to access the index contents, rather than relying only on a lock on the parent table.
2004-10-01 01:21:26 +02:00
{
Relation ind = Irel[nindexes];
if (lockmode != NoLock)
UnlockRelation(ind, lockmode);
index_close(ind);
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pfree(Irel);
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Shrinking and other things.
1996-11-27 08:27:20 +01:00
New-style vacuum neglected to update pg_class statistics about indexes if there were no deletions to do.
2001-07-18 02:46:25 +02:00
/*
* Is an index partial (ie, could it contain fewer tuples than the heap?)
*/
bool
vac_is_partial_index(Relation indrel)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If the index's AM doesn't support nulls, it's partial for our purposes
Back out unindented modification to file.
2004-10-07 16:19:58 +02:00
*/
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
if (!indrel->rd_am->amindexnulls)
Restructure index AM interface for index building and index tuple deletion, per previous discussion on pghackers. Most of the duplicate code in different AMs' ambuild routines has been moved out to a common routine in index.c; this means that all index types now do the right things about inserting recently-dead tuples, etc. (I also removed support for EXTEND INDEX in the ambuild routines, since that's about to go away anyway, and it cluttered the code a lot.) The retail indextuple deletion routines have been replaced by a "bulk delete" routine in which the indexscan is inside the access method. I haven't pushed this change as far as it should go yet, but it should allow considerable simplification of the internal bookkeeping for deletions. Also, add flag columns to pg_am to eliminate various hardcoded tests on AM OIDs, and remove unused pg_am columns. Fix rtree and gist index types to not attempt to store NULLs; before this, gist usually crashed, while rtree managed not to crash but computed wacko bounding boxes for NULL entries (which might have had something to do with the performance problems we've heard about occasionally). Add AtEOXact routines to hash, rtree, and gist, all of which have static state that needs to be reset after an error. We discovered this need long ago for btree, but missed the other guys. Oh, one more thing: concurrent VACUUM is now the default.
2001-07-16 00:48:19 +02:00
return true;
/* Otherwise, look to see if there's a partial-index predicate */
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
if (!heap_attisnull(indrel->rd_indextuple, Anum_pg_index_indpred))
return true;
return false;
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
static bool
More vacuum cleanup
2000-05-29 19:06:15 +02:00
enough_space(VacPage vacpage, Size len)
Shrinking and other things.
1996-11-27 08:27:20 +01:00
{
Install new alignment code to use MAXALIGN rather than DOUBLEALIGN where approproate.
1999-07-19 09:07:29 +02:00
len = MAXALIGN(len);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More vacuum cleanup
2000-05-29 19:06:15 +02:00
if (len > vacpage->free)
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return false;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
/* if there are free itemid(s) and len <= free_space... */
if (vacpage->offsets_used < vacpage->offsets_free)
return true;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment, because index locking issues are not handled correctly yet. Need to go work on the index AMs next.
2001-07-14 00:55:59 +02:00
/* noff_used >= noff_free and so we'll have to allocate new itemid */
if (len + sizeof(ItemIdData) <= vacpage->free)
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return true;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return false;
More vacuum renaming.
2000-05-29 03:46:00 +02:00
}
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
static Size
PageGetFreeSpaceWithFillFactor(Relation relation, Page page)
{
PageHeader pd = (PageHeader) page;
Size freespace = pd->pd_upper - pd->pd_lower;
Code review for FILLFACTOR patch. Change WITH grammar as per earlier discussion (including making def_arg allow reserved words), add missed opt_definition for UNIQUE case. Put the reloptions support code in a less random place (I chose to make a new file access/common/reloptions.c). Eliminate header inclusion creep. Make the index options functions safely user-callable (seems like client apps might like to be able to test validity of options before trying to make an index). Reduce overhead for normal case with no options by allowing rd_options to be NULL. Fix some unmaintainably klugy code, including getting rid of Natts_pg_class_fixed at long last. Some stylistic cleanup too, and pay attention to keeping comments in sync with code. Documentation still needs work, though I did fix the omissions in catalogs.sgml and indexam.sgml.
2006-07-04 00:45:41 +02:00
Size targetfree;
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
Code review for FILLFACTOR patch. Change WITH grammar as per earlier discussion (including making def_arg allow reserved words), add missed opt_definition for UNIQUE case. Put the reloptions support code in a less random place (I chose to make a new file access/common/reloptions.c). Eliminate header inclusion creep. Make the index options functions safely user-callable (seems like client apps might like to be able to test validity of options before trying to make an index). Reduce overhead for normal case with no options by allowing rd_options to be NULL. Fix some unmaintainably klugy code, including getting rid of Natts_pg_class_fixed at long last. Some stylistic cleanup too, and pay attention to keeping comments in sync with code. Documentation still needs work, though I did fix the omissions in catalogs.sgml and indexam.sgml.
2006-07-04 00:45:41 +02:00
targetfree = RelationGetTargetPageFreeSpace(relation,
HEAP_DEFAULT_FILLFACTOR);
if (freespace > targetfree)
return freespace - targetfree;
Add FILLFACTOR to CREATE INDEX. ITAGAKI Takahiro
2006-07-02 04:23:23 +02:00
else
return 0;
}
Repair subtle VACUUM bug that led to 'HEAP_MOVED_IN was not expected' errors. VACUUM normally compacts the table back-to-front, and stops as soon as it gets to a page that it has moved some tuples onto. (This logic doesn't make for a complete packing of the table, but it should be pretty close.) But the way it was checking whether it had got to a page with some moved-in tuples was to look at whether the current page was the same as the last page of the list of pages that have enough free space to be move-in targets. And there was other code that would remove pages from that list once they got full. There was a kluge that prevented the last list entry from being removed, but it didn't get the job done. Fixed by keeping a separate variable that contains the largest block number into which a tuple has been moved. There's no longer any need to protect the last element of the fraged_pages list. Also, fix NOTICE messages to describe elapsed user/system CPU time correctly.
2000-01-10 05:09:50 +01:00
Centralize implementation of delay code by creating a pg_usleep() subroutine in src/port/pgsleep.c. Remove platform dependencies from miscadmin.h and put them in port.h where they belong. Extend recent vacuum cost-based-delay patch to apply to VACUUM FULL, ANALYZE, and non-btree index vacuuming. By the way, where is the documentation for the cost-based-delay patch?
2004-02-10 04:42:45 +01:00
/*
* vacuum_delay_point --- check for interrupts and cost-based delay.
*
* This should be called in each major loop of VACUUM processing,
* typically once per page processed.
*/
void
vacuum_delay_point(void)
{
/* Always check for interrupts */
CHECK_FOR_INTERRUPTS();
/* Nap if appropriate */
if (VacuumCostActive && !InterruptPending &&
VacuumCostBalance >= VacuumCostLimit)
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
int msec;
Centralize implementation of delay code by creating a pg_usleep() subroutine in src/port/pgsleep.c. Remove platform dependencies from miscadmin.h and put them in port.h where they belong. Extend recent vacuum cost-based-delay patch to apply to VACUUM FULL, ANALYZE, and non-btree index vacuuming. By the way, where is the documentation for the cost-based-delay patch?
2004-02-10 04:42:45 +01:00
Rename vacuum_cost_naptime to vacuum_cost_delay, with agreement from Jan.
2004-08-06 06:15:09 +02:00
msec = VacuumCostDelay * VacuumCostBalance / VacuumCostLimit;
if (msec > VacuumCostDelay * 4)
msec = VacuumCostDelay * 4;
Centralize implementation of delay code by creating a pg_usleep() subroutine in src/port/pgsleep.c. Remove platform dependencies from miscadmin.h and put them in port.h where they belong. Extend recent vacuum cost-based-delay patch to apply to VACUUM FULL, ANALYZE, and non-btree index vacuuming. By the way, where is the documentation for the cost-based-delay patch?
2004-02-10 04:42:45 +01:00
pg_usleep(msec * 1000L);
VacuumCostBalance = 0;
/* Might have gotten an interrupt while sleeping */
CHECK_FOR_INTERRUPTS();
}
}