Improve vacuum code to track minimum Xids per table instead of per database.

To this end, add a couple of columns to pg_class, relminxid and relvacuumxid,
based on which we calculate the pg_database columns after each vacuum.

We now force all databases to be vacuumed, even template ones.  A backend
noticing too old a database (meaning pg_database.datminxid is in danger of
falling behind Xid wraparound) will signal the postmaster, which in turn will
start an autovacuum iteration to process the offending database.  In principle
this is only there to cope with frozen (non-connectable) databases without
forcing users to set them to connectable, but it could force regular user
database to go through a database-wide vacuum at any time.  Maybe we should
warn users about this somehow.  Of course the real solution will be to use
autovacuum all the time ;-)

There are some additional improvements we could have in this area: for example
the vacuum code could be smarter about not updating pg_database for each table
when called by autovacuum, and do it only once the whole autovacuum iteration
is done.

I updated the system catalogs documentation, but I didn't modify the
maintenance section.  Also having some regression tests for this would be nice
but it's not really a very straightforward thing to do.

Catalog version bumped due to system catalog changes.
This commit is contained in:
Alvaro Herrera 2006-07-10 16:20:52 +00:00
parent e627c9b9a6
commit d4cef0aa2a
19 changed files with 543 additions and 264 deletions

View File

@ -1,4 +1,4 @@
<!-- $PostgreSQL: pgsql/doc/src/sgml/catalogs.sgml,v 2.125 2006/07/03 22:45:36 tgl Exp $ -->
<!-- $PostgreSQL: pgsql/doc/src/sgml/catalogs.sgml,v 2.126 2006/07/10 16:20:49 alvherre Exp $ -->
<!--
Documentation of the system catalogs, directed toward PostgreSQL developers
-->
@ -1639,6 +1639,30 @@
<entry>True if table has (or once had) any inheritance children</entry>
</row>
<row>
<entry><structfield>relminxid</structfield></entry>
<entry><type>xid</type></entry>
<entry></entry>
<entry>
The minimum transaction ID present in all rows in this table. This
value is used to determine the database-global
<structname>pg_database</>.<structfield>datminxid</> value.
</entry>
</row>
<row>
<entry><structfield>relvacuumxid</structfield></entry>
<entry><type>xid</type></entry>
<entry></entry>
<entry>
The transaction ID that was used as cleaning point as of the last vacuum
operation. All rows inserted, updated or deleted in this table by
transactions whose IDs are below this one have been marked as known good
or deleted. This is used to determine the database-global
<structname>pg_database</>.<structfield>datvacuumxid</> value.
</entry>
</row>
<row>
<entry><structfield>relacl</structfield></entry>
<entry><type>aclitem[]</type></entry>
@ -2022,21 +2046,27 @@
<entry><type>xid</type></entry>
<entry></entry>
<entry>
All rows inserted or deleted by transaction IDs before this one
have been marked as known committed or known aborted in this database.
This is used to determine when commit-log space can be recycled.
The transaction ID that was used as cleaning point as of the last vacuum
operation. All rows inserted or deleted by transaction IDs before this one
have been marked as known good or deleted. This
is used to determine when commit-log space can be recycled.
If InvalidTransactionId, then the minimum is unknown and can be
determined by scanning <structname>pg_class</>.<structfield>relvacuumxid</>.
</entry>
</row>
<row>
<entry><structfield>datfrozenxid</structfield></entry>
<entry><structfield>datminxid</structfield></entry>
<entry><type>xid</type></entry>
<entry></entry>
<entry>
The minimum transaction ID present in all tables in this database.
All rows inserted by transaction IDs before this one have been
relabeled with a permanent (<quote>frozen</>) transaction ID in this
database. This is useful to check whether a database must be vacuumed
soon to avoid transaction ID wrap-around problems.
database. This is useful to check whether a database must be
vacuumed soon to avoid transaction ID wrap-around problems.
If InvalidTransactionId, then the minimum is unknown and can be
determined by scanning <structname>pg_class</>.<structfield>relminxid</>.
</entry>
</row>

View File

@ -6,7 +6,7 @@
* Copyright (c) 2000-2006, PostgreSQL Global Development Group
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/access/transam/varsup.c,v 1.70 2006/03/05 15:58:22 momjian Exp $
* $PostgreSQL: pgsql/src/backend/access/transam/varsup.c,v 1.71 2006/07/10 16:20:49 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -168,11 +168,11 @@ ReadNewTransactionId(void)
/*
* Determine the last safe XID to allocate given the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
* datminxid (ie, the oldest XID that might exist in any database
* of our cluster).
*/
void
SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
SetTransactionIdLimit(TransactionId oldest_datminxid,
Name oldest_datname)
{
TransactionId xidWarnLimit;
@ -180,16 +180,16 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
TransactionId xidWrapLimit;
TransactionId curXid;
Assert(TransactionIdIsValid(oldest_datfrozenxid));
Assert(TransactionIdIsValid(oldest_datminxid));
/*
* The place where we actually get into deep trouble is halfway around
* from the oldest potentially-existing XID. (This calculation is
* probably off by one or two counts, because the special XIDs reduce the
* size of the loop a little bit. But we throw in plenty of slop below,
* so it doesn't matter.)
* from the oldest existing XID. (This calculation is probably off by one
* or two counts, because the special XIDs reduce the size of the loop a
* little bit. But we throw in plenty of slop below, so it doesn't
* matter.)
*/
xidWrapLimit = oldest_datfrozenxid + (MaxTransactionId >> 1);
xidWrapLimit = oldest_datminxid + (MaxTransactionId >> 1);
if (xidWrapLimit < FirstNormalTransactionId)
xidWrapLimit += FirstNormalTransactionId;

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/catalog/heap.c,v 1.305 2006/07/08 20:45:38 alvherre Exp $
* $PostgreSQL: pgsql/src/backend/catalog/heap.c,v 1.306 2006/07/10 16:20:49 alvherre Exp $
*
*
* INTERFACE ROUTINES
@ -597,6 +597,8 @@ InsertPgClassTuple(Relation pg_class_desc,
values[Anum_pg_class_relhaspkey - 1] = BoolGetDatum(rd_rel->relhaspkey);
values[Anum_pg_class_relhasrules - 1] = BoolGetDatum(rd_rel->relhasrules);
values[Anum_pg_class_relhassubclass - 1] = BoolGetDatum(rd_rel->relhassubclass);
values[Anum_pg_class_relminxid - 1] = TransactionIdGetDatum(rd_rel->relminxid);
values[Anum_pg_class_relvacuumxid - 1] = TransactionIdGetDatum(rd_rel->relvacuumxid);
/* start out with empty permissions */
nulls[Anum_pg_class_relacl - 1] = 'n';
if (reloptions != (Datum) 0)
@ -644,6 +646,35 @@ AddNewRelationTuple(Relation pg_class_desc,
*/
new_rel_reltup = new_rel_desc->rd_rel;
/* Initialize relminxid and relvacuumxid */
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_TOASTVALUE)
{
/*
* Only real tables have Xids stored in them; initialize our known
* value to the minimum Xid that could put tuples in the new table.
*/
if (!IsBootstrapProcessingMode())
{
new_rel_reltup->relminxid = RecentXmin;
new_rel_reltup->relvacuumxid = RecentXmin;
}
else
{
new_rel_reltup->relminxid = FirstNormalTransactionId;
new_rel_reltup->relvacuumxid = FirstNormalTransactionId;
}
}
else
{
/*
* Other relations will not have Xids in them, so set the initial value
* to InvalidTransactionId.
*/
new_rel_reltup->relminxid = InvalidTransactionId;
new_rel_reltup->relvacuumxid = InvalidTransactionId;
}
switch (relkind)
{
case RELKIND_RELATION:

View File

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/commands/analyze.c,v 1.93 2006/03/23 00:19:28 tgl Exp $
* $PostgreSQL: pgsql/src/backend/commands/analyze.c,v 1.94 2006/07/10 16:20:50 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -424,8 +424,9 @@ analyze_rel(Oid relid, VacuumStmt *vacstmt)
{
vac_update_relstats(RelationGetRelid(onerel),
RelationGetNumberOfBlocks(onerel),
totalrows,
hasindex);
totalrows, hasindex,
InvalidTransactionId, InvalidTransactionId);
for (ind = 0; ind < nindexes; ind++)
{
AnlIndexData *thisdata = &indexdata[ind];
@ -434,8 +435,8 @@ analyze_rel(Oid relid, VacuumStmt *vacstmt)
totalindexrows = ceil(thisdata->tupleFract * totalrows);
vac_update_relstats(RelationGetRelid(Irel[ind]),
RelationGetNumberOfBlocks(Irel[ind]),
totalindexrows,
false);
totalindexrows, false,
InvalidTransactionId, InvalidTransactionId);
}
/* report results to the stats collector, too */

View File

@ -13,7 +13,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/commands/dbcommands.c,v 1.181 2006/05/04 16:07:29 tgl Exp $
* $PostgreSQL: pgsql/src/backend/commands/dbcommands.c,v 1.182 2006/07/10 16:20:50 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -55,7 +55,7 @@ static bool get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP,
Oid *dbLastSysOidP,
TransactionId *dbVacuumXidP, TransactionId *dbFrozenXidP,
TransactionId *dbVacuumXidP, TransactionId *dbMinXidP,
Oid *dbTablespace);
static bool have_createdb_privilege(void);
static void remove_dbtablespaces(Oid db_id);
@ -76,7 +76,7 @@ createdb(const CreatedbStmt *stmt)
bool src_allowconn;
Oid src_lastsysoid;
TransactionId src_vacuumxid;
TransactionId src_frozenxid;
TransactionId src_minxid;
Oid src_deftablespace;
volatile Oid dst_deftablespace;
Relation pg_database_rel;
@ -228,7 +228,7 @@ createdb(const CreatedbStmt *stmt)
if (!get_db_info(dbtemplate, ShareLock,
&src_dboid, &src_owner, &src_encoding,
&src_istemplate, &src_allowconn, &src_lastsysoid,
&src_vacuumxid, &src_frozenxid, &src_deftablespace))
&src_vacuumxid, &src_minxid, &src_deftablespace))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_DATABASE),
errmsg("template database \"%s\" does not exist",
@ -326,16 +326,6 @@ createdb(const CreatedbStmt *stmt)
/* Note there is no additional permission check in this path */
}
/*
* Normally we mark the new database with the same datvacuumxid and
* datfrozenxid as the source. However, if the source is not allowing
* connections then we assume it is fully frozen, and we can set the
* current transaction ID as the xid limits. This avoids immediately
* starting to generate warnings after cloning template0.
*/
if (!src_allowconn)
src_vacuumxid = src_frozenxid = GetCurrentTransactionId();
/*
* Check for db name conflict. This is just to give a more friendly
* error message than "unique index violation". There's a race condition
@ -367,7 +357,7 @@ createdb(const CreatedbStmt *stmt)
new_record[Anum_pg_database_datconnlimit - 1] = Int32GetDatum(dbconnlimit);
new_record[Anum_pg_database_datlastsysoid - 1] = ObjectIdGetDatum(src_lastsysoid);
new_record[Anum_pg_database_datvacuumxid - 1] = TransactionIdGetDatum(src_vacuumxid);
new_record[Anum_pg_database_datfrozenxid - 1] = TransactionIdGetDatum(src_frozenxid);
new_record[Anum_pg_database_datminxid - 1] = TransactionIdGetDatum(src_minxid);
new_record[Anum_pg_database_dattablespace - 1] = ObjectIdGetDatum(dst_deftablespace);
/*
@ -1066,7 +1056,7 @@ get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP,
Oid *dbLastSysOidP,
TransactionId *dbVacuumXidP, TransactionId *dbFrozenXidP,
TransactionId *dbVacuumXidP, TransactionId *dbMinXidP,
Oid *dbTablespace)
{
bool result = false;
@ -1155,9 +1145,9 @@ get_db_info(const char *name, LOCKMODE lockmode,
/* limit of vacuumed XIDs */
if (dbVacuumXidP)
*dbVacuumXidP = dbform->datvacuumxid;
/* limit of frozen XIDs */
if (dbFrozenXidP)
*dbFrozenXidP = dbform->datfrozenxid;
/* limit of min XIDs */
if (dbMinXidP)
*dbMinXidP = dbform->datminxid;
/* default tablespace for this database */
if (dbTablespace)
*dbTablespace = dbform->dattablespace;

View File

@ -13,7 +13,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/commands/vacuum.c,v 1.332 2006/07/03 22:45:38 tgl Exp $
* $PostgreSQL: pgsql/src/backend/commands/vacuum.c,v 1.333 2006/07/10 16:20:50 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -25,6 +25,7 @@
#include "access/clog.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@ -38,6 +39,7 @@
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
#include "storage/freespace.h"
#include "storage/pmsignal.h"
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "tcop/pquery.h"
@ -127,6 +129,7 @@ typedef struct VRelStats
Size min_tlen;
Size max_tlen;
bool hasindex;
TransactionId minxid; /* Minimum Xid present anywhere on table */
/* vtlinks array for tuple chain following - sorted by new_tid */
int num_vtlinks;
VTupleLink vtlinks;
@ -194,25 +197,22 @@ static MemoryContext vac_context = NULL;
static int elevel = -1;
static TransactionId OldestXmin;
static TransactionId FreezeLimit;
/* non-export function prototypes */
static List *get_rel_oids(List *relids, const RangeVar *vacrel,
const char *stmttype);
static void vac_update_dbstats(Oid dbid,
TransactionId vacuumXID,
TransactionId frozenXID);
static void vac_truncate_clog(TransactionId vacuumXID,
TransactionId frozenXID);
static bool vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind);
static void vac_update_dbminxid(Oid dbid,
TransactionId *minxid,
TransactionId *vacuumxid);
static void vac_truncate_clog(TransactionId myminxid, TransactionId myvacxid);
static void vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind);
static void full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt);
static void scan_heap(VRelStats *vacrelstats, Relation onerel,
VacPageList vacuum_pages, VacPageList fraged_pages);
VacPageList vacuum_pages, VacPageList fraged_pages,
TransactionId FreezeLimit, TransactionId OldestXmin);
static void repair_frag(VRelStats *vacrelstats, Relation onerel,
VacPageList vacuum_pages, VacPageList fraged_pages,
int nindexes, Relation *Irel);
int nindexes, Relation *Irel, TransactionId OldestXmin);
static void move_chain_tuple(Relation rel,
Buffer old_buf, Page old_page, HeapTuple old_tup,
Buffer dst_buf, Page dst_page, VacPage dst_vacpage,
@ -269,8 +269,6 @@ void
vacuum(VacuumStmt *vacstmt, List *relids)
{
const char *stmttype = vacstmt->vacuum ? "VACUUM" : "ANALYZE";
TransactionId initialOldestXmin = InvalidTransactionId;
TransactionId initialFreezeLimit = InvalidTransactionId;
volatile MemoryContext anl_context = NULL;
volatile bool all_rels,
in_outer_xact,
@ -353,32 +351,6 @@ vacuum(VacuumStmt *vacstmt, List *relids)
*/
relations = get_rel_oids(relids, vacstmt->relation, stmttype);
if (vacstmt->vacuum && all_rels)
{
/*
* It's a database-wide VACUUM.
*
* Compute the initially applicable OldestXmin and FreezeLimit XIDs,
* so that we can record these values at the end of the VACUUM. Note
* that individual tables may well be processed with newer values, but
* we can guarantee that no (non-shared) relations are processed with
* older ones.
*
* It is okay to record non-shared values in pg_database, even though
* we may vacuum shared relations with older cutoffs, because only the
* minimum of the values present in pg_database matters. We can be
* sure that shared relations have at some time been vacuumed with
* cutoffs no worse than the global minimum; for, if there is a
* backend in some other DB with xmin = OLDXMIN that's determining the
* cutoff with which we vacuum shared relations, it is not possible
* for that database to have a cutoff newer than OLDXMIN recorded in
* pg_database.
*/
vacuum_set_xid_limits(vacstmt, false,
&initialOldestXmin,
&initialFreezeLimit);
}
/*
* Decide whether we need to start/commit our own transactions.
*
@ -446,10 +418,8 @@ vacuum(VacuumStmt *vacstmt, List *relids)
Oid relid = lfirst_oid(cur);
if (vacstmt->vacuum)
{
if (!vacuum_rel(relid, vacstmt, RELKIND_RELATION))
all_rels = false; /* forget about updating dbstats */
}
vacuum_rel(relid, vacstmt, RELKIND_RELATION);
if (vacstmt->analyze)
{
MemoryContext old_context = NULL;
@ -525,6 +495,9 @@ vacuum(VacuumStmt *vacstmt, List *relids)
if (vacstmt->vacuum)
{
TransactionId minxid,
vacuumxid;
/*
* If it was a database-wide VACUUM, print FSM usage statistics (we
* don't make you be superuser to see these).
@ -532,17 +505,11 @@ vacuum(VacuumStmt *vacstmt, List *relids)
if (all_rels)
PrintFreeSpaceMapStatistics(elevel);
/*
* If we completed a database-wide VACUUM without skipping any
* relations, update the database's pg_database row with info about
* the transaction IDs used, and try to truncate pg_clog.
*/
if (all_rels)
{
vac_update_dbstats(MyDatabaseId,
initialOldestXmin, initialFreezeLimit);
vac_truncate_clog(initialOldestXmin, initialFreezeLimit);
}
/* Update pg_database.datminxid and datvacuumxid */
vac_update_dbminxid(MyDatabaseId, &minxid, &vacuumxid);
/* Try to truncate pg_clog. */
vac_truncate_clog(minxid, vacuumxid);
}
/*
@ -688,7 +655,8 @@ vacuum_set_xid_limits(VacuumStmt *vacstmt, bool sharedRel,
*/
void
vac_update_relstats(Oid relid, BlockNumber num_pages, double num_tuples,
bool hasindex)
bool hasindex, TransactionId minxid,
TransactionId vacuumxid)
{
Relation rd;
HeapTuple ctup;
@ -736,6 +704,16 @@ vac_update_relstats(Oid relid, BlockNumber num_pages, double num_tuples,
dirty = true;
}
}
if (TransactionIdIsValid(minxid) && pgcform->relminxid != minxid)
{
pgcform->relminxid = minxid;
dirty = true;
}
if (TransactionIdIsValid(vacuumxid) && pgcform->relvacuumxid != vacuumxid)
{
pgcform->relvacuumxid = vacuumxid;
dirty = true;
}
/*
* If anything changed, write out the tuple
@ -748,10 +726,13 @@ vac_update_relstats(Oid relid, BlockNumber num_pages, double num_tuples,
/*
* vac_update_dbstats() -- update statistics for one database
* vac_update_dbminxid() -- update the minimum Xid present in one database
*
* Update the whole-database statistics that are kept in its pg_database
* row, and the flat-file copy of pg_database.
* Update pg_database's datminxid and datvacuumxid, and the flat-file copy
* of it. datminxid is updated to the minimum of all relminxid found in
* pg_class. datvacuumxid is updated to the minimum of all relvacuumxid
* found in pg_class. The values are also returned in minxid and
* vacuumxid, respectively.
*
* We violate transaction semantics here by overwriting the database's
* existing pg_database tuple with the new values. This is reasonably
@ -759,18 +740,67 @@ vac_update_relstats(Oid relid, BlockNumber num_pages, double num_tuples,
* commits. As with vac_update_relstats, this avoids leaving dead tuples
* behind after a VACUUM.
*
* This routine is shared by full and lazy VACUUM. Note that it is only
* applied after a database-wide VACUUM operation.
* This routine is shared by full and lazy VACUUM.
*/
static void
vac_update_dbstats(Oid dbid,
TransactionId vacuumXID,
TransactionId frozenXID)
vac_update_dbminxid(Oid dbid, TransactionId *minxid, TransactionId *vacuumxid)
{
Relation relation;
HeapTuple tuple;
Form_pg_database dbform;
Relation relation;
SysScanDesc scan;
HeapTuple classTup;
TransactionId newMinXid = InvalidTransactionId;
TransactionId newVacXid = InvalidTransactionId;
bool dirty = false;
/*
* We must seqscan pg_class to find the minimum Xid, because there
* is no index that can help us here.
*/
relation = heap_open(RelationRelationId, AccessShareLock);
scan = systable_beginscan(relation, InvalidOid, false,
SnapshotNow, 0, NULL);
while ((classTup = systable_getnext(scan)) != NULL)
{
Form_pg_class classForm;
classForm = (Form_pg_class) GETSTRUCT(classTup);
/*
* Only consider heap and TOAST tables (anything else should have
* InvalidTransactionId in both fields anyway.)
*/
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_TOASTVALUE)
continue;
Assert(TransactionIdIsNormal(classForm->relminxid));
Assert(TransactionIdIsNormal(classForm->relvacuumxid));
/*
* Compute the minimum relminxid in all the tables in the database.
*/
if ((!TransactionIdIsValid(newMinXid) ||
TransactionIdPrecedes(classForm->relminxid, newMinXid)))
newMinXid = classForm->relminxid;
/* ditto, for relvacuumxid */
if ((!TransactionIdIsValid(newVacXid) ||
TransactionIdPrecedes(classForm->relvacuumxid, newVacXid)))
newVacXid = classForm->relvacuumxid;
}
/* we're done with pg_class */
systable_endscan(scan);
heap_close(relation, AccessShareLock);
Assert(TransactionIdIsNormal(newMinXid));
Assert(TransactionIdIsNormal(newVacXid));
/* Now fetch the pg_database tuple we need to update. */
relation = heap_open(DatabaseRelationId, RowExclusiveLock);
/* Fetch a copy of the tuple to scribble on */
@ -781,16 +811,29 @@ vac_update_dbstats(Oid dbid,
elog(ERROR, "could not find tuple for database %u", dbid);
dbform = (Form_pg_database) GETSTRUCT(tuple);
/* overwrite the existing statistics in the tuple */
dbform->datvacuumxid = vacuumXID;
dbform->datfrozenxid = frozenXID;
if (TransactionIdPrecedes(dbform->datminxid, newMinXid))
{
dbform->datminxid = newMinXid;
dirty = true;
}
if (TransactionIdPrecedes(dbform->datvacuumxid, newVacXid))
{
dbform->datvacuumxid = newVacXid;
dirty = true;
}
heap_inplace_update(relation, tuple);
if (dirty)
heap_inplace_update(relation, tuple);
heap_freetuple(tuple);
heap_close(relation, RowExclusiveLock);
/* set return values */
*minxid = newMinXid;
*vacuumxid = newVacXid;
/* Mark the flat-file copy of pg_database for update at commit */
database_file_update_needed();
database_file_update_needed();
}
@ -814,18 +857,22 @@ vac_update_dbstats(Oid dbid,
* applied after a database-wide VACUUM operation.
*/
static void
vac_truncate_clog(TransactionId vacuumXID, TransactionId frozenXID)
vac_truncate_clog(TransactionId myminxid, TransactionId myvacxid)
{
TransactionId myXID = GetCurrentTransactionId();
TransactionId minXID;
TransactionId vacuumXID;
Relation relation;
HeapScanDesc scan;
HeapTuple tuple;
int32 age;
NameData oldest_datname;
bool vacuumAlreadyWrapped = false;
bool frozenAlreadyWrapped = false;
bool minAlreadyWrapped = false;
/* init oldest_datname to sync with my frozenXID */
/* Initialize the minimum values. */
minXID = myminxid;
vacuumXID = myvacxid;
namestrcpy(&oldest_datname, get_database_name(MyDatabaseId));
/*
@ -840,27 +887,20 @@ vac_truncate_clog(TransactionId vacuumXID, TransactionId frozenXID)
{
Form_pg_database dbform = (Form_pg_database) GETSTRUCT(tuple);
/* Ignore non-connectable databases (eg, template0) */
/* It's assumed that these have been frozen correctly */
if (!dbform->datallowconn)
continue;
Assert(TransactionIdIsNormal(dbform->datvacuumxid));
Assert(TransactionIdIsNormal(dbform->datminxid));
if (TransactionIdIsNormal(dbform->datvacuumxid))
if (TransactionIdPrecedes(myXID, dbform->datvacuumxid))
vacuumAlreadyWrapped = true;
else if (TransactionIdPrecedes(dbform->datvacuumxid, vacuumXID))
vacuumXID = dbform->datvacuumxid;
if (TransactionIdPrecedes(myXID, dbform->datminxid))
minAlreadyWrapped = true;
else if (TransactionIdPrecedes(dbform->datminxid, minXID))
{
if (TransactionIdPrecedes(myXID, dbform->datvacuumxid))
vacuumAlreadyWrapped = true;
else if (TransactionIdPrecedes(dbform->datvacuumxid, vacuumXID))
vacuumXID = dbform->datvacuumxid;
}
if (TransactionIdIsNormal(dbform->datfrozenxid))
{
if (TransactionIdPrecedes(myXID, dbform->datfrozenxid))
frozenAlreadyWrapped = true;
else if (TransactionIdPrecedes(dbform->datfrozenxid, frozenXID))
{
frozenXID = dbform->datfrozenxid;
namecpy(&oldest_datname, &dbform->datname);
}
minXID = dbform->datminxid;
namecpy(&oldest_datname, &dbform->datname);
}
}
@ -887,7 +927,7 @@ vac_truncate_clog(TransactionId vacuumXID, TransactionId frozenXID)
* Do not update varsup.c if we seem to have suffered wraparound already;
* the computed XID might be bogus.
*/
if (frozenAlreadyWrapped)
if (minAlreadyWrapped)
{
ereport(WARNING,
(errmsg("some databases have not been vacuumed in over 1 billion transactions"),
@ -896,10 +936,10 @@ vac_truncate_clog(TransactionId vacuumXID, TransactionId frozenXID)
}
/* Update the wrap limit for GetNewTransactionId */
SetTransactionIdLimit(frozenXID, &oldest_datname);
SetTransactionIdLimit(minXID, &oldest_datname);
/* Give warning about impending wraparound problems */
age = (int32) (myXID - frozenXID);
age = (int32) (myXID - minXID);
if (age > (int32) ((MaxTransactionId >> 3) * 3))
ereport(WARNING,
(errmsg("database \"%s\" must be vacuumed within %u transactions",
@ -907,6 +947,28 @@ vac_truncate_clog(TransactionId vacuumXID, TransactionId frozenXID)
(MaxTransactionId >> 1) - age),
errhint("To avoid a database shutdown, execute a full-database VACUUM in \"%s\".",
NameStr(oldest_datname))));
/*
* Have the postmaster start an autovacuum iteration. If the user has
* autovacuum configured, this is not needed; otherwise, we need to make
* sure we have some mechanism to cope with transaction Id wraparound.
* Ideally this would only be needed for template databases, because all
* other databases should be kept nicely pruned by regular vacuuming.
*
* XXX -- the test we use here is fairly arbitrary. Note that in the
* autovacuum database-wide code, a template database is always processed
* with VACUUM FREEZE, so we can be sure that it will be truly frozen so
* it won't be need to be processed here again soon.
*
* FIXME -- here we could get into a kind of loop if the database being
* chosen is not actually a template database, because we'll not freeze
* it, so its age may not really decrease if there are any live
* non-freezable tuples. Consider forcing a vacuum freeze if autovacuum
* is invoked by a backend. On the other hand, forcing a vacuum freeze
* on a user database may not a be a very polite thing to do.
*/
if (!AutoVacuumingActive() && age > (int32) ((MaxTransactionId >> 3) * 3))
SendPostmasterSignal(PMSIGNAL_START_AUTOVAC);
}
@ -921,11 +983,6 @@ vac_truncate_clog(TransactionId vacuumXID, TransactionId frozenXID)
/*
* vacuum_rel() -- vacuum one heap relation
*
* Returns TRUE if we actually processed the relation (or can ignore it
* for some reason), FALSE if we failed to process it due to permissions
* or other reasons. (A FALSE result really means that some data
* may have been left unvacuumed, so we can't update XID stats.)
*
* Doing one heap at a time incurs extra overhead, since we need to
* check that the heap exists again just before we vacuum it. The
* reason that we do this is so that vacuuming can be spread across
@ -934,14 +991,13 @@ vac_truncate_clog(TransactionId vacuumXID, TransactionId frozenXID)
*
* At entry and exit, we are not inside a transaction.
*/
static bool
static void
vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
{
LOCKMODE lmode;
Relation onerel;
LockRelId onerelid;
Oid toast_relid;
bool result;
/* Begin a transaction for vacuuming this relation */
StartTransactionCommand();
@ -970,7 +1026,7 @@ vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
{
StrategyHintVacuum(false);
CommitTransactionCommand();
return true; /* okay 'cause no data there */
return;
}
/*
@ -1001,7 +1057,7 @@ vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
relation_close(onerel, lmode);
StrategyHintVacuum(false);
CommitTransactionCommand();
return false;
return;
}
/*
@ -1016,7 +1072,7 @@ vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
relation_close(onerel, lmode);
StrategyHintVacuum(false);
CommitTransactionCommand();
return false;
return;
}
/*
@ -1031,7 +1087,7 @@ vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
relation_close(onerel, lmode);
StrategyHintVacuum(false);
CommitTransactionCommand();
return true; /* assume no long-lived data in temp tables */
return; /* assume no long-lived data in temp tables */
}
/*
@ -1060,8 +1116,6 @@ vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
else
lazy_vacuum_rel(onerel, vacstmt);
result = true; /* did the vacuum */
/* all done with this class, but hold lock until commit */
relation_close(onerel, NoLock);
@ -1079,17 +1133,14 @@ vacuum_rel(Oid relid, VacuumStmt *vacstmt, char expected_relkind)
* totally unimportant for toast relations.
*/
if (toast_relid != InvalidOid)
{
if (!vacuum_rel(toast_relid, vacstmt, RELKIND_TOASTVALUE))
result = false; /* failed to vacuum the TOAST table? */
}
vacuum_rel(toast_relid, vacstmt, RELKIND_TOASTVALUE);
/*
* Now release the session-level lock on the master table.
*/
UnlockRelationForSession(&onerelid, lmode);
return result;
return;
}
@ -1121,6 +1172,8 @@ full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
int nindexes,
i;
VRelStats *vacrelstats;
TransactionId FreezeLimit,
OldestXmin;
vacuum_set_xid_limits(vacstmt, onerel->rd_rel->relisshared,
&OldestXmin, &FreezeLimit);
@ -1133,9 +1186,21 @@ full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
vacrelstats->rel_tuples = 0;
vacrelstats->hasindex = false;
/*
* Set initial minimum Xid, which will be updated if a smaller Xid is found
* in the relation by scan_heap.
*
* We use RecentXmin here (the minimum Xid that belongs to a transaction
* that is still open according to our snapshot), because it is the
* earliest transaction that could insert new tuples in the table after our
* VACUUM is done.
*/
vacrelstats->minxid = RecentXmin;
/* scan the heap */
vacuum_pages.num_pages = fraged_pages.num_pages = 0;
scan_heap(vacrelstats, onerel, &vacuum_pages, &fraged_pages);
scan_heap(vacrelstats, onerel, &vacuum_pages, &fraged_pages, FreezeLimit,
OldestXmin);
/* Now open all indexes of the relation */
vac_open_indexes(onerel, AccessExclusiveLock, &nindexes, &Irel);
@ -1163,7 +1228,7 @@ full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
{
/* Try to shrink heap */
repair_frag(vacrelstats, onerel, &vacuum_pages, &fraged_pages,
nindexes, Irel);
nindexes, Irel, OldestXmin);
vac_close_indexes(nindexes, Irel, NoLock);
}
else
@ -1181,7 +1246,8 @@ full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
/* update statistics in pg_class */
vac_update_relstats(RelationGetRelid(onerel), vacrelstats->rel_pages,
vacrelstats->rel_tuples, vacrelstats->hasindex);
vacrelstats->rel_tuples, vacrelstats->hasindex,
vacrelstats->minxid, OldestXmin);
/* report results to the stats collector, too */
pgstat_report_vacuum(RelationGetRelid(onerel), onerel->rd_rel->relisshared,
@ -1197,14 +1263,17 @@ full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
* deleted tuples), constructs fraged_pages (list of pages with free
* space that tuples could be moved into), and calculates statistics
* on the number of live tuples in the heap.
*
* It also updates the minimum Xid found anywhere on the table in
* vacrelstats->minxid, for later storing it in pg_class.relminxid.
*/
static void
scan_heap(VRelStats *vacrelstats, Relation onerel,
VacPageList vacuum_pages, VacPageList fraged_pages)
VacPageList vacuum_pages, VacPageList fraged_pages,
TransactionId FreezeLimit, TransactionId OldestXmin)
{
BlockNumber nblocks,
blkno;
HeapTupleData tuple;
char *relname;
VacPage vacpage;
BlockNumber empty_pages,
@ -1318,6 +1387,7 @@ scan_heap(VRelStats *vacrelstats, Relation onerel,
{
ItemId itemid = PageGetItemId(page, offnum);
bool tupgone = false;
HeapTupleData tuple;
/*
* Collect un-used items too - it's possible to have indexes
@ -1453,12 +1523,23 @@ scan_heap(VRelStats *vacrelstats, Relation onerel,
}
else
{
TransactionId min;
num_tuples += 1;
notup = false;
if (tuple.t_len < min_tlen)
min_tlen = tuple.t_len;
if (tuple.t_len > max_tlen)
max_tlen = tuple.t_len;
/*
* If the tuple is alive, we consider it for the "minxid"
* calculations.
*/
min = vactuple_get_minxid(&tuple);
if (TransactionIdIsValid(min) &&
TransactionIdPrecedes(min, vacrelstats->minxid))
vacrelstats->minxid = min;
}
} /* scan along page */
@ -1584,6 +1665,63 @@ scan_heap(VRelStats *vacrelstats, Relation onerel,
pg_rusage_show(&ru0))));
}
/*
* vactuple_get_minxid
*
* Get the minimum relevant Xid for a tuple, not considering FrozenXid.
* Return InvalidXid if none (i.e., xmin=FrozenXid, xmax=InvalidXid).
* This is for the purpose of calculating pg_class.relminxid for a table
* we're vacuuming.
*/
TransactionId
vactuple_get_minxid(HeapTuple tuple)
{
TransactionId min = InvalidTransactionId;
/*
* Initialize calculations with Xmin. NB -- may be FrozenXid and
* we don't want that one.
*/
if (TransactionIdIsNormal(HeapTupleHeaderGetXmin(tuple->t_data)))
min = HeapTupleHeaderGetXmin(tuple->t_data);
/*
* If Xmax is not marked INVALID, we assume it's valid without making
* further checks on it --- it must be recently obsoleted or still running,
* else HeapTupleSatisfiesVacuum would have deemed it removable.
*/
if (!(tuple->t_data->t_infomask | HEAP_XMAX_INVALID))
{
TransactionId xmax = HeapTupleHeaderGetXmax(tuple->t_data);
/* If xmax is a plain Xid, consider it by itself */
if (!(tuple->t_data->t_infomask | HEAP_XMAX_IS_MULTI))
{
if (!TransactionIdIsValid(min) ||
(TransactionIdIsNormal(xmax) &&
TransactionIdPrecedes(xmax, min)))
min = xmax;
}
else
{
/* If it's a MultiXactId, consider each of its members */
TransactionId *members;
int nmembers,
membno;
nmembers = GetMultiXactIdMembers(xmax, &members);
for (membno = 0; membno < nmembers; membno++)
{
if (!TransactionIdIsValid(min) ||
TransactionIdPrecedes(members[membno], min))
min = members[membno];
}
}
}
return min;
}
/*
* repair_frag() -- try to repair relation's fragmentation
@ -1598,7 +1736,7 @@ scan_heap(VRelStats *vacrelstats, Relation onerel,
static void
repair_frag(VRelStats *vacrelstats, Relation onerel,
VacPageList vacuum_pages, VacPageList fraged_pages,
int nindexes, Relation *Irel)
int nindexes, Relation *Irel, TransactionId OldestXmin)
{
TransactionId myXID = GetCurrentTransactionId();
Buffer dst_buffer = InvalidBuffer;
@ -2940,7 +3078,7 @@ scan_index(Relation indrel, double num_tuples)
/* now update statistics in pg_class */
vac_update_relstats(RelationGetRelid(indrel),
stats->num_pages, stats->num_index_tuples,
false);
false, InvalidTransactionId, InvalidTransactionId);
ereport(elevel,
(errmsg("index \"%s\" now contains %.0f row versions in %u pages",
@ -3009,7 +3147,7 @@ vacuum_index(VacPageList vacpagelist, Relation indrel,
/* now update statistics in pg_class */
vac_update_relstats(RelationGetRelid(indrel),
stats->num_pages, stats->num_index_tuples,
false);
false, InvalidTransactionId, InvalidTransactionId);
ereport(elevel,
(errmsg("index \"%s\" now contains %.0f row versions in %u pages",

View File

@ -31,7 +31,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/commands/vacuumlazy.c,v 1.72 2006/07/03 22:45:38 tgl Exp $
* $PostgreSQL: pgsql/src/backend/commands/vacuumlazy.c,v 1.73 2006/07/10 16:20:50 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -42,6 +42,7 @@
#include "access/genam.h"
#include "access/heapam.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
#include "commands/vacuum.h"
#include "miscadmin.h"
#include "pgstat.h"
@ -72,6 +73,7 @@ typedef struct LVRelStats
double tuples_deleted;
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
Size threshold; /* minimum interesting free space */
TransactionId minxid; /* minimum Xid present anywhere in table */
/* List of TIDs of tuples we intend to delete */
/* NB: this list is ordered by TID address */
int num_dead_tuples; /* current # of entries */
@ -88,13 +90,11 @@ typedef struct LVRelStats
static int elevel = -1;
static TransactionId OldestXmin;
static TransactionId FreezeLimit;
/* non-export function prototypes */
static void lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
Relation *Irel, int nindexes);
Relation *Irel, int nindexes, TransactionId FreezeLimit,
TransactionId OldestXmin);
static void lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats);
static void lazy_vacuum_index(Relation indrel,
IndexBulkDeleteResult **stats,
@ -104,9 +104,10 @@ static void lazy_cleanup_index(Relation indrel,
LVRelStats *vacrelstats);
static int lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int tupindex, LVRelStats *vacrelstats);
static void lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats);
static void lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats,
TransactionId OldestXmin);
static BlockNumber count_nondeletable_pages(Relation onerel,
LVRelStats *vacrelstats);
LVRelStats *vacrelstats, TransactionId OldestXmin);
static void lazy_space_alloc(LVRelStats *vacrelstats, BlockNumber relblocks);
static void lazy_record_dead_tuple(LVRelStats *vacrelstats,
ItemPointer itemptr);
@ -122,7 +123,8 @@ static int vac_cmp_page_spaces(const void *left, const void *right);
* lazy_vacuum_rel() -- perform LAZY VACUUM for one heap relation
*
* This routine vacuums a single heap, cleans out its indexes, and
* updates its num_pages and num_tuples statistics.
* updates its relpages and reltuples statistics, as well as the
* relminxid and relvacuumxid information.
*
* At entry, we have already established a transaction and opened
* and locked the relation.
@ -135,6 +137,8 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
int nindexes;
bool hasindex;
BlockNumber possibly_freeable;
TransactionId OldestXmin,
FreezeLimit;
if (vacstmt->verbose)
elevel = INFO;
@ -150,12 +154,23 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
/* XXX should we scale it up or down? Adjust vacuum.c too, if so */
vacrelstats->threshold = GetAvgFSMRequestSize(&onerel->rd_node);
/*
* Set initial minimum Xid, which will be updated if a smaller Xid is found
* in the relation by lazy_scan_heap.
*
* We use RecentXmin here (the minimum Xid that belongs to a transaction
* that is still open according to our snapshot), because it is the
* earliest transaction that could concurrently insert new tuples in the
* table.
*/
vacrelstats->minxid = RecentXmin;
/* Open all indexes of the relation */
vac_open_indexes(onerel, ShareUpdateExclusiveLock, &nindexes, &Irel);
hasindex = (nindexes > 0);
/* Do the vacuuming */
lazy_scan_heap(onerel, vacrelstats, Irel, nindexes);
lazy_scan_heap(onerel, vacrelstats, Irel, nindexes, FreezeLimit, OldestXmin);
/* Done with indexes */
vac_close_indexes(nindexes, Irel, NoLock);
@ -169,7 +184,7 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
possibly_freeable = vacrelstats->rel_pages - vacrelstats->nonempty_pages;
if (possibly_freeable >= REL_TRUNCATE_MINIMUM ||
possibly_freeable >= vacrelstats->rel_pages / REL_TRUNCATE_FRACTION)
lazy_truncate_heap(onerel, vacrelstats);
lazy_truncate_heap(onerel, vacrelstats, OldestXmin);
/* Update shared free space map with final free space info */
lazy_update_fsm(onerel, vacrelstats);
@ -178,7 +193,8 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
vac_update_relstats(RelationGetRelid(onerel),
vacrelstats->rel_pages,
vacrelstats->rel_tuples,
hasindex);
hasindex,
vacrelstats->minxid, OldestXmin);
/* report results to the stats collector, too */
pgstat_report_vacuum(RelationGetRelid(onerel), onerel->rd_rel->relisshared,
@ -193,10 +209,14 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
* and pages with free space, and calculates statistics on the number
* of live tuples in the heap. When done, or when we run low on space
* for dead-tuple TIDs, invoke vacuuming of indexes and heap.
*
* It also updates the minimum Xid found anywhere on the table in
* vacrelstats->minxid, for later storing it in pg_class.relminxid.
*/
static void
lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
Relation *Irel, int nindexes)
Relation *Irel, int nindexes, TransactionId FreezeLimit,
TransactionId OldestXmin)
{
BlockNumber nblocks,
blkno;
@ -406,8 +426,19 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
}
else
{
TransactionId min;
num_tuples += 1;
hastup = true;
/*
* If the tuple is alive, we consider it for the "minxid"
* calculations.
*/
min = vactuple_get_minxid(&tuple);
if (TransactionIdIsValid(min) &&
TransactionIdPrecedes(min, vacrelstats->minxid))
vacrelstats->minxid = min;
}
} /* scan along page */
@ -670,7 +701,7 @@ lazy_cleanup_index(Relation indrel,
vac_update_relstats(RelationGetRelid(indrel),
stats->num_pages,
stats->num_index_tuples,
false);
false, InvalidTransactionId, InvalidTransactionId);
ereport(elevel,
(errmsg("index \"%s\" now contains %.0f row versions in %u pages",
@ -691,7 +722,8 @@ lazy_cleanup_index(Relation indrel,
* lazy_truncate_heap - try to truncate off any empty pages at the end
*/
static void
lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats,
TransactionId OldestXmin)
{
BlockNumber old_rel_pages = vacrelstats->rel_pages;
BlockNumber new_rel_pages;
@ -732,7 +764,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
* because other backends could have added tuples to these pages whilst we
* were vacuuming.
*/
new_rel_pages = count_nondeletable_pages(onerel, vacrelstats);
new_rel_pages = count_nondeletable_pages(onerel, vacrelstats, OldestXmin);
if (new_rel_pages >= old_rel_pages)
{
@ -787,7 +819,8 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
* Returns number of nondeletable pages (last nonempty page + 1).
*/
static BlockNumber
count_nondeletable_pages(Relation onerel, LVRelStats *vacrelstats)
count_nondeletable_pages(Relation onerel, LVRelStats *vacrelstats,
TransactionId OldestXmin)
{
BlockNumber blkno;
HeapTupleData tuple;

View File

@ -10,7 +10,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/libpq/hba.c,v 1.152 2006/06/20 19:56:52 tgl Exp $
* $PostgreSQL: pgsql/src/backend/libpq/hba.c,v 1.153 2006/07/10 16:20:50 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -1004,7 +1004,7 @@ load_hba(void)
* dbname: gets database name (must be of size NAMEDATALEN bytes)
* dboid: gets database OID
* dbtablespace: gets database's default tablespace's OID
* dbfrozenxid: gets database's frozen XID
* dbminxid: gets database's minimum XID
* dbvacuumxid: gets database's vacuum XID
*
* This is not much related to the other functions in hba.c, but we put it
@ -1012,7 +1012,7 @@ load_hba(void)
*/
bool
read_pg_database_line(FILE *fp, char *dbname, Oid *dboid,
Oid *dbtablespace, TransactionId *dbfrozenxid,
Oid *dbtablespace, TransactionId *dbminxid,
TransactionId *dbvacuumxid)
{
char buf[MAX_TOKEN];
@ -1035,7 +1035,7 @@ read_pg_database_line(FILE *fp, char *dbname, Oid *dboid,
next_token(fp, buf, sizeof(buf));
if (!isdigit((unsigned char) buf[0]))
elog(FATAL, "bad data in flat pg_database file");
*dbfrozenxid = atoxid(buf);
*dbminxid = atoxid(buf);
next_token(fp, buf, sizeof(buf));
if (!isdigit((unsigned char) buf[0]))
elog(FATAL, "bad data in flat pg_database file");

View File

@ -10,7 +10,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/postmaster/autovacuum.c,v 1.21 2006/06/27 22:16:43 momjian Exp $
* $PostgreSQL: pgsql/src/backend/postmaster/autovacuum.c,v 1.22 2006/07/10 16:20:50 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -79,7 +79,7 @@ typedef struct autovac_dbase
{
Oid oid;
char *name;
TransactionId frozenxid;
TransactionId minxid;
TransactionId vacuumxid;
PgStat_StatDBEntry *entry;
int32 age;
@ -126,10 +126,6 @@ autovac_start(void)
time_t curtime;
pid_t AutoVacPID;
/* Do nothing if no autovacuum process needed */
if (!AutoVacuumingActive())
return 0;
/*
* Do nothing if too soon since last autovacuum exit. This limits how
* often the daemon runs. Since the time per iteration can be quite
@ -144,6 +140,7 @@ autovac_start(void)
*
* XXX todo: implement sleep scale factor that existed in contrib code.
*/
curtime = time(NULL);
if ((unsigned int) (curtime - last_autovac_stop_time) <
(unsigned int) autovacuum_naptime)
@ -334,6 +331,14 @@ AutoVacMain(int argc, char *argv[])
* connected to it since the stats were last initialized, it doesn't need
* vacuuming.
*
* Note that if we are called when autovacuum is nominally disabled in
* postgresql.conf, we assume the postmaster has invoked us because a
* database is in danger of Xid wraparound. In that case, we only
* consider vacuuming whole databases, not individual tables; and we pick
* the oldest one, regardless of it's true age. So the criteria for
* deciding that a database needs a database-wide vacuum is elsewhere
* (currently in vac_truncate_clog).
*
* XXX This could be improved if we had more info about whether it needs
* vacuuming before connecting to it. Perhaps look through the pgstats
* data for the database's tables? One idea is to keep track of the
@ -344,13 +349,8 @@ AutoVacMain(int argc, char *argv[])
db = NULL;
whole_db = false;
foreach(cell, dblist)
if (AutoVacuumingActive())
{
autovac_dbase *tmp = lfirst(cell);
bool this_whole_db;
int32 freeze_age,
vacuum_age;
/*
* We look for the database that most urgently needs a database-wide
* vacuum. We decide that a database-wide vacuum is needed 100000
@ -361,38 +361,70 @@ AutoVacMain(int argc, char *argv[])
* Unlike vacuum.c, we also look at vacuumxid. This is so that
* pg_clog can be kept trimmed to a reasonable size.
*/
freeze_age = (int32) (nextXid - tmp->frozenxid);
vacuum_age = (int32) (nextXid - tmp->vacuumxid);
tmp->age = Max(freeze_age, vacuum_age);
this_whole_db = (tmp->age >
(int32) ((MaxTransactionId >> 3) * 3 - 100000));
if (whole_db || this_whole_db)
foreach(cell, dblist)
{
if (!this_whole_db)
continue;
if (db == NULL || tmp->age > db->age)
autovac_dbase *tmp = lfirst(cell);
bool this_whole_db;
int32 true_age,
vacuum_age;
true_age = (int32) (nextXid - tmp->minxid);
vacuum_age = (int32) (nextXid - tmp->vacuumxid);
tmp->age = Max(true_age, vacuum_age);
this_whole_db = (tmp->age >
(int32) ((MaxTransactionId >> 3) * 3 - 100000));
if (whole_db || this_whole_db)
{
db = tmp;
whole_db = true;
if (!this_whole_db)
continue;
if (db == NULL || tmp->age > db->age)
{
db = tmp;
whole_db = true;
}
continue;
}
continue;
/*
* Otherwise, skip a database with no pgstat entry; it means it hasn't
* seen any activity.
*/
tmp->entry = pgstat_fetch_stat_dbentry(tmp->oid);
if (!tmp->entry)
continue;
/*
* Remember the db with oldest autovac time.
*/
if (db == NULL ||
tmp->entry->last_autovac_time < db->entry->last_autovac_time)
db = tmp;
}
}
else
{
/*
* Otherwise, skip a database with no pgstat entry; it means it hasn't
* seen any activity.
* If autovacuuming is not active, we must have gotten here because a
* backend signalled the postmaster. Pick up the database with the
* greatest age, and apply a database-wide vacuum on it.
*/
tmp->entry = pgstat_fetch_stat_dbentry(tmp->oid);
if (!tmp->entry)
continue;
int32 oldest = 0;
/*
* Remember the db with oldest autovac time.
*/
if (db == NULL ||
tmp->entry->last_autovac_time < db->entry->last_autovac_time)
db = tmp;
whole_db = true;
foreach(cell, dblist)
{
autovac_dbase *tmp = lfirst(cell);
int32 age = (int32) (nextXid - tmp->minxid);
if (age > oldest)
{
oldest = age;
db = tmp;
}
}
Assert(db);
}
if (db)
@ -454,7 +486,7 @@ autovac_get_database_list(void)
FILE *db_file;
Oid db_id;
Oid db_tablespace;
TransactionId db_frozenxid;
TransactionId db_minxid;
TransactionId db_vacuumxid;
filename = database_getflatfilename();
@ -465,7 +497,7 @@ autovac_get_database_list(void)
errmsg("could not open file \"%s\": %m", filename)));
while (read_pg_database_line(db_file, thisname, &db_id,
&db_tablespace, &db_frozenxid,
&db_tablespace, &db_minxid,
&db_vacuumxid))
{
autovac_dbase *db;
@ -474,7 +506,7 @@ autovac_get_database_list(void)
db->oid = db_id;
db->name = pstrdup(thisname);
db->frozenxid = db_frozenxid;
db->minxid = db_minxid;
db->vacuumxid = db_vacuumxid;
/* these get set later: */
db->entry = NULL;

View File

@ -37,7 +37,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/postmaster/postmaster.c,v 1.490 2006/06/29 20:00:08 tgl Exp $
* $PostgreSQL: pgsql/src/backend/postmaster/postmaster.c,v 1.491 2006/07/10 16:20:51 alvherre Exp $
*
* NOTES
*
@ -3417,6 +3417,16 @@ sigusr1_handler(SIGNAL_ARGS)
kill(SysLoggerPID, SIGUSR1);
}
if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC))
{
/* start one iteration of the autovacuum daemon */
if (Shutdown == NoShutdown)
{
Assert(!AutoVacuumingActive());
AutoVacPID = autovac_start();
}
}
PG_SETMASK(&UnBlockSig);
errno = save_errno;

View File

@ -23,7 +23,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/backend/utils/init/flatfiles.c,v 1.18 2006/05/04 16:07:29 tgl Exp $
* $PostgreSQL: pgsql/src/backend/utils/init/flatfiles.c,v 1.19 2006/07/10 16:20:51 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -163,7 +163,7 @@ name_okay(const char *str)
/*
* write_database_file: update the flat database file
*
* A side effect is to determine the oldest database's datfrozenxid
* A side effect is to determine the oldest database's datminxid
* so we can set or update the XID wrap limit.
*/
static void
@ -177,7 +177,7 @@ write_database_file(Relation drel)
HeapScanDesc scan;
HeapTuple tuple;
NameData oldest_datname;
TransactionId oldest_datfrozenxid = InvalidTransactionId;
TransactionId oldest_datminxid = InvalidTransactionId;
/*
* Create a temporary filename to be renamed later. This prevents the
@ -208,27 +208,27 @@ write_database_file(Relation drel)
char *datname;
Oid datoid;
Oid dattablespace;
TransactionId datfrozenxid,
TransactionId datminxid,
datvacuumxid;
datname = NameStr(dbform->datname);
datoid = HeapTupleGetOid(tuple);
dattablespace = dbform->dattablespace;
datfrozenxid = dbform->datfrozenxid;
datminxid = dbform->datminxid;
datvacuumxid = dbform->datvacuumxid;
/*
* Identify the oldest datfrozenxid, ignoring databases that are not
* Identify the oldest datminxid, ignoring databases that are not
* connectable (we assume they are safely frozen). This must match
* the logic in vac_truncate_clog() in vacuum.c.
*/
if (dbform->datallowconn &&
TransactionIdIsNormal(datfrozenxid))
TransactionIdIsNormal(datminxid))
{
if (oldest_datfrozenxid == InvalidTransactionId ||
TransactionIdPrecedes(datfrozenxid, oldest_datfrozenxid))
if (oldest_datminxid == InvalidTransactionId ||
TransactionIdPrecedes(datminxid, oldest_datminxid))
{
oldest_datfrozenxid = datfrozenxid;
oldest_datminxid = datminxid;
namestrcpy(&oldest_datname, datname);
}
}
@ -244,14 +244,14 @@ write_database_file(Relation drel)
}
/*
* The file format is: "dbname" oid tablespace frozenxid vacuumxid
* The file format is: "dbname" oid tablespace minxid vacuumxid
*
* The xids are not needed for backend startup, but are of use to
* autovacuum, and might also be helpful for forensic purposes.
*/
fputs_quote(datname, fp);
fprintf(fp, " %u %u %u %u\n",
datoid, dattablespace, datfrozenxid, datvacuumxid);
datoid, dattablespace, datminxid, datvacuumxid);
}
heap_endscan(scan);
@ -272,10 +272,10 @@ write_database_file(Relation drel)
tempname, filename)));
/*
* Set the transaction ID wrap limit using the oldest datfrozenxid
* Set the transaction ID wrap limit using the oldest datminxid
*/
if (oldest_datfrozenxid != InvalidTransactionId)
SetTransactionIdLimit(oldest_datfrozenxid, &oldest_datname);
if (oldest_datminxid != InvalidTransactionId)
SetTransactionIdLimit(oldest_datminxid, &oldest_datname);
}

View File

@ -7,7 +7,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/access/transam.h,v 1.57 2006/03/05 15:58:53 momjian Exp $
* $PostgreSQL: pgsql/src/include/access/transam.h,v 1.58 2006/07/10 16:20:51 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -23,6 +23,7 @@
* always be considered valid.
*
* FirstNormalTransactionId is the first "normal" transaction id.
* Note: if you need to change it, you must change it in pg_class.h as well.
* ----------------
*/
#define InvalidTransactionId ((TransactionId) 0)
@ -123,7 +124,7 @@ extern bool TransactionIdFollowsOrEquals(TransactionId id1, TransactionId id2);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
extern void SetTransactionIdLimit(TransactionId oldest_datminxid,
Name oldest_datname);
extern Oid GetNewObjectId(void);

View File

@ -37,7 +37,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/catalog/catversion.h,v 1.336 2006/07/03 22:45:40 tgl Exp $
* $PostgreSQL: pgsql/src/include/catalog/catversion.h,v 1.337 2006/07/10 16:20:51 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -53,6 +53,6 @@
*/
/* yyyymmddN */
#define CATALOG_VERSION_NO 200607021
#define CATALOG_VERSION_NO 200607101
#endif

View File

@ -8,7 +8,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/catalog/pg_attribute.h,v 1.122 2006/07/03 22:45:40 tgl Exp $
* $PostgreSQL: pgsql/src/include/catalog/pg_attribute.h,v 1.123 2006/07/10 16:20:51 alvherre Exp $
*
* NOTES
* the genbki.sh script reads this file and generates .bki
@ -404,8 +404,10 @@ DATA(insert ( 1249 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
{ 1259, {"relhaspkey"}, 16, -1, 1, 22, 0, -1, -1, true, 'p', 'c', true, false, false, true, 0 }, \
{ 1259, {"relhasrules"}, 16, -1, 1, 23, 0, -1, -1, true, 'p', 'c', true, false, false, true, 0 }, \
{ 1259, {"relhassubclass"},16, -1, 1, 24, 0, -1, -1, true, 'p', 'c', true, false, false, true, 0 }, \
{ 1259, {"relacl"}, 1034, -1, -1, 25, 1, -1, -1, false, 'x', 'i', false, false, false, true, 0 }, \
{ 1259, {"reloptions"}, 1009, -1, -1, 26, 1, -1, -1, false, 'x', 'i', false, false, false, true, 0 }
{ 1259, {"relminxid"}, 28, -1, 4, 25, 0, -1, -1, true, 'p', 'i', true, false, false, true, 0 }, \
{ 1259, {"relvacuumxid"}, 28, -1, 4, 26, 0, -1, -1, true, 'p', 'i', true, false, false, true, 0 }, \
{ 1259, {"relacl"}, 1034, -1, -1, 27, 1, -1, -1, false, 'x', 'i', false, false, false, true, 0 }, \
{ 1259, {"reloptions"}, 1009, -1, -1, 28, 1, -1, -1, false, 'x', 'i', false, false, false, true, 0 }
DATA(insert ( 1259 relname 19 -1 NAMEDATALEN 1 0 -1 -1 f p i t f f t 0));
DATA(insert ( 1259 relnamespace 26 -1 4 2 0 -1 -1 t p i t f f t 0));
@ -431,8 +433,10 @@ DATA(insert ( 1259 relhasoids 16 -1 1 21 0 -1 -1 t p c t f f t 0));
DATA(insert ( 1259 relhaspkey 16 -1 1 22 0 -1 -1 t p c t f f t 0));
DATA(insert ( 1259 relhasrules 16 -1 1 23 0 -1 -1 t p c t f f t 0));
DATA(insert ( 1259 relhassubclass 16 -1 1 24 0 -1 -1 t p c t f f t 0));
DATA(insert ( 1259 relacl 1034 -1 -1 25 1 -1 -1 f x i f f f t 0));
DATA(insert ( 1259 reloptions 1009 -1 -1 26 1 -1 -1 f x i f f f t 0));
DATA(insert ( 1259 relminxid 28 -1 4 25 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1259 relvacuumxid 28 -1 4 26 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1259 relacl 1034 -1 -1 27 1 -1 -1 f x i f f f t 0));
DATA(insert ( 1259 reloptions 1009 -1 -1 28 1 -1 -1 f x i f f f t 0));
DATA(insert ( 1259 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
DATA(insert ( 1259 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1259 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));

View File

@ -8,7 +8,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/catalog/pg_class.h,v 1.94 2006/07/03 22:45:40 tgl Exp $
* $PostgreSQL: pgsql/src/include/catalog/pg_class.h,v 1.95 2006/07/10 16:20:51 alvherre Exp $
*
* NOTES
* the genbki.sh script reads this file and generates .bki
@ -65,6 +65,8 @@ CATALOG(pg_class,1259) BKI_BOOTSTRAP
bool relhaspkey; /* has PRIMARY KEY index */
bool relhasrules; /* has associated rules */
bool relhassubclass; /* has derived classes */
TransactionId relminxid; /* minimum Xid present in table */
TransactionId relvacuumxid; /* Xid used as last vacuum OldestXmin */
/*
* VARIABLE LENGTH FIELDS start here. These fields may be NULL, too.
@ -78,7 +80,7 @@ CATALOG(pg_class,1259) BKI_BOOTSTRAP
/* Size of fixed part of pg_class tuples, not counting var-length fields */
#define CLASS_TUPLE_SIZE \
(offsetof(FormData_pg_class,relhassubclass) + sizeof(bool))
(offsetof(FormData_pg_class,relvacuumxid) + sizeof(TransactionId))
/* ----------------
* Form_pg_class corresponds to a pointer to a tuple with
@ -92,7 +94,7 @@ typedef FormData_pg_class *Form_pg_class;
* ----------------
*/
#define Natts_pg_class 26
#define Natts_pg_class 28
#define Anum_pg_class_relname 1
#define Anum_pg_class_relnamespace 2
#define Anum_pg_class_reltype 3
@ -117,8 +119,10 @@ typedef FormData_pg_class *Form_pg_class;
#define Anum_pg_class_relhaspkey 22
#define Anum_pg_class_relhasrules 23
#define Anum_pg_class_relhassubclass 24
#define Anum_pg_class_relacl 25
#define Anum_pg_class_reloptions 26
#define Anum_pg_class_relminxid 25
#define Anum_pg_class_relvacuumxid 26
#define Anum_pg_class_relacl 27
#define Anum_pg_class_reloptions 28
/* ----------------
* initial contents of pg_class
@ -128,13 +132,14 @@ typedef FormData_pg_class *Form_pg_class;
* ----------------
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 PGUID 0 1247 0 0 0 0 0 f f r 23 0 0 0 0 0 t f f f _null_ _null_ ));
/* Note: the "3" here stands for FirstNormalTransactionId */
DATA(insert OID = 1247 ( pg_type PGNSP 71 PGUID 0 1247 0 0 0 0 0 f f r 23 0 0 0 0 0 t f f f 3 3 _null_ _null_ ));
DESCR("");
DATA(insert OID = 1249 ( pg_attribute PGNSP 75 PGUID 0 1249 0 0 0 0 0 f f r 17 0 0 0 0 0 f f f f _null_ _null_ ));
DATA(insert OID = 1249 ( pg_attribute PGNSP 75 PGUID 0 1249 0 0 0 0 0 f f r 17 0 0 0 0 0 f f f f 3 3 _null_ _null_ ));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 PGUID 0 1255 0 0 0 0 0 f f r 18 0 0 0 0 0 t f f f _null_ _null_ ));
DATA(insert OID = 1255 ( pg_proc PGNSP 81 PGUID 0 1255 0 0 0 0 0 f f r 18 0 0 0 0 0 t f f f 3 3 _null_ _null_ ));
DESCR("");
DATA(insert OID = 1259 ( pg_class PGNSP 83 PGUID 0 1259 0 0 0 0 0 f f r 26 0 0 0 0 0 t f f f _null_ _null_ ));
DATA(insert OID = 1259 ( pg_class PGNSP 83 PGUID 0 1259 0 0 0 0 0 f f r 28 0 0 0 0 0 t f f f 3 3 _null_ _null_ ));
DESCR("");
#define RELKIND_INDEX 'i' /* secondary index */

View File

@ -8,7 +8,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/catalog/pg_database.h,v 1.40 2006/03/05 15:58:54 momjian Exp $
* $PostgreSQL: pgsql/src/include/catalog/pg_database.h,v 1.41 2006/07/10 16:20:51 alvherre Exp $
*
* NOTES
* the genbki.sh script reads this file and generates .bki
@ -43,7 +43,7 @@ CATALOG(pg_database,1262) BKI_SHARED_RELATION
int4 datconnlimit; /* max connections allowed (-1=no limit) */
Oid datlastsysoid; /* highest OID to consider a system OID */
TransactionId datvacuumxid; /* all XIDs before this are vacuumed */
TransactionId datfrozenxid; /* all XIDs before this are frozen */
TransactionId datminxid; /* minimum XID present anywhere in the DB */
Oid dattablespace; /* default table space for this DB */
text datconfig[1]; /* database-specific GUC (VAR LENGTH) */
aclitem datacl[1]; /* access permissions (VAR LENGTH) */
@ -69,7 +69,7 @@ typedef FormData_pg_database *Form_pg_database;
#define Anum_pg_database_datconnlimit 6
#define Anum_pg_database_datlastsysoid 7
#define Anum_pg_database_datvacuumxid 8
#define Anum_pg_database_datfrozenxid 9
#define Anum_pg_database_datminxid 9
#define Anum_pg_database_dattablespace 10
#define Anum_pg_database_datconfig 11
#define Anum_pg_database_datacl 12

View File

@ -7,7 +7,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/commands/vacuum.h,v 1.63 2006/03/05 15:58:55 momjian Exp $
* $PostgreSQL: pgsql/src/include/commands/vacuum.h,v 1.64 2006/07/10 16:20:51 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -117,12 +117,15 @@ extern void vac_close_indexes(int nindexes, Relation *Irel, LOCKMODE lockmode);
extern void vac_update_relstats(Oid relid,
BlockNumber num_pages,
double num_tuples,
bool hasindex);
bool hasindex,
TransactionId minxid,
TransactionId vacuumxid);
extern void vacuum_set_xid_limits(VacuumStmt *vacstmt, bool sharedRel,
TransactionId *oldestXmin,
TransactionId *freezeLimit);
extern bool vac_is_partial_index(Relation indrel);
extern void vacuum_delay_point(void);
extern TransactionId vactuple_get_minxid(HeapTuple tuple);
/* in commands/vacuumlazy.c */
extern void lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt);

View File

@ -4,7 +4,7 @@
* Interface to hba.c
*
*
* $PostgreSQL: pgsql/src/include/libpq/hba.h,v 1.42 2006/03/06 17:41:44 momjian Exp $
* $PostgreSQL: pgsql/src/include/libpq/hba.h,v 1.43 2006/07/10 16:20:52 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -40,7 +40,7 @@ extern void load_role(void);
extern int hba_getauthmethod(hbaPort *port);
extern int authident(hbaPort *port);
extern bool read_pg_database_line(FILE *fp, char *dbname, Oid *dboid,
Oid *dbtablespace, TransactionId *dbfrozenxid,
Oid *dbtablespace, TransactionId *dbminxid,
TransactionId *dbvacuumxid);
#endif /* HBA_H */

View File

@ -7,7 +7,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/storage/pmsignal.h,v 1.14 2006/03/05 15:58:59 momjian Exp $
* $PostgreSQL: pgsql/src/include/storage/pmsignal.h,v 1.15 2006/07/10 16:20:52 alvherre Exp $
*
*-------------------------------------------------------------------------
*/
@ -26,6 +26,7 @@ typedef enum
PMSIGNAL_WAKEN_CHILDREN, /* send a SIGUSR1 signal to all backends */
PMSIGNAL_WAKEN_ARCHIVER, /* send a NOTIFY signal to xlog archiver */
PMSIGNAL_ROTATE_LOGFILE, /* send SIGUSR1 to syslogger to rotate logfile */
PMSIGNAL_START_AUTOVAC, /* start an autovacuum iteration */
NUM_PMSIGNALS /* Must be last value of enum! */
} PMSignalReason;