1996-07-09 08:22:35 +02:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
1999-02-14 00:22:53 +01:00
|
|
|
* varsup.c
|
2000-11-30 09:46:26 +01:00
|
|
|
* postgres OID & XID variables support routines
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
2015-01-06 17:43:47 +01:00
|
|
|
* Copyright (c) 2000-2015, PostgreSQL Global Development Group
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
2010-09-20 22:08:53 +02:00
|
|
|
* src/backend/access/transam/varsup.c
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
2000-11-03 12:39:36 +01:00
|
|
|
|
2000-11-30 09:46:26 +01:00
|
|
|
#include "postgres.h"
|
1996-10-21 09:15:18 +02:00
|
|
|
|
2001-08-25 20:52:43 +02:00
|
|
|
#include "access/clog.h"
|
Keep track of transaction commit timestamps
Transactions can now set their commit timestamp directly as they commit,
or an external transaction commit timestamp can be fed from an outside
system using the new function TransactionTreeSetCommitTsData(). This
data is crash-safe, and truncated at Xid freeze point, same as pg_clog.
This module is disabled by default because it causes a performance hit,
but can be enabled in postgresql.conf requiring only a server restart.
A new test in src/test/modules is included.
Catalog version bumped due to the new subdirectory within PGDATA and a
couple of new SQL functions.
Authors: Álvaro Herrera and Petr Jelínek
Reviewed to varying degrees by Michael Paquier, Andres Freund, Robert
Haas, Amit Kapila, Fujii Masao, Jaime Casanova, Simon Riggs, Steven
Singer, Peter Eisentraut
2014-12-03 15:53:02 +01:00
|
|
|
#include "access/commit_ts.h"
|
2004-07-01 02:52:04 +02:00
|
|
|
#include "access/subtrans.h"
|
2000-11-30 09:46:26 +01:00
|
|
|
#include "access/transam.h"
|
2010-10-20 18:48:51 +02:00
|
|
|
#include "access/xact.h"
|
2014-11-06 12:52:08 +01:00
|
|
|
#include "access/xlog.h"
|
2009-08-31 04:23:23 +02:00
|
|
|
#include "commands/dbcommands.h"
|
2005-02-20 03:22:07 +01:00
|
|
|
#include "miscadmin.h"
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
#include "postmaster/autovacuum.h"
|
|
|
|
#include "storage/pmsignal.h"
|
2011-09-04 07:13:16 +02:00
|
|
|
#include "storage/proc.h"
|
2009-08-31 04:23:23 +02:00
|
|
|
#include "utils/syscache.h"
|
1996-07-09 08:22:35 +02:00
|
|
|
|
1997-08-19 23:40:56 +02:00
|
|
|
|
2001-07-12 06:11:13 +02:00
|
|
|
/* Number of OIDs to prefetch (preallocate) per XLOG write */
|
XLOG (and related) changes:
* Store two past checkpoint locations, not just one, in pg_control.
On startup, we fall back to the older checkpoint if the newer one
is unreadable. Also, a physical copy of the newest checkpoint record
is kept in pg_control for possible use in disaster recovery (ie,
complete loss of pg_xlog). Also add a version number for pg_control
itself. Remove archdir from pg_control; it ought to be a GUC
parameter, not a special case (not that it's implemented yet anyway).
* Suppress successive checkpoint records when nothing has been entered
in the WAL log since the last one. This is not so much to avoid I/O
as to make it actually useful to keep track of the last two
checkpoints. If the things are right next to each other then there's
not a lot of redundancy gained...
* Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs
on alternate bytes. Polynomial borrowed from ECMA DLT1 standard.
* Fix XLOG record length handling so that it will work at BLCKSZ = 32k.
* Change XID allocation to work more like OID allocation. (This is of
dubious necessity, but I think it's a good idea anyway.)
* Fix a number of minor bugs, such as off-by-one logic for XLOG file
wraparound at the 4 gig mark.
* Add documentation and clean up some coding infelicities; move file
format declarations out to include files where planned contrib
utilities can get at them.
* Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or
every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also
possible to force a checkpoint by sending SIGUSR1 to the postmaster
(undocumented feature...)
* Defend against kill -9 postmaster by storing shmem block's key and ID
in postmaster.pid lockfile, and checking at startup to ensure that no
processes are still connected to old shmem block (if it still exists).
* Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency
stop, for symmetry with postmaster and xlog utilities. Clean up signal
handling in bootstrap.c so that xlog utilities launched by postmaster
will react to signals better.
* Standalone bootstrap now grabs lockfile in target directory, as added
insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
|
|
|
#define VAR_OID_PREFETCH 8192
|
|
|
|
|
2000-11-30 09:46:26 +01:00
|
|
|
/* pointer to "variable cache" in shared memory (set up by shmem.c) */
|
1998-09-01 06:40:42 +02:00
|
|
|
VariableCache ShmemVariableCache = NULL;
|
1998-07-21 08:17:39 +02:00
|
|
|
|
2001-07-12 06:11:13 +02:00
|
|
|
|
|
|
|
/*
|
2009-08-31 04:23:23 +02:00
|
|
|
* Allocate the next XID for a new transaction or subtransaction.
|
2007-09-08 22:31:15 +02:00
|
|
|
*
|
2012-05-14 09:22:44 +02:00
|
|
|
* The new XID is also stored into MyPgXact before returning.
|
2009-08-31 04:23:23 +02:00
|
|
|
*
|
|
|
|
* Note: when this is called, we are actually already inside a valid
|
|
|
|
* transaction, since XIDs are now not allocated until the transaction
|
2014-05-06 18:12:18 +02:00
|
|
|
* does something. So it is safe to do a database lookup if we want to
|
2009-08-31 04:23:23 +02:00
|
|
|
* issue a warning about XID wrap.
|
2001-07-12 06:11:13 +02:00
|
|
|
*/
|
2001-08-25 20:52:43 +02:00
|
|
|
TransactionId
|
2004-07-01 02:52:04 +02:00
|
|
|
GetNewTransactionId(bool isSubXact)
|
1996-07-09 08:22:35 +02:00
|
|
|
{
|
2001-10-25 07:50:21 +02:00
|
|
|
TransactionId xid;
|
2001-08-25 20:52:43 +02:00
|
|
|
|
Create an infrastructure for parallel computation in PostgreSQL.
This does four basic things. First, it provides convenience routines
to coordinate the startup and shutdown of parallel workers. Second,
it synchronizes various pieces of state (e.g. GUCs, combo CID
mappings, transaction snapshot) from the parallel group leader to the
worker processes. Third, it prohibits various operations that would
result in unsafe changes to that state while parallelism is active.
Finally, it propagates events that would result in an ErrorResponse,
NoticeResponse, or NotifyResponse message being sent to the client
from the parallel workers back to the master, from which they can then
be sent on to the client.
Robert Haas, Amit Kapila, Noah Misch, Rushabh Lathia, Jeevan Chalke.
Suggestions and review from Andres Freund, Heikki Linnakangas, Noah
Misch, Simon Riggs, Euler Taveira, and Jim Nasby.
2015-04-30 21:02:14 +02:00
|
|
|
/*
|
|
|
|
* Workers synchronize transaction state at the beginning of each parallel
|
|
|
|
* operation, so we can't account for new XIDs after that point.
|
|
|
|
*/
|
|
|
|
if (IsInParallelMode())
|
|
|
|
elog(ERROR, "cannot assign TransactionIds during a parallel operation");
|
|
|
|
|
2000-11-30 09:46:26 +01:00
|
|
|
/*
|
2001-03-22 05:01:46 +01:00
|
|
|
* During bootstrap initialization, we return the special bootstrap
|
|
|
|
* transaction id.
|
1996-07-09 08:22:35 +02:00
|
|
|
*/
|
2005-02-20 22:46:50 +01:00
|
|
|
if (IsBootstrapProcessingMode())
|
2007-09-08 22:31:15 +02:00
|
|
|
{
|
|
|
|
Assert(!isSubXact);
|
2011-11-25 14:02:10 +01:00
|
|
|
MyPgXact->xid = BootstrapTransactionId;
|
2001-08-25 20:52:43 +02:00
|
|
|
return BootstrapTransactionId;
|
2007-09-08 22:31:15 +02:00
|
|
|
}
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2010-02-20 22:24:02 +01:00
|
|
|
/* safety check, we should never get this far in a HS slave */
|
|
|
|
if (RecoveryInProgress())
|
|
|
|
elog(ERROR, "cannot assign TransactionIds during recovery");
|
|
|
|
|
2001-09-29 06:02:27 +02:00
|
|
|
LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
|
XLOG (and related) changes:
* Store two past checkpoint locations, not just one, in pg_control.
On startup, we fall back to the older checkpoint if the newer one
is unreadable. Also, a physical copy of the newest checkpoint record
is kept in pg_control for possible use in disaster recovery (ie,
complete loss of pg_xlog). Also add a version number for pg_control
itself. Remove archdir from pg_control; it ought to be a GUC
parameter, not a special case (not that it's implemented yet anyway).
* Suppress successive checkpoint records when nothing has been entered
in the WAL log since the last one. This is not so much to avoid I/O
as to make it actually useful to keep track of the last two
checkpoints. If the things are right next to each other then there's
not a lot of redundancy gained...
* Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs
on alternate bytes. Polynomial borrowed from ECMA DLT1 standard.
* Fix XLOG record length handling so that it will work at BLCKSZ = 32k.
* Change XID allocation to work more like OID allocation. (This is of
dubious necessity, but I think it's a good idea anyway.)
* Fix a number of minor bugs, such as off-by-one logic for XLOG file
wraparound at the 4 gig mark.
* Add documentation and clean up some coding infelicities; move file
format declarations out to include files where planned contrib
utilities can get at them.
* Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or
every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also
possible to force a checkpoint by sending SIGUSR1 to the postmaster
(undocumented feature...)
* Defend against kill -9 postmaster by storing shmem block's key and ID
in postmaster.pid lockfile, and checking at startup to ensure that no
processes are still connected to old shmem block (if it still exists).
* Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency
stop, for symmetry with postmaster and xlog utilities. Clean up signal
handling in bootstrap.c so that xlog utilities launched by postmaster
will react to signals better.
* Standalone bootstrap now grabs lockfile in target directory, as added
insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
|
|
|
|
2001-08-25 20:52:43 +02:00
|
|
|
xid = ShmemVariableCache->nextXid;
|
1998-09-01 06:40:42 +02:00
|
|
|
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
/*----------
|
2005-10-15 04:49:52 +02:00
|
|
|
* Check to see if it's safe to assign another XID. This protects against
|
|
|
|
* catastrophic data loss due to XID wraparound. The basic rules are:
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
*
|
|
|
|
* If we're past xidVacLimit, start trying to force autovacuum cycles.
|
|
|
|
* If we're past xidWarnLimit, start issuing warnings.
|
|
|
|
* If we're past xidStopLimit, refuse to execute transactions, unless
|
2013-06-21 05:03:18 +02:00
|
|
|
* we are running in single-user mode (which gives an escape hatch
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
* to the DBA who somehow got past the earlier defenses).
|
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
|
|
|
*
|
|
|
|
* Note that this coding also appears in GetNewMultiXactId.
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
*----------
|
2005-02-20 03:22:07 +01:00
|
|
|
*/
|
2010-02-17 04:10:33 +01:00
|
|
|
if (TransactionIdFollowsOrEquals(xid, ShmemVariableCache->xidVacLimit))
|
2005-02-20 03:22:07 +01:00
|
|
|
{
|
2009-08-31 04:23:23 +02:00
|
|
|
/*
|
|
|
|
* For safety's sake, we release XidGenLock while sending signals,
|
|
|
|
* warnings, etc. This is not so much because we care about
|
|
|
|
* preserving concurrency in this situation, as to avoid any
|
2010-02-26 03:01:40 +01:00
|
|
|
* possibility of deadlock while doing get_database_name(). First,
|
|
|
|
* copy all the shared values we'll need in this path.
|
2009-08-31 04:23:23 +02:00
|
|
|
*/
|
|
|
|
TransactionId xidWarnLimit = ShmemVariableCache->xidWarnLimit;
|
|
|
|
TransactionId xidStopLimit = ShmemVariableCache->xidStopLimit;
|
|
|
|
TransactionId xidWrapLimit = ShmemVariableCache->xidWrapLimit;
|
2010-02-26 03:01:40 +01:00
|
|
|
Oid oldest_datoid = ShmemVariableCache->oldestXidDB;
|
2009-08-31 04:23:23 +02:00
|
|
|
|
|
|
|
LWLockRelease(XidGenLock);
|
|
|
|
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
/*
|
2007-11-15 22:14:46 +01:00
|
|
|
* To avoid swamping the postmaster with signals, we issue the autovac
|
|
|
|
* request only once per 64K transaction starts. This still gives
|
|
|
|
* plenty of chances before we get into real trouble.
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
*/
|
|
|
|
if (IsUnderPostmaster && (xid % 65536) == 0)
|
2007-02-16 00:23:23 +01:00
|
|
|
SendPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER);
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
|
2005-02-20 03:22:07 +01:00
|
|
|
if (IsUnderPostmaster &&
|
2009-08-31 04:23:23 +02:00
|
|
|
TransactionIdFollowsOrEquals(xid, xidStopLimit))
|
|
|
|
{
|
2010-02-26 03:01:40 +01:00
|
|
|
char *oldest_datname = get_database_name(oldest_datoid);
|
2009-08-31 04:23:23 +02:00
|
|
|
|
|
|
|
/* complain even if that DB has disappeared */
|
|
|
|
if (oldest_datname)
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
|
|
|
|
errmsg("database is not accepting commands to avoid wraparound data loss in database \"%s\"",
|
|
|
|
oldest_datname),
|
2013-06-21 05:03:18 +02:00
|
|
|
errhint("Stop the postmaster and vacuum that database in single-user mode.\n"
|
2009-08-31 04:23:23 +02:00
|
|
|
"You might also need to commit or roll back old prepared transactions.")));
|
|
|
|
else
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
|
|
|
|
errmsg("database is not accepting commands to avoid wraparound data loss in database with OID %u",
|
|
|
|
oldest_datoid),
|
2013-06-21 05:03:18 +02:00
|
|
|
errhint("Stop the postmaster and vacuum that database in single-user mode.\n"
|
2009-08-31 04:23:23 +02:00
|
|
|
"You might also need to commit or roll back old prepared transactions.")));
|
|
|
|
}
|
|
|
|
else if (TransactionIdFollowsOrEquals(xid, xidWarnLimit))
|
|
|
|
{
|
2010-02-26 03:01:40 +01:00
|
|
|
char *oldest_datname = get_database_name(oldest_datoid);
|
2009-08-31 04:23:23 +02:00
|
|
|
|
|
|
|
/* complain even if that DB has disappeared */
|
|
|
|
if (oldest_datname)
|
|
|
|
ereport(WARNING,
|
|
|
|
(errmsg("database \"%s\" must be vacuumed within %u transactions",
|
|
|
|
oldest_datname,
|
|
|
|
xidWrapLimit - xid),
|
|
|
|
errhint("To avoid a database shutdown, execute a database-wide VACUUM in that database.\n"
|
|
|
|
"You might also need to commit or roll back old prepared transactions.")));
|
|
|
|
else
|
|
|
|
ereport(WARNING,
|
|
|
|
(errmsg("database with OID %u must be vacuumed within %u transactions",
|
|
|
|
oldest_datoid,
|
|
|
|
xidWrapLimit - xid),
|
|
|
|
errhint("To avoid a database shutdown, execute a database-wide VACUUM in that database.\n"
|
|
|
|
"You might also need to commit or roll back old prepared transactions.")));
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Re-acquire lock and start over */
|
|
|
|
LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
|
|
|
|
xid = ShmemVariableCache->nextXid;
|
2005-02-20 03:22:07 +01:00
|
|
|
}
|
|
|
|
|
2001-08-25 20:52:43 +02:00
|
|
|
/*
|
2004-08-29 07:07:03 +02:00
|
|
|
* If we are allocating the first XID of a new page of the commit log,
|
2005-10-15 04:49:52 +02:00
|
|
|
* zero out that commit-log page before returning. We must do this while
|
|
|
|
* holding XidGenLock, else another xact could acquire and commit a later
|
|
|
|
* XID before we zero the page. Fortunately, a page of the commit log
|
|
|
|
* holds 32K or more transactions, so we don't have to do this very often.
|
2004-07-01 02:52:04 +02:00
|
|
|
*
|
Keep track of transaction commit timestamps
Transactions can now set their commit timestamp directly as they commit,
or an external transaction commit timestamp can be fed from an outside
system using the new function TransactionTreeSetCommitTsData(). This
data is crash-safe, and truncated at Xid freeze point, same as pg_clog.
This module is disabled by default because it causes a performance hit,
but can be enabled in postgresql.conf requiring only a server restart.
A new test in src/test/modules is included.
Catalog version bumped due to the new subdirectory within PGDATA and a
couple of new SQL functions.
Authors: Álvaro Herrera and Petr Jelínek
Reviewed to varying degrees by Michael Paquier, Andres Freund, Robert
Haas, Amit Kapila, Fujii Masao, Jaime Casanova, Simon Riggs, Steven
Singer, Peter Eisentraut
2014-12-03 15:53:02 +01:00
|
|
|
* Extend pg_subtrans and pg_commit_ts too.
|
2001-08-25 20:52:43 +02:00
|
|
|
*/
|
|
|
|
ExtendCLOG(xid);
|
Keep track of transaction commit timestamps
Transactions can now set their commit timestamp directly as they commit,
or an external transaction commit timestamp can be fed from an outside
system using the new function TransactionTreeSetCommitTsData(). This
data is crash-safe, and truncated at Xid freeze point, same as pg_clog.
This module is disabled by default because it causes a performance hit,
but can be enabled in postgresql.conf requiring only a server restart.
A new test in src/test/modules is included.
Catalog version bumped due to the new subdirectory within PGDATA and a
couple of new SQL functions.
Authors: Álvaro Herrera and Petr Jelínek
Reviewed to varying degrees by Michael Paquier, Andres Freund, Robert
Haas, Amit Kapila, Fujii Masao, Jaime Casanova, Simon Riggs, Steven
Singer, Peter Eisentraut
2014-12-03 15:53:02 +01:00
|
|
|
ExtendCommitTs(xid);
|
2004-07-01 02:52:04 +02:00
|
|
|
ExtendSUBTRANS(xid);
|
2001-08-25 20:52:43 +02:00
|
|
|
|
2004-01-26 20:15:59 +01:00
|
|
|
/*
|
2005-10-15 04:49:52 +02:00
|
|
|
* Now advance the nextXid counter. This must not happen until after we
|
|
|
|
* have successfully completed ExtendCLOG() --- if that routine fails, we
|
2014-05-06 18:12:18 +02:00
|
|
|
* want the next incoming transaction to try it again. We cannot assign
|
2005-10-15 04:49:52 +02:00
|
|
|
* more XIDs until there is CLOG space for them.
|
2004-01-26 20:15:59 +01:00
|
|
|
*/
|
|
|
|
TransactionIdAdvance(ShmemVariableCache->nextXid);
|
|
|
|
|
2001-07-12 06:11:13 +02:00
|
|
|
/*
|
2007-09-08 22:31:15 +02:00
|
|
|
* We must store the new XID into the shared ProcArray before releasing
|
2014-05-06 18:12:18 +02:00
|
|
|
* XidGenLock. This ensures that every active XID older than
|
2007-11-15 22:14:46 +01:00
|
|
|
* latestCompletedXid is present in the ProcArray, which is essential for
|
|
|
|
* correct OldestXmin tracking; see src/backend/access/transam/README.
|
2001-07-12 06:11:13 +02:00
|
|
|
*
|
2012-06-10 21:20:04 +02:00
|
|
|
* XXX by storing xid into MyPgXact without acquiring ProcArrayLock, we
|
|
|
|
* are relying on fetch/store of an xid to be atomic, else other backends
|
2014-05-06 18:12:18 +02:00
|
|
|
* might see a partially-set xid here. But holding both locks at once
|
2007-09-08 22:31:15 +02:00
|
|
|
* would be a nasty concurrency hit. So for now, assume atomicity.
|
|
|
|
*
|
2012-05-14 09:22:44 +02:00
|
|
|
* Note that readers of PGXACT xid fields should be careful to fetch the
|
2007-09-08 22:31:15 +02:00
|
|
|
* value only once, rather than assume they can read a value multiple
|
|
|
|
* times and get the same answer each time.
|
2001-07-17 00:43:34 +02:00
|
|
|
*
|
2004-08-01 19:32:22 +02:00
|
|
|
* The same comments apply to the subxact xid count and overflow fields.
|
|
|
|
*
|
2012-05-14 09:22:44 +02:00
|
|
|
* A solution to the atomic-store problem would be to give each PGXACT its
|
|
|
|
* own spinlock used only for fetching/storing that PGXACT's xid and
|
2005-10-15 04:49:52 +02:00
|
|
|
* related fields.
|
2004-07-01 02:52:04 +02:00
|
|
|
*
|
2004-08-01 19:32:22 +02:00
|
|
|
* If there's no room to fit a subtransaction XID into PGPROC, set the
|
|
|
|
* cache-overflowed flag instead. This forces readers to look in
|
2005-10-15 04:49:52 +02:00
|
|
|
* pg_subtrans to map subtransaction XIDs up to top-level XIDs. There is a
|
|
|
|
* race-condition window, in that the new XID will not appear as running
|
|
|
|
* until its parent link has been placed into pg_subtrans. However, that
|
|
|
|
* will happen before anyone could possibly have a reason to inquire about
|
2007-09-08 22:31:15 +02:00
|
|
|
* the status of the XID, so it seems OK. (Snapshots taken during this
|
2005-10-15 04:49:52 +02:00
|
|
|
* window *will* include the parent XID, so they will deliver the correct
|
|
|
|
* answer later on when someone does have a reason to inquire.)
|
2001-07-12 06:11:13 +02:00
|
|
|
*/
|
2004-08-01 19:32:22 +02:00
|
|
|
{
|
2006-09-03 17:59:39 +02:00
|
|
|
/*
|
|
|
|
* Use volatile pointer to prevent code rearrangement; other backends
|
2006-10-04 02:30:14 +02:00
|
|
|
* could be examining my subxids info concurrently, and we don't want
|
|
|
|
* them to see an invalid intermediate state, such as incrementing
|
|
|
|
* nxids before filling the array entry. Note we are assuming that
|
|
|
|
* TransactionId and int fetch/store are atomic.
|
2006-09-03 17:59:39 +02:00
|
|
|
*/
|
|
|
|
volatile PGPROC *myproc = MyProc;
|
2011-11-25 14:02:10 +01:00
|
|
|
volatile PGXACT *mypgxact = MyPgXact;
|
2006-09-03 17:59:39 +02:00
|
|
|
|
2004-08-01 19:32:22 +02:00
|
|
|
if (!isSubXact)
|
2011-11-25 14:02:10 +01:00
|
|
|
mypgxact->xid = xid;
|
2004-08-01 19:32:22 +02:00
|
|
|
else
|
|
|
|
{
|
2011-11-25 14:02:10 +01:00
|
|
|
int nxids = mypgxact->nxids;
|
2006-09-03 17:59:39 +02:00
|
|
|
|
|
|
|
if (nxids < PGPROC_MAX_CACHED_SUBXIDS)
|
2004-08-01 19:32:22 +02:00
|
|
|
{
|
2006-09-03 17:59:39 +02:00
|
|
|
myproc->subxids.xids[nxids] = xid;
|
2011-11-25 14:02:10 +01:00
|
|
|
mypgxact->nxids = nxids + 1;
|
2004-08-01 19:32:22 +02:00
|
|
|
}
|
|
|
|
else
|
2011-11-25 14:02:10 +01:00
|
|
|
mypgxact->overflowed = true;
|
2004-08-01 19:32:22 +02:00
|
|
|
}
|
|
|
|
}
|
2001-07-12 06:11:13 +02:00
|
|
|
|
2001-09-29 06:02:27 +02:00
|
|
|
LWLockRelease(XidGenLock);
|
2001-08-25 20:52:43 +02:00
|
|
|
|
|
|
|
return xid;
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
|
|
|
|
1998-12-18 10:10:39 +01:00
|
|
|
/*
|
XLOG (and related) changes:
* Store two past checkpoint locations, not just one, in pg_control.
On startup, we fall back to the older checkpoint if the newer one
is unreadable. Also, a physical copy of the newest checkpoint record
is kept in pg_control for possible use in disaster recovery (ie,
complete loss of pg_xlog). Also add a version number for pg_control
itself. Remove archdir from pg_control; it ought to be a GUC
parameter, not a special case (not that it's implemented yet anyway).
* Suppress successive checkpoint records when nothing has been entered
in the WAL log since the last one. This is not so much to avoid I/O
as to make it actually useful to keep track of the last two
checkpoints. If the things are right next to each other then there's
not a lot of redundancy gained...
* Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs
on alternate bytes. Polynomial borrowed from ECMA DLT1 standard.
* Fix XLOG record length handling so that it will work at BLCKSZ = 32k.
* Change XID allocation to work more like OID allocation. (This is of
dubious necessity, but I think it's a good idea anyway.)
* Fix a number of minor bugs, such as off-by-one logic for XLOG file
wraparound at the 4 gig mark.
* Add documentation and clean up some coding infelicities; move file
format declarations out to include files where planned contrib
utilities can get at them.
* Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or
every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also
possible to force a checkpoint by sending SIGUSR1 to the postmaster
(undocumented feature...)
* Defend against kill -9 postmaster by storing shmem block's key and ID
in postmaster.pid lockfile, and checking at startup to ensure that no
processes are still connected to old shmem block (if it still exists).
* Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency
stop, for symmetry with postmaster and xlog utilities. Clean up signal
handling in bootstrap.c so that xlog utilities launched by postmaster
will react to signals better.
* Standalone bootstrap now grabs lockfile in target directory, as added
insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
|
|
|
* Read nextXid but don't allocate it.
|
1998-12-18 10:10:39 +01:00
|
|
|
*/
|
2001-08-25 20:52:43 +02:00
|
|
|
TransactionId
|
|
|
|
ReadNewTransactionId(void)
|
1998-12-18 10:10:39 +01:00
|
|
|
{
|
2001-10-25 07:50:21 +02:00
|
|
|
TransactionId xid;
|
2001-08-25 20:52:43 +02:00
|
|
|
|
2001-09-29 06:02:27 +02:00
|
|
|
LWLockAcquire(XidGenLock, LW_SHARED);
|
2001-08-25 20:52:43 +02:00
|
|
|
xid = ShmemVariableCache->nextXid;
|
2001-09-29 06:02:27 +02:00
|
|
|
LWLockRelease(XidGenLock);
|
2001-08-25 20:52:43 +02:00
|
|
|
|
|
|
|
return xid;
|
1998-12-18 10:10:39 +01:00
|
|
|
}
|
|
|
|
|
2005-02-20 03:22:07 +01:00
|
|
|
/*
|
|
|
|
* Determine the last safe XID to allocate given the currently oldest
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
* datfrozenxid (ie, the oldest XID that might exist in any database
|
2009-08-31 04:23:23 +02:00
|
|
|
* of our cluster), and the OID of the (or a) database with that value.
|
2005-02-20 03:22:07 +01:00
|
|
|
*/
|
|
|
|
void
|
2009-08-31 04:23:23 +02:00
|
|
|
SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
|
2005-02-20 03:22:07 +01:00
|
|
|
{
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
TransactionId xidVacLimit;
|
2005-02-20 03:22:07 +01:00
|
|
|
TransactionId xidWarnLimit;
|
|
|
|
TransactionId xidStopLimit;
|
|
|
|
TransactionId xidWrapLimit;
|
|
|
|
TransactionId curXid;
|
|
|
|
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
Assert(TransactionIdIsNormal(oldest_datfrozenxid));
|
2005-02-20 03:22:07 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* The place where we actually get into deep trouble is halfway around
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
* from the oldest potentially-existing XID. (This calculation is
|
|
|
|
* probably off by one or two counts, because the special XIDs reduce the
|
|
|
|
* size of the loop a little bit. But we throw in plenty of slop below,
|
|
|
|
* so it doesn't matter.)
|
2005-02-20 03:22:07 +01:00
|
|
|
*/
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
xidWrapLimit = oldest_datfrozenxid + (MaxTransactionId >> 1);
|
2005-02-20 03:22:07 +01:00
|
|
|
if (xidWrapLimit < FirstNormalTransactionId)
|
|
|
|
xidWrapLimit += FirstNormalTransactionId;
|
|
|
|
|
|
|
|
/*
|
2005-10-15 04:49:52 +02:00
|
|
|
* We'll refuse to continue assigning XIDs in interactive mode once we get
|
|
|
|
* within 1M transactions of data loss. This leaves lots of room for the
|
|
|
|
* DBA to fool around fixing things in a standalone backend, while not
|
|
|
|
* being significant compared to total XID space. (Note that since
|
|
|
|
* vacuuming requires one transaction per table cleaned, we had better be
|
|
|
|
* sure there's lots of XIDs left...)
|
2005-02-20 03:22:07 +01:00
|
|
|
*/
|
|
|
|
xidStopLimit = xidWrapLimit - 1000000;
|
|
|
|
if (xidStopLimit < FirstNormalTransactionId)
|
|
|
|
xidStopLimit -= FirstNormalTransactionId;
|
|
|
|
|
|
|
|
/*
|
2005-10-15 04:49:52 +02:00
|
|
|
* We'll start complaining loudly when we get within 10M transactions of
|
2014-05-06 18:12:18 +02:00
|
|
|
* the stop point. This is kind of arbitrary, but if you let your gas
|
2005-10-15 04:49:52 +02:00
|
|
|
* gauge get down to 1% of full, would you be looking for the next gas
|
|
|
|
* station? We need to be fairly liberal about this number because there
|
|
|
|
* are lots of scenarios where most transactions are done by automatic
|
|
|
|
* clients that won't pay attention to warnings. (No, we're not gonna make
|
|
|
|
* this configurable. If you know enough to configure it, you know enough
|
|
|
|
* to not get in this kind of trouble in the first place.)
|
2005-02-20 03:22:07 +01:00
|
|
|
*/
|
|
|
|
xidWarnLimit = xidStopLimit - 10000000;
|
|
|
|
if (xidWarnLimit < FirstNormalTransactionId)
|
|
|
|
xidWarnLimit -= FirstNormalTransactionId;
|
|
|
|
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
/*
|
2007-11-15 22:14:46 +01:00
|
|
|
* We'll start trying to force autovacuums when oldest_datfrozenxid gets
|
|
|
|
* to be more than autovacuum_freeze_max_age transactions old.
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
*
|
2007-11-15 22:14:46 +01:00
|
|
|
* Note: guc.c ensures that autovacuum_freeze_max_age is in a sane range,
|
|
|
|
* so that xidVacLimit will be well before xidWarnLimit.
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
*
|
|
|
|
* Note: autovacuum_freeze_max_age is a PGC_POSTMASTER parameter so that
|
|
|
|
* we don't have to worry about dealing with on-the-fly changes in its
|
|
|
|
* value. It doesn't look practical to update shared state from a GUC
|
|
|
|
* assign hook (too many processes would try to execute the hook,
|
2007-11-15 22:14:46 +01:00
|
|
|
* resulting in race conditions as well as crashes of those not connected
|
Separate multixact freezing parameters from xid's
Previously we were piggybacking on transaction ID parameters to freeze
multixacts; but since there isn't necessarily any relationship between
rates of Xid and multixact consumption, this turns out not to be a good
idea.
Therefore, we now have multixact-specific freezing parameters:
vacuum_multixact_freeze_min_age: when to remove multis as we come across
them in vacuum (default to 5 million, i.e. early in comparison to Xid's
default of 50 million)
vacuum_multixact_freeze_table_age: when to force whole-table scans
instead of scanning only the pages marked as not all visible in
visibility map (default to 150 million, same as for Xids). Whichever of
both which reaches the 150 million mark earlier will cause a whole-table
scan.
autovacuum_multixact_freeze_max_age: when for cause emergency,
uninterruptible whole-table scans (default to 400 million, double as
that for Xids). This means there shouldn't be more frequent emergency
vacuuming than previously, unless multixacts are being used very
rapidly.
Backpatch to 9.3 where multixacts were made to persist enough to require
freezing. To avoid an ABI break in 9.3, VacuumStmt has a couple of
fields in an unnatural place, and StdRdOptions is split in two so that
the newly added fields can go at the end.
Patch by me, reviewed by Robert Haas, with additional input from Andres
Freund and Tom Lane.
2014-02-13 23:30:30 +01:00
|
|
|
* to shared memory). Perhaps this can be improved someday. See also
|
|
|
|
* SetMultiXactIdLimit.
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
*/
|
|
|
|
xidVacLimit = oldest_datfrozenxid + autovacuum_freeze_max_age;
|
|
|
|
if (xidVacLimit < FirstNormalTransactionId)
|
|
|
|
xidVacLimit += FirstNormalTransactionId;
|
|
|
|
|
2005-02-20 03:22:07 +01:00
|
|
|
/* Grab lock for just long enough to set the new limit values */
|
|
|
|
LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
ShmemVariableCache->oldestXid = oldest_datfrozenxid;
|
|
|
|
ShmemVariableCache->xidVacLimit = xidVacLimit;
|
2005-02-20 03:22:07 +01:00
|
|
|
ShmemVariableCache->xidWarnLimit = xidWarnLimit;
|
|
|
|
ShmemVariableCache->xidStopLimit = xidStopLimit;
|
|
|
|
ShmemVariableCache->xidWrapLimit = xidWrapLimit;
|
2009-08-31 04:23:23 +02:00
|
|
|
ShmemVariableCache->oldestXidDB = oldest_datoid;
|
2005-02-20 03:22:07 +01:00
|
|
|
curXid = ShmemVariableCache->nextXid;
|
|
|
|
LWLockRelease(XidGenLock);
|
|
|
|
|
|
|
|
/* Log the info */
|
2006-09-26 19:21:39 +02:00
|
|
|
ereport(DEBUG1,
|
2010-02-26 03:01:40 +01:00
|
|
|
(errmsg("transaction ID wrap limit is %u, limited by database with OID %u",
|
|
|
|
xidWrapLimit, oldest_datoid)));
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If past the autovacuum force point, immediately signal an autovac
|
|
|
|
* request. The reason for this is that autovac only processes one
|
|
|
|
* database per invocation. Once it's finished cleaning up the oldest
|
|
|
|
* database, it'll call here, and we'll signal the postmaster to start
|
|
|
|
* another iteration immediately if there are still any old databases.
|
|
|
|
*/
|
|
|
|
if (TransactionIdFollowsOrEquals(curXid, xidVacLimit) &&
|
2010-02-17 04:10:33 +01:00
|
|
|
IsUnderPostmaster && !InRecovery)
|
2007-02-16 00:23:23 +01:00
|
|
|
SendPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER);
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
|
2005-02-20 03:22:07 +01:00
|
|
|
/* Give an immediate warning if past the wrap warn point */
|
2010-02-17 04:10:33 +01:00
|
|
|
if (TransactionIdFollowsOrEquals(curXid, xidWarnLimit) && !InRecovery)
|
2009-08-31 04:23:23 +02:00
|
|
|
{
|
2010-10-20 18:48:51 +02:00
|
|
|
char *oldest_datname;
|
2009-08-31 04:23:23 +02:00
|
|
|
|
|
|
|
/*
|
2011-04-10 17:42:00 +02:00
|
|
|
* We can be called when not inside a transaction, for example during
|
|
|
|
* StartupXLOG(). In such a case we cannot do database access, so we
|
|
|
|
* must just report the oldest DB's OID.
|
2010-10-20 18:48:51 +02:00
|
|
|
*
|
|
|
|
* Note: it's also possible that get_database_name fails and returns
|
|
|
|
* NULL, for example because the database just got dropped. We'll
|
|
|
|
* still warn, even though the warning might now be unnecessary.
|
2009-08-31 04:23:23 +02:00
|
|
|
*/
|
2010-10-20 18:48:51 +02:00
|
|
|
if (IsTransactionState())
|
|
|
|
oldest_datname = get_database_name(oldest_datoid);
|
|
|
|
else
|
|
|
|
oldest_datname = NULL;
|
|
|
|
|
2009-08-31 04:23:23 +02:00
|
|
|
if (oldest_datname)
|
|
|
|
ereport(WARNING,
|
2010-02-26 03:01:40 +01:00
|
|
|
(errmsg("database \"%s\" must be vacuumed within %u transactions",
|
|
|
|
oldest_datname,
|
|
|
|
xidWrapLimit - curXid),
|
|
|
|
errhint("To avoid a database shutdown, execute a database-wide VACUUM in that database.\n"
|
|
|
|
"You might also need to commit or roll back old prepared transactions.")));
|
2009-08-31 04:23:23 +02:00
|
|
|
else
|
|
|
|
ereport(WARNING,
|
|
|
|
(errmsg("database with OID %u must be vacuumed within %u transactions",
|
|
|
|
oldest_datoid,
|
|
|
|
xidWrapLimit - curXid),
|
|
|
|
errhint("To avoid a database shutdown, execute a database-wide VACUUM in that database.\n"
|
|
|
|
"You might also need to commit or roll back old prepared transactions.")));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
2009-09-01 06:46:49 +02:00
|
|
|
* ForceTransactionIdLimitUpdate -- does the XID wrap-limit data need updating?
|
2009-08-31 04:23:23 +02:00
|
|
|
*
|
|
|
|
* We primarily check whether oldestXidDB is valid. The cases we have in
|
|
|
|
* mind are that that database was dropped, or the field was reset to zero
|
|
|
|
* by pg_resetxlog. In either case we should force recalculation of the
|
2014-05-06 18:12:18 +02:00
|
|
|
* wrap limit. Also do it if oldestXid is old enough to be forcing
|
2009-09-01 06:46:49 +02:00
|
|
|
* autovacuums or other actions; this ensures we update our state as soon
|
|
|
|
* as possible once extra overhead is being incurred.
|
2009-08-31 04:23:23 +02:00
|
|
|
*/
|
|
|
|
bool
|
2009-09-01 06:46:49 +02:00
|
|
|
ForceTransactionIdLimitUpdate(void)
|
2009-08-31 04:23:23 +02:00
|
|
|
{
|
2009-09-01 06:46:49 +02:00
|
|
|
TransactionId nextXid;
|
|
|
|
TransactionId xidVacLimit;
|
2009-08-31 04:23:23 +02:00
|
|
|
TransactionId oldestXid;
|
|
|
|
Oid oldestXidDB;
|
|
|
|
|
|
|
|
/* Locking is probably not really necessary, but let's be careful */
|
|
|
|
LWLockAcquire(XidGenLock, LW_SHARED);
|
2009-09-01 06:46:49 +02:00
|
|
|
nextXid = ShmemVariableCache->nextXid;
|
|
|
|
xidVacLimit = ShmemVariableCache->xidVacLimit;
|
2009-08-31 04:23:23 +02:00
|
|
|
oldestXid = ShmemVariableCache->oldestXid;
|
|
|
|
oldestXidDB = ShmemVariableCache->oldestXidDB;
|
|
|
|
LWLockRelease(XidGenLock);
|
|
|
|
|
|
|
|
if (!TransactionIdIsNormal(oldestXid))
|
2009-09-01 06:46:49 +02:00
|
|
|
return true; /* shouldn't happen, but just in case */
|
2010-02-17 04:10:33 +01:00
|
|
|
if (!TransactionIdIsValid(xidVacLimit))
|
|
|
|
return true; /* this shouldn't happen anymore either */
|
|
|
|
if (TransactionIdFollowsOrEquals(nextXid, xidVacLimit))
|
2009-09-01 06:46:49 +02:00
|
|
|
return true; /* past VacLimit, don't delay updating */
|
2010-02-14 19:42:19 +01:00
|
|
|
if (!SearchSysCacheExists1(DATABASEOID, ObjectIdGetDatum(oldestXidDB)))
|
2009-09-01 06:46:49 +02:00
|
|
|
return true; /* could happen, per comments above */
|
|
|
|
return false;
|
2005-02-20 03:22:07 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
2005-08-12 03:36:05 +02:00
|
|
|
/*
|
|
|
|
* GetNewObjectId -- allocate a new OID
|
|
|
|
*
|
|
|
|
* OIDs are generated by a cluster-wide counter. Since they are only 32 bits
|
|
|
|
* wide, counter wraparound will occur eventually, and therefore it is unwise
|
|
|
|
* to assume they are unique unless precautions are taken to make them so.
|
|
|
|
* Hence, this routine should generally not be used directly. The only
|
|
|
|
* direct callers should be GetNewOid() and GetNewRelFileNode() in
|
|
|
|
* catalog/catalog.c.
|
1996-07-09 08:22:35 +02:00
|
|
|
*/
|
2001-08-10 20:57:42 +02:00
|
|
|
Oid
|
|
|
|
GetNewObjectId(void)
|
1996-07-09 08:22:35 +02:00
|
|
|
{
|
2001-10-25 07:50:21 +02:00
|
|
|
Oid result;
|
2001-08-10 20:57:42 +02:00
|
|
|
|
2010-02-20 22:24:02 +01:00
|
|
|
/* safety check, we should never get this far in a HS slave */
|
|
|
|
if (RecoveryInProgress())
|
|
|
|
elog(ERROR, "cannot assign OIDs during recovery");
|
|
|
|
|
2001-09-29 06:02:27 +02:00
|
|
|
LWLockAcquire(OidGenLock, LW_EXCLUSIVE);
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2001-08-10 20:57:42 +02:00
|
|
|
/*
|
|
|
|
* Check for wraparound of the OID counter. We *must* not return 0
|
|
|
|
* (InvalidOid); and as long as we have to check that, it seems a good
|
2005-04-13 20:54:57 +02:00
|
|
|
* idea to skip over everything below FirstNormalObjectId too. (This
|
2005-08-12 03:36:05 +02:00
|
|
|
* basically just avoids lots of collisions with bootstrap-assigned OIDs
|
|
|
|
* right after a wrap occurs, so as to avoid a possibly large number of
|
|
|
|
* iterations in GetNewOid.) Note we are relying on unsigned comparison.
|
2005-04-13 20:54:57 +02:00
|
|
|
*
|
2005-11-22 19:17:34 +01:00
|
|
|
* During initdb, we start the OID generator at FirstBootstrapObjectId, so
|
2013-05-13 21:40:16 +02:00
|
|
|
* we only wrap if before that point when in bootstrap or standalone mode.
|
|
|
|
* The first time through this routine after normal postmaster start, the
|
|
|
|
* counter will be forced up to FirstNormalObjectId. This mechanism
|
|
|
|
* leaves the OIDs between FirstBootstrapObjectId and FirstNormalObjectId
|
|
|
|
* available for automatic assignment during initdb, while ensuring they
|
|
|
|
* will never conflict with user-assigned OIDs.
|
2001-08-10 20:57:42 +02:00
|
|
|
*/
|
2005-04-13 20:54:57 +02:00
|
|
|
if (ShmemVariableCache->nextOid < ((Oid) FirstNormalObjectId))
|
2001-08-10 20:57:42 +02:00
|
|
|
{
|
2005-04-13 20:54:57 +02:00
|
|
|
if (IsPostmasterEnvironment)
|
|
|
|
{
|
2013-05-13 21:40:16 +02:00
|
|
|
/* wraparound, or first post-initdb assignment, in normal mode */
|
2005-04-13 20:54:57 +02:00
|
|
|
ShmemVariableCache->nextOid = FirstNormalObjectId;
|
|
|
|
ShmemVariableCache->oidCount = 0;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* we may be bootstrapping, so don't enforce the full range */
|
|
|
|
if (ShmemVariableCache->nextOid < ((Oid) FirstBootstrapObjectId))
|
|
|
|
{
|
2013-05-13 21:40:16 +02:00
|
|
|
/* wraparound in standalone mode (unlikely but possible) */
|
|
|
|
ShmemVariableCache->nextOid = FirstNormalObjectId;
|
2005-04-13 20:54:57 +02:00
|
|
|
ShmemVariableCache->oidCount = 0;
|
|
|
|
}
|
|
|
|
}
|
2001-08-10 20:57:42 +02:00
|
|
|
}
|
|
|
|
|
XLOG (and related) changes:
* Store two past checkpoint locations, not just one, in pg_control.
On startup, we fall back to the older checkpoint if the newer one
is unreadable. Also, a physical copy of the newest checkpoint record
is kept in pg_control for possible use in disaster recovery (ie,
complete loss of pg_xlog). Also add a version number for pg_control
itself. Remove archdir from pg_control; it ought to be a GUC
parameter, not a special case (not that it's implemented yet anyway).
* Suppress successive checkpoint records when nothing has been entered
in the WAL log since the last one. This is not so much to avoid I/O
as to make it actually useful to keep track of the last two
checkpoints. If the things are right next to each other then there's
not a lot of redundancy gained...
* Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs
on alternate bytes. Polynomial borrowed from ECMA DLT1 standard.
* Fix XLOG record length handling so that it will work at BLCKSZ = 32k.
* Change XID allocation to work more like OID allocation. (This is of
dubious necessity, but I think it's a good idea anyway.)
* Fix a number of minor bugs, such as off-by-one logic for XLOG file
wraparound at the 4 gig mark.
* Add documentation and clean up some coding infelicities; move file
format declarations out to include files where planned contrib
utilities can get at them.
* Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or
every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also
possible to force a checkpoint by sending SIGUSR1 to the postmaster
(undocumented feature...)
* Defend against kill -9 postmaster by storing shmem block's key and ID
in postmaster.pid lockfile, and checking at startup to ensure that no
processes are still connected to old shmem block (if it still exists).
* Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency
stop, for symmetry with postmaster and xlog utilities. Clean up signal
handling in bootstrap.c so that xlog utilities launched by postmaster
will react to signals better.
* Standalone bootstrap now grabs lockfile in target directory, as added
insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
|
|
|
/* If we run out of logged for use oids then we must log more */
|
2000-11-30 09:46:26 +01:00
|
|
|
if (ShmemVariableCache->oidCount == 0)
|
1997-09-07 07:04:48 +02:00
|
|
|
{
|
2000-11-30 09:46:26 +01:00
|
|
|
XLogPutNextOid(ShmemVariableCache->nextOid + VAR_OID_PREFETCH);
|
|
|
|
ShmemVariableCache->oidCount = VAR_OID_PREFETCH;
|
1997-09-07 07:04:48 +02:00
|
|
|
}
|
|
|
|
|
2001-08-10 20:57:42 +02:00
|
|
|
result = ShmemVariableCache->nextOid;
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2000-11-30 09:46:26 +01:00
|
|
|
(ShmemVariableCache->nextOid)++;
|
|
|
|
(ShmemVariableCache->oidCount)--;
|
|
|
|
|
2001-09-29 06:02:27 +02:00
|
|
|
LWLockRelease(OidGenLock);
|
2001-08-10 20:57:42 +02:00
|
|
|
|
|
|
|
return result;
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|