2010-09-20 22:08:53 +02:00
|
|
|
<!-- doc/src/sgml/maintenance.sgml -->
|
2001-08-26 23:17:12 +02:00
|
|
|
|
|
|
|
<chapter id="maintenance">
|
|
|
|
<title>Routine Database Maintenance Tasks</title>
|
|
|
|
|
2003-08-31 19:32:24 +02:00
|
|
|
<indexterm zone="maintenance">
|
|
|
|
<primary>maintenance</primary>
|
|
|
|
</indexterm>
|
|
|
|
|
2006-08-01 02:09:06 +02:00
|
|
|
<indexterm zone="maintenance">
|
2006-08-01 21:17:18 +02:00
|
|
|
<primary>routine maintenance</primary>
|
2006-08-01 02:09:06 +02:00
|
|
|
</indexterm>
|
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
2006-08-01 21:17:18 +02:00
|
|
|
<productname>PostgreSQL</>, like any database software, requires that certain tasks
|
|
|
|
be performed regularly to achieve optimum performance. The tasks
|
2006-08-01 02:09:06 +02:00
|
|
|
discussed here are <emphasis>required</emphasis>, but they
|
2006-08-01 21:17:18 +02:00
|
|
|
are repetitive in nature and can easily be automated using standard
|
2008-06-16 05:13:14 +02:00
|
|
|
tools such as <application>cron</application> scripts or
|
2010-02-03 18:25:06 +01:00
|
|
|
Windows' <application>Task Scheduler</>. It is the database
|
2006-08-01 21:17:18 +02:00
|
|
|
administrator's responsibility to set up appropriate scripts, and to
|
|
|
|
check that they execute successfully.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
2006-08-01 21:17:18 +02:00
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
2010-02-03 18:25:06 +01:00
|
|
|
One obvious maintenance task is the creation of backup copies of the data on a
|
2006-08-01 21:17:18 +02:00
|
|
|
regular schedule. Without a recent backup, you have no chance of recovery
|
|
|
|
after a catastrophe (disk failure, fire, mistakenly dropping a critical
|
|
|
|
table, etc.). The backup and recovery mechanisms available in
|
|
|
|
<productname>PostgreSQL</productname> are discussed at length in
|
|
|
|
<xref linkend="backup">.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2006-08-01 21:17:18 +02:00
|
|
|
The other main category of maintenance task is periodic <quote>vacuuming</>
|
|
|
|
of the database. This activity is discussed in
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
<xref linkend="routine-vacuuming">. Closely related to this is updating
|
2006-08-01 21:17:18 +02:00
|
|
|
the statistics that will be used by the query planner, as discussed in
|
|
|
|
<xref linkend="vacuum-for-statistics">.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
2001-09-23 06:06:24 +02:00
|
|
|
<para>
|
2006-08-01 21:17:18 +02:00
|
|
|
Another task that might need periodic attention is log file management.
|
2001-11-20 05:27:49 +01:00
|
|
|
This is discussed in <xref linkend="logfile-maintenance">.
|
2001-09-23 06:06:24 +02:00
|
|
|
</para>
|
2001-08-26 23:17:12 +02:00
|
|
|
|
2009-04-06 19:55:19 +02:00
|
|
|
<para>
|
|
|
|
<ulink
|
2010-03-17 18:12:31 +01:00
|
|
|
url="http://bucardo.org/wiki/Check_postgres"><application>check_postgres</></ulink>
|
2009-04-06 19:55:19 +02:00
|
|
|
is available for monitoring database health and reporting unusual
|
2010-03-17 18:12:31 +01:00
|
|
|
conditions. <application>check_postgres</> integrates with
|
2009-04-06 19:56:31 +02:00
|
|
|
Nagios and MRTG, but can be run standalone too.
|
2009-04-06 19:55:19 +02:00
|
|
|
</para>
|
|
|
|
|
2006-08-01 21:17:18 +02:00
|
|
|
<para>
|
|
|
|
<productname>PostgreSQL</productname> is low-maintenance compared
|
|
|
|
to some other database management systems. Nonetheless,
|
|
|
|
appropriate attention to these tasks will go far towards ensuring a
|
|
|
|
pleasant and productive experience with the system.
|
|
|
|
</para>
|
2001-08-26 23:17:12 +02:00
|
|
|
|
|
|
|
<sect1 id="routine-vacuuming">
|
|
|
|
<title>Routine Vacuuming</title>
|
|
|
|
|
2001-11-12 20:19:39 +01:00
|
|
|
<indexterm zone="routine-vacuuming">
|
|
|
|
<primary>vacuum</primary>
|
|
|
|
</indexterm>
|
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
<productname>PostgreSQL</productname> databases require periodic
|
|
|
|
maintenance known as <firstterm>vacuuming</>. For many installations, it
|
|
|
|
is sufficient to let vacuuming be performed by the <firstterm>autovacuum
|
|
|
|
daemon</>, which is described in <xref linkend="autovacuum">. You might
|
|
|
|
need to adjust the autovacuuming parameters described there to obtain best
|
|
|
|
results for your situation. Some database administrators will want to
|
|
|
|
supplement or replace the daemon's activities with manually-managed
|
|
|
|
<command>VACUUM</> commands, which typically are executed according to a
|
|
|
|
schedule by <application>cron</application> or <application>Task
|
|
|
|
Scheduler</> scripts. To set up manually-managed vacuuming properly,
|
|
|
|
it is essential to understand the issues discussed in the next few
|
|
|
|
subsections. Administrators who rely on autovacuuming may still wish
|
|
|
|
to skim this material to help them understand and adjust autovacuuming.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<sect2 id="vacuum-basics">
|
|
|
|
<title>Vacuuming Basics</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
<productname>PostgreSQL</productname>'s
|
2010-04-03 09:23:02 +02:00
|
|
|
<xref linkend="sql-vacuum"> command has to
|
2008-06-16 05:13:14 +02:00
|
|
|
process each table on a regular basis for several reasons:
|
2001-08-26 23:17:12 +02:00
|
|
|
|
|
|
|
<orderedlist>
|
|
|
|
<listitem>
|
2006-08-01 02:09:06 +02:00
|
|
|
<simpara>To recover or reuse disk space occupied by updated or deleted
|
2001-08-26 23:17:12 +02:00
|
|
|
rows.</simpara>
|
|
|
|
</listitem>
|
|
|
|
|
|
|
|
<listitem>
|
|
|
|
<simpara>To update data statistics used by the
|
|
|
|
<productname>PostgreSQL</productname> query planner.</simpara>
|
|
|
|
</listitem>
|
|
|
|
|
|
|
|
<listitem>
|
|
|
|
<simpara>To protect against loss of very old data due to
|
|
|
|
<firstterm>transaction ID wraparound</>.</simpara>
|
|
|
|
</listitem>
|
|
|
|
</orderedlist>
|
|
|
|
|
2008-06-16 05:13:14 +02:00
|
|
|
Each of these reasons dictates performing <command>VACUUM</> operations
|
|
|
|
of varying frequency and scope, as explained in the following subsections.
|
|
|
|
</para>
|
2004-12-27 23:30:10 +01:00
|
|
|
|
2008-06-16 05:13:14 +02:00
|
|
|
<para>
|
|
|
|
There are two variants of <command>VACUUM</>: standard <command>VACUUM</>
|
|
|
|
and <command>VACUUM FULL</>. <command>VACUUM FULL</> can reclaim more
|
|
|
|
disk space but runs much more slowly. Also,
|
|
|
|
the standard form of <command>VACUUM</> can run in parallel with production
|
|
|
|
database operations. (Commands such as <command>SELECT</command>,
|
|
|
|
<command>INSERT</command>, <command>UPDATE</command>, and
|
2010-02-03 18:25:06 +01:00
|
|
|
<command>DELETE</command> will continue to function normally, though you
|
2008-06-16 05:13:14 +02:00
|
|
|
will not be able to modify the definition of a table with commands such as
|
|
|
|
<command>ALTER TABLE</command> while it is being vacuumed.)
|
|
|
|
<command>VACUUM FULL</> requires exclusive lock on the table it is
|
|
|
|
working on, and therefore cannot be done in parallel with other use
|
2010-02-08 05:33:55 +01:00
|
|
|
of the table. Generally, therefore,
|
2008-06-16 05:13:14 +02:00
|
|
|
administrators should strive to use standard <command>VACUUM</> and
|
|
|
|
avoid <command>VACUUM FULL</>.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
<command>VACUUM</command> creates a substantial amount of I/O
|
|
|
|
traffic, which can cause poor performance for other active sessions.
|
|
|
|
There are configuration parameters that can be adjusted to reduce the
|
|
|
|
performance impact of background vacuuming — see
|
|
|
|
<xref linkend="runtime-config-resource-vacuum-cost">.
|
|
|
|
</para>
|
|
|
|
</sect2>
|
2005-09-13 03:51:18 +02:00
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<sect2 id="vacuum-for-space-recovery">
|
2007-09-14 01:43:35 +02:00
|
|
|
<title>Recovering Disk Space</title>
|
2001-08-26 23:17:12 +02:00
|
|
|
|
2001-11-12 20:19:39 +01:00
|
|
|
<indexterm zone="vacuum-for-space-recovery">
|
|
|
|
<primary>disk space</primary>
|
|
|
|
</indexterm>
|
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
In <productname>PostgreSQL</productname>, an
|
2002-06-23 05:37:12 +02:00
|
|
|
<command>UPDATE</> or <command>DELETE</> of a row does not
|
2003-11-01 02:56:29 +01:00
|
|
|
immediately remove the old version of the row.
|
2002-06-23 05:37:12 +02:00
|
|
|
This approach is necessary to gain the benefits of multiversion
|
2010-02-03 18:25:06 +01:00
|
|
|
concurrency control (<acronym>MVCC</>, see <xref linkend="mvcc">): the row version
|
2002-06-23 05:37:12 +02:00
|
|
|
must not be deleted while it is still potentially visible to other
|
2003-11-01 02:56:29 +01:00
|
|
|
transactions. But eventually, an outdated or deleted row version is no
|
2008-06-16 05:13:14 +02:00
|
|
|
longer of interest to any transaction. The space it occupies must then be
|
2010-02-03 18:25:06 +01:00
|
|
|
reclaimed for reuse by new rows, to avoid unbounded growth of disk
|
2002-06-23 05:37:12 +02:00
|
|
|
space requirements. This is done by running <command>VACUUM</>.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
The standard form of <command>VACUUM</command> removes dead row
|
|
|
|
versions in tables and indexes and marks the space available for
|
|
|
|
future reuse. However, it will not return the space to the operating
|
|
|
|
system, except in the special case where one or more pages at the
|
|
|
|
end of a table become entirely free and an exclusive table lock can be
|
|
|
|
easily obtained. In contrast, <command>VACUUM FULL</> actively compacts
|
2010-02-08 05:33:55 +01:00
|
|
|
tables by writing a complete new version of the table file with no dead
|
|
|
|
space. This minimizes the size of the table, but can take a long time.
|
|
|
|
It also requires extra disk space for the new copy of the table, until
|
|
|
|
the operation completes.
|
2003-12-14 01:10:32 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
The usual goal of routine vacuuming is to do standard <command>VACUUM</>s
|
|
|
|
often enough to avoid needing <command>VACUUM FULL</>. The
|
|
|
|
autovacuum daemon attempts to work this way, and in fact will
|
|
|
|
never issue <command>VACUUM FULL</>. In this approach, the idea
|
|
|
|
is not to keep tables at their minimum size, but to maintain steady-state
|
|
|
|
usage of disk space: each table occupies space equivalent to its
|
|
|
|
minimum size plus however much space gets used up between vacuumings.
|
|
|
|
Although <command>VACUUM FULL</> can be used to shrink a table back
|
|
|
|
to its minimum size and return the disk space to the operating system,
|
|
|
|
there is not much point in this if the table will just grow again in the
|
|
|
|
future. Thus, moderately-frequent standard <command>VACUUM</> runs are a
|
|
|
|
better approach than infrequent <command>VACUUM FULL</> runs for
|
|
|
|
maintaining heavily-updated tables.
|
2007-09-14 01:43:35 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
Some administrators prefer to schedule vacuuming themselves, for example
|
|
|
|
doing all the work at night when load is low.
|
|
|
|
The difficulty with doing vacuuming according to a fixed schedule
|
|
|
|
is that if a table has an unexpected spike in update activity, it may
|
|
|
|
get bloated to the point that <command>VACUUM FULL</> is really necessary
|
|
|
|
to reclaim space. Using the autovacuum daemon alleviates this problem,
|
|
|
|
since the daemon schedules vacuuming dynamically in response to update
|
|
|
|
activity. It is unwise to disable the daemon completely unless you
|
|
|
|
have an extremely predictable workload. One possible compromise is
|
|
|
|
to set the daemon's parameters so that it will only react to unusually
|
|
|
|
heavy update activity, thus keeping things from getting out of hand,
|
|
|
|
while scheduled <command>VACUUM</>s are expected to do the bulk of the
|
|
|
|
work when the load is typical.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
For those not using autovacuum, a typical approach is to schedule a
|
|
|
|
database-wide <command>VACUUM</> once a day during a low-usage period,
|
|
|
|
supplemented by more frequent vacuuming of heavily-updated tables as
|
2007-09-14 01:43:35 +02:00
|
|
|
necessary. (Some installations with extremely high update rates vacuum
|
|
|
|
their busiest tables as often as once every few minutes.) If you have
|
|
|
|
multiple databases in a cluster, don't forget to
|
|
|
|
<command>VACUUM</command> each one; the program <xref
|
2010-04-03 09:23:02 +02:00
|
|
|
linkend="app-vacuumdb"> might be helpful.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
2008-06-16 05:13:14 +02:00
|
|
|
<tip>
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
2010-02-08 05:33:55 +01:00
|
|
|
Plain <command>VACUUM</> may not be satisfactory when
|
2008-06-16 05:13:14 +02:00
|
|
|
a table contains large numbers of dead row versions as a result of
|
|
|
|
massive update or delete activity. If you have such a table and
|
2010-02-08 05:33:55 +01:00
|
|
|
you need to reclaim the excess disk space it occupies, you will need
|
|
|
|
to use <command>VACUUM FULL</>, or alternatively
|
2010-04-03 09:23:02 +02:00
|
|
|
<xref linkend="sql-cluster">
|
2008-06-16 05:13:14 +02:00
|
|
|
or one of the table-rewriting variants of
|
2010-04-03 09:23:02 +02:00
|
|
|
<xref linkend="sql-altertable">.
|
2008-06-16 05:13:14 +02:00
|
|
|
These commands rewrite an entire new copy of the table and build
|
2010-02-08 05:33:55 +01:00
|
|
|
new indexes for it. All these options require exclusive lock. Note that
|
|
|
|
they also temporarily use extra disk space approximately equal to the size
|
|
|
|
of the table, since the old copies of the table and indexes can't be
|
|
|
|
released until the new ones are complete.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
2008-06-16 05:13:14 +02:00
|
|
|
</tip>
|
2001-08-26 23:17:12 +02:00
|
|
|
|
2008-06-16 05:13:14 +02:00
|
|
|
<tip>
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
2006-08-01 02:09:06 +02:00
|
|
|
If you have a table whose entire contents are deleted on a periodic
|
2008-06-16 05:13:14 +02:00
|
|
|
basis, consider doing it with
|
2010-04-03 09:23:02 +02:00
|
|
|
<xref linkend="sql-truncate"> rather
|
2006-08-01 21:17:18 +02:00
|
|
|
than using <command>DELETE</command> followed by
|
2003-12-14 01:10:32 +01:00
|
|
|
<command>VACUUM</command>. <command>TRUNCATE</command> removes the
|
2004-12-13 19:05:10 +01:00
|
|
|
entire content of the table immediately, without requiring a
|
2003-12-14 01:10:32 +01:00
|
|
|
subsequent <command>VACUUM</command> or <command>VACUUM
|
|
|
|
FULL</command> to reclaim the now-unused disk space.
|
2008-06-16 05:13:14 +02:00
|
|
|
The disadvantage is that strict MVCC semantics are violated.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
2008-06-16 05:13:14 +02:00
|
|
|
</tip>
|
2001-08-26 23:17:12 +02:00
|
|
|
</sect2>
|
|
|
|
|
|
|
|
<sect2 id="vacuum-for-statistics">
|
2010-04-03 09:23:02 +02:00
|
|
|
<title>Updating Planner Statistics</title>
|
2001-08-26 23:17:12 +02:00
|
|
|
|
2003-08-31 19:32:24 +02:00
|
|
|
<indexterm zone="vacuum-for-statistics">
|
|
|
|
<primary>statistics</primary>
|
|
|
|
<secondary>of the planner</secondary>
|
|
|
|
</indexterm>
|
|
|
|
|
|
|
|
<indexterm zone="vacuum-for-statistics">
|
|
|
|
<primary>ANALYZE</primary>
|
|
|
|
</indexterm>
|
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
|
|
|
The <productname>PostgreSQL</productname> query planner relies on
|
|
|
|
statistical information about the contents of tables in order to
|
|
|
|
generate good plans for queries. These statistics are gathered by
|
2010-04-03 09:23:02 +02:00
|
|
|
the <xref linkend="sql-analyze"> command,
|
2007-10-07 03:16:42 +02:00
|
|
|
which can be invoked by itself or
|
2001-08-26 23:17:12 +02:00
|
|
|
as an optional step in <command>VACUUM</>. It is important to have
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
reasonably accurate statistics, otherwise poor choices of plans might
|
2001-08-26 23:17:12 +02:00
|
|
|
degrade database performance.
|
|
|
|
</para>
|
|
|
|
|
2008-06-16 05:13:14 +02:00
|
|
|
<para>
|
|
|
|
The autovacuum daemon, if enabled, will automatically issue
|
|
|
|
<command>ANALYZE</> commands whenever the content of a table has
|
|
|
|
changed sufficiently. However, administrators might prefer to rely
|
|
|
|
on manually-scheduled <command>ANALYZE</> operations, particularly
|
|
|
|
if it is known that update activity on a table will not affect the
|
|
|
|
statistics of <quote>interesting</> columns. The daemon schedules
|
|
|
|
<command>ANALYZE</> strictly as a function of the number of rows
|
|
|
|
inserted or updated; it has no knowledge of whether that will lead
|
|
|
|
to meaningful statistical changes.
|
|
|
|
</para>
|
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
|
|
|
As with vacuuming for space recovery, frequent updates of statistics
|
2002-06-23 05:37:12 +02:00
|
|
|
are more useful for heavily-updated tables than for seldom-updated
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
ones. But even for a heavily-updated table, there might be no need for
|
2002-06-23 05:37:12 +02:00
|
|
|
statistics updates if the statistical distribution of the data is
|
|
|
|
not changing much. A simple rule of thumb is to think about how much
|
2001-08-26 23:17:12 +02:00
|
|
|
the minimum and maximum values of the columns in the table change.
|
2002-06-23 05:37:12 +02:00
|
|
|
For example, a <type>timestamp</type> column that contains the time
|
|
|
|
of row update will have a constantly-increasing maximum value as
|
|
|
|
rows are added and updated; such a column will probably need more
|
|
|
|
frequent statistics updates than, say, a column containing URLs for
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
pages accessed on a website. The URL column might receive changes just
|
2002-06-23 05:37:12 +02:00
|
|
|
as often, but the statistical distribution of its values probably
|
|
|
|
changes relatively slowly.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
It is possible to run <command>ANALYZE</> on specific tables and even
|
|
|
|
just specific columns of a table, so the flexibility exists to update some
|
|
|
|
statistics more frequently than others if your application requires it.
|
2008-06-16 05:13:14 +02:00
|
|
|
In practice, however, it is usually best to just analyze the entire
|
|
|
|
database, because it is a fast operation. <command>ANALYZE</> uses a
|
2010-02-03 18:25:06 +01:00
|
|
|
statistically random sampling of the rows of a table rather than reading
|
2008-06-16 05:13:14 +02:00
|
|
|
every single row.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<tip>
|
|
|
|
<para>
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
Although per-column tweaking of <command>ANALYZE</> frequency might not be
|
2010-02-03 18:25:06 +01:00
|
|
|
very productive, you might find it worthwhile to do per-column
|
2001-08-26 23:17:12 +02:00
|
|
|
adjustment of the level of detail of the statistics collected by
|
2008-06-16 05:13:14 +02:00
|
|
|
<command>ANALYZE</>. Columns that are heavily used in <literal>WHERE</>
|
|
|
|
clauses and have highly irregular data distributions might require a
|
|
|
|
finer-grain data histogram than other columns. See <command>ALTER TABLE
|
|
|
|
SET STATISTICS</>, or change the database-wide default using the <xref
|
|
|
|
linkend="guc-default-statistics-target"> configuration parameter.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
2010-02-23 03:47:27 +01:00
|
|
|
|
|
|
|
<para>
|
|
|
|
Also, by default there is limited information available about
|
|
|
|
the selectivity of functions. However, if you create an expression
|
|
|
|
index that uses a function call, useful statistics will be
|
|
|
|
gathered about the function, which can greatly improve query
|
|
|
|
plans that use the expression index.
|
|
|
|
</para>
|
2001-08-26 23:17:12 +02:00
|
|
|
</tip>
|
|
|
|
</sect2>
|
|
|
|
|
|
|
|
<sect2 id="vacuum-for-wraparound">
|
2007-09-24 05:12:23 +02:00
|
|
|
<title>Preventing Transaction ID Wraparound Failures</title>
|
2001-08-26 23:17:12 +02:00
|
|
|
|
2001-11-12 20:19:39 +01:00
|
|
|
<indexterm zone="vacuum-for-wraparound">
|
|
|
|
<primary>transaction ID</primary>
|
|
|
|
<secondary>wraparound</secondary>
|
|
|
|
</indexterm>
|
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<para>
|
|
|
|
<productname>PostgreSQL</productname>'s MVCC transaction semantics
|
2003-03-24 15:32:51 +01:00
|
|
|
depend on being able to compare transaction ID (<acronym>XID</>)
|
2003-11-01 02:56:29 +01:00
|
|
|
numbers: a row version with an insertion XID greater than the current
|
2001-08-26 23:17:12 +02:00
|
|
|
transaction's XID is <quote>in the future</> and should not be visible
|
|
|
|
to the current transaction. But since transaction IDs have limited size
|
2010-02-03 18:25:06 +01:00
|
|
|
(32 bits) a cluster that runs for a long time (more
|
2005-02-20 03:22:07 +01:00
|
|
|
than 4 billion transactions) would suffer <firstterm>transaction ID
|
2001-08-26 23:17:12 +02:00
|
|
|
wraparound</>: the XID counter wraps around to zero, and all of a sudden
|
2004-11-15 07:32:15 +01:00
|
|
|
transactions that were in the past appear to be in the future — which
|
2010-02-03 18:25:06 +01:00
|
|
|
means their output become invisible. In short, catastrophic data loss.
|
Wording cleanup for error messages. Also change can't -> cannot.
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
2007-02-01 20:10:30 +01:00
|
|
|
(Actually the data is still there, but that's cold comfort if you cannot
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
get at it.) To avoid this, it is necessary to vacuum every table
|
|
|
|
in every database at least once every two billion transactions.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
The reason that periodic vacuuming solves the problem is that
|
2010-02-03 18:25:06 +01:00
|
|
|
<productname>PostgreSQL</productname> reserves a special XID
|
|
|
|
as <literal>FrozenXID</>. This XID does not follow the normal XID
|
|
|
|
comparison rules and is always considered older
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
than every normal XID. Normal XIDs are
|
2002-06-23 05:37:12 +02:00
|
|
|
compared using modulo-2<superscript>31</> arithmetic. This means
|
2001-08-26 23:17:12 +02:00
|
|
|
that for every normal XID, there are two billion XIDs that are
|
2002-06-23 05:37:12 +02:00
|
|
|
<quote>older</> and two billion that are <quote>newer</>; another
|
|
|
|
way to say it is that the normal XID space is circular with no
|
2003-11-01 02:56:29 +01:00
|
|
|
endpoint. Therefore, once a row version has been created with a particular
|
|
|
|
normal XID, the row version will appear to be <quote>in the past</> for
|
2002-06-23 05:37:12 +02:00
|
|
|
the next two billion transactions, no matter which normal XID we are
|
2003-11-01 02:56:29 +01:00
|
|
|
talking about. If the row version still exists after more than two billion
|
2002-06-23 05:37:12 +02:00
|
|
|
transactions, it will suddenly appear to be in the future. To
|
2010-02-03 18:25:06 +01:00
|
|
|
prevent this, old row versions must be reassigned the XID
|
2002-06-23 05:37:12 +02:00
|
|
|
<literal>FrozenXID</> sometime before they reach the
|
|
|
|
two-billion-transactions-old mark. Once they are assigned this
|
|
|
|
special XID, they will appear to be <quote>in the past</> to all
|
|
|
|
normal transactions regardless of wraparound issues, and so such
|
2010-02-03 18:25:06 +01:00
|
|
|
row versions will be valid until deleted, no matter how long that is.
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
This reassignment of old XIDs is handled by <command>VACUUM</>.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2009-05-17 00:51:24 +02:00
|
|
|
<xref linkend="guc-vacuum-freeze-min-age">
|
2009-01-16 14:27:24 +01:00
|
|
|
controls how old an XID value has to be before it's replaced with
|
2009-04-23 12:09:11 +02:00
|
|
|
<literal>FrozenXID</>. Larger values of this setting
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
preserve transactional information longer, while smaller values increase
|
|
|
|
the number of transactions that can elapse before the table must be
|
|
|
|
vacuumed again.
|
|
|
|
</para>
|
|
|
|
|
2009-05-17 00:51:24 +02:00
|
|
|
<para>
|
|
|
|
<command>VACUUM</> normally skips pages that don't have any dead row
|
|
|
|
versions, but those pages might still have row versions with old XID
|
|
|
|
values. To ensure all old XIDs have been replaced by
|
|
|
|
<literal>FrozenXID</>, a scan of the whole table is needed.
|
|
|
|
<xref linkend="guc-vacuum-freeze-table-age"> controls when
|
|
|
|
<command>VACUUM</> does that: a whole table sweep is forced if
|
|
|
|
the table hasn't been fully scanned for <varname>vacuum_freeze_table_age</>
|
|
|
|
minus <varname>vacuum_freeze_min_age</> transactions. Setting it to 0
|
|
|
|
forces <command>VACUUM</> to always scan all pages, effectively ignoring
|
|
|
|
the visibility map.
|
|
|
|
</para>
|
|
|
|
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
<para>
|
|
|
|
The maximum time that a table can go unvacuumed is two billion
|
2010-02-03 18:25:06 +01:00
|
|
|
transactions minus the <varname>vacuum_freeze_min_age</> value at
|
|
|
|
the time <command>VACUUM</> last scanned the whole table. If it were to go
|
2009-01-16 14:27:24 +01:00
|
|
|
unvacuumed for longer than
|
2008-06-16 05:13:14 +02:00
|
|
|
that, data loss could result. To ensure that this does not happen,
|
|
|
|
autovacuum is invoked on any table that might contain XIDs older than the
|
|
|
|
age specified by the configuration parameter <xref
|
|
|
|
linkend="guc-autovacuum-freeze-max-age">. (This will happen even if
|
2010-02-03 18:25:06 +01:00
|
|
|
autovacuum is disabled.)
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
This implies that if a table is not otherwise vacuumed,
|
|
|
|
autovacuum will be invoked on it approximately once every
|
|
|
|
<varname>autovacuum_freeze_max_age</> minus
|
|
|
|
<varname>vacuum_freeze_min_age</> transactions.
|
|
|
|
For tables that are regularly vacuumed for space reclamation purposes,
|
|
|
|
this is of little importance. However, for static tables
|
|
|
|
(including tables that receive inserts, but no updates or deletes),
|
2010-02-03 18:25:06 +01:00
|
|
|
there is no need to vacuum for space reclamation, so it can
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
be useful to try to maximize the interval between forced autovacuums
|
|
|
|
on very large static tables. Obviously one can do this either by
|
2010-02-03 18:25:06 +01:00
|
|
|
increasing <varname>autovacuum_freeze_max_age</> or decreasing
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
<varname>vacuum_freeze_min_age</>.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2009-05-17 00:51:24 +02:00
|
|
|
The effective maximum for <varname>vacuum_freeze_table_age</> is 0.95 *
|
2009-04-23 12:09:11 +02:00
|
|
|
<varname>autovacuum_freeze_max_age</>; a setting higher than that will be
|
2009-05-17 00:51:24 +02:00
|
|
|
capped to the maximum. A value higher than
|
2009-04-23 12:09:11 +02:00
|
|
|
<varname>autovacuum_freeze_max_age</> wouldn't make sense because an
|
|
|
|
anti-wraparound autovacuum would be triggered at that point anyway, and
|
|
|
|
the 0.95 multiplier leaves some breathing room to run a manual
|
|
|
|
<command>VACUUM</> before that happens. As a rule of thumb,
|
|
|
|
<command>vacuum_freeze_table_age</> should be set to a value somewhat
|
|
|
|
below <varname>autovacuum_freeze_max_age</>, leaving enough gap so that
|
|
|
|
a regularly scheduled <command>VACUUM</> or an autovacuum triggered by
|
|
|
|
normal delete and update activity is run in that window. Setting it too
|
|
|
|
close could lead to anti-wraparound autovacuums, even though the table
|
|
|
|
was recently vacuumed to reclaim space, whereas lower values lead to more
|
|
|
|
frequent whole-table scans.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
The sole disadvantage of increasing <varname>autovacuum_freeze_max_age</>
|
|
|
|
(and <varname>vacuum_freeze_table_age</> along with it)
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
is that the <filename>pg_clog</> subdirectory of the database cluster
|
2010-02-03 18:25:06 +01:00
|
|
|
will take more space, because it must store the commit status of all
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
transactions back to the <varname>autovacuum_freeze_max_age</> horizon.
|
|
|
|
The commit status uses two bits per transaction, so if
|
2010-02-03 18:25:06 +01:00
|
|
|
<varname>autovacuum_freeze_max_age</> is set to its maximum allowed value of
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
a little less than two billion, <filename>pg_clog</> can be expected to
|
|
|
|
grow to about half a gigabyte. If this is trivial compared to your
|
2009-04-23 12:09:11 +02:00
|
|
|
total database size, setting <varname>autovacuum_freeze_max_age</> to
|
|
|
|
its maximum allowed value is recommended. Otherwise, set it depending
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
on what you are willing to allow for <filename>pg_clog</> storage.
|
|
|
|
(The default, 200 million transactions, translates to about 50MB of
|
|
|
|
<filename>pg_clog</> storage.)
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
One disadvantage of decreasing <varname>vacuum_freeze_min_age</> is that
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
it might cause <command>VACUUM</> to do useless work: changing a table row's
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
XID to <literal>FrozenXID</> is a waste of time if the row is modified
|
|
|
|
soon thereafter (causing it to acquire a new XID). So the setting should
|
|
|
|
be large enough that rows are not frozen until they are unlikely to change
|
|
|
|
any more. Another disadvantage of decreasing this setting is
|
|
|
|
that details about exactly which transaction inserted or modified a
|
|
|
|
row will be lost sooner. This information sometimes comes in handy,
|
|
|
|
particularly when trying to analyze what went wrong after a database
|
|
|
|
failure. For these two reasons, decreasing this setting is not
|
|
|
|
recommended except for completely static tables.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
To track the age of the oldest XIDs in a database,
|
|
|
|
<command>VACUUM</> stores XID
|
|
|
|
statistics in the system tables <structname>pg_class</> and
|
|
|
|
<structname>pg_database</>. In particular,
|
|
|
|
the <structfield>relfrozenxid</> column of a table's
|
|
|
|
<structname>pg_class</> row contains the freeze cutoff XID that was used
|
2009-05-17 00:51:24 +02:00
|
|
|
by the last whole-table <command>VACUUM</> for that table. All normal
|
2001-08-26 23:17:12 +02:00
|
|
|
XIDs older than this cutoff XID are guaranteed to have been replaced by
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
<literal>FrozenXID</> within the table. Similarly,
|
|
|
|
the <structfield>datfrozenxid</> column of a database's
|
|
|
|
<structname>pg_database</> row is a lower bound on the normal XIDs
|
|
|
|
appearing in that database — it is just the minimum of the
|
|
|
|
per-table <structfield>relfrozenxid</> values within the database.
|
|
|
|
A convenient way to
|
2007-02-01 01:28:19 +01:00
|
|
|
examine this information is to execute queries such as:
|
2002-11-11 21:14:04 +01:00
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<programlisting>
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
SELECT relname, age(relfrozenxid) FROM pg_class WHERE relkind = 'r';
|
2001-10-13 01:32:34 +02:00
|
|
|
SELECT datname, age(datfrozenxid) FROM pg_database;
|
2001-08-26 23:17:12 +02:00
|
|
|
</programlisting>
|
2002-11-11 21:14:04 +01:00
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
The <literal>age</> column measures the number of transactions from the
|
2009-05-17 00:51:24 +02:00
|
|
|
cutoff XID to the current transaction's XID.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
<command>VACUUM</> normally
|
|
|
|
only scans pages that have been modified since the last vacuum, but
|
2009-04-23 12:09:11 +02:00
|
|
|
<structfield>relfrozenxid</> can only be advanced when the whole table is
|
|
|
|
scanned. The whole table is scanned when <structfield>relfrozenxid</> is
|
2009-11-16 22:32:07 +01:00
|
|
|
more than <varname>vacuum_freeze_table_age</> transactions old, when
|
|
|
|
<command>VACUUM</>'s <literal>FREEZE</> option is used, or when all pages
|
|
|
|
happen to
|
2009-04-23 12:09:11 +02:00
|
|
|
require vacuuming to remove dead row versions. When <command>VACUUM</>
|
2009-01-16 14:27:24 +01:00
|
|
|
scans the whole table, after it's finished <literal>age(relfrozenxid)</>
|
|
|
|
should be a little more than the <varname>vacuum_freeze_min_age</> setting
|
|
|
|
that was used (more by the number of transactions started since the
|
2009-04-23 12:09:11 +02:00
|
|
|
<command>VACUUM</> started). If no whole-table-scanning <command>VACUUM</>
|
|
|
|
is issued on the table until <varname>autovacuum_freeze_max_age</> is
|
|
|
|
reached, an autovacuum will soon be forced for the table.
|
2001-08-26 23:17:12 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
If for some reason autovacuum fails to clear old XIDs from a table,
|
|
|
|
the system will begin to emit warning messages like this when the
|
|
|
|
database's oldest XIDs reach ten million transactions from the wraparound
|
|
|
|
point:
|
2002-11-11 21:14:04 +01:00
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
<programlisting>
|
2005-02-20 03:22:07 +01:00
|
|
|
WARNING: database "mydb" must be vacuumed within 177009986 transactions
|
2008-12-11 19:16:18 +01:00
|
|
|
HINT: To avoid a database shutdown, execute a database-wide VACUUM in "mydb".
|
2001-08-26 23:17:12 +02:00
|
|
|
</programlisting>
|
|
|
|
|
2008-12-11 19:16:18 +01:00
|
|
|
(A manual <command>VACUUM</> should fix the problem, as suggested by the
|
|
|
|
hint; but note that the <command>VACUUM</> must be performed by a
|
|
|
|
superuser, else it will fail to process system catalogs and thus not
|
|
|
|
be able to advance the database's <structfield>datfrozenxid</>.)
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
If these warnings are
|
2010-02-03 18:25:06 +01:00
|
|
|
ignored, the system will shut down and refuse to start any new
|
2005-02-20 03:22:07 +01:00
|
|
|
transactions once there are fewer than 1 million transactions left
|
|
|
|
until wraparound:
|
|
|
|
|
|
|
|
<programlisting>
|
2007-09-14 04:43:18 +02:00
|
|
|
ERROR: database is not accepting commands to avoid wraparound data loss in database "mydb"
|
2005-02-20 03:22:07 +01:00
|
|
|
HINT: Stop the postmaster and use a standalone backend to VACUUM in "mydb".
|
|
|
|
</programlisting>
|
|
|
|
|
|
|
|
The 1-million-transaction safety margin exists to let the
|
|
|
|
administrator recover without data loss, by manually executing the
|
|
|
|
required <command>VACUUM</> commands. However, since the system will not
|
|
|
|
execute commands once it has gone into the safety shutdown mode,
|
2006-06-18 17:38:37 +02:00
|
|
|
the only way to do this is to stop the server and use a single-user
|
2005-02-20 03:22:07 +01:00
|
|
|
backend to execute <command>VACUUM</>. The shutdown mode is not enforced
|
2006-06-18 17:38:37 +02:00
|
|
|
by a single-user backend. See the <xref linkend="app-postgres"> reference
|
|
|
|
page for details about using a single-user backend.
|
2005-02-20 03:22:07 +01:00
|
|
|
</para>
|
|
|
|
|
2001-08-26 23:17:12 +02:00
|
|
|
</sect2>
|
2005-09-13 03:51:18 +02:00
|
|
|
|
|
|
|
<sect2 id="autovacuum">
|
2010-04-03 09:23:02 +02:00
|
|
|
<title>The Autovacuum Daemon</title>
|
2005-09-13 03:51:18 +02:00
|
|
|
|
|
|
|
<indexterm>
|
|
|
|
<primary>autovacuum</primary>
|
|
|
|
<secondary>general information</secondary>
|
|
|
|
</indexterm>
|
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
<productname>PostgreSQL</productname> has an optional but highly
|
|
|
|
recommended feature called <firstterm>autovacuum</firstterm>,
|
2007-04-16 20:30:04 +02:00
|
|
|
whose purpose is to automate the execution of
|
2005-09-13 03:51:18 +02:00
|
|
|
<command>VACUUM</command> and <command>ANALYZE </command> commands.
|
2007-04-16 20:30:04 +02:00
|
|
|
When enabled, autovacuum checks for
|
2005-09-13 03:51:18 +02:00
|
|
|
tables that have had a large number of inserted, updated or deleted
|
2007-09-24 05:12:23 +02:00
|
|
|
tuples. These checks use the statistics collection facility;
|
2007-04-16 20:30:04 +02:00
|
|
|
therefore, autovacuum cannot be used unless <xref
|
2007-09-24 05:12:23 +02:00
|
|
|
linkend="guc-track-counts"> is set to <literal>true</literal>.
|
2007-04-16 20:30:04 +02:00
|
|
|
In the default configuration, autovacuuming is enabled and the related
|
2007-01-16 19:26:02 +01:00
|
|
|
configuration parameters are appropriately set.
|
2005-09-13 03:51:18 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2008-06-16 05:13:14 +02:00
|
|
|
The <quote>autovacuum daemon</> actually consists of multiple processes.
|
|
|
|
There is a persistent daemon process, called the
|
2007-04-18 22:44:53 +02:00
|
|
|
<firstterm>autovacuum launcher</firstterm>, which is in charge of starting
|
2008-12-08 21:30:58 +01:00
|
|
|
<firstterm>autovacuum worker</firstterm> processes for all databases. The
|
2009-05-17 00:51:24 +02:00
|
|
|
launcher will distribute the work across time, attempting to start one
|
2010-02-08 05:33:55 +01:00
|
|
|
worker within each database every <xref linkend="guc-autovacuum-naptime">
|
|
|
|
seconds. (Therefore, if the installation has <replaceable>N</> databases,
|
|
|
|
a new worker will be launched every
|
|
|
|
<varname>autovacuum_naptime</>/<replaceable>N</> seconds.)
|
|
|
|
A maximum of <xref linkend="guc-autovacuum-max-workers"> worker processes
|
|
|
|
are allowed to run at the same time. If there are more than
|
|
|
|
<varname>autovacuum_max_workers</> databases to be processed,
|
2008-12-08 21:30:58 +01:00
|
|
|
the next database will be processed as soon as the first worker finishes.
|
2009-05-17 00:51:24 +02:00
|
|
|
Each worker process will check each table within its database and
|
2008-12-08 21:30:58 +01:00
|
|
|
execute <command>VACUUM</> and/or <command>ANALYZE</> as needed.
|
2007-04-16 20:30:04 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2010-02-08 05:33:55 +01:00
|
|
|
If several large tables all become eligible for vacuuming in a short
|
|
|
|
amount of time, all autovacuum workers might become occupied with
|
|
|
|
vacuuming those tables for a long period. This would result
|
2007-07-23 19:22:00 +02:00
|
|
|
in other tables and databases not being vacuumed until a worker became
|
2010-02-03 18:25:06 +01:00
|
|
|
available. There is no limit on how many workers might be in a
|
2007-09-24 05:12:23 +02:00
|
|
|
single database, but workers do try to avoid repeating work that has
|
2007-07-23 19:22:00 +02:00
|
|
|
already been done by other workers. Note that the number of running
|
2010-02-03 18:25:06 +01:00
|
|
|
workers does not count towards <xref linkend="guc-max-connections"> or
|
|
|
|
<xref linkend="guc-superuser-reserved-connections"> limits.
|
2007-04-16 20:30:04 +02:00
|
|
|
</para>
|
|
|
|
|
2005-09-13 03:51:18 +02:00
|
|
|
<para>
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
Tables whose <structfield>relfrozenxid</> value is more than
|
2009-06-17 15:59:28 +02:00
|
|
|
<xref linkend="guc-autovacuum-freeze-max-age"> transactions old are always
|
2009-02-09 21:57:59 +01:00
|
|
|
vacuumed (this also applies to those tables whose freeze max age has
|
|
|
|
been modified via storage parameters; see below). Otherwise, if the
|
|
|
|
number of tuples obsoleted since the last
|
2005-09-13 03:51:18 +02:00
|
|
|
<command>VACUUM</command> exceeds the <quote>vacuum threshold</quote>, the
|
2005-10-21 21:39:08 +02:00
|
|
|
table is vacuumed. The vacuum threshold is defined as:
|
2005-09-13 03:51:18 +02:00
|
|
|
<programlisting>
|
|
|
|
vacuum threshold = vacuum base threshold + vacuum scale factor * number of tuples
|
|
|
|
</programlisting>
|
|
|
|
where the vacuum base threshold is
|
2005-10-21 21:39:08 +02:00
|
|
|
<xref linkend="guc-autovacuum-vacuum-threshold">,
|
2005-09-13 03:51:18 +02:00
|
|
|
the vacuum scale factor is
|
2005-10-21 21:39:08 +02:00
|
|
|
<xref linkend="guc-autovacuum-vacuum-scale-factor">,
|
2005-09-13 03:51:18 +02:00
|
|
|
and the number of tuples is
|
|
|
|
<structname>pg_class</structname>.<structfield>reltuples</structfield>.
|
2005-10-21 21:39:08 +02:00
|
|
|
The number of obsolete tuples is obtained from the statistics
|
|
|
|
collector; it is a semi-accurate count updated by each
|
2005-09-13 03:51:18 +02:00
|
|
|
<command>UPDATE</command> and <command>DELETE</command> operation. (It
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
is only semi-accurate because some information might be lost under heavy
|
2009-04-23 12:09:11 +02:00
|
|
|
load.) If the <structfield>relfrozenxid</> value of the table is more
|
|
|
|
than <varname>vacuum_freeze_table_age</> transactions old, the whole
|
|
|
|
table is scanned to freeze old tuples and advance
|
|
|
|
<structfield>relfrozenxid</>, otherwise only pages that have been modified
|
2009-05-17 00:51:24 +02:00
|
|
|
since the last vacuum are scanned.
|
2008-06-16 05:13:14 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
For analyze, a similar condition is used: the threshold, defined as:
|
2005-10-21 21:39:08 +02:00
|
|
|
<programlisting>
|
|
|
|
analyze threshold = analyze base threshold + analyze scale factor * number of tuples
|
|
|
|
</programlisting>
|
2010-09-21 22:40:41 +02:00
|
|
|
is compared to the total number of tuples inserted, updated, or deleted
|
2005-10-21 21:39:08 +02:00
|
|
|
since the last <command>ANALYZE</command>.
|
2005-09-13 03:51:18 +02:00
|
|
|
</para>
|
|
|
|
|
2010-04-16 04:22:33 +02:00
|
|
|
<para>
|
|
|
|
Temporary tables cannot be accessed by autovacuum. Therefore,
|
|
|
|
appropriate vacuum and analyze operations should be performed via
|
|
|
|
session SQL commands.
|
|
|
|
</para>
|
|
|
|
|
2005-09-13 03:51:18 +02:00
|
|
|
<para>
|
2005-10-21 21:39:08 +02:00
|
|
|
The default thresholds and scale factors are taken from
|
|
|
|
<filename>postgresql.conf</filename>, but it is possible to override them
|
2009-05-17 00:51:24 +02:00
|
|
|
on a table-by-table basis; see
|
2009-02-09 21:57:59 +01:00
|
|
|
<xref linkend="sql-createtable-storage-parameters"
|
|
|
|
endterm="sql-createtable-storage-parameters-title"> for more information.
|
|
|
|
If a setting
|
|
|
|
has been changed via storage parameters, that value is used; otherwise the
|
|
|
|
global settings are used. See <xref linkend="runtime-config-autovacuum"> for
|
2005-10-21 21:39:08 +02:00
|
|
|
more details on the global settings.
|
2005-09-13 03:51:18 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2009-02-09 21:57:59 +01:00
|
|
|
Besides the base threshold values and scale factors, there are six
|
|
|
|
more autovacuum parameters that can be set for each table via
|
|
|
|
storage parameters.
|
|
|
|
The first parameter, <literal>autovacuum_enabled</>,
|
2005-10-21 21:39:08 +02:00
|
|
|
can be set to <literal>false</literal> to instruct the autovacuum daemon
|
|
|
|
to skip that particular table entirely. In this case
|
Fix recently-understood problems with handling of XID freezing, particularly
in PITR scenarios. We now WAL-log the replacement of old XIDs with
FrozenTransactionId, so that such replacement is guaranteed to propagate to
PITR slave databases. Also, rather than relying on hint-bit updates to be
preserved, pg_clog is not truncated until all instances of an XID are known to
have been replaced by FrozenTransactionId. Add new GUC variables and
pg_autovacuum columns to allow management of the freezing policy, so that
users can trade off the size of pg_clog against the amount of freezing work
done. Revise the already-existing code that forces autovacuum of tables
approaching the wraparound point to make it more bulletproof; also, revise the
autovacuum logic so that anti-wraparound vacuuming is done per-table rather
than per-database. initdb forced because of changes in pg_class, pg_database,
and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
|
|
|
autovacuum will only touch the table if it must do so
|
2005-10-21 21:39:08 +02:00
|
|
|
to prevent transaction ID wraparound.
|
2009-02-09 21:57:59 +01:00
|
|
|
Another two parameters,
|
2011-02-03 00:06:38 +01:00
|
|
|
<varname>autovacuum_vacuum_cost_delay</> and
|
|
|
|
<varname>autovacuum_vacuum_cost_limit</>, are used to set
|
2010-04-03 09:23:02 +02:00
|
|
|
table-specific values for the cost-based vacuum delay feature
|
|
|
|
(see <xref linkend="runtime-config-resource-vacuum-cost">).
|
2011-02-03 00:06:38 +01:00
|
|
|
<varname>autovacuum_freeze_min_age</>,
|
|
|
|
<varname>autovacuum_freeze_max_age</> and
|
|
|
|
<varname>autovacuum_freeze_table_age</> are used to set
|
2009-02-09 21:57:59 +01:00
|
|
|
values for <xref linkend="guc-vacuum-freeze-min-age">,
|
|
|
|
<xref linkend="guc-autovacuum-freeze-max-age"> and
|
|
|
|
<xref linkend="guc-vacuum-freeze-table-age"> respectively.
|
2005-09-13 03:51:18 +02:00
|
|
|
</para>
|
|
|
|
|
2007-04-16 20:30:04 +02:00
|
|
|
<para>
|
2007-05-15 17:52:40 +02:00
|
|
|
When multiple workers are running, the cost limit is
|
|
|
|
<quote>balanced</quote> among all the running workers, so that the
|
|
|
|
total impact on the system is the same, regardless of the number
|
|
|
|
of workers actually running.
|
2007-04-16 20:30:04 +02:00
|
|
|
</para>
|
2005-09-13 03:51:18 +02:00
|
|
|
</sect2>
|
2001-08-26 23:17:12 +02:00
|
|
|
</sect1>
|
2001-11-20 05:27:49 +01:00
|
|
|
|
2002-06-13 06:36:50 +02:00
|
|
|
|
2002-06-22 06:08:07 +02:00
|
|
|
<sect1 id="routine-reindex">
|
|
|
|
<title>Routine Reindexing</title>
|
|
|
|
|
|
|
|
<indexterm zone="routine-reindex">
|
|
|
|
<primary>reindex</primary>
|
|
|
|
</indexterm>
|
|
|
|
|
|
|
|
<para>
|
2003-09-13 00:17:24 +02:00
|
|
|
In some situations it is worthwhile to rebuild indexes periodically
|
2010-04-03 09:23:02 +02:00
|
|
|
with the <xref linkend="sql-reindex">
|
2005-11-01 22:09:51 +01:00
|
|
|
command.
|
2005-10-21 21:39:08 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2010-02-08 05:33:55 +01:00
|
|
|
B-tree index pages that have become completely empty are reclaimed for
|
|
|
|
re-use. However, there is still a possibility
|
2010-02-03 18:25:06 +01:00
|
|
|
of inefficient use of space: if all but a few index keys on a page have
|
|
|
|
been deleted, the page remains allocated. Therefore, a usage
|
|
|
|
pattern in which most, but not all, keys in each range are eventually
|
|
|
|
deleted will see poor use of space. For such usage patterns,
|
|
|
|
periodic reindexing is recommended.
|
2005-10-21 21:39:08 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2005-11-05 00:14:02 +01:00
|
|
|
The potential for bloat in non-B-tree indexes has not been well
|
2010-02-03 18:25:06 +01:00
|
|
|
researched. It is a good idea to periodically monitor the index's physical
|
2005-11-05 00:14:02 +01:00
|
|
|
size when using any non-B-tree index type.
|
2005-10-21 21:39:08 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2010-02-03 18:25:06 +01:00
|
|
|
Also, for B-tree indexes, a freshly-constructed index is slightly faster to
|
|
|
|
access than one that has been updated many times because logically
|
2005-10-21 21:39:08 +02:00
|
|
|
adjacent pages are usually also physically adjacent in a newly built index.
|
2010-02-03 18:25:06 +01:00
|
|
|
(This consideration does not apply to non-B-tree indexes.) It
|
2005-10-21 21:39:08 +02:00
|
|
|
might be worthwhile to reindex periodically just to improve access speed.
|
2002-06-22 06:08:07 +02:00
|
|
|
</para>
|
|
|
|
</sect1>
|
|
|
|
|
|
|
|
|
2001-11-20 05:27:49 +01:00
|
|
|
<sect1 id="logfile-maintenance">
|
|
|
|
<title>Log File Maintenance</title>
|
|
|
|
|
|
|
|
<indexterm zone="logfile-maintenance">
|
2003-08-31 19:32:24 +02:00
|
|
|
<primary>server log</primary>
|
|
|
|
<secondary>log file maintenance</secondary>
|
2001-11-20 05:27:49 +01:00
|
|
|
</indexterm>
|
|
|
|
|
|
|
|
<para>
|
2003-12-14 01:10:32 +01:00
|
|
|
It is a good idea to save the database server's log output
|
2010-02-03 18:25:06 +01:00
|
|
|
somewhere, rather than just discarding it via <filename>/dev/null</>.
|
|
|
|
The log output is invaluable when diagnosing
|
2003-12-14 01:10:32 +01:00
|
|
|
problems. However, the log output tends to be voluminous
|
2010-02-03 18:25:06 +01:00
|
|
|
(especially at higher debug levels) so you won't want to save it
|
|
|
|
indefinitely. You need to <emphasis>rotate</> the log files so that
|
2003-12-14 01:10:32 +01:00
|
|
|
new log files are started and old ones removed after a reasonable
|
|
|
|
period of time.
|
2001-11-20 05:27:49 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2006-06-18 17:38:37 +02:00
|
|
|
If you simply direct the <systemitem>stderr</> of
|
|
|
|
<command>postgres</command> into a
|
2004-08-06 01:32:13 +02:00
|
|
|
file, you will have log output, but
|
|
|
|
the only way to truncate the log file is to stop and restart
|
2010-02-03 18:25:06 +01:00
|
|
|
the server. This might be acceptable if you are using
|
2003-12-14 01:10:32 +01:00
|
|
|
<productname>PostgreSQL</productname> in a development environment,
|
|
|
|
but few production servers would find this behavior acceptable.
|
2001-11-20 05:27:49 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2006-06-18 17:38:37 +02:00
|
|
|
A better approach is to send the server's
|
2004-08-06 01:32:13 +02:00
|
|
|
<systemitem>stderr</> output to some type of log rotation program.
|
2010-02-03 18:25:06 +01:00
|
|
|
There is a built-in log rotation facility, which you can use by
|
2011-02-03 00:06:38 +01:00
|
|
|
setting the configuration parameter <varname>logging_collector</> to
|
2004-08-06 01:32:13 +02:00
|
|
|
<literal>true</> in <filename>postgresql.conf</>. The control
|
|
|
|
parameters for this program are described in <xref
|
2007-08-19 03:41:25 +02:00
|
|
|
linkend="runtime-config-logging-where">. You can also use this approach
|
2010-02-03 18:25:06 +01:00
|
|
|
to capture the log data in machine readable <acronym>CSV</>
|
|
|
|
(comma-separated values) format.
|
2004-08-06 01:32:13 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Alternatively, you might prefer to use an external log rotation
|
2010-02-03 18:25:06 +01:00
|
|
|
program if you have one that you are already using with other
|
2004-08-06 01:32:13 +02:00
|
|
|
server software. For example, the <application>rotatelogs</application>
|
|
|
|
tool included in the <productname>Apache</productname> distribution
|
|
|
|
can be used with <productname>PostgreSQL</productname>. To do this,
|
2006-06-18 17:38:37 +02:00
|
|
|
just pipe the server's
|
2004-08-06 01:32:13 +02:00
|
|
|
<systemitem>stderr</> output to the desired program.
|
|
|
|
If you start the server with
|
|
|
|
<command>pg_ctl</>, then <systemitem>stderr</>
|
|
|
|
is already redirected to <systemitem>stdout</>, so you just need a
|
2004-12-27 23:30:10 +01:00
|
|
|
pipe command, for example:
|
2004-08-06 01:32:13 +02:00
|
|
|
|
|
|
|
<programlisting>
|
|
|
|
pg_ctl start | rotatelogs /var/log/pgsql_log 86400
|
|
|
|
</programlisting>
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Another production-grade approach to managing log output is to
|
2010-02-03 18:25:06 +01:00
|
|
|
send it to <application>syslog</> and let
|
2003-11-01 02:56:29 +01:00
|
|
|
<application>syslog</> deal with file rotation. To do this, set the
|
2011-02-03 00:06:38 +01:00
|
|
|
configuration parameter <varname>log_destination</> to <literal>syslog</>
|
2004-08-06 01:32:13 +02:00
|
|
|
(to log to <application>syslog</> only) in
|
|
|
|
<filename>postgresql.conf</>. Then you can send a <literal>SIGHUP</literal>
|
|
|
|
signal to the <application>syslog</> daemon whenever you want to force it
|
|
|
|
to start writing a new log file. If you want to automate log
|
2004-05-16 21:34:46 +02:00
|
|
|
rotation, the <application>logrotate</application> program can be
|
2003-11-01 02:56:29 +01:00
|
|
|
configured to work with log files from
|
|
|
|
<application>syslog</application>.
|
2004-03-15 15:15:45 +01:00
|
|
|
</para>
|
2003-10-09 21:05:09 +02:00
|
|
|
|
2004-03-15 15:15:45 +01:00
|
|
|
<para>
|
|
|
|
On many systems, however, <application>syslog</> is not very reliable,
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
particularly with large log messages; it might truncate or drop messages
|
2004-12-27 23:30:10 +01:00
|
|
|
just when you need them the most. Also, on <productname>Linux</>,
|
2010-02-03 18:25:06 +01:00
|
|
|
<application>syslog</> will flush each message to disk, yielding poor
|
|
|
|
performance. (You can use a <quote><literal>-</></> at the start of the file name
|
2009-05-17 00:51:24 +02:00
|
|
|
in the <application>syslog</> configuration file to disable syncing.)
|
2001-11-20 05:27:49 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2004-08-06 01:32:13 +02:00
|
|
|
Note that all the solutions described above take care of starting new
|
|
|
|
log files at configurable intervals, but they do not handle deletion
|
2010-02-03 18:25:06 +01:00
|
|
|
of old, no-longer-useful log files. You will probably want to set
|
2004-12-27 23:30:10 +01:00
|
|
|
up a batch job to periodically delete old log files. Another possibility
|
|
|
|
is to configure the rotation program so that old log files are overwritten
|
|
|
|
cyclically.
|
2001-11-20 05:27:49 +01:00
|
|
|
</para>
|
2010-05-27 01:29:45 +02:00
|
|
|
|
|
|
|
<para>
|
|
|
|
<ulink url="http://pgfouine.projects.postgresql.org/">pgFouine</ulink>
|
|
|
|
is an external project that does sophisticated log file analysis.
|
|
|
|
<ulink
|
2010-05-27 01:55:51 +02:00
|
|
|
url="http://bucardo.org/wiki/Check_postgres">check_postgres</ulink>
|
2010-05-27 01:29:45 +02:00
|
|
|
provides Nagios alerts when important messages appear in the log
|
|
|
|
files, as well as detection of many other extraordinary conditions.
|
|
|
|
</para>
|
2001-11-20 05:27:49 +01:00
|
|
|
</sect1>
|
2001-08-26 23:17:12 +02:00
|
|
|
</chapter>
|