mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-08-25 08:27:20 +02:00
command line. We find this useful because we frequently deal with thousands of tables in an environment where neither the databases nor the tables are updated frequently. This helps allow us to cut down on the overhead of updating the list for every other primary loop of pg_autovacuum. I chose -i as the command-line argument and documented it briefly in the README. The patch was applied to the 7.4.7 version of pg_autovacuum in contrib. Thomas F.O'Connell
247 lines
9.8 KiB
Plaintext
247 lines
9.8 KiB
Plaintext
pg_autovacuum README
|
|
--------------------
|
|
|
|
pg_autovacuum is a libpq client program that monitors all the
|
|
databases associated with a PostgreSQL server. It uses the statistics
|
|
collector to monitor insert, update and delete activity.
|
|
|
|
When a table exceeds a insert or delete threshold (for more detail on
|
|
thresholds, see "Vacuum and Analyze" below) then that table will be
|
|
vacuumed and/or analyzed.
|
|
|
|
This allows PostgreSQL to keep the FSM (Free Space Map) and table
|
|
statistics up to date, and eliminates the need to schedule periodic
|
|
vacuums.
|
|
|
|
The primary benefit of pg_autovacuum is that the FSM and table
|
|
statistic information are updated more nearly as frequently as needed.
|
|
When a table is actively changing, pg_autovacuum will perform the
|
|
VACUUMs and ANALYZEs that such a table needs, whereas if a table
|
|
remains static, no cycles will be wasted performing this
|
|
unnecessarily.
|
|
|
|
A secondary benefit of pg_autovacuum is that it ensures that a
|
|
database wide vacuum is performed prior to XID wraparound. This is an
|
|
important, if rare, problem, as failing to do so can result in major
|
|
data loss. (See the section in the _Administrator's Guide_ entitled
|
|
"Preventing transaction ID wraparound failures" for more details.)
|
|
|
|
KNOWN ISSUES:
|
|
-------------
|
|
|
|
pg_autovacuum has been tested under Redhat Linux (by me) and Debian
|
|
GNU/Linux, Solaris, and AIX (by Christopher B. Browne) and all known
|
|
bugs have been resolved. Please report any problems to the hackers
|
|
list.
|
|
|
|
pg_autovacuum requires that the statistics system be enabled and
|
|
reporting row level stats. The overhead of the stats system has been
|
|
shown to be significant under certain workloads. For instance, a
|
|
tight loop of queries performing "select 1" was found to run nearly
|
|
30% slower when row-level stats were enabled. However, in practice,
|
|
with more realistic workloads, the stats system overhead is usually
|
|
nominal.
|
|
|
|
pg_autovacuum does not get started automatically by either the
|
|
postmaster or by pg_ctl. Similarly, when the postmaster exits, no one
|
|
tells pg_autovacuum. The result of that is that at the start of the
|
|
next loop, pg_autovacuum will fail to connect to the server and
|
|
exit(). Any time it fails to connect pg_autovacuum exit()s.
|
|
|
|
While pg_autovacuum can manage vacuums for as many databases as you
|
|
may have tied to a particular PostgreSQL postmaster, it can only
|
|
connect to a single PostgreSQL postmaster. Thus, if you have multiple
|
|
postmasters on a particular host, you will need multiple pg_autovacuum
|
|
instances, and they have no way, at present, to coordinate between one
|
|
another to ensure that they do not concurrently vacuum big tables.
|
|
|
|
When installed as a service under Windows, there is currently no way to
|
|
know the name of the PostgreSQL server service (if there even is one)
|
|
so it is not possible to specify a startup dependency. It is therefore
|
|
possible for pg_autovacuum to start before the server.
|
|
|
|
When installed as a service under Windows, if the -P option is used to
|
|
specify the connection password, this option (and the password) is
|
|
stored in plain text in the registry.
|
|
|
|
TODO:
|
|
-----
|
|
|
|
At present, there are no sample scripts to automatically start up
|
|
pg_autovacuum along with the database. It would be desirable to have
|
|
a SysV script to start up pg_autovacuum after PostgreSQL has been
|
|
started.
|
|
|
|
Some users have expressed interest in making pg_autovacuum more
|
|
configurable so that certain tables known to be inactive could be
|
|
excluded from being vacuumed. It would probably make sense to
|
|
introduce this sort of functionality by providing arguments to specify
|
|
the database and schema in which to find a configuration table.
|
|
|
|
It would also be desirable for the daemon to monitor how busy the
|
|
system is, with a view to deferring vacuums until there is less other
|
|
activity.
|
|
|
|
INSTALL:
|
|
--------
|
|
|
|
As of postgresql v7.4 pg_autovacuum is included in the main source
|
|
tree under contrib. Therefore you merely need to "make && make
|
|
install" (similar to most other contrib modules) and it will be
|
|
installed for you.
|
|
|
|
If you are using an earlier version of PostgreSQL, uncompress the
|
|
tar.gz file into the contrib directory and modify the contrib/Makefile
|
|
to include the pg_autovacuum directory. pg_autovacuum will then be
|
|
built as part of the standard postgresql install. It is known to work
|
|
with v7.3 releases; it is not presently compatible with v7.2.
|
|
|
|
make sure that the following are set in postgresql.conf:
|
|
|
|
stats_start_collector = true
|
|
stats_row_level = true
|
|
|
|
Start up the postmaster, then execute the pg_autovacuum executable.
|
|
|
|
If you have a script that automatically starts up the PostgreSQL
|
|
instance, you might add in, after that, something similar to the
|
|
following:
|
|
|
|
sleep 10 # To give the database some time to start up
|
|
$PGBINS/pg_autovacuum -D -s $SBASE -S $SSCALE ... [other arguments]
|
|
|
|
Command line arguments:
|
|
-----------------------
|
|
|
|
pg_autovacuum has the following optional arguments:
|
|
|
|
-d debug: 0 silent, 1 basic info, 2 more debug info, etc...
|
|
-D daemonize: Detach from tty and run in background.
|
|
-s sleep base value: see "Sleeping" below.
|
|
-S sleep scaling factor: see "Sleeping" below.
|
|
-v vacuum base threshold: see "Vacuum and Analyze" below.
|
|
-V vacuum scaling factor: see "Vacuum and Analyze" below.
|
|
-a analyze base threshold: see "Vacuum and Analyze" below.
|
|
-A analyze scaling factor: see "Vacuum and Analyze" below.
|
|
-i update interval: how often (in terms of iterations of the primary loop
|
|
over the database list) to update the database list. The default is 2,
|
|
which means the list will be updated before every other pass through
|
|
the database list.
|
|
-L log file: Name of file to which output is submitted, otherwise STDERR
|
|
-U username: Username pg_autovacuum will use to connect with, if not
|
|
specified the current username is used.
|
|
-P password: Password pg_autovacuum will use to connect with. *WARNING*
|
|
This option is insecure. When installed as a Windows Service, this
|
|
option will be stored in plain text in the registry. When used with
|
|
most Unix variants, other users will be able to see the argument to
|
|
the "-P" option via ps(1). The ~/.pgpass file can be used to
|
|
specify a password more securely.
|
|
-H host: host name or IP to connect to.
|
|
-p port: port used for connection.
|
|
-h help: list of command line options.
|
|
|
|
The following 5 autovacuum command line options correspond to the various
|
|
cost-based vacuum settings. If not given, then the cluster default values
|
|
will be used.
|
|
|
|
-c vacuum_cost_delay
|
|
-C vacuum_cost_page_hit
|
|
-m vacuum_cost_page_miss
|
|
-n vacuum_cost_page_dirty
|
|
-l vacuum_cost_limit
|
|
|
|
|
|
Numerous arguments have default values defined in pg_autovacuum.h. At
|
|
the time of writing they are:
|
|
|
|
-d 1
|
|
-v 1000
|
|
-V 2
|
|
-a 500 (half of -v if not specified)
|
|
-A 1 (half of -V if not specified)
|
|
-s 300 (5 minutes)
|
|
-S 2
|
|
-i 2
|
|
|
|
The following arguments are used on Windows only:
|
|
|
|
-I Install the executable as a Windows service. Other appropriate command
|
|
line options will be stored in the registry and passed to the service
|
|
at startup. *WARNING* This includes the connection password which will
|
|
be stored in plain text.
|
|
|
|
-N service user: Name of the Windows user account under which the service
|
|
will run. Only used when installing as a Windows service.
|
|
|
|
-W service password: The password for the service account. Only used when
|
|
installing as a Windows service.
|
|
|
|
-R Uninstall pg_autovacuum as a service.
|
|
|
|
-E Dependent service that must start before this service. Normally this will be
|
|
a PostgreSQL instance, e.g. "-E pgsql-8.0.0". Only used when installing as
|
|
a Windows service.
|
|
|
|
Vacuum and Analyze:
|
|
-------------------
|
|
|
|
pg_autovacuum performs either a VACUUM ANALYZE or just ANALYZE
|
|
depending on the mixture of table activity (insert, update, or
|
|
delete):
|
|
|
|
- If the number of (inserts + updates + deletes) > AnalyzeThreshold, then
|
|
only an analyze is performed.
|
|
|
|
- If the number of (deletes + updates) > VacuumThreshold, then a
|
|
vacuum analyze is performed.
|
|
|
|
VacuumThreshold is equal to:
|
|
vacuum_base_value + (vacuum_scaling_factor * "number of tuples in the table")
|
|
|
|
AnalyzeThreshold is equal to:
|
|
analyze_base_value + (analyze_scaling_factor * "number of tuples in the table")
|
|
|
|
The AnalyzeThreshold defaults to half of the VacuumThreshold since it
|
|
represents a much less expensive operation (approx 5%-10% of vacuum),
|
|
and running ANALYZE more often should not substantially degrade system
|
|
performance.
|
|
|
|
Sleeping:
|
|
---------
|
|
|
|
pg_autovacuum sleeps for a while after it is done checking all the
|
|
databases. It does this in order to limit the amount of system
|
|
resources it consumes. This allows the system administrator to
|
|
configure pg_autovacuum to be more or less aggressive.
|
|
|
|
Reducing the sleep time will cause pg_autovacuum to respond more
|
|
quickly to changes, whether they be database addition/removal, table
|
|
addition/removal, or just normal table activity.
|
|
|
|
On the other hand, setting pg_autovacuum to sleep values too
|
|
aggressively (to too short periods of time) can have a negative effect
|
|
on server performance. For instance, if a table gets vacuumed 5 times
|
|
during the course of a large set of updates, this is likely to take a
|
|
lot more work than if the table was vacuumed just once, at the end.
|
|
|
|
The total time it sleeps is equal to:
|
|
|
|
base_sleep_value + sleep_scaling_factor * "duration of the previous
|
|
loop"
|
|
|
|
Note that timing measurements are made in seconds; specifying
|
|
"pg_vacuum -s 1" means pg_autovacuum could poll the database up to 60
|
|
times minute. In a system with large tables where vacuums may run for
|
|
several minutes, rather longer times between vacuums are likely to be
|
|
appropriate.
|
|
|
|
What pg_autovacuum monitors:
|
|
----------------------------
|
|
|
|
pg_autovacuum dynamically generates a list of all databases and tables
|
|
that exist on the server. It will dynamically add and remove
|
|
databases and tables that are removed from the database server while
|
|
pg_autovacuum is running. Overhead is fairly small per object. For
|
|
example: 10 databases with 10 tables each appears to less than 10k of
|
|
memory on my Linux box.
|