Commit Graph

96 Commits

Author SHA1 Message Date
Alvaro Herrera e8f28cb25d Dynamically set a lower bound on autovacuum nap time so that we don't rebuild
the database list too often.

Per bug report from Łukasz Jagiełło and ensuing discussion on
pgsql-performance.
2009-06-09 16:41:02 +00:00
Tom Lane 4616d57dad Fix all the server-side SIGQUIT handlers (grumble ... why so many identical
copies?) to ensure they really don't run proc_exit/shmem_exit callbacks,
as was intended.  I broke this behavior recently by installing atexit
callbacks without thinking about the one case where we truly don't want
to run those callback functions.  Noted in an example from Dave Page.
2009-05-15 15:56:39 +00:00
Tom Lane 948d6ec90f Modify the relcache to record the temp status of both local and nonlocal
temp relations; this is no more expensive than before, now that we have
pg_class.relistemp.  Insert tests into bufmgr.c to prevent attempting
to fetch pages from nonlocal temp relations.  This provides a low-level
defense against bugs-of-omission allowing temp pages to be loaded into shared
buffers, as in the contrib/pgstattuple problem reported by Stuart Bishop.
While at it, tweak a bunch of places to use new relcache tests (instead of
expensive probes into pg_namespace) to detect local or nonlocal temp tables.
2009-03-31 22:12:48 +00:00
Alvaro Herrera 834a6da4f7 Update autovacuum to use reloptions instead of a system catalog, for
per-table overrides of parameters.

This removes a whole class of problems related to misusing the catalog,
and perhaps more importantly, gives us pg_dump support for the parameters.

Based on a patch by Euler Taveira de Oliveira, heavily reworked by me.
2009-02-09 20:57:59 +00:00
Heikki Linnakangas 6587818542 Add vacuum_freeze_table_age GUC option, to control when VACUUM should
ignore the visibility map and scan the whole table, to advance
relfrozenxid.
2009-01-16 13:27:24 +00:00
Bruce Momjian 511db38ace Update copyright for 2009. 2009-01-01 17:24:05 +00:00
Heikki Linnakangas dea81a6cf6 Revert SIGUSR1 multiplexing patch, per Tom's objection. 2008-12-09 15:59:39 +00:00
Heikki Linnakangas 7b05b3fa39 Provide support for multiplexing SIGUSR1 signal. The upcoming synchronous
replication patch needs a signal, but we've already used SIGUSR1 and
SIGUSR2 in normal backends. This patch allows reusing SIGUSR1 for that,
and for other purposes too if the need arises.
2008-12-09 14:28:20 +00:00
Heikki Linnakangas 7537f52a00 Utilize the visibility map in autovacuum, too. There was an oversight in
the visibility map patch that because autovacuum always sets
VacuumStmt->freeze_min_age, visibility map was never used for autovacuum,
only for manually launched vacuums. This patch introduces a new scan_all
field to VacuumStmt, indicating explicitly whether the visibility map
should be used, or the whole relation should be scanned, to advance
relfrozenxid. Anti-wraparound vacuums still need to scan all pages.
2008-12-04 11:42:24 +00:00
Heikki Linnakangas 4c22564471 Fix off-by-one error in autovacuum shmem struct sizing. This could lead to
autovacuum worker sending SIGUSR1 signal to wrong process, per Zou Yong's
report.

Backpatch to 8.3.
2008-11-12 10:10:32 +00:00
Alvaro Herrera 88dd4b0a0d Reduce the acceptable staleness of pgstat data for autovacuum, per the
longstanding note in the source that this patch removes.
2008-11-03 19:03:41 +00:00
Tom Lane d7112cfa88 Remove the last vestiges of the MAKE_PTR/MAKE_OFFSET mechanism. We haven't
allowed different processes to have different addresses for the shmem segment
in quite a long time, but there were still a few places left that used the
old coding convention.  Clean them up to reduce confusion and improve the
compiler's ability to detect pointer type mismatches.

Kris Jurka
2008-11-02 21:24:52 +00:00
Alvaro Herrera 3ccde312ec Have autovacuum consider processing TOAST tables separately from their
main tables.

This requires vacuum() to accept processing a toast table standalone, so
there's a user-visible change in that it's now possible (for a superuser) to
execute "VACUUM pg_toast.pg_toast_XXX".
2008-08-13 00:07:50 +00:00
Alvaro Herrera 85dfe376d9 Ratchet up patch to improve autovacuum wraparound messages.
Simon Riggs
2008-07-23 20:20:10 +00:00
Alvaro Herrera 0d09688f88 Publish more openly the fact that autovacuum is working for wraparound
protection.

Simon Riggs
2008-07-21 15:27:02 +00:00
Alvaro Herrera 46c5a212ec Avoid crashing when a table is deleted while we're on the process of checking
it.

Per report from Tom Lane based on buildfarm evidence.
2008-07-17 21:02:31 +00:00
Tom Lane 5b965bf08b Teach autovacuum how to determine whether a temp table belongs to a crashed
backend.  If so, send a LOG message to the postmaster log, and if the table
is beyond the vacuum-for-wraparound horizon, forcibly drop it.  Per recent
discussions.  Perhaps we ought to back-patch this, but it probably needs
to age a bit in HEAD first.
2008-07-01 02:09:34 +00:00
Alvaro Herrera 9319fd89e1 Modify vacuum() to accept a single relation OID instead of a list (which we
always pass as a single element anyway.)  In passing, fix an outdated comment.
2008-06-05 15:47:32 +00:00
Tom Lane 93c701edc6 Add support for tracking call counts and elapsed runtime for user-defined
functions.

Note that because this patch changes FmgrInfo, any external C functions
you might be testing with 8.4 will need to be recompiled.

Patch by Martin Pihlak, some editorialization by me (principally, removing
tracking of getrusage() numbers)
2008-05-15 00:17:41 +00:00
Alvaro Herrera f8c4d7db60 Restructure some header files a bit, in particular heapam.h, by removing some
unnecessary #include lines in it.  Also, move some tuple routine prototypes and
macros to htup.h, which allows removal of heapam.h inclusion from some .c
files.

For this to work, a new header file access/sysattr.h needed to be created,
initially containing attribute numbers of system columns, for pg_dump usage.

While at it, make contrib ltree, intarray and hstore header files more
consistent with our header style.
2008-05-12 00:00:54 +00:00
Alvaro Herrera 73b0300b2a Move the HTSU_Result enum definition into snapshot.h, to avoid including
tqual.h into heapam.h.  This makes all inclusion of tqual.h explicit.

I also sorted alphabetically the includes on some source files.
2008-03-26 21:10:39 +00:00
Alvaro Herrera 470c6c12a1 Remove another useless snapshot creation. 2008-03-19 21:14:20 +00:00
Tom Lane 4873c96ff3 Fix inappropriately-timed memory context switch in autovacuum_do_vac_analyze.
This accidentally failed to fail before 8.3, because the context we were
switching back to was long-lived anyway; but it sure looks risky as can be
now.  Well spotted by Pavan Deolasee.
2008-03-14 23:49:28 +00:00
Alvaro Herrera adc4e1e635 Fix vacuum so that autovacuum is really not cancelled when doing an emergency
job (i.e. to prevent Xid wraparound problems.)  Bug reported by ITAGAKI
Takahiro in 20080314103837.63D3.52131E4D@oss.ntt.co.jp, though I didn't use his
patch.
2008-03-14 17:25:59 +00:00
Alvaro Herrera bccc8e3608 Change error message to be able to differentiate the two cases. Per suggestion
from Jaime Casanova.
2008-02-20 14:01:45 +00:00
Alvaro Herrera 7aa4164363 Mark autovacuum entries in pg_stat_activity so that they can be easily
distinguished from user-invoked commands.  Per suggestion from Tom Lane.
2008-01-14 13:39:25 +00:00
Bruce Momjian 9098ab9e32 Update copyrights in source tree to 2008. 2008-01-01 19:46:01 +00:00
Bruce Momjian f6e8730d11 Re-run pgindent with updated list of typedefs. (Updated README should
avoid this problem in the future.)
2007-11-15 22:25:18 +00:00
Bruce Momjian fdf5a5efb7 pgindent run for 8.3. 2007-11-15 21:14:46 +00:00
Alvaro Herrera 92be6991ad In the previous commit message, I forgot to credit Simon Riggs who provided
the patch, which I modified only slightly before applying.
2007-10-29 22:17:41 +00:00
Alvaro Herrera acac68b2bc Allow an autovacuum worker to be interrupted automatically when it is found
to be locking another process (except when it's working to prevent Xid
wraparound problems).
2007-10-26 20:45:10 +00:00
Alvaro Herrera dd7444cfe2 Fix memory management for new variables -- they must actually survive
transaction end, in case we decide to do a vacuum analyze (which is done in two
xacts).
2007-10-25 19:13:37 +00:00
Alvaro Herrera dbe48765cd Extract catalog info for error reporting before an error actually happens.
Also, remove redundant reset of for-wraparound PGPROC flag.

Thanks to Tom Lane for noticing both bogosities.
2007-10-25 14:45:55 +00:00
Alvaro Herrera 745c1b2c2a Rearrange vacuum-related bits in PGPROC as a bitmask, to better support
having several of them.  Add two more flags: whether the process is
executing an ANALYZE, and whether a vacuum is for Xid wraparound (which
is obviously only set by autovacuum).

Sneakily move the worker's recently-acquired PostAuthDelay to a more useful
place.
2007-10-24 20:55:36 +00:00
Alvaro Herrera 2627c83561 Minor changes to autovacuum worker: change error handling so that it continues
with the next table on schedule instead of exiting, in all cases instead of
just on query cancel.

Add a errcontext() line indicating the activity of the worker to the error
message when it is cancelled.

Change the WorkerInfo struct to contain a pointer to the worker's PGPROC
instead of just the PID.

Add forgotten post-auth delays, per Simon Riggs.  Also to autovac launcher.
2007-10-24 19:08:25 +00:00
Alvaro Herrera 45cbdaa038 Avoid having autovacuum read pgstats data too many times in quick succession.
This is problematic for the autovac launcher when there are many databases,
so we keep data for a full second before reading it again.
2007-09-24 04:12:01 +00:00
Tom Lane 48f7e64395 Simplify and rename some GUC variables, per various recent discussions:
* stats_start_collector goes away; we always start the collector process,
unless prevented by a problem with setting up the stats UDP socket.

* stats_reset_on_server_start goes away; it seems useless in view of the
availability of pg_stat_reset().

* stats_block_level and stats_row_level are merged into a single variable
"track_counts", which controls all reports sent to the collector process.

* stats_command_string is renamed to track_activities.

* log_autovacuum is renamed to log_autovacuum_min_duration to better reflect
its meaning.

The log_autovacuum change is not a compatibility issue since it didn't exist
before 8.3 anyway.  The other changes need to be release-noted.
2007-09-24 03:12:23 +00:00
Tom Lane ea72d37ff7 Make autovacuum report the start time of its current activity in
pg_stat_activity.  Per gripe from Jim Nasby.
2007-09-23 20:07:33 +00:00
Alvaro Herrera 945ba50dc3 Fix a memory leak in the autovacuum launcher code. Noted by Darcy Buskermolen,
who reported it privately to me.
2007-09-12 22:14:59 +00:00
Tom Lane f181f9e1e4 Make sure that open hash table scans are cleaned up when bgwriter tries to
recover from elog(ERROR).  Problem was created by introduction of hash seq
search tracking awhile back, and affects all branches that have bgwriter;
in HEAD the disease has snuck into autovacuum and walwriter too.  (Not sure
that the latter two use hash_seq_search at the moment, but surely they might
someday.)  Per report from Sergey Koposov.
2007-09-11 17:15:33 +00:00
Andrew Dunstan 63872601e8 Move session_start out of MyProcPort stucture and make it a global called MyStartTime,
so that we will be able to create a cookie for all processes for CSVlogs.
It is set wherever MyProcPid is set. Take the opportunity to remove the now
unnecessary session-only restriction on the %s and %c escapes in log_line_prefix.
2007-08-02 23:39:45 +00:00
Tom Lane bce7bacdf2 Reduce the maximum sleep interval in the autovac launcher to 1 second,
so that it responds to SIGQUIT reasonably promptly even on machines where
SA_RESTART signals restart a sleep from scratch.  (This whole area could
stand some rethinking, but for now make it work like the other processes
do.)  Also some marginal stylistic cleanups.
2007-07-01 18:30:54 +00:00
Tom Lane 070907b241 Add 'volatile' to suppress 'variable might be clobbered by longjmp'
warning emitted by some versions of gcc.
2007-07-01 02:20:59 +00:00
Alvaro Herrera 2910ccefb4 Avoid crash in interrupted autovacuum worker, caused by leaving the current
memory context pointing at a context not long lived enough.

Also, create a fake PortalContext where to store the vac_context, if only
to avoid having it be a top-level memory context.
2007-06-30 04:08:05 +00:00
Alvaro Herrera 10af02b912 Arrange for SIGINT in autovacuum workers to cancel the current table and
continue with the schedule.  Change current uses of SIGINT to abort a worker
into SIGTERM, which keeps the old behaviour of terminating the process.

Patch from ITAGAKI Takahiro, with some editorializing of my own.
2007-06-29 17:07:39 +00:00
Alvaro Herrera bae0b56880 Improve autovacuum launcher's ability to detect a problem in worker startup,
by having the postmaster signal it when certain failures occur.  This requires
the postmaster setting a flag in shared memory, but should be as safe as the
pmsignal.c code is.

Also make sure the launcher honor's a postgresql.conf change turning it off
on SIGHUP.
2007-06-25 16:09:03 +00:00
Alvaro Herrera a0a26c47d4 Avoid integer overflow issues in autovacuum. 2007-06-13 21:24:56 +00:00
Alvaro Herrera a4d5872719 Disallow the cost balancing code from resulting in a zero cost limit, which
causes a division-by-zero error in the vacuum code.  This can happen when there
are more workers than cost limit units.

Per report from Galy Lee in
<200705310914.l4V9E6JA094603@wwwmaster.postgresql.org>.
2007-06-08 21:21:28 +00:00
Alvaro Herrera 2b438c12cc Avoid passing zero as a value for vacuum_cost_limit, because it's not a valid
value for the vacuum code.  Instead, make zero signify getting the value from a
higher level configuration facility, just like -1 in the original coding.  We
still document that -1 is the value that disables the feature, to avoid
confusing the user unnecessarily.

Reported by Galy Lee in <200705310914.l4V9E6JA094603@wwwmaster.postgresql.org>;
per subsequent discussion.
2007-06-08 21:09:49 +00:00
Tom Lane d526575f89 Make large sequential scans and VACUUMs work in a limited-size "ring" of
buffers, rather than blowing out the whole shared-buffer arena.  Aside from
avoiding cache spoliation, this fixes the problem that VACUUM formerly tended
to cause a WAL flush for every page it modified, because we had it hacked to
use only a single buffer.  Those flushes will now occur only once per
ring-ful.  The exact ring size, and the threshold for seqscans to switch into
the ring usage pattern, remain under debate; but the infrastructure seems
done.  The key bit of infrastructure is a new optional BufferAccessStrategy
object that can be passed to ReadBuffer operations; this replaces the former
StrategyHintVacuum API.

This patch also changes the buffer usage-count methodology a bit: we now
advance usage_count when first pinning a buffer, rather than when last
unpinning it.  To preserve the behavior that a buffer's lifetime starts to
decrease when it's released, the clock sweep code is modified to not decrement
usage_count of pinned buffers.

Work not done in this commit: teach GiST and GIN indexes to use the vacuum
BufferAccessStrategy for vacuum-driven fetches.

Original patch by Simon, reworked by Heikki and again by Tom.
2007-05-30 20:12:03 +00:00