Add auto-vacuum emails.
This commit is contained in:
Bruce Momjian
parent
2609ccc34b
commit
eb978845e4
|
|
@ -1403,7 +1403,7 @@ From owner-pgsql-hackers@hub.org Sat Jan 22 02:31:03 2000
|
|||
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
|
||||
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA06743
|
||||
for <pgman@candle.pha.pa.us>; Sat, 22 Jan 2000 03:31:02 -0500 (EST)
|
||||
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id DAA07529 for <pgman@candle.pha.pa.us>; Sat, 22 Jan 2000 03:25:13 -0500 (EST)
|
||||
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.7 $) with ESMTP id DAA07529 for <pgman@candle.pha.pa.us>; Sat, 22 Jan 2000 03:25:13 -0500 (EST)
|
||||
Received: from localhost (majordom@localhost)
|
||||
by hub.org (8.9.3/8.9.3) with SMTP id DAA31900;
|
||||
Sat, 22 Jan 2000 03:19:53 -0500 (EST)
|
||||
|
|
@ -1475,7 +1475,7 @@ From tgl@sss.pgh.pa.us Sat Jan 22 10:31:02 2000
|
|||
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
|
||||
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id LAA20882
|
||||
for <pgman@candle.pha.pa.us>; Sat, 22 Jan 2000 11:31:00 -0500 (EST)
|
||||
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id LAA26612 for <pgman@candle.pha.pa.us>; Sat, 22 Jan 2000 11:12:44 -0500 (EST)
|
||||
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2]) by renoir.op.net (o1/$Revision: 1.7 $) with ESMTP id LAA26612 for <pgman@candle.pha.pa.us>; Sat, 22 Jan 2000 11:12:44 -0500 (EST)
|
||||
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
||||
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id LAA20569;
|
||||
Sat, 22 Jan 2000 11:11:26 -0500 (EST)
|
||||
|
|
@ -1868,3 +1868,259 @@ Http://www.rhyme.com.au | / \|
|
|||
PGP key available upon request, | /
|
||||
and from pgp5.ai.mit.edu:11371 |/
|
||||
|
||||
From pgsql-hackers-owner+M29308@postgresql.org Mon Sep 23 09:47:54 2002
|
||||
Return-path: <pgsql-hackers-owner+M29308@postgresql.org>
|
||||
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
||||
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8NDlqd00289
|
||||
for <pgman@candle.pha.pa.us>; Mon, 23 Sep 2002 09:47:53 -0400 (EDT)
|
||||
Received: from localhost (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with ESMTP
|
||||
id 7CA64476497; Mon, 23 Sep 2002 09:43:28 -0400 (EDT)
|
||||
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with SMTP
|
||||
id EDA70475BC3; Mon, 23 Sep 2002 09:43:20 -0400 (EDT)
|
||||
Received: from localhost (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with ESMTP id 85264476479
|
||||
for <pgsql-hackers@postgresql.org>; Mon, 23 Sep 2002 09:43:15 -0400 (EDT)
|
||||
Received: from www.pspl.co.in (www.pspl.co.in [202.54.11.65])
|
||||
by postgresql.org (Postfix) with ESMTP id C7899476477
|
||||
for <pgsql-hackers@postgresql.org>; Mon, 23 Sep 2002 09:43:12 -0400 (EDT)
|
||||
Received: (from root@localhost)
|
||||
by www.pspl.co.in (8.11.6/8.11.6) id g8NDiQ030526
|
||||
for <pgsql-hackers@postgresql.org>; Mon, 23 Sep 2002 19:14:26 +0530
|
||||
Received: from daithan (daithan.intranet.pspl.co.in [192.168.7.161])
|
||||
by www.pspl.co.in (8.11.6/8.11.0) with ESMTP id g8NDiQ330521;
|
||||
Mon, 23 Sep 2002 19:14:26 +0530
|
||||
From: "Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
|
||||
To: pgsql-hackers@postgresql.org, pgsql-general@postgresql.org
|
||||
Date: Mon, 23 Sep 2002 19:13:44 +0530
|
||||
MIME-Version: 1.0
|
||||
Subject: [HACKERS] Postgresql Automatic vacuum
|
||||
Reply-To: shridhar_daithankar@persistent.co.in
|
||||
Message-ID: <3D8F67E8.7500.4E0E180@localhost>
|
||||
X-Mailer: Pegasus Mail for Windows (v4.02)
|
||||
Content-Type: text/plain; charset=US-ASCII
|
||||
Content-Transfer-Encoding: 7BIT
|
||||
Content-Description: Mail message body
|
||||
X-Virus-Scanned: by AMaViS new-20020517
|
||||
Precedence: bulk
|
||||
Sender: pgsql-hackers-owner@postgresql.org
|
||||
X-Virus-Scanned: by AMaViS new-20020517
|
||||
Status: OR
|
||||
|
||||
Hello All,
|
||||
|
||||
I have written a small daemon that can automatically vacuum PostgreSQL
|
||||
database, depending upon activity per table.
|
||||
|
||||
It sits on top of postgres statistics collector. The postgres installation
|
||||
should have per row statistics collection enabled.
|
||||
|
||||
Features are,
|
||||
|
||||
* Vacuuming based on activity on the table
|
||||
* Per table vacuum. So only heavily updated tables are vacuumed.
|
||||
* multiple databases supported
|
||||
* Performs 'vacuum analyze' only, so it will not block the database
|
||||
|
||||
|
||||
The project location is
|
||||
http://gborg.postgresql.org/project/pgavd/projdisplay.php
|
||||
|
||||
Let me know for bugs/improvements and comments..
|
||||
|
||||
I am sure real world postgres installations has some sort of scripts doing
|
||||
similar thing. This is an attempt to provide a generic interface to periodic
|
||||
vacuum.
|
||||
|
||||
|
||||
Bye
|
||||
Shridhar
|
||||
|
||||
--
|
||||
The Abrams' Principle: The shortest distance between two points is off the
|
||||
wall.
|
||||
|
||||
|
||||
---------------------------(end of broadcast)---------------------------
|
||||
TIP 3: if posting/reading through Usenet, please send an appropriate
|
||||
subscribe-nomail command to majordomo@postgresql.org so that your
|
||||
message can get through to the mailing list cleanly
|
||||
|
||||
From pgsql-hackers-owner+M29344@postgresql.org Tue Sep 24 02:42:36 2002
|
||||
Return-path: <pgsql-hackers-owner+M29344@postgresql.org>
|
||||
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
||||
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8O6gYg19416
|
||||
for <pgman@candle.pha.pa.us>; Tue, 24 Sep 2002 02:42:35 -0400 (EDT)
|
||||
Received: from localhost (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with ESMTP
|
||||
id 128704762AF; Tue, 24 Sep 2002 02:42:36 -0400 (EDT)
|
||||
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with SMTP
|
||||
id DE80C4760F5; Tue, 24 Sep 2002 02:42:32 -0400 (EDT)
|
||||
Received: from localhost (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with ESMTP id 40A8A475DBC
|
||||
for <pgsql-hackers@postgresql.org>; Tue, 24 Sep 2002 02:42:25 -0400 (EDT)
|
||||
Received: from relay.icomedias.com (relay.icomedias.com [62.99.232.66])
|
||||
by postgresql.org (Postfix) with ESMTP id 7ECC8475DAD
|
||||
for <pgsql-hackers@postgresql.org>; Tue, 24 Sep 2002 02:42:23 -0400 (EDT)
|
||||
Received: from loki ([10.192.17.128])
|
||||
by relay.icomedias.com (8.12.5/8.12.5) with ESMTP id g8O6g8BX014226;
|
||||
Tue, 24 Sep 2002 08:42:09 +0200
|
||||
Content-Type: text/plain;
|
||||
charset="iso-8859-1"
|
||||
From: Mario Weilguni <mweilguni@sime.com>
|
||||
To: shridhar_daithankar@persistent.co.in, matthew@zeut.net
|
||||
Subject: Re: [HACKERS] Postgresql Automatic vacuum
|
||||
Date: Tue, 24 Sep 2002 08:42:06 +0200
|
||||
User-Agent: KMail/1.4.3
|
||||
cc: pgsql-hackers@postgresql.org
|
||||
References: <3D8F67E8.7500.4E0E180@localhost> <3D9050B2.9782.86E55C0@localhost>
|
||||
In-Reply-To: <3D9050B2.9782.86E55C0@localhost>
|
||||
MIME-Version: 1.0
|
||||
Message-ID: <200209240842.06459.mweilguni@sime.com>
|
||||
avpresult: 0, ok, ok
|
||||
X-Scanned-By: MIMEDefang 2.16 (www . roaringpenguin . com / mimedefang)
|
||||
X-Virus-Scanned: by AMaViS new-20020517
|
||||
Precedence: bulk
|
||||
Sender: pgsql-hackers-owner@postgresql.org
|
||||
X-Virus-Scanned: by AMaViS new-20020517
|
||||
Content-Transfer-Encoding: 8bit
|
||||
X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g8O6gYg19416
|
||||
Status: OR
|
||||
|
||||
Am Dienstag, 24. September 2002 08:16 schrieb Shridhar Daithankar:
|
||||
>
|
||||
> > I will play with it more and give you some more feedback.
|
||||
>
|
||||
> Awaiting that.
|
||||
>
|
||||
|
||||
IMO there are still several problems with that approach, namely:
|
||||
* every database will get "polluted" with the autovacuum table, which is undesired
|
||||
* the biggest problem is the ~/.pgavrc file. I think it should work like other postgres utils do, e.g. supporting -U, -d, ....
|
||||
* it's not possible to use without activly administration the config file. it should be able to work without
|
||||
adminstrator assistance.
|
||||
|
||||
When this is a daemon, why not store the data in memory? Even with several thousands of tables the memory footprint would
|
||||
still be small. And it should be possible to use for all databases without modifying a config file.
|
||||
|
||||
Two weeks ago I began writing a similar daemon, but had no time yet to finish it. I've tried to avoid using fixed numbers (namely "vacuum table
|
||||
after 1000 updates") and tried to make my own heuristic based on the statistics data and the size of the table. The reason is, for a large table 1000 entries might be
|
||||
a small percentage and vacuum is not necessary, while for small tables 10 updates might be sufficient.
|
||||
|
||||
Best regards,
|
||||
Mario Weilguni
|
||||
|
||||
|
||||
---------------------------(end of broadcast)---------------------------
|
||||
TIP 2: you can get off all lists at once with the unregister command
|
||||
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
||||
|
||||
From pgsql-hackers-owner+M29345@postgresql.org Tue Sep 24 03:02:50 2002
|
||||
Return-path: <pgsql-hackers-owner+M29345@postgresql.org>
|
||||
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
||||
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8O72lg21051
|
||||
for <pgman@candle.pha.pa.us>; Tue, 24 Sep 2002 03:02:48 -0400 (EDT)
|
||||
Received: from localhost (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with ESMTP
|
||||
id 9B3EA4762F6; Tue, 24 Sep 2002 03:02:48 -0400 (EDT)
|
||||
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with SMTP
|
||||
id 902EA476020; Tue, 24 Sep 2002 03:02:45 -0400 (EDT)
|
||||
Received: from localhost (postgresql.org [64.49.215.8])
|
||||
by postgresql.org (Postfix) with ESMTP id 98689475DAD
|
||||
for <pgsql-hackers@postgresql.org>; Tue, 24 Sep 2002 03:02:18 -0400 (EDT)
|
||||
Received: from www.pspl.co.in (www.pspl.co.in [202.54.11.65])
|
||||
by postgresql.org (Postfix) with ESMTP id 47B8647592C
|
||||
for <pgsql-hackers@postgresql.org>; Tue, 24 Sep 2002 03:02:16 -0400 (EDT)
|
||||
Received: (from root@localhost)
|
||||
by www.pspl.co.in (8.11.6/8.11.6) id g8O73QQ16318
|
||||
for <pgsql-hackers@postgresql.org>; Tue, 24 Sep 2002 12:33:26 +0530
|
||||
Received: from daithan (daithan.intranet.pspl.co.in [192.168.7.161])
|
||||
by www.pspl.co.in (8.11.6/8.11.0) with ESMTP id g8O73Q316313
|
||||
for <pgsql-hackers@postgresql.org>; Tue, 24 Sep 2002 12:33:26 +0530
|
||||
From: "Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
|
||||
To: pgsql-hackers@postgresql.org
|
||||
Date: Tue, 24 Sep 2002 12:32:43 +0530
|
||||
MIME-Version: 1.0
|
||||
Subject: Re: [HACKERS] Postgresql Automatic vacuum
|
||||
Reply-To: shridhar_daithankar@persistent.co.in
|
||||
Message-ID: <3D905B6B.1635.898382A@localhost>
|
||||
References: <3D9050B2.9782.86E55C0@localhost>
|
||||
In-Reply-To: <200209240842.06459.mweilguni@sime.com>
|
||||
X-Mailer: Pegasus Mail for Windows (v4.02)
|
||||
Content-Type: text/plain; charset=US-ASCII
|
||||
Content-Transfer-Encoding: 7BIT
|
||||
Content-Description: Mail message body
|
||||
X-Virus-Scanned: by AMaViS new-20020517
|
||||
Precedence: bulk
|
||||
Sender: pgsql-hackers-owner@postgresql.org
|
||||
X-Virus-Scanned: by AMaViS new-20020517
|
||||
Status: OR
|
||||
|
||||
On 24 Sep 2002 at 8:42, Mario Weilguni wrote:
|
||||
|
||||
> Am Dienstag, 24. September 2002 08:16 schrieb Shridhar Daithankar:
|
||||
> IMO there are still several problems with that approach, namely:
|
||||
> * every database will get "polluted" with the autovacuum table, which is undesired
|
||||
|
||||
I agree. But that was the best alternative I could see. explanation
|
||||
follows..Besides I didn't want to touch PG meta data..
|
||||
|
||||
> * the biggest problem is the ~/.pgavrc file. I think it should work like other postgres utils do, e.g. supporting -U, -d, ....
|
||||
|
||||
Shouldn't be a problem. The config stuff is working and I can add that. I would
|
||||
rather term it a minor issue. On personal preference, I would just fire it
|
||||
without any arguments. It's not a thing that you change daily. Configure it in
|
||||
config file and done..
|
||||
|
||||
> * it's not possible to use without activly administration the config file. it should be able to work without
|
||||
> adminstrator assistance.
|
||||
|
||||
Well. I would call that tuning. Each admin can tune it. Yes it's an effort but
|
||||
certainly not an active administration.
|
||||
|
||||
> When this is a daemon, why not store the data in memory? Even with several thousands of tables the memory footprint would
|
||||
> still be small. And it should be possible to use for all databases without modifying a config file.
|
||||
|
||||
Well. When postgresql has ability to deal with arbitrary number of rows, it
|
||||
seemed redundant to me to duplicate all those functionality. Why write lists
|
||||
and arrays again and again? Let postgresql do it.
|
||||
|
||||
|
||||
> Two weeks ago I began writing a similar daemon, but had no time yet to finish it. I've tried to avoid using fixed numbers (namely "vacuum table
|
||||
> after 1000 updates") and tried to make my own heuristic based on the statistics data and the size of the table. The reason is, for a large table 1000 entries might be
|
||||
> a small percentage and vacuum is not necessary, while for small tables 10 updates might be sufficient.
|
||||
|
||||
Well, that fixed number is not really fixed but admin tunable, that too per
|
||||
database. These are just defaults. Tune it to suit your needs.
|
||||
|
||||
The objective of whole exercise is to get rid of periodic vacuum as this app.
|
||||
shifts threshold to activity rather than time.
|
||||
|
||||
Besides a table should be vacuumed when it starts affecting performance. On an
|
||||
installation if a table a 1M rows and change 1K rows affects performance, there
|
||||
will be a similar performance hit for a 100K rows table for 1K rows update.
|
||||
Because overhead involved would be almost same.(Not disk space. pgavd does not
|
||||
target vacuum full but tuple size should matter).
|
||||
|
||||
At least me thinks so..
|
||||
|
||||
I plan to implement per table threshold in addition to per database thresholds.
|
||||
But right now, it seems like overhead to me. Besides there is an item in TODO,
|
||||
to shift unit of work from rows to blocks affected. I guess that takes care of
|
||||
some of your points..
|
||||
Bye
|
||||
Shridhar
|
||||
|
||||
--
|
||||
Jones' Second Law: The man who smiles when things go wrong has thought of
|
||||
someone to blame it on.
|
||||
|
||||
|
||||
---------------------------(end of broadcast)---------------------------
|
||||
TIP 5: Have you checked our extensive FAQ?
|
||||
|
||||
http://www.postgresql.org/users-lounge/docs/faq.html
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue