2223 lines
99 KiB
Plaintext
2223 lines
99 KiB
Plaintext
From pgsql-hackers-owner+M3040@hub.org Thu Jun 8 00:31:01 2000
|
|
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA13157
|
|
for <pgman@candle.pha.pa.us>; Thu, 8 Jun 2000 00:31:00 -0400 (EDT)
|
|
Received: from hub.org (root@hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id AAA01089 for <pgman@candle.pha.pa.us>; Thu, 8 Jun 2000 00:17:19 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e5846ib99782;
|
|
Thu, 8 Jun 2000 00:06:44 -0400 (EDT)
|
|
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e5846Xb99707
|
|
for <pgsql-hackers@postgreSQL.org>; Thu, 8 Jun 2000 00:06:33 -0400 (EDT)
|
|
Received: from cadzone ([126.0.1.40] (may be forged))
|
|
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
|
|
id NAA01145; Thu, 08 Jun 2000 13:05:42 +0900
|
|
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
To: "Bruce Momjian" <pgman@candle.pha.pa.us>
|
|
Cc: "PostgreSQL-development" <pgsql-hackers@postgreSQL.org>
|
|
Subject: RE: [HACKERS] DROP COLUMN status
|
|
Date: Thu, 8 Jun 2000 13:07:44 +0900
|
|
Message-ID: <000d01bfd0ff$194d56c0$2801007e@tpf.co.jp>
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3 (Normal)
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
|
|
In-Reply-To: <200006080309.XAA10305@candle.pha.pa.us>
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
|
|
Importance: Normal
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
> -----Original Message-----
|
|
> From: pgsql-hackers-owner@hub.org [mailto:pgsql-hackers-owner@hub.org]On
|
|
> Behalf Of Bruce Momjian
|
|
>
|
|
> Can someone comment on where we are with DROP COLUMN?
|
|
>
|
|
|
|
I've already committed my trial implementation 3 months ago.
|
|
They are $ifdef'd by _DROP_COLUMN_HACK__.
|
|
Please enable the feature and evaluate it.
|
|
You could enable the feature without initdb.
|
|
|
|
Regards.
|
|
|
|
Hiroshi Inoue
|
|
Inoue@tpf.co.jp
|
|
|
|
|
|
From Inoue@tpf.co.jp Thu Jun 8 02:03:27 2000
|
|
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id CAA14243
|
|
for <pgman@candle.pha.pa.us>; Thu, 8 Jun 2000 02:03:25 -0400 (EDT)
|
|
Received: from cadzone ([126.0.1.40] (may be forged))
|
|
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
|
|
id PAA01221; Thu, 08 Jun 2000 15:03:23 +0900
|
|
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
To: "Bruce Momjian" <pgman@candle.pha.pa.us>
|
|
Cc: "PostgreSQL-development" <pgsql-hackers@postgreSQL.org>
|
|
Subject: RE: [HACKERS] DROP COLUMN status
|
|
Date: Thu, 8 Jun 2000 15:05:24 +0900
|
|
Message-ID: <000f01bfd10f$893798a0$2801007e@tpf.co.jp>
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3 (Normal)
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
|
|
In-Reply-To: <200006080457.AAA13430@candle.pha.pa.us>
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
|
|
Importance: Normal
|
|
Status: OR
|
|
|
|
> -----Original Message-----
|
|
> From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
|
|
> Sent: Thursday, June 08, 2000 1:58 PM
|
|
>
|
|
> [ Charset ISO-8859-1 unsupported, converting... ]
|
|
> > > -----Original Message-----
|
|
> > > From: pgsql-hackers-owner@hub.org
|
|
> [mailto:pgsql-hackers-owner@hub.org]On
|
|
> > > Behalf Of Bruce Momjian
|
|
> > >
|
|
> > > Can someone comment on where we are with DROP COLUMN?
|
|
> > >
|
|
> >
|
|
> > I've already committed my trial implementation 3 months ago.
|
|
> > They are $ifdef'd by _DROP_COLUMN_HACK__.
|
|
> > Please enable the feature and evaluate it.
|
|
> > You could enable the feature without initdb.
|
|
>
|
|
> OK, can you explain how it works, and add any needed documentation so we
|
|
> can enable it.
|
|
>
|
|
|
|
First it's only a trial so I don't implement it completely.
|
|
Especially I don't completely drop related objects
|
|
(FK_constraint,triggers,views etc). I don't know whether
|
|
we could drop them properly or not.
|
|
|
|
The implementation makes the dropped column invisible by
|
|
changing its attnum to -attnum - offset(currently 20) and
|
|
attnam to ("*already Dropped%d",attnum). It doesn't touch
|
|
the table at all. After dropping a column insert/update
|
|
operation regards the column as NULL and other related
|
|
stuff simply ignores the column.
|
|
|
|
Regards.
|
|
|
|
Hiroshi Inoue
|
|
Inoue@tpf.co.jp
|
|
|
|
From tgl@sss.pgh.pa.us Thu Jun 8 10:20:34 2000
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA29148
|
|
for <pgman@candle.pha.pa.us>; Thu, 8 Jun 2000 10:20:33 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id KAA15725;
|
|
Thu, 8 Jun 2000 10:20:11 -0400 (EDT)
|
|
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"PostgreSQL-development" <pgsql-hackers@postgresql.org>
|
|
Subject: Re: [HACKERS] DROP COLUMN status
|
|
In-reply-to: <000f01bfd10f$893798a0$2801007e@tpf.co.jp>
|
|
References: <000f01bfd10f$893798a0$2801007e@tpf.co.jp>
|
|
Comments: In-reply-to "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
message dated "Thu, 08 Jun 2000 15:05:24 +0900"
|
|
Date: Thu, 08 Jun 2000 10:20:11 -0400
|
|
Message-ID: <15722.960474011@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Status: ORr
|
|
|
|
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
|
|
> The implementation makes the dropped column invisible by
|
|
> changing its attnum to -attnum - offset(currently 20) and
|
|
> attnam to ("*already Dropped%d",attnum).
|
|
|
|
Ugh. No wonder you had to hack so many places in such an ugly fashion.
|
|
Why not leave the attnum as-is, and just add a bool saying "column is
|
|
dropped" to pg_attribute? As long as the parser ignores columns marked
|
|
that way for field lookup and expansion of *, it seems the rest of the
|
|
system wouldn't need to treat dropped columns specially in any way.
|
|
|
|
regards, tom lane
|
|
|
|
From pgsql-hackers-owner+M3094@hub.org Thu Jun 8 15:58:30 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id PAA25109
|
|
for <pgman@candle.pha.pa.us>; Thu, 8 Jun 2000 15:58:28 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e58JrqT91713;
|
|
Thu, 8 Jun 2000 15:53:52 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e58JqpT91436
|
|
for <pgsql-hackers@postgreSQL.org>; Thu, 8 Jun 2000 15:52:51 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id PAA19690;
|
|
Thu, 8 Jun 2000 15:52:43 -0400 (EDT)
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: Hiroshi Inoue <Inoue@tpf.co.jp>,
|
|
PostgreSQL-development <pgsql-hackers@postgreSQL.org>
|
|
Subject: Re: [HACKERS] DROP COLUMN status
|
|
In-reply-to: <200006081541.LAA01566@candle.pha.pa.us>
|
|
References: <200006081541.LAA01566@candle.pha.pa.us>
|
|
Comments: In-reply-to Bruce Momjian <pgman@candle.pha.pa.us>
|
|
message dated "Thu, 08 Jun 2000 11:41:43 -0400"
|
|
Date: Thu, 08 Jun 2000 15:52:43 -0400
|
|
Message-ID: <19687.960493963@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
>>>> The implementation makes the dropped column invisible by
|
|
>>>> changing its attnum to -attnum - offset(currently 20) and
|
|
>>>> attnam to ("*already Dropped%d",attnum).
|
|
>>
|
|
>> Ugh. No wonder you had to hack so many places in such an ugly fashion.
|
|
>> Why not leave the attnum as-is, and just add a bool saying "column is
|
|
>> dropped" to pg_attribute? As long as the parser ignores columns marked
|
|
>> that way for field lookup and expansion of *, it seems the rest of the
|
|
>> system wouldn't need to treat dropped columns specially in any way.
|
|
|
|
> If we leave it as positive, don't we have to change user applications
|
|
> that query pg_attribute so they also know to skip it?
|
|
|
|
Good point, but I think user applications that query pg_attribute
|
|
are likely to have trouble anyway: if they're expecting a consecutive
|
|
series of attnums then they're going to lose no matter what.
|
|
|
|
regards, tom lane
|
|
|
|
From hannu@tm.ee Sat Jun 10 01:02:57 2000
|
|
Received: from me.tm.ee (ppp15.tele2.ee [212.107.33.15])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA10377
|
|
for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 01:02:55 -0400 (EDT)
|
|
Received: from tm.ee (IDENT:hannu@localhost.localdomain [127.0.0.1])
|
|
by me.tm.ee (8.9.3/8.9.3) with ESMTP id GAA00940;
|
|
Sat, 10 Jun 2000 06:59:33 +0300
|
|
Sender: hannu@me.tm.ee
|
|
Message-ID: <3941BD25.96760D2E@tm.ee>
|
|
Date: Sat, 10 Jun 2000 06:59:33 +0300
|
|
From: Hannu Krosing <hannu@tm.ee>
|
|
X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.12-20 i686)
|
|
X-Accept-Language: en
|
|
MIME-Version: 1.0
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
CC: Tom Lane <tgl@sss.pgh.pa.us>, Peter Eisentraut <peter_e@gmx.net>,
|
|
PostgreSQL Development <pgsql-hackers@postgresql.org>
|
|
Subject: Re: [HACKERS] ALTER TABLE DROP COLUMN
|
|
References: <200006091249.IAA18730@candle.pha.pa.us>
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Transfer-Encoding: 7bit
|
|
Status: OR
|
|
|
|
Bruce Momjian wrote:
|
|
>
|
|
> Seems we have 4 DROP COLUMN ideas:
|
|
>
|
|
> Method Advantage
|
|
> -----------------------------------------------------------------
|
|
> 1 invisible column marked by negative attnum fast
|
|
> 2 invisible column marked by is_dropped column fast
|
|
> 3 make copy of table without column col removed
|
|
> 4 make new tuples in existing table without column col removed
|
|
|
|
IIRC there was a fifth idea, a variation of 2 that would work better
|
|
with
|
|
inheritance -
|
|
|
|
5 all columns have is_real_column attribute that is true for all
|
|
coluns
|
|
present in that relation, so situations like
|
|
|
|
create table tab_a(a_i int);
|
|
create table tab_b(b_i int) inherits(tab_a);
|
|
alter table tab_a add column c_i int;
|
|
|
|
can be made to work.
|
|
|
|
It would also require clients to ignore all missing columns that backend
|
|
can
|
|
pass to them as nulls (which is usually quite cheap in bandwith usage)
|
|
in
|
|
case of "SELECT **" queries.
|
|
|
|
We could even rename attno to attid to make folks aware that it is not
|
|
be
|
|
assumed to be continuous.
|
|
|
|
> Folks, we had better choose one and get started.
|
|
>
|
|
> Number 1 Hiroshi has ifdef'ed out in the code. Items 1 and 2 have
|
|
> problems with backend code and 3rd party code not seeing the dropped
|
|
> columns, or having gaps in the attno numbering.
|
|
|
|
If we want to make ADD COLUMN to work with inheritance wihout having to
|
|
rewrite every single tuple in both parent and inherited tables, we will
|
|
have to accept the fact that there are caps in in attno numbering.
|
|
|
|
> Number 3 has problems
|
|
> with making it an atomic operation, and number 4 is described below.
|
|
|
|
Nr 4 has still problems with attno numbering _changing_ during drop
|
|
which
|
|
could either be better or worse for client software than having gaps -
|
|
in both cases client must be prepared to deal with runtime changes in
|
|
attribute definition.
|
|
|
|
--------------
|
|
Hannu
|
|
|
|
From Inoue@tpf.co.jp Sat Jun 10 01:01:01 2000
|
|
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA10355
|
|
for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 01:01:00 -0400 (EDT)
|
|
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id AAA25467 for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 00:41:32 -0400 (EDT)
|
|
Received: from mcadnote1 (ppm110.noc.fukui.nsk.ne.jp [210.161.188.29] (may be forged))
|
|
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
|
|
id NAA03125; Sat, 10 Jun 2000 13:40:40 +0900
|
|
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
To: "Bruce Momjian" <pgman@candle.pha.pa.us>, "Tom Lane" <tgl@sss.pgh.pa.us>
|
|
Cc: "Peter Eisentraut" <peter_e@gmx.net>,
|
|
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
|
|
Subject: RE: [HACKERS] ALTER TABLE DROP COLUMN
|
|
Date: Sat, 10 Jun 2000 13:43:26 +0900
|
|
Message-ID: <EKEJJICOHDIEMGPNIFIJEELACBAA.Inoue@tpf.co.jp>
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="US-ASCII"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3 (Normal)
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
|
|
In-Reply-To: <200006091249.IAA18730@candle.pha.pa.us>
|
|
Importance: Normal
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
|
|
Status: ORr
|
|
|
|
> -----Original Message-----
|
|
> From: pgsql-hackers-owner@hub.org
|
|
> [mailto:pgsql-hackers-owner@hub.org]On Behalf Of Bruce Momjian
|
|
>
|
|
> Seems we have 4 DROP COLUMN ideas:
|
|
>
|
|
> Method Advantage
|
|
> -----------------------------------------------------------------
|
|
> 1 invisible column marked by negative attnum fast
|
|
> 2 invisible column marked by is_dropped column fast
|
|
> 3 make copy of table without column col removed
|
|
> 4 make new tuples in existing table without column col removed
|
|
>
|
|
> Folks, we had better choose one and get started.
|
|
>
|
|
> Number 1 Hiroshi has ifdef'ed out in the code. Items 1 and 2 have
|
|
> problems with backend code and 3rd party code not seeing the dropped
|
|
> columns,
|
|
|
|
Hmm,doesn't *not seeing* mean the column is dropped ?
|
|
|
|
> or having gaps in the attno numbering. Number 3 has problems
|
|
> with making it an atomic operation, and number 4 is described below.
|
|
>
|
|
|
|
Don't forget another important point.
|
|
|
|
Currently even DROP TABLE doesn't remove related objects completely.
|
|
And I don't think I could remove objects related to the dropping column
|
|
completely using 1)2) in ALTER TABLE DROP COLUMN implementation.
|
|
|
|
Using 3)4) we should not only remove objects as 1)2) but also
|
|
change attnum-s in all objects related to the relation. Otherwise
|
|
PostgreSQL would do the wrong thing silently.
|
|
|
|
Regards.
|
|
|
|
Hiroshi Inoue
|
|
Inoue@tpf.co.jp
|
|
|
|
From dhogaza@pacifier.com Sat Jun 10 01:01:06 2000
|
|
Received: from smtp.pacifier.com (comet.pacifier.com [199.2.117.155])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA10370
|
|
for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 01:01:04 -0400 (EDT)
|
|
Received: from desktop (dsl-dhogaza.pacifier.net [207.202.226.68])
|
|
by smtp.pacifier.com (8.9.3/8.9.3pop) with SMTP id WAA08521;
|
|
Fri, 9 Jun 2000 22:01:00 -0700 (PDT)
|
|
Message-Id: <3.0.1.32.20000609215758.0116d850@mail.pacifier.com>
|
|
X-Sender: dhogaza@mail.pacifier.com
|
|
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
|
|
Date: Fri, 09 Jun 2000 21:57:58 -0700
|
|
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>,
|
|
"Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Tom Lane" <tgl@sss.pgh.pa.us>
|
|
From: Don Baccus <dhogaza@pacifier.com>
|
|
Subject: RE: [HACKERS] ALTER TABLE DROP COLUMN
|
|
Cc: "Peter Eisentraut" <peter_e@gmx.net>,
|
|
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
|
|
In-Reply-To: <EKEJJICOHDIEMGPNIFIJEELACBAA.Inoue@tpf.co.jp>
|
|
References: <200006091249.IAA18730@candle.pha.pa.us>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset="us-ascii"
|
|
Status: OR
|
|
|
|
At 01:43 PM 6/10/00 +0900, Hiroshi Inoue wrote:
|
|
>> -----Original Message-----
|
|
>> From: pgsql-hackers-owner@hub.org
|
|
>> [mailto:pgsql-hackers-owner@hub.org]On Behalf Of Bruce Momjian
|
|
>>
|
|
>> Seems we have 4 DROP COLUMN ideas:
|
|
>>
|
|
>> Method Advantage
|
|
>> -----------------------------------------------------------------
|
|
>> 1 invisible column marked by negative attnum fast
|
|
>> 2 invisible column marked by is_dropped column fast
|
|
>> 3 make copy of table without column col removed
|
|
>> 4 make new tuples in existing table without column col removed
|
|
>>
|
|
>> Folks, we had better choose one and get started.
|
|
|
|
Oracle gives you the choice between the "cheating" fast method(s) and
|
|
the "real" slow (really slow?) real method.
|
|
|
|
So there's at least real world experience by virtue of example by
|
|
the world's most successful database supplier that user control
|
|
over "hide the column" and "really delete the column" is valuable.
|
|
|
|
It really makes a lot of sense to give such a choice. If one
|
|
does so by "hiding", at a later date one would think the choice
|
|
of "really deleting" would be a possibility. I don't know if
|
|
Oracle does this...
|
|
|
|
If not, they might not care. In today's world, there are bazillions
|
|
of dollars for Oracle to scoop up from users who could just as easily
|
|
be PG users - all those "we'll fail if don't IPO 'cause we'll never
|
|
have any customers" database-backed websites :)
|
|
|
|
|
|
|
|
- Don Baccus, Portland OR <dhogaza@pacifier.com>
|
|
Nature photos, on-line guides, Pacific Northwest
|
|
Rare Bird Alert Service and other goodies at
|
|
http://donb.photo.net.
|
|
|
|
From tgl@sss.pgh.pa.us Sat Jun 10 01:31:04 2000
|
|
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA10922
|
|
for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 01:31:03 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id BAA27265 for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 01:16:07 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id BAA06206;
|
|
Sat, 10 Jun 2000 01:14:37 -0400 (EDT)
|
|
To: Don Baccus <dhogaza@pacifier.com>
|
|
cc: "Hiroshi Inoue" <Inoue@tpf.co.jp>,
|
|
"Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Peter Eisentraut" <peter_e@gmx.net>,
|
|
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
|
|
Subject: Re: [HACKERS] ALTER TABLE DROP COLUMN
|
|
In-reply-to: <3.0.1.32.20000609215758.0116d850@mail.pacifier.com>
|
|
References: <200006091249.IAA18730@candle.pha.pa.us> <3.0.1.32.20000609215758.0116d850@mail.pacifier.com>
|
|
Comments: In-reply-to Don Baccus <dhogaza@pacifier.com>
|
|
message dated "Fri, 09 Jun 2000 21:57:58 -0700"
|
|
Date: Sat, 10 Jun 2000 01:14:37 -0400
|
|
Message-ID: <6203.960614077@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Status: OR
|
|
|
|
Don Baccus <dhogaza@pacifier.com> writes:
|
|
> Oracle gives you the choice between the "cheating" fast method(s) and
|
|
> the "real" slow (really slow?) real method.
|
|
|
|
> So there's at least real world experience by virtue of example by
|
|
> the world's most successful database supplier that user control
|
|
> over "hide the column" and "really delete the column" is valuable.
|
|
|
|
Sure, but you don't need any help from the database to do "really delete
|
|
the column". SELECT INTO... is enough, and it's not even any slower
|
|
than the implementations under discussion.
|
|
|
|
So I'm satisfied if we offer the "hide the column" approach.
|
|
|
|
Has anyone thought about what happens to table constraints that use the
|
|
doomed column? Triggers, RI rules, yadda yadda?
|
|
|
|
Has anyone thought about undoing a DELETE COLUMN? The data is still
|
|
there, at least in tuples that have not been updated, so it's not
|
|
totally unreasonable.
|
|
|
|
regards, tom lane
|
|
|
|
From dhogaza@pacifier.com Sat Jun 10 09:30:59 2000
|
|
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id JAA25987
|
|
for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 09:30:58 -0400 (EDT)
|
|
Received: from smtp.pacifier.com (comet.pacifier.com [199.2.117.155]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id JAA18716 for <pgman@candle.pha.pa.us>; Sat, 10 Jun 2000 09:15:08 -0400 (EDT)
|
|
Received: from desktop (dsl-dhogaza.pacifier.net [207.202.226.68])
|
|
by smtp.pacifier.com (8.9.3/8.9.3pop) with SMTP id GAA15799;
|
|
Sat, 10 Jun 2000 06:14:28 -0700 (PDT)
|
|
Message-Id: <3.0.1.32.20000610054306.0115f020@mail.pacifier.com>
|
|
X-Sender: dhogaza@mail.pacifier.com
|
|
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
|
|
Date: Sat, 10 Jun 2000 05:43:06 -0700
|
|
To: Tom Lane <tgl@sss.pgh.pa.us>
|
|
From: Don Baccus <dhogaza@pacifier.com>
|
|
Subject: Re: [HACKERS] ALTER TABLE DROP COLUMN
|
|
Cc: "Hiroshi Inoue" <Inoue@tpf.co.jp>,
|
|
"Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Peter Eisentraut" <peter_e@gmx.net>,
|
|
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
|
|
In-Reply-To: <6203.960614077@sss.pgh.pa.us>
|
|
References: <3.0.1.32.20000609215758.0116d850@mail.pacifier.com>
|
|
<200006091249.IAA18730@candle.pha.pa.us>
|
|
<3.0.1.32.20000609215758.0116d850@mail.pacifier.com>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset="us-ascii"
|
|
Status: OR
|
|
|
|
At 01:14 AM 6/10/00 -0400, Tom Lane wrote:
|
|
>Don Baccus <dhogaza@pacifier.com> writes:
|
|
>> Oracle gives you the choice between the "cheating" fast method(s) and
|
|
>> the "real" slow (really slow?) real method.
|
|
>
|
|
>> So there's at least real world experience by virtue of example by
|
|
>> the world's most successful database supplier that user control
|
|
>> over "hide the column" and "really delete the column" is valuable.
|
|
>
|
|
>Sure, but you don't need any help from the database to do "really delete
|
|
>the column". SELECT INTO... is enough, and it's not even any slower
|
|
>than the implementations under discussion.
|
|
>
|
|
>So I'm satisfied if we offer the "hide the column" approach.
|
|
|
|
<shrug> I wouldn't put a "real" drop column at the top of my list
|
|
of priorities, but there is something to be said for user convenience.
|
|
|
|
|
|
|
|
- Don Baccus, Portland OR <dhogaza@pacifier.com>
|
|
Nature photos, on-line guides, Pacific Northwest
|
|
Rare Bird Alert Service and other goodies at
|
|
http://donb.photo.net.
|
|
|
|
From tgl@sss.pgh.pa.us Sun Jun 11 12:31:03 2000
|
|
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id MAA05771
|
|
for <pgman@candle.pha.pa.us>; Sun, 11 Jun 2000 12:31:01 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id MAA19315 for <pgman@candle.pha.pa.us>; Sun, 11 Jun 2000 12:24:06 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id MAA09503;
|
|
Sun, 11 Jun 2000 12:22:42 -0400 (EDT)
|
|
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Peter Eisentraut" <peter_e@gmx.net>,
|
|
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
|
|
Subject: Re: [HACKERS] ALTER TABLE DROP COLUMN
|
|
In-reply-to: <EKEJJICOHDIEMGPNIFIJEELACBAA.Inoue@tpf.co.jp>
|
|
References: <EKEJJICOHDIEMGPNIFIJEELACBAA.Inoue@tpf.co.jp>
|
|
Comments: In-reply-to "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
message dated "Sat, 10 Jun 2000 13:43:26 +0900"
|
|
Date: Sun, 11 Jun 2000 12:22:42 -0400
|
|
Message-ID: <9500.960740562@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Status: ORr
|
|
|
|
>> Seems we have 4 DROP COLUMN ideas:
|
|
>> Method Advantage
|
|
>> -----------------------------------------------------------------
|
|
>> 1 invisible column marked by negative attnum fast
|
|
>> 2 invisible column marked by is_dropped column fast
|
|
>> 3 make copy of table without column col removed
|
|
>> 4 make new tuples in existing table without column col removed
|
|
|
|
Bruce and I talked about this by phone yesterday, and we realized that
|
|
none of these are very satisfactory. #1 and #2 both have the flaw that
|
|
applications that examine pg_attribute will probably break: they will
|
|
see a sequence of attnum values with gaps in it. And what should the
|
|
rel's relnatts field be set to? #3 and #4 are better on that point,
|
|
but they leave us with the problem of renumbering references to columns
|
|
after the dropped one in constraints, rules, PL functions, etc.
|
|
|
|
Furthermore, there is a closely related problem that none of these
|
|
approaches give us much help on: recursive ALTER TABLE ADD COLUMN.
|
|
Right now, ADD puts the new column at the end of each table it's added
|
|
to, which often means that it gets a different column number in child
|
|
tables than in parent tables. That leads to havoc for pg_dump.
|
|
|
|
I think the only clean solution is to create a clear distinction between
|
|
physical and logical column numbers. Each pg_attribute tuple would need
|
|
two attnum fields, and pg_class would need two relnatts fields as well.
|
|
A column once created would never change its physical column number, but
|
|
its logical column number might change as a consequence of adding or
|
|
dropping columns before it. ADD COLUMN would ensure that a column added
|
|
to child tables receives the same logical column number as it has in the
|
|
parent table, thus solving the dump/reload problem. DROP COLUMN would
|
|
assign an invalid logical column number to dropped columns. They could
|
|
be numbered zero except that we'd probably still want a unique index on
|
|
attrelid+attnum, and the index would complain. I'd suggest using
|
|
Hiroshi's idea: give a dropped column a logical attnum equal to
|
|
-(physical_attnum + offset).
|
|
|
|
With this approach, internal operations on tuples would all use
|
|
physical column numbers, but operations that interface to the outside
|
|
world would present a view of only the valid logical columns. For
|
|
example, the parser would only allow logical columns to be referenced
|
|
by name; "SELECT *" would expand to valid logical columns in logical-
|
|
column-number order; COPY would send or receive valid logical columns
|
|
in logical-column-number order; etc.
|
|
|
|
Stored rules and so forth probably should store physical column numbers
|
|
so that they need not be modified during column add/drop.
|
|
|
|
This would require looking at all the places in the backend to determine
|
|
whether they should be working with logical or physical column numbers,
|
|
but the design is such that most all places would want to be using
|
|
physical numbers, so I don't think it'd be too painful.
|
|
|
|
Although I'd prefer to give the replacement columns two new names
|
|
(eg, "attlnum" and "attpnum") to ensure we find all uses, this would
|
|
surely break applications that examine pg_attribute. For compatibility
|
|
we'd have to recycle "attnum" and "relnatts" to indicate logical column
|
|
number and logical column count, while adding new fields (say "attpnum"
|
|
and "relnpatts") for the physical number and count.
|
|
|
|
Comments?
|
|
|
|
regards, tom lane
|
|
|
|
From pgsql-hackers-owner+M3184@hub.org Mon Jun 12 09:29:17 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id JAA16538
|
|
for <pgman@candle.pha.pa.us>; Mon, 12 Jun 2000 09:29:15 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e5C9RxT92685;
|
|
Mon, 12 Jun 2000 05:27:59 -0400 (EDT)
|
|
Received: from clio.trends.ca (root@clio.trends.ca [209.47.148.2])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e5C8YWT89945
|
|
for <pgsql-hackers@postgreSQL.org>; Mon, 12 Jun 2000 04:34:32 -0400 (EDT)
|
|
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
|
|
by clio.trends.ca (8.9.3+Sun/8.9.3) with ESMTP id VAA17711
|
|
for <pgsql-hackers@postgreSQL.org>; Sun, 11 Jun 2000 21:40:28 -0400 (EDT)
|
|
Received: from cadzone ([126.0.1.40] (may be forged))
|
|
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
|
|
id KAA03734; Mon, 12 Jun 2000 10:38:42 +0900
|
|
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
|
|
To: "Tom Lane" <tgl@sss.pgh.pa.us>
|
|
Cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Peter Eisentraut" <peter_e@gmx.net>,
|
|
"PostgreSQL Development" <pgsql-hackers@postgresql.org>
|
|
Subject: RE: [HACKERS] ALTER TABLE DROP COLUMN
|
|
Date: Mon, 12 Jun 2000 10:40:47 +0900
|
|
Message-ID: <000b01bfd40f$3b3091e0$2801007e@tpf.co.jp>
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="iso-2022-jp"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3 (Normal)
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
|
|
In-Reply-To: <9500.960740562@sss.pgh.pa.us>
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
|
|
Importance: Normal
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
> -----Original Message-----
|
|
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
|
|
>
|
|
> >> Seems we have 4 DROP COLUMN ideas:
|
|
> >> Method Advantage
|
|
> >> -----------------------------------------------------------------
|
|
> >> 1 invisible column marked by negative attnum fast
|
|
> >> 2 invisible column marked by is_dropped column fast
|
|
> >> 3 make copy of table without column col removed
|
|
> >> 4 make new tuples in existing table without column col removed
|
|
>
|
|
|
|
Hmm,I've received no pg-ML mails for more than 1 day.
|
|
What's happened with pgsql ML ?
|
|
|
|
> Bruce and I talked about this by phone yesterday, and we realized that
|
|
> none of these are very satisfactory. #1 and #2 both have the flaw that
|
|
> applications that examine pg_attribute will probably break: they will
|
|
> see a sequence of attnum values with gaps in it. And what should the
|
|
> rel's relnatts field be set to? #3 and #4 are better on that point,
|
|
> but they leave us with the problem of renumbering references to columns
|
|
> after the dropped one in constraints, rules, PL functions, etc.
|
|
>
|
|
> Furthermore, there is a closely related problem that none of these
|
|
> approaches give us much help on: recursive ALTER TABLE ADD COLUMN.
|
|
> Right now, ADD puts the new column at the end of each table it's added
|
|
> to, which often means that it gets a different column number in child
|
|
> tables than in parent tables. That leads to havoc for pg_dump.
|
|
>
|
|
|
|
Inheritance is one of the reason why I didn't take #2. I don't understand
|
|
marking is_dropped is needed or not when pg_attribute is overhauled
|
|
for inheritance.
|
|
I myself have never wanted to use current inheritance functionality
|
|
mainly because of this big flaw. Judging from the recent discussion
|
|
about oo(though I don't understand details),the change seems to be
|
|
needed in order to make inheritance functionality really useful.
|
|
|
|
> I think the only clean solution is to create a clear distinction between
|
|
> physical and logical column numbers. Each pg_attribute tuple would need
|
|
> two attnum fields, and pg_class would need two relnatts fields as well.
|
|
> A column once created would never change its physical column number, but
|
|
|
|
I don't understand inheritance well. In the near future wouldn't the
|
|
implementation require e.g. attid which is common to all children
|
|
of a parent and is never changed ? If so,we would need the third
|
|
attid field which is irrevalent to physical/logical position. If not,
|
|
physical column number would be sufficient .
|
|
|
|
> its logical column number might change as a consequence of adding or
|
|
> dropping columns before it. ADD COLUMN would ensure that a column added
|
|
> to child tables receives the same logical column number as it has in the
|
|
> parent table, thus solving the dump/reload problem. DROP COLUMN would
|
|
> assign an invalid logical column number to dropped columns. They could
|
|
> be numbered zero except that we'd probably still want a unique index on
|
|
> attrelid+attnum, and the index would complain. I'd suggest using
|
|
> Hiroshi's idea: give a dropped column a logical attnum equal to
|
|
> -(physical_attnum + offset).
|
|
>
|
|
> With this approach, internal operations on tuples would all use
|
|
> physical column numbers, but operations that interface to the outside
|
|
> world would present a view of only the valid logical columns. For
|
|
> example, the parser would only allow logical columns to be referenced
|
|
> by name; "SELECT *" would expand to valid logical columns in logical-
|
|
> column-number order; COPY would send or receive valid logical columns
|
|
> in logical-column-number order; etc.
|
|
>
|
|
> Stored rules and so forth probably should store physical column numbers
|
|
> so that they need not be modified during column add/drop.
|
|
>
|
|
> This would require looking at all the places in the backend to determine
|
|
> whether they should be working with logical or physical column numbers,
|
|
> but the design is such that most all places would want to be using
|
|
> physical numbers, so I don't think it'd be too painful.
|
|
>
|
|
> Although I'd prefer to give the replacement columns two new names
|
|
> (eg, "attlnum" and "attpnum") to ensure we find all uses, this would
|
|
> surely break applications that examine pg_attribute. For compatibility
|
|
> we'd have to recycle "attnum" and "relnatts" to indicate logical column
|
|
> number and logical column count, while adding new fields (say "attpnum"
|
|
> and "relnpatts") for the physical number and count.
|
|
>
|
|
|
|
I agree with you that we would add attpnum and change the meaing of
|
|
attnum as logical column number for backward compatibility.
|
|
|
|
Regards.
|
|
|
|
Hiroshi Inoue
|
|
Inoue@tpf.co.jp
|
|
|
|
From pgsql-hackers-owner+M3050@postgresql.org Thu Jan 11 21:49:43 2001
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA20277
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Jan 2001 21:49:42 -0500 (EST)
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0C2lhp74989;
|
|
Thu, 11 Jan 2001 21:47:43 -0500 (EST)
|
|
(envelope-from pgsql-hackers-owner+M3050@postgresql.org)
|
|
Received: from dynworks.com (adsl-63-206-168-198.dsl.sktn01.pacbell.net [63.206.168.198])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f0C2lNp74855
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Jan 2001 21:47:23 -0500 (EST)
|
|
(envelope-from jdavis@dynworks.com)
|
|
Received: from localhost.localdomain (localhost.localdomain [127.0.0.1])
|
|
by dynworks.com (Postfix) with ESMTP id CC44F31FAB
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Jan 2001 18:48:36 -0800 (PST)
|
|
Date: Thu, 11 Jan 2001 18:48:36 PST
|
|
From: Jeff Davis <jdavis@dynworks.com>
|
|
To: pgsql-hackers@postgresql.org
|
|
Subject: [HACKERS] alter table drop column
|
|
Reply-To: jdavis@dynworks.com
|
|
X-Mailer: Spruce 0.6.5 for X11 w/smtpio 0.7.9
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain; charset="iso-8859-1"
|
|
Content-Transfer-Encoding: 8bit
|
|
Message-Id: <20010112024836.CC44F31FAB@dynworks.com>
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
|
|
I read the transcript of the alter table drop column discussion (old
|
|
discussion) at http://www.postgresql.org/docs/pgsql/doc/TODO.detail/drop,
|
|
and I have something to add:
|
|
|
|
People mentioned such ideas as a hidden column and a really deleted column,
|
|
and it occurred to me that perhaps "vacuum" would be a good option to use.
|
|
When a delete was issued, the column would be hidden (by a negative/invalid
|
|
logical column number, it appears was the consensus). Upon issuing a
|
|
vacuum, it could perform a complete deletion. This method would allow users
|
|
to know that the process may take a while (I think the agreed method for a
|
|
complete delete was to "select into..." the right columns and leave out the
|
|
deleted ones, then delete the old table).
|
|
|
|
Furthermore, I liked the idea of some kind of "undelete", as long as it was
|
|
just hidden. This could apply to anything that is cleaned out with a vacuum
|
|
(before it is cleaned out), although I am not sure how feasible this is,
|
|
and it isn't particularly important to me.
|
|
|
|
Regards,
|
|
Jeff
|
|
|
|
--
|
|
Jeff Davis
|
|
Dynamic Works
|
|
jdavis@dynworks.com
|
|
http://dynworks.com
|
|
|
|
|
|
From owner-pgsql-hackers@hub.org Sat Feb 26 01:07:45 2000
|
|
Received: from hub.org (hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA17776
|
|
for <pgman@candle.pha.pa.us>; Sat, 26 Feb 2000 01:07:43 -0500 (EST)
|
|
Received: from localhost (majordom@localhost)
|
|
by hub.org (8.9.3/8.9.3) with SMTP id BAA06232;
|
|
Sat, 26 Feb 2000 01:03:53 -0500 (EST)
|
|
(envelope-from owner-pgsql-hackers)
|
|
Received: by hub.org (bulk_mailer v1.5); Sat, 26 Feb 2000 01:03:26 -0500
|
|
Received: (from majordom@localhost)
|
|
by hub.org (8.9.3/8.9.3) id BAA05808
|
|
for pgsql-hackers-outgoing; Sat, 26 Feb 2000 01:02:28 -0500 (EST)
|
|
(envelope-from owner-pgsql-hackers@postgreSQL.org)
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
|
|
by hub.org (8.9.3/8.9.3) with ESMTP id BAA05426
|
|
for <pgsql-hackers@postgreSQL.org>; Sat, 26 Feb 2000 01:01:46 -0500 (EST)
|
|
(envelope-from tgl@sss.pgh.pa.us)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id BAA14228;
|
|
Sat, 26 Feb 2000 01:01:34 -0500 (EST)
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: Peter Eisentraut <peter_e@gmx.net>,
|
|
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
|
|
Subject: Re: [HACKERS] ALTER TABLE DROP COLUMN
|
|
In-reply-to: <200002260412.XAA14752@candle.pha.pa.us>
|
|
References: <200002260412.XAA14752@candle.pha.pa.us>
|
|
Comments: In-reply-to Bruce Momjian <pgman@candle.pha.pa.us>
|
|
message dated "Fri, 25 Feb 2000 23:12:26 -0500"
|
|
Date: Sat, 26 Feb 2000 01:01:33 -0500
|
|
Message-ID: <14225.951544893@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Sender: owner-pgsql-hackers@postgreSQL.org
|
|
Status: ORr
|
|
|
|
Bruce Momjian <pgman@candle.pha.pa.us> writes:
|
|
> You can exclusively lock the table, then do a heap_getnext() scan over
|
|
> the entire table, remove the dropped column, do a heap_insert(), then a
|
|
> heap_delete() on the current tuple, making sure to skip over the tuples
|
|
> inserted by the current transaction. When completed, remove the column
|
|
> from pg_attribute, mark the transaction as committed (if desired), and
|
|
> run vacuum over the table to remove the deleted rows.
|
|
|
|
Hmm, that would work --- the new tuples commit at the same instant that
|
|
the schema updates commit, so it should be correct. You have the 2x
|
|
disk usage problem, but there's no way around that without losing
|
|
rollback ability.
|
|
|
|
A potentially tricky bit will be persuading the tuple-reading and tuple-
|
|
writing subroutines to pay attention to different versions of the tuple
|
|
structure for the same table. I haven't looked to see if this will be
|
|
difficult or not. If you can pass the TupleDesc explicitly then it
|
|
shouldn't be a problem.
|
|
|
|
I'd suggest that the cleanup vacuum *not* be an automatic part of
|
|
the operation; just recommend that people do it ASAP after dropping
|
|
a column. Consider needing to drop several columns...
|
|
|
|
regards, tom lane
|
|
|
|
************
|
|
|
|
From pgsql-hackers-owner+M18768=candle.pha.pa.us=pgman@postgresql.org Wed Feb 13 03:52:00 2002
|
|
Return-path: <pgsql-hackers-owner+M18768=candle.pha.pa.us=pgman@postgresql.org>
|
|
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g1D8pxP21056
|
|
for <pgman@candle.pha.pa.us>; Wed, 13 Feb 2002 03:52:00 -0500 (EST)
|
|
Received: (qmail 97959 invoked by alias); 13 Feb 2002 08:51:46 -0000
|
|
Received: from unknown (HELO postgresql.org) (64.49.215.8)
|
|
by www.postgresql.org with SMTP; 13 Feb 2002 08:51:46 -0000
|
|
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
|
|
by postgresql.org (8.11.3/8.11.4) with SMTP id g1D8mRE97432
|
|
for <pgsql-hackers@postgresql.org>; Wed, 13 Feb 2002 03:48:28 -0500 (EST)
|
|
(envelope-from Inoue@tpf.co.jp)
|
|
Received: (qmail 26891 invoked from network); 13 Feb 2002 08:48:27 -0000
|
|
Received: from unknown (HELO viscomail.tpf.co.jp) (100.0.0.108)
|
|
by sd2.tpf-fw-c.co.jp with SMTP; 13 Feb 2002 08:48:27 -0000
|
|
Received: from tpf.co.jp (3dgateway1 [126.0.1.60])
|
|
by viscomail.tpf.co.jp (8.8.8+Sun/8.8.8) with ESMTP id RAA01019;
|
|
Wed, 13 Feb 2002 17:48:20 +0900 (JST)
|
|
Message-ID: <3C6A2861.6E71A124@tpf.co.jp>
|
|
Date: Wed, 13 Feb 2002 17:48:33 +0900
|
|
From: Hiroshi Inoue <Inoue@tpf.co.jp>
|
|
X-Mailer: Mozilla 4.73 [ja] (Windows NT 5.0; U)
|
|
X-Accept-Language: ja
|
|
MIME-Version: 1.0
|
|
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
cc: Tom Lane <tgl@sss.pgh.pa.us>,
|
|
Kovacs Zoltan <kovacsz@pc10.radnoti-szeged.sulinet.hu>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] alter table drop column status
|
|
References: <GNELIHDDFBOCMGBFGEFOMEFPCBAA.chriskl@familyhealth.com.au>
|
|
Content-Type: text/plain; charset=iso-2022-jp
|
|
Content-Transfer-Encoding: 7bit
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Christopher Kings-Lynne wrote:
|
|
>
|
|
> > No there was an unapplied hack which uses logical/physical
|
|
> > attribute numbers. I have synchronized it with cvs for a
|
|
> > year or so but stop it now. Though it had some flaws It
|
|
> > solved the following TODOs.
|
|
> >
|
|
> > * Add ALTER TABLE DROP COLUMN feature
|
|
> > * ALTER TABLE ADD COLUMN to inherited table put column in wrong place
|
|
> > * Prevent column dropping if column is used by foreign key
|
|
>
|
|
> This seems fantastic - why can't this be committed? Surely if it's
|
|
> committed then the flaws will fairly quickly be ironed out? Even if it has
|
|
> flaws, then if we say 'this function is not yet stable' at least people can
|
|
> start testing it and reporting the problems?
|
|
>
|
|
> > I gave up to apply the hack mainly because it may introduce
|
|
> > the maintenance headache.
|
|
>
|
|
> Is it a maintenance headache just for you to keep it up to date, or how
|
|
> would it be a maintenance headache if it were committed?
|
|
|
|
Probably(oops I don't remember well now sorry) the main
|
|
reason why I didn't insist to apply the patch was that
|
|
it wasn't so clean as I had expected.
|
|
My trial implementation uses logical(for clients) and
|
|
physical (for backend internal) attribute numbers but
|
|
there were many places where I wasn't able to judge which
|
|
to use immediately. I'm pretty suspicious if a developer
|
|
could be careful about the choise when he is implementing
|
|
an irrevant feature. (Un)fortunately the numbers have
|
|
the same values mostly and he could hardly notice the
|
|
mistake even if he chose the wrong attribute numbers.
|
|
I'm not sure if I myself chose the right attribute numbers
|
|
everywhere in my implementation.
|
|
In addtion (probably) there were some pretty essential
|
|
flaws. I intended to manage the backend internal
|
|
object references without the logical attribute
|
|
numbers but I found it difficult in some cases
|
|
(probably the handling of virtual(not existent
|
|
in any real table) tuples).
|
|
|
|
Sorry it was more than 1 year ago when I implemented
|
|
it and I can't remember well what I'd thougth then.
|
|
Though I'd kept my local branch up to date for
|
|
about a year, it's about half a year since I touched
|
|
the stuff last.
|
|
|
|
regards,
|
|
Hiroshi Inoue
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 2: you can get off all lists at once with the unregister command
|
|
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
|
|
|
From chriskl@familyhealth.com.au Thu Apr 11 12:00:22 2002
|
|
Return-path: <chriskl@familyhealth.com.au>
|
|
Received: from houston.familyhealth.com.au (root@i231-006.nv.iinet.net.au [203.59.231.6])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3BG0KS02910
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 12:00:20 -0400 (EDT)
|
|
Received: from localhost (chriskl@localhost)
|
|
by houston.familyhealth.com.au (8.11.6/8.11.6) with ESMTP id g3BG0GJ70765;
|
|
Fri, 12 Apr 2002 00:00:16 +0800 (WST)
|
|
(envelope-from chriskl@familyhealth.com.au)
|
|
Date: Fri, 12 Apr 2002 00:00:16 +0800 (WST)
|
|
From: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: Hiroshi Inoue <Inoue@tpf.co.jp>, Tom Lane <tgl@sss.pgh.pa.us>,
|
|
<pgsql-hackers@postgresql.org>
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <200204110419.g3B4J8v29682@candle.pha.pa.us>
|
|
Message-ID: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Status: OR
|
|
|
|
> Actually, what we need to do to reclaim space is to enable table
|
|
> recreation without the column, now that we have relfilenode for file
|
|
> renaming. It isn't hard to do, but no one has focused on it. I want to
|
|
> focus on it, but have not had the time, obviously, and would be very
|
|
> excited to assist someone else.
|
|
>
|
|
> Hiroshi's fine idea of marking certain columns as unused would not have
|
|
> reclaimed the missing space, just as my idea of physical/logical column
|
|
> distinction would not reclaim the space either. Again, my
|
|
> physical/logical idea is more for fixing other problems and
|
|
> optimization, not DROP COLUMN.
|
|
|
|
Hmmm. Personally, I think that a DROP COLUMN that cannot reclaim space is
|
|
kinda useless - you may as well just use a view!!!
|
|
|
|
So how would this occur?:
|
|
|
|
1. Lock target table for writing (allow reads)
|
|
2. Begin a table scan on target table, writing
|
|
a new file with a particular filenode
|
|
3. Delete the attribute row from pg_attribute
|
|
4. Point the table in the catalog to the new filenode
|
|
5. Release locks
|
|
6. Commit transaction
|
|
7. Delete orhpan filenode
|
|
|
|
i. Upon postmaster startup, remove any orphaned filenodes
|
|
|
|
The real problem here is the fact that there are now missing attnos in
|
|
pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
also quite hard?
|
|
|
|
This, of course, suffers from the double size data problem - but I believe
|
|
that it does not matter - we just need to document it.
|
|
|
|
Interestingly enough, Oracle support
|
|
|
|
ALTER TABLE foo SET UNUSED col;
|
|
|
|
Which invalidates the attribute entry, and:
|
|
|
|
ALTER TABLE foo DROP col CHECKPOINT 1000;
|
|
|
|
Which actually reclaims the space. The optional CHECKPOINT [n] clause
|
|
tells Oracle to do a checkpoint every [n] rows.
|
|
|
|
"Checkpointing cuts down the amount of undo logs accumulated during the
|
|
drop column operation to avoid running out of rollback segment space.
|
|
However, if this statement is interrupted after a checkpoint has been
|
|
applied, the table remains in an unusable state. While the table is
|
|
unusable, the only operations allowed on it are DROP TABLE, TRUNCATE
|
|
TABLE, and ALTER TABLE DROP COLUMNS CONTINUE (described below). "
|
|
|
|
Chris
|
|
|
|
|
|
|
|
From pgsql-hackers-owner+M21180@postgresql.org Thu Apr 11 12:02:54 2002
|
|
Return-path: <pgsql-hackers-owner+M21180@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3BG2sS03611
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 12:02:54 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 6446B478F0A; Thu, 11 Apr 2002 12:01:19 -0400 (EDT)
|
|
Received: from houston.familyhealth.com.au (i231-006.nv.iinet.net.au [203.59.231.6])
|
|
by postgresql.org (Postfix) with ESMTP id B6271478E4C
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Apr 2002 12:00:24 -0400 (EDT)
|
|
Received: from localhost (chriskl@localhost)
|
|
by houston.familyhealth.com.au (8.11.6/8.11.6) with ESMTP id g3BG0GJ70765;
|
|
Fri, 12 Apr 2002 00:00:16 +0800 (WST)
|
|
(envelope-from chriskl@familyhealth.com.au)
|
|
Date: Fri, 12 Apr 2002 00:00:16 +0800 (WST)
|
|
From: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: Hiroshi Inoue <Inoue@tpf.co.jp>, Tom Lane <tgl@sss.pgh.pa.us>,
|
|
<pgsql-hackers@postgresql.org>
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <200204110419.g3B4J8v29682@candle.pha.pa.us>
|
|
Message-ID: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: ORr
|
|
|
|
> Actually, what we need to do to reclaim space is to enable table
|
|
> recreation without the column, now that we have relfilenode for file
|
|
> renaming. It isn't hard to do, but no one has focused on it. I want to
|
|
> focus on it, but have not had the time, obviously, and would be very
|
|
> excited to assist someone else.
|
|
>
|
|
> Hiroshi's fine idea of marking certain columns as unused would not have
|
|
> reclaimed the missing space, just as my idea of physical/logical column
|
|
> distinction would not reclaim the space either. Again, my
|
|
> physical/logical idea is more for fixing other problems and
|
|
> optimization, not DROP COLUMN.
|
|
|
|
Hmmm. Personally, I think that a DROP COLUMN that cannot reclaim space is
|
|
kinda useless - you may as well just use a view!!!
|
|
|
|
So how would this occur?:
|
|
|
|
1. Lock target table for writing (allow reads)
|
|
2. Begin a table scan on target table, writing
|
|
a new file with a particular filenode
|
|
3. Delete the attribute row from pg_attribute
|
|
4. Point the table in the catalog to the new filenode
|
|
5. Release locks
|
|
6. Commit transaction
|
|
7. Delete orhpan filenode
|
|
|
|
i. Upon postmaster startup, remove any orphaned filenodes
|
|
|
|
The real problem here is the fact that there are now missing attnos in
|
|
pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
also quite hard?
|
|
|
|
This, of course, suffers from the double size data problem - but I believe
|
|
that it does not matter - we just need to document it.
|
|
|
|
Interestingly enough, Oracle support
|
|
|
|
ALTER TABLE foo SET UNUSED col;
|
|
|
|
Which invalidates the attribute entry, and:
|
|
|
|
ALTER TABLE foo DROP col CHECKPOINT 1000;
|
|
|
|
Which actually reclaims the space. The optional CHECKPOINT [n] clause
|
|
tells Oracle to do a checkpoint every [n] rows.
|
|
|
|
"Checkpointing cuts down the amount of undo logs accumulated during the
|
|
drop column operation to avoid running out of rollback segment space.
|
|
However, if this statement is interrupted after a checkpoint has been
|
|
applied, the table remains in an unusable state. While the table is
|
|
unusable, the only operations allowed on it are DROP TABLE, TRUNCATE
|
|
TABLE, and ALTER TABLE DROP COLUMNS CONTINUE (described below). "
|
|
|
|
Chris
|
|
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 2: you can get off all lists at once with the unregister command
|
|
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
|
|
|
From tgl@sss.pgh.pa.us Thu Apr 11 12:22:44 2002
|
|
Return-path: <tgl@sss.pgh.pa.us>
|
|
Received: from sss.pgh.pa.us (root@[192.204.191.242])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3BGMhS05541
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 12:22:43 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3BGMaF01827;
|
|
Thu, 11 Apr 2002 12:22:36 -0400 (EDT)
|
|
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
cc: Bruce Momjian <pgman@candle.pha.pa.us>, Hiroshi Inoue <Inoue@tpf.co.jp>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
Comments: In-reply-to Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
message dated "Fri, 12 Apr 2002 00:00:16 +0800"
|
|
Date: Thu, 11 Apr 2002 12:22:35 -0400
|
|
Message-ID: <1824.1018542155@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Status: ORr
|
|
|
|
Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
|
|
> The real problem here is the fact that there are now missing attnos in
|
|
> pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
> also quite hard?
|
|
|
|
Updating pg_attribute per se is not so hard --- just store new copies of
|
|
all the rows for the table. However, propagating the changes into other
|
|
places could be quite painful (I'm thinking of column numbers in stored
|
|
constraints, rules, etc).
|
|
|
|
It seems to me that reducing the column to NULLs already gets you the
|
|
majority of the space savings. I don't think there is a case to be made
|
|
that getting back that last bit is worth the pain involved, either in
|
|
implementation effort or direct runtime costs (do you really want a DROP
|
|
COLUMN to force an immediate rewrite of the whole table?)
|
|
|
|
regards, tom lane
|
|
|
|
From pgsql-hackers-owner+M21186@postgresql.org Thu Apr 11 13:03:13 2002
|
|
Return-path: <pgsql-hackers-owner+M21186@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3BH3DS08942
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 13:03:13 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 517ED479415; Thu, 11 Apr 2002 12:29:32 -0400 (EDT)
|
|
Received: from sss.pgh.pa.us (unknown [192.204.191.242])
|
|
by postgresql.org (Postfix) with ESMTP id B87BC479327
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Apr 2002 12:22:51 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3BGMaF01827;
|
|
Thu, 11 Apr 2002 12:22:36 -0400 (EDT)
|
|
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
cc: Bruce Momjian <pgman@candle.pha.pa.us>, Hiroshi Inoue <Inoue@tpf.co.jp>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
Comments: In-reply-to Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
message dated "Fri, 12 Apr 2002 00:00:16 +0800"
|
|
Date: Thu, 11 Apr 2002 12:22:35 -0400
|
|
Message-ID: <1824.1018542155@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
|
|
> The real problem here is the fact that there are now missing attnos in
|
|
> pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
> also quite hard?
|
|
|
|
Updating pg_attribute per se is not so hard --- just store new copies of
|
|
all the rows for the table. However, propagating the changes into other
|
|
places could be quite painful (I'm thinking of column numbers in stored
|
|
constraints, rules, etc).
|
|
|
|
It seems to me that reducing the column to NULLs already gets you the
|
|
majority of the space savings. I don't think there is a case to be made
|
|
that getting back that last bit is worth the pain involved, either in
|
|
implementation effort or direct runtime costs (do you really want a DROP
|
|
COLUMN to force an immediate rewrite of the whole table?)
|
|
|
|
regards, tom lane
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 4: Don't 'kill -9' the postmaster
|
|
|
|
From pgsql-hackers-owner+M21187@postgresql.org Thu Apr 11 13:25:05 2002
|
|
Return-path: <pgsql-hackers-owner+M21187@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3BHP4S10960
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 13:25:05 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 2BC27479442; Thu, 11 Apr 2002 12:30:28 -0400 (EDT)
|
|
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
|
|
by postgresql.org (Postfix) with ESMTP id 265E5479340
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Apr 2002 12:23:30 -0400 (EDT)
|
|
Received: (from pgman@localhost)
|
|
by candle.pha.pa.us (8.11.6/8.10.1) id g3BGNS405576;
|
|
Thu, 11 Apr 2002 12:23:28 -0400 (EDT)
|
|
From: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
Message-ID: <200204111623.g3BGNS405576@candle.pha.pa.us>
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
Date: Thu, 11 Apr 2002 12:23:28 -0400 (EDT)
|
|
cc: Hiroshi Inoue <Inoue@tpf.co.jp>, Tom Lane <tgl@sss.pgh.pa.us>,
|
|
pgsql-hackers@postgresql.org
|
|
X-Mailer: ELM [version 2.4ME+ PL97 (25)]
|
|
MIME-Version: 1.0
|
|
Content-Transfer-Encoding: 7bit
|
|
Content-Type: text/plain; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Christopher Kings-Lynne wrote:
|
|
> > Actually, what we need to do to reclaim space is to enable table
|
|
> > recreation without the column, now that we have relfilenode for file
|
|
> > renaming. It isn't hard to do, but no one has focused on it. I want to
|
|
> > focus on it, but have not had the time, obviously, and would be very
|
|
> > excited to assist someone else.
|
|
> >
|
|
> > Hiroshi's fine idea of marking certain columns as unused would not have
|
|
> > reclaimed the missing space, just as my idea of physical/logical column
|
|
> > distinction would not reclaim the space either. Again, my
|
|
> > physical/logical idea is more for fixing other problems and
|
|
> > optimization, not DROP COLUMN.
|
|
>
|
|
> Hmmm. Personally, I think that a DROP COLUMN that cannot reclaim space is
|
|
> kinda useless - you may as well just use a view!!!
|
|
|
|
Yep, kind of a problem. It is a tradeoff between double diskspace/speed
|
|
and removing column from disk. I guess that's why Oracle has both.
|
|
|
|
>
|
|
> So how would this occur?:
|
|
>
|
|
> 1. Lock target table for writing (allow reads)
|
|
> 2. Begin a table scan on target table, writing
|
|
> a new file with a particular filenode
|
|
> 3. Delete the attribute row from pg_attribute
|
|
> 4. Point the table in the catalog to the new filenode
|
|
> 5. Release locks
|
|
> 6. Commit transaction
|
|
> 7. Delete orhpan filenode
|
|
|
|
Yep, something like that. CLUSTER is a good start. DROP COLUMN just
|
|
deals with the attno too. You would have to renumber them to fill the
|
|
gap.
|
|
|
|
> i. Upon postmaster startup, remove any orphaned filenodes
|
|
|
|
Actually, we don't have a good solution for finding orphaned filenodes
|
|
right now. I do have some code that tries to do this as part of VACUUM
|
|
but it was not 100% perfect, so it was rejected. I am willing to open
|
|
the discussion to see if a perfect solution can be found.
|
|
|
|
|
|
--
|
|
Bruce Momjian | http://candle.pha.pa.us
|
|
pgman@candle.pha.pa.us | (610) 853-3000
|
|
+ If your life is a hard drive, | 830 Blythe Avenue
|
|
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 2: you can get off all lists at once with the unregister command
|
|
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
|
|
|
From pgsql-hackers-owner+M21190@postgresql.org Thu Apr 11 13:40:34 2002
|
|
Return-path: <pgsql-hackers-owner+M21190@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3BHeXS12137
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 13:40:33 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 2BD6C479604; Thu, 11 Apr 2002 12:35:51 -0400 (EDT)
|
|
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
|
|
by postgresql.org (Postfix) with ESMTP id 2DF9D47946A
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Apr 2002 12:31:25 -0400 (EDT)
|
|
Received: (from pgman@localhost)
|
|
by candle.pha.pa.us (8.11.6/8.10.1) id g3BGVM806114;
|
|
Thu, 11 Apr 2002 12:31:22 -0400 (EDT)
|
|
From: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
Message-ID: <200204111631.g3BGVM806114@candle.pha.pa.us>
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <1824.1018542155@sss.pgh.pa.us>
|
|
To: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Date: Thu, 11 Apr 2002 12:31:22 -0400 (EDT)
|
|
cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
Hiroshi Inoue <Inoue@tpf.co.jp>, pgsql-hackers@postgresql.org
|
|
X-Mailer: ELM [version 2.4ME+ PL97 (25)]
|
|
MIME-Version: 1.0
|
|
Content-Transfer-Encoding: 7bit
|
|
Content-Type: text/plain; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Tom Lane wrote:
|
|
> Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
|
|
> > The real problem here is the fact that there are now missing attnos in
|
|
> > pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
> > also quite hard?
|
|
>
|
|
> Updating pg_attribute per se is not so hard --- just store new copies of
|
|
> all the rows for the table. However, propagating the changes into other
|
|
> places could be quite painful (I'm thinking of column numbers in stored
|
|
> constraints, rules, etc).
|
|
>
|
|
> It seems to me that reducing the column to NULLs already gets you the
|
|
> majority of the space savings. I don't think there is a case to be made
|
|
> that getting back that last bit is worth the pain involved, either in
|
|
> implementation effort or direct runtime costs (do you really want a DROP
|
|
> COLUMN to force an immediate rewrite of the whole table?)
|
|
|
|
That is an excellent point about having to fix all the places that refer
|
|
to attno. In fact, we have been moving away from attname references to
|
|
attno references for a while, so it only gets worse. Tom is also
|
|
correct that setting it to NULL removes the problem of disk space usage
|
|
quite easily.
|
|
|
|
That only leaves the problem of having gaps in the pg_attribute for that
|
|
relation, and as I remember, that was the problem for Hiroshi's DROP
|
|
COLUMN change, but at this point, after years of delay with no great
|
|
solution on the horizon, we may as well just get this working.
|
|
|
|
--
|
|
Bruce Momjian | http://candle.pha.pa.us
|
|
pgman@candle.pha.pa.us | (610) 853-3000
|
|
+ If your life is a hard drive, | 830 Blythe Avenue
|
|
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 3: if posting/reading through Usenet, please send an appropriate
|
|
subscribe-nomail command to majordomo@postgresql.org so that your
|
|
message can get through to the mailing list cleanly
|
|
|
|
From Inoue@tpf.co.jp Thu Apr 11 19:55:08 2002
|
|
Return-path: <Inoue@tpf.co.jp>
|
|
Received: from sd.tpf.co.jp (IDENT:qmailr@sd.tpf.co.jp [210.161.239.34])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g3BNt6S19759
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 19:55:06 -0400 (EDT)
|
|
Received: (qmail 31013 invoked from network); 11 Apr 2002 23:55:06 -0000
|
|
Received: from unknown (HELO viscomail.tpf.co.jp) (100.0.0.108)
|
|
by sd2.tpf-fw-c.co.jp with SMTP; 11 Apr 2002 23:55:06 -0000
|
|
Received: from tpf.co.jp (3dgateway1 [126.0.1.60])
|
|
by viscomail.tpf.co.jp (8.8.8+Sun/8.8.8) with ESMTP id IAA22335;
|
|
Fri, 12 Apr 2002 08:55:05 +0900 (JST)
|
|
Message-ID: <3CB62298.88565A54@tpf.co.jp>
|
|
Date: Fri, 12 Apr 2002 08:56:08 +0900
|
|
From: Hiroshi Inoue <Inoue@tpf.co.jp>
|
|
X-Mailer: Mozilla 4.73 [ja] (Windows NT 5.0; U)
|
|
X-Accept-Language: ja
|
|
MIME-Version: 1.0
|
|
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
cc: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
Content-Type: text/plain; charset=iso-2022-jp
|
|
Content-Transfer-Encoding: 7bit
|
|
Status: OR
|
|
|
|
Christopher Kings-Lynne wrote:
|
|
>
|
|
> Hmmm. Personally, I think that a DROP COLUMN that cannot reclaim space is
|
|
> kinda useless - you may as well just use a view!!!
|
|
>
|
|
> So how would this occur?:
|
|
>
|
|
> 1. Lock target table for writing (allow reads)
|
|
> 2. Begin a table scan on target table, writing
|
|
> a new file with a particular filenode
|
|
> 3. Delete the attribute row from pg_attribute
|
|
> 4. Point the table in the catalog to the new filenode
|
|
> 5. Release locks
|
|
> 6. Commit transaction
|
|
> 7. Delete orhpan filenode
|
|
>
|
|
> i. Upon postmaster startup, remove any orphaned filenodes
|
|
>
|
|
> The real problem here is the fact that there are now missing attnos in
|
|
> pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
> also quite hard?
|
|
|
|
The attnos should be renumbered and it's easy.
|
|
But the above seems only 20% of the total implementation.
|
|
If the attnos are renumbered, all objects which refer to
|
|
the numbers must be invalidated or re-compiled ...
|
|
For example the parameters of foreign key constraints
|
|
triggers are consist of relname and colnames currently.
|
|
There has been a proposal that change to use relid or
|
|
column numbers instead. Certainly it makes RENAME happy
|
|
but DROP COLUMN unhappy. If there's a foreign key a_rel/1/3
|
|
and the second column of the relation is dropped the
|
|
parameter must be changed to be a_rel/1/2. If neither
|
|
foreign key stuff nor DROP COLUMN take the other into
|
|
account, the consistency is easily broken.
|
|
|
|
regards,
|
|
Hiroshi Inoue
|
|
|
|
From pgsql-hackers-owner+M21205@postgresql.org Thu Apr 11 19:56:20 2002
|
|
Return-path: <pgsql-hackers-owner+M21205@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3BNuJS19855
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 19:56:19 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 2B38A47656E; Thu, 11 Apr 2002 19:55:57 -0400 (EDT)
|
|
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
|
|
by postgresql.org (Postfix) with SMTP id 6C92E475C96
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Apr 2002 19:55:04 -0400 (EDT)
|
|
Received: (qmail 31013 invoked from network); 11 Apr 2002 23:55:06 -0000
|
|
Received: from unknown (HELO viscomail.tpf.co.jp) (100.0.0.108)
|
|
by sd2.tpf-fw-c.co.jp with SMTP; 11 Apr 2002 23:55:06 -0000
|
|
Received: from tpf.co.jp (3dgateway1 [126.0.1.60])
|
|
by viscomail.tpf.co.jp (8.8.8+Sun/8.8.8) with ESMTP id IAA22335;
|
|
Fri, 12 Apr 2002 08:55:05 +0900 (JST)
|
|
Message-ID: <3CB62298.88565A54@tpf.co.jp>
|
|
Date: Fri, 12 Apr 2002 08:56:08 +0900
|
|
From: Hiroshi Inoue <Inoue@tpf.co.jp>
|
|
X-Mailer: Mozilla 4.73 [ja] (Windows NT 5.0; U)
|
|
X-Accept-Language: ja
|
|
MIME-Version: 1.0
|
|
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
cc: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
Content-Type: text/plain; charset=iso-2022-jp
|
|
Content-Transfer-Encoding: 7bit
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: ORr
|
|
|
|
Christopher Kings-Lynne wrote:
|
|
>
|
|
> Hmmm. Personally, I think that a DROP COLUMN that cannot reclaim space is
|
|
> kinda useless - you may as well just use a view!!!
|
|
>
|
|
> So how would this occur?:
|
|
>
|
|
> 1. Lock target table for writing (allow reads)
|
|
> 2. Begin a table scan on target table, writing
|
|
> a new file with a particular filenode
|
|
> 3. Delete the attribute row from pg_attribute
|
|
> 4. Point the table in the catalog to the new filenode
|
|
> 5. Release locks
|
|
> 6. Commit transaction
|
|
> 7. Delete orhpan filenode
|
|
>
|
|
> i. Upon postmaster startup, remove any orphaned filenodes
|
|
>
|
|
> The real problem here is the fact that there are now missing attnos in
|
|
> pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
> also quite hard?
|
|
|
|
The attnos should be renumbered and it's easy.
|
|
But the above seems only 20% of the total implementation.
|
|
If the attnos are renumbered, all objects which refer to
|
|
the numbers must be invalidated or re-compiled ...
|
|
For example the parameters of foreign key constraints
|
|
triggers are consist of relname and colnames currently.
|
|
There has been a proposal that change to use relid or
|
|
column numbers instead. Certainly it makes RENAME happy
|
|
but DROP COLUMN unhappy. If there's a foreign key a_rel/1/3
|
|
and the second column of the relation is dropped the
|
|
parameter must be changed to be a_rel/1/2. If neither
|
|
foreign key stuff nor DROP COLUMN take the other into
|
|
account, the consistency is easily broken.
|
|
|
|
regards,
|
|
Hiroshi Inoue
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 6: Have you searched our list archives?
|
|
|
|
http://archives.postgresql.org
|
|
|
|
From pgsql-hackers-owner+M21209@postgresql.org Thu Apr 11 22:27:40 2002
|
|
Return-path: <pgsql-hackers-owner+M21209@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3C2ReS27660
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 22:27:40 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id A89AF4766D0; Thu, 11 Apr 2002 22:27:35 -0400 (EDT)
|
|
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
|
|
by postgresql.org (Postfix) with ESMTP id 4CE13475EB9
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Apr 2002 22:26:25 -0400 (EDT)
|
|
Received: (from pgman@localhost)
|
|
by candle.pha.pa.us (8.11.6/8.10.1) id g3C2Q5I27551;
|
|
Thu, 11 Apr 2002 22:26:05 -0400 (EDT)
|
|
From: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
Message-ID: <200204120226.g3C2Q5I27551@candle.pha.pa.us>
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <3CB62298.88565A54@tpf.co.jp>
|
|
To: Hiroshi Inoue <Inoue@tpf.co.jp>
|
|
Date: Thu, 11 Apr 2002 22:26:05 -0400 (EDT)
|
|
cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org
|
|
X-Mailer: ELM [version 2.4ME+ PL97 (25)]
|
|
MIME-Version: 1.0
|
|
Content-Transfer-Encoding: 7bit
|
|
Content-Type: text/plain; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Hiroshi Inoue wrote:
|
|
> Christopher Kings-Lynne wrote:
|
|
> >
|
|
> > Hmmm. Personally, I think that a DROP COLUMN that cannot reclaim space is
|
|
> > kinda useless - you may as well just use a view!!!
|
|
> >
|
|
> > So how would this occur?:
|
|
> >
|
|
> > 1. Lock target table for writing (allow reads)
|
|
> > 2. Begin a table scan on target table, writing
|
|
> > a new file with a particular filenode
|
|
> > 3. Delete the attribute row from pg_attribute
|
|
> > 4. Point the table in the catalog to the new filenode
|
|
> > 5. Release locks
|
|
> > 6. Commit transaction
|
|
> > 7. Delete orhpan filenode
|
|
> >
|
|
> > i. Upon postmaster startup, remove any orphaned filenodes
|
|
> >
|
|
> > The real problem here is the fact that there are now missing attnos in
|
|
> > pg_attribute. Either that's handled or we renumber the attnos - which is
|
|
> > also quite hard?
|
|
>
|
|
> The attnos should be renumbered and it's easy.
|
|
> But the above seems only 20% of the total implementation.
|
|
> If the attnos are renumbered, all objects which refer to
|
|
> the numbers must be invalidated or re-compiled ...
|
|
> For example the parameters of foreign key constraints
|
|
> triggers are consist of relname and colnames currently.
|
|
> There has been a proposal that change to use relid or
|
|
> column numbers instead. Certainly it makes RENAME happy
|
|
> but DROP COLUMN unhappy. If there's a foreign key a_rel/1/3
|
|
> and the second column of the relation is dropped the
|
|
> parameter must be changed to be a_rel/1/2. If neither
|
|
> foreign key stuff nor DROP COLUMN take the other into
|
|
> account, the consistency is easily broken.
|
|
|
|
I think that is why Tom was suggesting making all the column values NULL
|
|
and removing the pg_attribute row for the column. With a NULL value, it
|
|
doesn't take up any room in the tuple, and with the pg_attribute column
|
|
gone, no one will see that row. The only problem is the gap in attno
|
|
numbering. How big a problem is that?
|
|
|
|
--
|
|
Bruce Momjian | http://candle.pha.pa.us
|
|
pgman@candle.pha.pa.us | (610) 853-3000
|
|
+ If your life is a hard drive, | 830 Blythe Avenue
|
|
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 2: you can get off all lists at once with the unregister command
|
|
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
|
|
|
From pgsql-hackers-owner+M21211@postgresql.org Thu Apr 11 22:55:44 2002
|
|
Return-path: <pgsql-hackers-owner+M21211@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3C2tiS29394
|
|
for <pgman@candle.pha.pa.us>; Thu, 11 Apr 2002 22:55:44 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id B86AF476739; Thu, 11 Apr 2002 22:55:39 -0400 (EDT)
|
|
Received: from sss.pgh.pa.us (unknown [192.204.191.242])
|
|
by postgresql.org (Postfix) with ESMTP id 56D8747593C
|
|
for <pgsql-hackers@postgresql.org>; Thu, 11 Apr 2002 22:54:26 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3C2s1F24007;
|
|
Thu, 11 Apr 2002 22:54:01 -0400 (EDT)
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: Hiroshi Inoue <Inoue@tpf.co.jp>,
|
|
Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
In-Reply-To: <200204120226.g3C2Q5I27551@candle.pha.pa.us>
|
|
References: <200204120226.g3C2Q5I27551@candle.pha.pa.us>
|
|
Comments: In-reply-to Bruce Momjian <pgman@candle.pha.pa.us>
|
|
message dated "Thu, 11 Apr 2002 22:26:05 -0400"
|
|
Date: Thu, 11 Apr 2002 22:54:01 -0400
|
|
Message-ID: <24004.1018580041@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Bruce Momjian <pgman@candle.pha.pa.us> writes:
|
|
> I think that is why Tom was suggesting making all the column values NULL
|
|
> and removing the pg_attribute row for the column.
|
|
|
|
That was not my suggestion.
|
|
|
|
> With a NULL value, it
|
|
> doesn't take up any room in the tuple, and with the pg_attribute column
|
|
> gone, no one will see that row. The only problem is the gap in attno
|
|
> numbering. How big a problem is that?
|
|
|
|
You can't do it that way unless you're intending to rewrite all rows of
|
|
the relation before committing the ALTER; which would be the worst of
|
|
both worlds. The pg_attribute row *must* be retained to show the
|
|
datatype of the former column, so that we can correctly skip over it
|
|
in tuples where the column isn't yet nulled out.
|
|
|
|
Hiroshi did this by renumbering the attnum; I propose leaving attnum
|
|
alone and instead adding an attisdropped flag. That would avoid
|
|
creating a gap in the column numbers, but either way is likely to affect
|
|
some applications that inspect pg_attribute.
|
|
|
|
regards, tom lane
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
|
|
|
|
From pgsql-hackers-owner+M21214@postgresql.org Fri Apr 12 00:09:26 2002
|
|
Return-path: <pgsql-hackers-owner+M21214@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3C49PS05093
|
|
for <pgman@candle.pha.pa.us>; Fri, 12 Apr 2002 00:09:25 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id B1BE6476810; Fri, 12 Apr 2002 00:09:20 -0400 (EDT)
|
|
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
|
|
by postgresql.org (Postfix) with SMTP id A8E07476444
|
|
for <pgsql-hackers@postgresql.org>; Fri, 12 Apr 2002 00:08:22 -0400 (EDT)
|
|
Received: (qmail 25808 invoked from network); 12 Apr 2002 04:08:26 -0000
|
|
Received: from unknown (HELO viscomail.tpf.co.jp) (100.0.0.108)
|
|
by sd2.tpf-fw-c.co.jp with SMTP; 12 Apr 2002 04:08:26 -0000
|
|
Received: from tpf.co.jp (3dgateway1 [126.0.1.60])
|
|
by viscomail.tpf.co.jp (8.8.8+Sun/8.8.8) with ESMTP id NAA22497;
|
|
Fri, 12 Apr 2002 13:08:24 +0900 (JST)
|
|
Message-ID: <3CB65DF7.8FCFC024@tpf.co.jp>
|
|
Date: Fri, 12 Apr 2002 13:09:28 +0900
|
|
From: Hiroshi Inoue <Inoue@tpf.co.jp>
|
|
X-Mailer: Mozilla 4.73 [ja] (Windows NT 5.0; U)
|
|
X-Accept-Language: ja
|
|
MIME-Version: 1.0
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
References: <200204120226.g3C2Q5I27551@candle.pha.pa.us>
|
|
Content-Type: text/plain; charset=iso-2022-jp
|
|
Content-Transfer-Encoding: 7bit
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Bruce Momjian wrote:
|
|
>
|
|
> Hiroshi Inoue wrote:
|
|
> > Christopher Kings-Lynne wrote:
|
|
> > >
|
|
> I think that is why Tom was suggesting making all the column values NULL
|
|
> and removing the pg_attribute row for the column. With a NULL value, it
|
|
> doesn't take up any room in the tuple, and with the pg_attribute column
|
|
> gone, no one will see that row. The only problem is the gap in attno
|
|
> numbering. How big a problem is that?
|
|
|
|
There's no problem with applications which don't inquire
|
|
of system catalogs(pg_attribute). Unfortunately we have
|
|
been very tolerant of users' access on system tables and
|
|
there would be pretty many applications which inquire of
|
|
pg_attribute.
|
|
|
|
regards,
|
|
Hiroshi Inoue
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 5: Have you checked our extensive FAQ?
|
|
|
|
http://www.postgresql.org/users-lounge/docs/faq.html
|
|
|
|
From pgsql-hackers-owner+M21221@postgresql.org Fri Apr 12 05:11:00 2002
|
|
Return-path: <pgsql-hackers-owner+M21221@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3C9AxS28516
|
|
for <pgman@candle.pha.pa.us>; Fri, 12 Apr 2002 05:11:00 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 28FF0476B9D; Fri, 12 Apr 2002 04:35:54 -0400 (EDT)
|
|
Received: from tele-post-20.mail.demon.net (tele-post-20.mail.demon.net [194.217.242.20])
|
|
by postgresql.org (Postfix) with ESMTP id BFDE74769AC
|
|
for <pgsql-hackers@postgresql.org>; Fri, 12 Apr 2002 04:30:29 -0400 (EDT)
|
|
Received: from mailgate.vale-housing.co.uk ([193.195.77.162] helo=dogbert.vale-housing.co.uk)
|
|
by tele-post-20.mail.demon.net with esmtp (Exim 3.35 #1)
|
|
id 16vwRc-0006GP-0K; Fri, 12 Apr 2002 08:30:08 +0000
|
|
Received: by dogbert.vale-housing.co.uk with Internet Mail Service (5.5.2650.21)
|
|
id <2H2ZS6HB>; Fri, 12 Apr 2002 09:35:53 +0100
|
|
Message-ID: <FED2B709E3270E4B903EB0175A49BCB1293387@dogbert.vale-housing.co.uk>
|
|
From: Dave Page <dpage@vale-housing.co.uk>
|
|
To: "'Tom Lane'" <tgl@sss.pgh.pa.us>, Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: Hiroshi Inoue <Inoue@tpf.co.jp>,
|
|
Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
Date: Fri, 12 Apr 2002 09:35:52 +0100
|
|
MIME-Version: 1.0
|
|
X-Mailer: Internet Mail Service (5.5.2650.21)
|
|
Content-Type: text/plain
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
|
|
|
|
> -----Original Message-----
|
|
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
|
|
> Sent: 12 April 2002 03:54
|
|
> To: Bruce Momjian
|
|
> Cc: Hiroshi Inoue; Christopher Kings-Lynne;
|
|
> pgsql-hackers@postgresql.org
|
|
> Subject: Re: RFC: Restructuring pg_aggregate
|
|
>
|
|
>
|
|
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
|
|
> > I think that is why Tom was suggesting making all the column values
|
|
> > NULL and removing the pg_attribute row for the column.
|
|
>
|
|
> That was not my suggestion.
|
|
>
|
|
> > With a NULL value, it
|
|
> > doesn't take up any room in the tuple, and with the pg_attribute
|
|
> > column gone, no one will see that row. The only problem is
|
|
> the gap in
|
|
> > attno numbering. How big a problem is that?
|
|
>
|
|
> You can't do it that way unless you're intending to rewrite
|
|
> all rows of the relation before committing the ALTER; which
|
|
> would be the worst of both worlds. The pg_attribute row
|
|
> *must* be retained to show the datatype of the former column,
|
|
> so that we can correctly skip over it in tuples where the
|
|
> column isn't yet nulled out.
|
|
>
|
|
> Hiroshi did this by renumbering the attnum; I propose leaving
|
|
> attnum alone and instead adding an attisdropped flag. That
|
|
> would avoid creating a gap in the column numbers, but either
|
|
> way is likely to affect some applications that inspect pg_attribute.
|
|
|
|
Applications like pgAdmin that inspect pg_attribute are being seriously
|
|
hacked to incorporate schema support anyway for 7.3. Personnally I'd be glad
|
|
to spend some time re-coding to allow for this, just to not have to answer
|
|
the numerous 'how do I drop a column' emails I get reguarly.
|
|
|
|
Regards, Dave.
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 2: you can get off all lists at once with the unregister command
|
|
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
|
|
|
From chriskl@familyhealth.com.au Sat Apr 13 02:25:23 2002
|
|
Return-path: <chriskl@familyhealth.com.au>
|
|
Received: from mail.iinet.net.au (symphony-01.iinet.net.au [203.59.3.33])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g3D6PLS06807
|
|
for <pgman@candle.pha.pa.us>; Sat, 13 Apr 2002 02:25:22 -0400 (EDT)
|
|
Received: (qmail 7569 invoked by uid 666); 13 Apr 2002 06:25:20 -0000
|
|
Received: from unknown (HELO SOL) (203.59.103.193)
|
|
by mail.iinet.net.au with SMTP; 13 Apr 2002 06:25:20 -0000
|
|
Message-ID: <001701c1e2b2$e7b10a40$0200a8c0@SOL>
|
|
From: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
To: "Tom Lane" <tgl@sss.pgh.pa.us>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Hiroshi Inoue" <Inoue@tpf.co.jp>, <pgsql-hackers@postgresql.org>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us>
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
Date: Sat, 13 Apr 2002 14:17:34 +0800
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook Express 5.50.4522.1200
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200
|
|
Status: OR
|
|
|
|
> Updating pg_attribute per se is not so hard --- just store new copies of
|
|
> all the rows for the table. However, propagating the changes into other
|
|
> places could be quite painful (I'm thinking of column numbers in stored
|
|
> constraints, rules, etc).
|
|
>
|
|
> It seems to me that reducing the column to NULLs already gets you the
|
|
> majority of the space savings. I don't think there is a case to be made
|
|
> that getting back that last bit is worth the pain involved, either in
|
|
> implementation effort or direct runtime costs (do you really want a DROP
|
|
> COLUMN to force an immediate rewrite of the whole table?)
|
|
|
|
OK, sounds fair. However, is there a more aggressive way of reclaiming the
|
|
space? The problem with updating all the rows to null for that column is
|
|
that the on-disk size is doubled anyway, right? So, could a VACUUM FULL
|
|
process do the nulling for us? Vacuum works outside of normal transaction
|
|
constraints anyway...?
|
|
|
|
Also, it seems to me that at some point we are forced to break client
|
|
compatibility. Either we add attisdropped field to pg_attribute, or we use
|
|
Hiroshi's (-1 * attnum - offset) idea. Both Tom and Hiroshi have good
|
|
reasons for each of these - would it be possible for you guys to post with
|
|
your reasons for and against both the techniques. I just want to get to an
|
|
implementation specification we all agree on that can be written up and then
|
|
the coding can proceed. Maybe we should add it to the 'Postgres Major
|
|
Projects' page - and remove those old ones that have already been
|
|
implemented.
|
|
|
|
Chris
|
|
|
|
|
|
|
|
From Inoue@tpf.co.jp Sun Apr 14 23:47:08 2002
|
|
Return-path: <Inoue@tpf.co.jp>
|
|
Received: from sd.tpf.co.jp (IDENT:qmailr@sd.tpf.co.jp [210.161.239.34])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g3F3l6S23155
|
|
for <pgman@candle.pha.pa.us>; Sun, 14 Apr 2002 23:47:07 -0400 (EDT)
|
|
Received: (qmail 9638 invoked from network); 15 Apr 2002 03:47:06 -0000
|
|
Received: from unknown (HELO viscomail.tpf.co.jp) (100.0.0.108)
|
|
by sd2.tpf-fw-c.co.jp with SMTP; 15 Apr 2002 03:47:06 -0000
|
|
Received: from tpf.co.jp (3dgateway1 [126.0.1.60])
|
|
by viscomail.tpf.co.jp (8.8.8+Sun/8.8.8) with ESMTP id MAA24068;
|
|
Mon, 15 Apr 2002 12:47:04 +0900 (JST)
|
|
Message-ID: <3CBA4D7A.9E61DECA@tpf.co.jp>
|
|
Date: Mon, 15 Apr 2002 12:48:10 +0900
|
|
From: Hiroshi Inoue <Inoue@tpf.co.jp>
|
|
X-Mailer: Mozilla 4.73 [ja] (Windows NT 5.0; U)
|
|
X-Accept-Language: ja
|
|
MIME-Version: 1.0
|
|
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
|
|
cc: Tom Lane <tgl@sss.pgh.pa.us>, Bruce Momjian <pgman@candle.pha.pa.us>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] RFC: Restructuring pg_aggregate
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL>
|
|
Content-Type: text/plain; charset=iso-2022-jp
|
|
Content-Transfer-Encoding: 7bit
|
|
Status: OR
|
|
|
|
Christopher Kings-Lynne wrote:
|
|
>
|
|
> Also, it seems to me that at some point we are forced to break client
|
|
> compatibility.
|
|
|
|
It's not a users' consensus at all. I'm suspicious if
|
|
DROP COLUMN is such a significant feature to break
|
|
client compatibility at our ease.
|
|
|
|
> Either we add attisdropped field to pg_attribute, or we use
|
|
> Hiroshi's (-1 * attnum - offset) idea. Both Tom and Hiroshi have good
|
|
> reasons for each of these - would it be possible for you guys to post with
|
|
> your reasons for and against both the techniques.
|
|
|
|
I don't object to adding attisdropped field. What
|
|
I meant to say is that the differene is very small.
|
|
|
|
regards,
|
|
Hiroshi Inoue
|
|
|
|
From tgl@sss.pgh.pa.us Sat Apr 13 11:30:17 2002
|
|
Return-path: <tgl@sss.pgh.pa.us>
|
|
Received: from sss.pgh.pa.us (root@[192.204.191.242])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3DFUGS26218
|
|
for <pgman@candle.pha.pa.us>; Sat, 13 Apr 2002 11:30:16 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3DFTjF15655;
|
|
Sat, 13 Apr 2002 11:29:45 -0400 (EDT)
|
|
To: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Hiroshi Inoue" <Inoue@tpf.co.jp>, pgsql-hackers@postgresql.org
|
|
Subject: Re: DROP COLUMN (was RFC: Restructuring pg_aggregate)
|
|
In-Reply-To: <001701c1e2b2$e7b10a40$0200a8c0@SOL>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL>
|
|
Comments: In-reply-to "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
message dated "Sat, 13 Apr 2002 14:17:34 +0800"
|
|
Date: Sat, 13 Apr 2002 11:29:45 -0400
|
|
Message-ID: <15652.1018711785@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Status: OR
|
|
|
|
[ way past time to change the title of this thread ]
|
|
|
|
"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:
|
|
> OK, sounds fair. However, is there a more aggressive way of reclaiming the
|
|
> space? The problem with updating all the rows to null for that column is
|
|
> that the on-disk size is doubled anyway, right? So, could a VACUUM FULL
|
|
> process do the nulling for us? Vacuum works outside of normal transaction
|
|
> constraints anyway...?
|
|
|
|
No, VACUUM has the same transactional constraints as everyone else
|
|
(unless you'd like a crash during VACUUM to trash your table...)
|
|
|
|
I do not think that we necessarily need to provide a special mechanism
|
|
for this at all. The docs for DROP COLUMN could simply explain that
|
|
the DROP itself doesn't reclaim the space, but that the space will be
|
|
reclaimed over time as extant rows are updated or deleted. If you want
|
|
to hurry the process along you could do
|
|
UPDATE table SET othercol = othercol
|
|
VACUUM FULL
|
|
to force all the rows to be updated and then reclaim space. But given
|
|
the peak-space-is-twice-as-much behavior, this is not obviously a win.
|
|
I'd sure object to an implementation that *forced* that approach on me,
|
|
whether during DROP itself or the next VACUUM.
|
|
|
|
> Also, it seems to me that at some point we are forced to break client
|
|
> compatibility. Either we add attisdropped field to pg_attribute, or we use
|
|
> Hiroshi's (-1 * attnum - offset) idea. Both Tom and Hiroshi have good
|
|
> reasons for each of these - would it be possible for you guys to post with
|
|
> your reasons for and against both the techniques.
|
|
|
|
Er, didn't we do that already?
|
|
|
|
regards, tom lane
|
|
|
|
From chriskl@familyhealth.com.au Sun Apr 14 01:06:31 2002
|
|
Return-path: <chriskl@familyhealth.com.au>
|
|
Received: from mail.iinet.net.au (symphony-03.iinet.net.au [203.59.3.35])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g3E56TS03274
|
|
for <pgman@candle.pha.pa.us>; Sun, 14 Apr 2002 01:06:30 -0400 (EDT)
|
|
Received: (qmail 20365 invoked by uid 666); 14 Apr 2002 05:06:31 -0000
|
|
Received: from unknown (HELO SOL) (203.59.168.230)
|
|
by mail.iinet.net.au with SMTP; 14 Apr 2002 05:06:31 -0000
|
|
Message-ID: <00c601c1e371$0e324670$0200a8c0@SOL>
|
|
From: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
To: "Tom Lane" <tgl@sss.pgh.pa.us>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Hiroshi Inoue" <Inoue@tpf.co.jp>, <pgsql-hackers@postgresql.org>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL> <15652.1018711785@sss.pgh.pa.us>
|
|
Subject: Re: DROP COLUMN (was RFC: Restructuring pg_aggregate)
|
|
Date: Sun, 14 Apr 2002 12:58:43 +0800
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook Express 5.50.4522.1200
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200
|
|
Status: OR
|
|
|
|
> No, VACUUM has the same transactional constraints as everyone else
|
|
> (unless you'd like a crash during VACUUM to trash your table...)
|
|
|
|
Seriously, you can run VACUUM in a transaction and rollback the movement of
|
|
a tuple on disk? What do you mean by same transactional constraints?
|
|
|
|
Chris
|
|
|
|
|
|
From pgsql-hackers-owner+M21278@postgresql.org Sat Apr 13 12:21:20 2002
|
|
Return-path: <pgsql-hackers-owner+M21278@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3DGLKS29823
|
|
for <pgman@candle.pha.pa.us>; Sat, 13 Apr 2002 12:21:20 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 9B4AF475CA6; Sat, 13 Apr 2002 12:21:12 -0400 (EDT)
|
|
Received: from sss.pgh.pa.us (unknown [192.204.191.242])
|
|
by postgresql.org (Postfix) with ESMTP id 1ED76474E71
|
|
for <pgsql-hackers@postgresql.org>; Sat, 13 Apr 2002 12:20:07 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3DGJeF15983;
|
|
Sat, 13 Apr 2002 12:19:40 -0400 (EDT)
|
|
To: Hannu Krosing <hannu@tm.ee>
|
|
cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
Bruce Momjian <pgman@candle.pha.pa.us>, Hiroshi Inoue <Inoue@tpf.co.jp>,
|
|
pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] DROP COLUMN (was RFC: Restructuring pg_aggregate)
|
|
In-Reply-To: <1018716432.3360.9.camel@taru.tm.ee>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL> <15652.1018711785@sss.pgh.pa.us> <1018716432.3360.9.camel@taru.tm.ee>
|
|
Comments: In-reply-to Hannu Krosing <hannu@tm.ee>
|
|
message dated "13 Apr 2002 18:47:07 +0200"
|
|
Date: Sat, 13 Apr 2002 12:19:40 -0400
|
|
Message-ID: <15980.1018714780@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Hannu Krosing <hannu@tm.ee> writes:
|
|
>> No, VACUUM has the same transactional constraints as everyone else
|
|
>> (unless you'd like a crash during VACUUM to trash your table...)
|
|
|
|
> But can't it do the SET TO NULL thing if it knows that the transaction
|
|
> that dropped the column has committed.
|
|
|
|
Hmm, you're thinking of allowing VACUUM to overwrite tuples in-place?
|
|
Strikes me as unsafe, but I'm not really sure.
|
|
|
|
In any case it's not that easy. If the column is wide enough
|
|
that reclaiming its space is actually worth doing, then presumably
|
|
most of its entries are just TOAST links, and what has to be done is
|
|
not just rewrite the main tuple but mark the TOAST rows deleted.
|
|
This is not something that VACUUM does now; I'd be rather concerned
|
|
about the locking implications (especially for lightweight VACUUM).
|
|
|
|
regards, tom lane
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
|
|
|
|
From pgsql-hackers-owner+M21277@postgresql.org Sat Apr 13 11:51:02 2002
|
|
Return-path: <pgsql-hackers-owner+M21277@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3DFp1S28016
|
|
for <pgman@candle.pha.pa.us>; Sat, 13 Apr 2002 11:51:01 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id B76F5475D68; Sat, 13 Apr 2002 11:47:59 -0400 (EDT)
|
|
Received: from gw.itmeedia.ee (gw.itmeedia.ee [213.180.3.226])
|
|
by postgresql.org (Postfix) with SMTP id 0635E475C6F
|
|
for <pgsql-hackers@postgresql.org>; Sat, 13 Apr 2002 11:47:01 -0400 (EDT)
|
|
Received: (qmail 12309 invoked from network); 13 Apr 2002 15:47:06 -0000
|
|
Received: from taru.itmeedia.ee (HELO taru.tm.ee) (213.180.3.230)
|
|
by gw.itmeedia.ee with SMTP; 13 Apr 2002 15:47:06 -0000
|
|
Subject: Re: [HACKERS] DROP COLUMN (was RFC: Restructuring pg_aggregate)
|
|
From: Hannu Krosing <hannu@tm.ee>
|
|
To: Tom Lane <tgl@sss.pgh.pa.us>
|
|
cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
Bruce Momjian <pgman@candle.pha.pa.us>, Hiroshi Inoue <Inoue@tpf.co.jp>,
|
|
pgsql-hackers@postgresql.org
|
|
In-Reply-To: <15652.1018711785@sss.pgh.pa.us>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au>
|
|
<1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL>
|
|
<15652.1018711785@sss.pgh.pa.us>
|
|
Content-Type: text/plain
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Mailer: Ximian Evolution 1.0.3.99
|
|
Date: 13 Apr 2002 18:47:07 +0200
|
|
Message-ID: <1018716432.3360.9.camel@taru.tm.ee>
|
|
MIME-Version: 1.0
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
On Sat, 2002-04-13 at 17:29, Tom Lane wrote:
|
|
> [ way past time to change the title of this thread ]
|
|
>
|
|
> "Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:
|
|
> > OK, sounds fair. However, is there a more aggressive way of reclaiming the
|
|
> > space? The problem with updating all the rows to null for that column is
|
|
> > that the on-disk size is doubled anyway, right? So, could a VACUUM FULL
|
|
> > process do the nulling for us? Vacuum works outside of normal transaction
|
|
> > constraints anyway...?
|
|
>
|
|
> No, VACUUM has the same transactional constraints as everyone else
|
|
> (unless you'd like a crash during VACUUM to trash your table...)
|
|
|
|
But can't it do the SET TO NULL thing if it knows that the transaction
|
|
that dropped the column has committed.
|
|
|
|
This could probably even be done in the light version of vacuum with a
|
|
special flag (VACUUM RECLAIM).
|
|
|
|
Of course running this this makes sense only if the dropped column had
|
|
some significant amount of data .
|
|
|
|
> I do not think that we necessarily need to provide a special mechanism
|
|
> for this at all. The docs for DROP COLUMN could simply explain that
|
|
> the DROP itself doesn't reclaim the space, but that the space will be
|
|
> reclaimed over time as extant rows are updated or deleted. If you want
|
|
> to hurry the process along you could do
|
|
> UPDATE table SET othercol = othercol
|
|
> VACUUM FULL
|
|
|
|
If only we could do it in namageable chunks:
|
|
|
|
FOR i IN 0 TO (size(table)/chunk) DO
|
|
UPDATE table SET othercol = othercol OFFSET i*chunk LIMIT chunk
|
|
VACUUM FULL;
|
|
END FOR;
|
|
|
|
or even better - "VACUUM FULL OFFSET i*chunk LIMIT chunk" and then make
|
|
chunk == 1 :)
|
|
|
|
--------------
|
|
Hannu
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 2: you can get off all lists at once with the unregister command
|
|
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
|
|
|
From pgsql-hackers-owner+M21292@postgresql.org Sun Apr 14 01:07:16 2002
|
|
Return-path: <pgsql-hackers-owner+M21292@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3E57FS03403
|
|
for <pgman@candle.pha.pa.us>; Sun, 14 Apr 2002 01:07:15 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 78A86475DD7; Sun, 14 Apr 2002 01:07:12 -0400 (EDT)
|
|
Received: from mail.iinet.net.au (symphony-03.iinet.net.au [203.59.3.35])
|
|
by postgresql.org (Postfix) with SMTP id DA1D447593E
|
|
for <pgsql-hackers@postgresql.org>; Sun, 14 Apr 2002 01:06:32 -0400 (EDT)
|
|
Received: (qmail 20365 invoked by uid 666); 14 Apr 2002 05:06:31 -0000
|
|
Received: from unknown (HELO SOL) (203.59.168.230)
|
|
by mail.iinet.net.au with SMTP; 14 Apr 2002 05:06:31 -0000
|
|
Message-ID: <00c601c1e371$0e324670$0200a8c0@SOL>
|
|
From: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
To: "Tom Lane" <tgl@sss.pgh.pa.us>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Hiroshi Inoue" <Inoue@tpf.co.jp>, <pgsql-hackers@postgresql.org>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL> <15652.1018711785@sss.pgh.pa.us>
|
|
Subject: Re: [HACKERS] DROP COLUMN (was RFC: Restructuring pg_aggregate)
|
|
Date: Sun, 14 Apr 2002 12:58:43 +0800
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook Express 5.50.4522.1200
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
> No, VACUUM has the same transactional constraints as everyone else
|
|
> (unless you'd like a crash during VACUUM to trash your table...)
|
|
|
|
Seriously, you can run VACUUM in a transaction and rollback the movement of
|
|
a tuple on disk? What do you mean by same transactional constraints?
|
|
|
|
Chris
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 4: Don't 'kill -9' the postmaster
|
|
|
|
From tgl@sss.pgh.pa.us Sun Apr 14 14:13:33 2002
|
|
Return-path: <tgl@sss.pgh.pa.us>
|
|
Received: from sss.pgh.pa.us (root@[192.204.191.242])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3EIDWS18224
|
|
for <pgman@candle.pha.pa.us>; Sun, 14 Apr 2002 14:13:32 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3EIDMF22681;
|
|
Sun, 14 Apr 2002 14:13:22 -0400 (EDT)
|
|
To: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Hiroshi Inoue" <Inoue@tpf.co.jp>, pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] DROP COLUMN (was RFC: Restructuring pg_aggregate)
|
|
In-Reply-To: <00c601c1e371$0e324670$0200a8c0@SOL>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL> <15652.1018711785@sss.pgh.pa.us> <00c601c1e371$0e324670$0200a8c0@SOL>
|
|
Comments: In-reply-to "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
message dated "Sun, 14 Apr 2002 12:58:43 +0800"
|
|
Date: Sun, 14 Apr 2002 14:13:21 -0400
|
|
Message-ID: <22678.1018808001@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Status: OR
|
|
|
|
"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:
|
|
>> No, VACUUM has the same transactional constraints as everyone else
|
|
>> (unless you'd like a crash during VACUUM to trash your table...)
|
|
|
|
> Seriously, you can run VACUUM in a transaction and rollback the movement of
|
|
> a tuple on disk? What do you mean by same transactional constraints?
|
|
|
|
In VACUUM FULL, tuples moved to compact the table aren't good until you
|
|
commit. In this hypothetical column-drop-implementing VACUUM, I think
|
|
there'd need to be some similar rule --- otherwise it's not clear what
|
|
happens to TOASTED data if you crash partway through. (In particular,
|
|
if we tried overwriting main tuples in place as Hannu was suggesting,
|
|
we'd need a way of being certain the deletion of the corresponding TOAST
|
|
rows occurs *before* we overwrite the only reference to them.)
|
|
|
|
regards, tom lane
|
|
|
|
From pgsql-hackers-owner+M21305@postgresql.org Sun Apr 14 14:14:46 2002
|
|
Return-path: <pgsql-hackers-owner+M21305@postgresql.org>
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3EIEkS18333
|
|
for <pgman@candle.pha.pa.us>; Sun, 14 Apr 2002 14:14:46 -0400 (EDT)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by postgresql.org (Postfix) with SMTP
|
|
id 8FA74475C4C; Sun, 14 Apr 2002 14:14:43 -0400 (EDT)
|
|
Received: from sss.pgh.pa.us (unknown [192.204.191.242])
|
|
by postgresql.org (Postfix) with ESMTP id 8AC04475892
|
|
for <pgsql-hackers@postgresql.org>; Sun, 14 Apr 2002 14:13:52 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3EIDMF22681;
|
|
Sun, 14 Apr 2002 14:13:22 -0400 (EDT)
|
|
To: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
cc: "Bruce Momjian" <pgman@candle.pha.pa.us>,
|
|
"Hiroshi Inoue" <Inoue@tpf.co.jp>, pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] DROP COLUMN (was RFC: Restructuring pg_aggregate)
|
|
In-Reply-To: <00c601c1e371$0e324670$0200a8c0@SOL>
|
|
References: <20020411233659.O69846-100000@houston.familyhealth.com.au> <1824.1018542155@sss.pgh.pa.us> <001701c1e2b2$e7b10a40$0200a8c0@SOL> <15652.1018711785@sss.pgh.pa.us> <00c601c1e371$0e324670$0200a8c0@SOL>
|
|
Comments: In-reply-to "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
|
|
message dated "Sun, 14 Apr 2002 12:58:43 +0800"
|
|
Date: Sun, 14 Apr 2002 14:13:21 -0400
|
|
Message-ID: <22678.1018808001@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:
|
|
>> No, VACUUM has the same transactional constraints as everyone else
|
|
>> (unless you'd like a crash during VACUUM to trash your table...)
|
|
|
|
> Seriously, you can run VACUUM in a transaction and rollback the movement of
|
|
> a tuple on disk? What do you mean by same transactional constraints?
|
|
|
|
In VACUUM FULL, tuples moved to compact the table aren't good until you
|
|
commit. In this hypothetical column-drop-implementing VACUUM, I think
|
|
there'd need to be some similar rule --- otherwise it's not clear what
|
|
happens to TOASTED data if you crash partway through. (In particular,
|
|
if we tried overwriting main tuples in place as Hannu was suggesting,
|
|
we'd need a way of being certain the deletion of the corresponding TOAST
|
|
rows occurs *before* we overwrite the only reference to them.)
|
|
|
|
regards, tom lane
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 5: Have you checked our extensive FAQ?
|
|
|
|
http://www.postgresql.org/users-lounge/docs/faq.html
|
|
|