mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-08-22 04:53:23 +02:00
1322 lines
58 KiB
Plaintext
1322 lines
58 KiB
Plaintext
From pgsql-hackers-owner+M4897@hub.org Wed Jul 12 00:15:33 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA06129
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 00:15:32 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C4FiW14410;
|
|
Wed, 12 Jul 2000 00:15:44 -0400 (EDT)
|
|
Received: from onyx-technologies.com (iron.onyx-technologies.com [216.205.44.194] (may be forged))
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C4ECW07902
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 00:14:12 -0400 (EDT)
|
|
Received: from onyx-technologies.com (collins.onyx-technologies.com [192.168.188.10])
|
|
by onyx-technologies.com (8.9.2/8.9.0) with ESMTP id AAA14868
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 00:11:43 -0400 (EDT)
|
|
Message-ID: <396BE1B6.F755C5CE@onyx-technologies.com>
|
|
Date: Tue, 11 Jul 2000 23:10:46 -0400
|
|
From: Jeffery Collins <collins@onyx-technologies.com>
|
|
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.14-15mdk i686)
|
|
X-Accept-Language: en
|
|
MIME-Version: 1.0
|
|
To: pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
References: <20000711185318.W25571@fw.wintelcom.net> <396BEA84.1A06F51F@nimrod.itg.telecom.com.au>
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: ORr
|
|
|
|
It seems like a first step would be to just have postmaster cache unused
|
|
connections. In other words if a client closes a connection, postmaster
|
|
keeps the connection and the child process around for the next connect
|
|
request. This has many of your advantages, but not all. However, it seems
|
|
like it would be simpler than attempting to multiplex a connection between
|
|
multiple clients.
|
|
|
|
Jeff
|
|
|
|
>
|
|
> Alfred Perlstein wrote:
|
|
> >
|
|
> > In an effort to complicate the postmaster beyond recognition I'm
|
|
> > proposing an idea that I hope can be useful to the developers.
|
|
> >
|
|
> > Connection pooling:
|
|
> >
|
|
> > The idea is to have the postmaster multiplex and do hand-offs of
|
|
> > database connections to other postgresql processes when the max
|
|
> > connections has been exceeded.
|
|
> >
|
|
> > This allows several gains:
|
|
> >
|
|
> > 1) Postgresql can support a large number of connections without
|
|
> > requiring a large amount of processes to do so.
|
|
> >
|
|
> > 2) Connection startup/finish will be cheaper because Postgresql
|
|
> > processes will not exit and need to reninit things such as shared
|
|
> > memory attachments and file opens. This will also reduce the load
|
|
> > on the supporting operating system and make postgresql much 'cheaper'
|
|
> > to run on systems that don't support the fork() model of execution
|
|
> > gracefully.
|
|
> >
|
|
> > 3) Long running connections can be preempted at transaction boundries
|
|
> > allowing other connections to gain process timeslices from the
|
|
> > connection pool.
|
|
> >
|
|
> > The idea is to make the postmaster that accepts connections a broker
|
|
> > for the connections. It will dole out descriptors using file
|
|
> > descriptor passing to children. If there's a demand for connections
|
|
> > meaning that all the postmasters are busy and there are pending
|
|
> > connections the postmaster can ask for a yeild on one of the
|
|
> > connections.
|
|
> >
|
|
> > A yeild involves the child postgresql process passing back the
|
|
> > client connection at a transaction boundry (between transactions)
|
|
> > so it can later be given to another (perhaps the same) child process.
|
|
> >
|
|
> > I spoke with Bruce briefly about this and he suggested that system
|
|
> > tables containing unique IDs could be used to identify passed
|
|
> > connections to the children and back to the postmaster.
|
|
> >
|
|
> > When a handoff occurs, the descriptor along with an ID referencing
|
|
> > things like temp tables and enviornment variables and authentication
|
|
> > information could be handed out as well allowing the child to resume
|
|
> > service to the interrupted connection.
|
|
> >
|
|
> > I really don't have the knowledge of Postgresql internals to
|
|
> > accomplish this, but the concepts are simple and the gains would
|
|
> > seem to be very high.
|
|
> >
|
|
> > Comments?
|
|
> >
|
|
> > --
|
|
> > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
|
|
> > "I have the heart of a child; I keep it in a jar on my desk."
|
|
|
|
|
|
From pgsql-hackers-owner+M4904@hub.org Wed Jul 12 01:24:09 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA06757
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 01:24:08 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C5OLW65679;
|
|
Wed, 12 Jul 2000 01:24:21 -0400 (EDT)
|
|
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C5MkW61040
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 01:22:46 -0400 (EDT)
|
|
Received: (from bright@localhost)
|
|
by fw.wintelcom.net (8.10.0/8.10.0) id e6C5Md429901;
|
|
Tue, 11 Jul 2000 22:22:39 -0700 (PDT)
|
|
Date: Tue, 11 Jul 2000 22:22:39 -0700
|
|
From: Alfred Perlstein <bright@wintelcom.net>
|
|
To: Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>
|
|
Cc: pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
Message-ID: <20000711222239.X25571@fw.wintelcom.net>
|
|
References: <20000711185318.W25571@fw.wintelcom.net> <396BEA84.1A06F51F@nimrod.itg.telecom.com.au>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Disposition: inline
|
|
User-Agent: Mutt/1.2i
|
|
In-Reply-To: <396BEA84.1A06F51F@nimrod.itg.telecom.com.au>; from chrisb@nimrod.itg.telstra.com.au on Wed, Jul 12, 2000 at 01:48:20PM +1000
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
* Chris Bitmead <chrisb@nimrod.itg.telstra.com.au> [000711 20:53] wrote:
|
|
>
|
|
> Seems a lot trickier than you think. A backend can only be running
|
|
> one transaction at a time, so you'd have to keep track of which backends
|
|
> are in the middle of a transaction. I can imagine race conditions here.
|
|
> And backends can have contexts that are set by various clients using
|
|
> SET and friends. Then you'd have to worry about authentication each
|
|
> time. And you'd have to have algorithms for cleaning up old processes
|
|
> and/or dead processes. It all really sounds a bit hard.
|
|
|
|
The backends can simply inform the postmaster when they are ready
|
|
either because they are done with a connection or because they
|
|
have just closed a transaction.
|
|
|
|
All the state (auth/temp tables) can be held in the system tables.
|
|
|
|
It's complicated, but no where on the order of something like
|
|
a new storage manager.
|
|
|
|
-Alfred
|
|
|
|
From bright@fw.wintelcom.net Wed Jul 12 01:34:30 2000
|
|
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA06793
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 01:34:29 -0400 (EDT)
|
|
Received: (from bright@localhost)
|
|
by fw.wintelcom.net (8.10.0/8.10.0) id e6C5Z1f00384;
|
|
Tue, 11 Jul 2000 22:35:01 -0700 (PDT)
|
|
Date: Tue, 11 Jul 2000 22:35:00 -0700
|
|
From: Alfred Perlstein <bright@wintelcom.net>
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
Cc: Jeffery Collins <collins@onyx-technologies.com>, pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
Message-ID: <20000711223500.Z25571@fw.wintelcom.net>
|
|
References: <396BE1B6.F755C5CE@onyx-technologies.com> <200007120428.AAA06357@candle.pha.pa.us>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Disposition: inline
|
|
User-Agent: Mutt/1.2i
|
|
In-Reply-To: <200007120428.AAA06357@candle.pha.pa.us>; from pgman@candle.pha.pa.us on Wed, Jul 12, 2000 at 12:28:46AM -0400
|
|
Status: OR
|
|
|
|
* Bruce Momjian <pgman@candle.pha.pa.us> [000711 21:31] wrote:
|
|
> > It seems like a first step would be to just have postmaster cache unused
|
|
> > connections. In other words if a client closes a connection, postmaster
|
|
> > keeps the connection and the child process around for the next connect
|
|
> > request. This has many of your advantages, but not all. However, it seems
|
|
> > like it would be simpler than attempting to multiplex a connection between
|
|
> > multiple clients.
|
|
> >
|
|
>
|
|
> This does seem like a good optimization.
|
|
|
|
I'm not sure if the postmaster is needed besideds just to fork/exec
|
|
the backend, if so then when a backend finishes it can just call
|
|
accept() on the listening socket inherited from the postmaster to
|
|
get the next incomming connection.
|
|
|
|
-Alfred
|
|
|
|
From pgsql-hackers-owner+M4906@hub.org Wed Jul 12 01:36:44 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA06806
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 01:36:44 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C5akW94517;
|
|
Wed, 12 Jul 2000 01:36:46 -0400 (EDT)
|
|
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C5ZCW88503
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 01:35:12 -0400 (EDT)
|
|
Received: (from bright@localhost)
|
|
by fw.wintelcom.net (8.10.0/8.10.0) id e6C5Z1f00384;
|
|
Tue, 11 Jul 2000 22:35:01 -0700 (PDT)
|
|
Date: Tue, 11 Jul 2000 22:35:00 -0700
|
|
From: Alfred Perlstein <bright@wintelcom.net>
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
Cc: Jeffery Collins <collins@onyx-technologies.com>, pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
Message-ID: <20000711223500.Z25571@fw.wintelcom.net>
|
|
References: <396BE1B6.F755C5CE@onyx-technologies.com> <200007120428.AAA06357@candle.pha.pa.us>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Disposition: inline
|
|
User-Agent: Mutt/1.2i
|
|
In-Reply-To: <200007120428.AAA06357@candle.pha.pa.us>; from pgman@candle.pha.pa.us on Wed, Jul 12, 2000 at 12:28:46AM -0400
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
* Bruce Momjian <pgman@candle.pha.pa.us> [000711 21:31] wrote:
|
|
> > It seems like a first step would be to just have postmaster cache unused
|
|
> > connections. In other words if a client closes a connection, postmaster
|
|
> > keeps the connection and the child process around for the next connect
|
|
> > request. This has many of your advantages, but not all. However, it seems
|
|
> > like it would be simpler than attempting to multiplex a connection between
|
|
> > multiple clients.
|
|
> >
|
|
>
|
|
> This does seem like a good optimization.
|
|
|
|
I'm not sure if the postmaster is needed besideds just to fork/exec
|
|
the backend, if so then when a backend finishes it can just call
|
|
accept() on the listening socket inherited from the postmaster to
|
|
get the next incomming connection.
|
|
|
|
-Alfred
|
|
|
|
From pgsql-hackers-owner+M4907@hub.org Wed Jul 12 01:55:39 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA06881
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 01:55:38 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C5tnW34576;
|
|
Wed, 12 Jul 2000 01:55:49 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C5rfW28119
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 01:53:42 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id BAA21895;
|
|
Wed, 12 Jul 2000 01:52:56 -0400 (EDT)
|
|
To: Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>
|
|
cc: Alfred Perlstein <bright@wintelcom.net>, pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
In-reply-to: <396BEA84.1A06F51F@nimrod.itg.telecom.com.au>
|
|
References: <20000711185318.W25571@fw.wintelcom.net> <396BEA84.1A06F51F@nimrod.itg.telecom.com.au>
|
|
Comments: In-reply-to Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>
|
|
message dated "Wed, 12 Jul 2000 13:48:20 +1000"
|
|
Date: Wed, 12 Jul 2000 01:52:56 -0400
|
|
Message-ID: <21892.963381176@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
Chris Bitmead <chrisb@nimrod.itg.telstra.com.au> writes:
|
|
> Seems a lot trickier than you think. A backend can only be running
|
|
> one transaction at a time, so you'd have to keep track of which backends
|
|
> are in the middle of a transaction. I can imagine race conditions here.
|
|
|
|
Aborting out of a transaction is no problem; we have code for that
|
|
anyway. More serious problems:
|
|
|
|
* We have no code for reassigning a backend to a different database,
|
|
so the pooling would have to be per-database.
|
|
|
|
* AFAIK there is no portable way to pass a socket connection from the
|
|
postmaster to an already-existing backend process. If you do a
|
|
fork() then the connection is inherited ... otherwise you've got a
|
|
problem. (You could work around this if the postmaster relays
|
|
every single byte in both directions between client and backend,
|
|
but the performance problems with that should be obvious.)
|
|
|
|
> And backends can have contexts that are set by various clients using
|
|
> SET and friends.
|
|
|
|
Resetting SET variables would be a problem, and there's also the
|
|
assigned user name to be reset. This doesn't seem impossible, but
|
|
it does seem tedious and error-prone. (OTOH, Peter E's recent work
|
|
on guc.c might have unified option-handling enough to bring it
|
|
within reason.)
|
|
|
|
The killer problem here is that you can't hand off a connection
|
|
accepted by the postmaster to a backend except by fork() --- at least
|
|
not with methods that work on a wide variety of Unixen. Unless someone
|
|
has a way around that, I think the idea is dead in the water; the lesser
|
|
issues don't matter.
|
|
|
|
regards, tom lane
|
|
|
|
From pgsql-hackers-owner+M4910@hub.org Wed Jul 12 02:24:16 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id CAA11184
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 02:24:15 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C6OAW98187;
|
|
Wed, 12 Jul 2000 02:24:10 -0400 (EDT)
|
|
Received: from acheron.rime.com.au (root@albatr.lnk.telstra.net [139.130.54.222])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C6MZW95741
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 02:22:36 -0400 (EDT)
|
|
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
|
|
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id QAA12845;
|
|
Wed, 12 Jul 2000 16:16:23 +1000
|
|
Message-Id: <3.0.5.32.20000712162210.0098fb00@mail.rhyme.com.au>
|
|
X-Sender: pjw@mail.rhyme.com.au
|
|
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
|
|
Date: Wed, 12 Jul 2000 16:22:10 +1000
|
|
To: Tom Lane <tgl@sss.pgh.pa.us>,
|
|
Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>
|
|
From: Philip Warner <pjw@rhyme.com.au>
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
Cc: Alfred Perlstein <bright@wintelcom.net>, pgsql-hackers@hub.org
|
|
In-Reply-To: <21892.963381176@sss.pgh.pa.us>
|
|
References: <396BEA84.1A06F51F@nimrod.itg.telecom.com.au>
|
|
<20000711185318.W25571@fw.wintelcom.net>
|
|
<396BEA84.1A06F51F@nimrod.itg.telecom.com.au>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset="us-ascii"
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
At 01:52 12/07/00 -0400, Tom Lane wrote:
|
|
>
|
|
>The killer problem here is that you can't hand off a connection
|
|
>accepted by the postmaster to a backend except by fork() --- at least
|
|
>not with methods that work on a wide variety of Unixen. Unless someone
|
|
>has a way around that, I think the idea is dead in the water; the lesser
|
|
>issues don't matter.
|
|
>
|
|
|
|
My understanding of pg client interfaces is that the client uses ont of the
|
|
pg interface libraries to make a connection to the db; they specify host &
|
|
port and get back some kind of connection object.
|
|
|
|
What stops the interface library from using the host & port to talk to the
|
|
postmaster, find the host & port the spare db server, then connect directly
|
|
to the server? This second connection is passed back in the connection object.
|
|
|
|
When the client disconnects from the server, it tells the postmaster it's
|
|
available again etc.
|
|
|
|
ie. in very rough terms:
|
|
|
|
client calls interface to connect
|
|
|
|
interface talks to postmaster on port 5432, says "I want a server for
|
|
xyz db"
|
|
|
|
postmaster replies with "Try port ABCD" OR "no servers available"
|
|
postmaster marks the nominated server as 'used'
|
|
postmaster disconnects from client
|
|
|
|
interface connects to port ABCD as per normal protocols
|
|
interface fills in connection object & returns
|
|
|
|
...client does some work...
|
|
|
|
client disconnects
|
|
|
|
db server tells postmaster it's available again.
|
|
|
|
|
|
There would also need to be timeout code to handle the case where the
|
|
interface did not do the second connect.
|
|
|
|
You could also have the interface allocate a port send it's number to the
|
|
postmaster then listen on it, but I think that would represent a potential
|
|
security hole.
|
|
|
|
|
|
----------------------------------------------------------------
|
|
Philip Warner | __---_____
|
|
Albatross Consulting Pty. Ltd. |----/ - \
|
|
(A.C.N. 008 659 498) | /(@) ______---_
|
|
Tel: (+61) 0500 83 82 81 | _________ \
|
|
Fax: (+61) 0500 83 82 82 | ___________ |
|
|
Http://www.rhyme.com.au | / \|
|
|
| --________--
|
|
PGP key available upon request, | /
|
|
and from pgp5.ai.mit.edu:11371 |/
|
|
|
|
From pgsql-hackers-owner+M4912@hub.org Wed Jul 12 02:32:21 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id CAA11228
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 02:32:20 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C6WWW18412;
|
|
Wed, 12 Jul 2000 02:32:32 -0400 (EDT)
|
|
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C6UwW16062
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 02:30:58 -0400 (EDT)
|
|
Received: (from bright@localhost)
|
|
by fw.wintelcom.net (8.10.0/8.10.0) id e6C6Uov01852;
|
|
Tue, 11 Jul 2000 23:30:50 -0700 (PDT)
|
|
Date: Tue, 11 Jul 2000 23:30:49 -0700
|
|
From: Alfred Perlstein <bright@wintelcom.net>
|
|
To: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Cc: Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>, pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
Message-ID: <20000711233049.A25571@fw.wintelcom.net>
|
|
References: <20000711185318.W25571@fw.wintelcom.net> <396BEA84.1A06F51F@nimrod.itg.telecom.com.au> <21892.963381176@sss.pgh.pa.us>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Disposition: inline
|
|
User-Agent: Mutt/1.2i
|
|
In-Reply-To: <21892.963381176@sss.pgh.pa.us>; from tgl@sss.pgh.pa.us on Wed, Jul 12, 2000 at 01:52:56AM -0400
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
* Tom Lane <tgl@sss.pgh.pa.us> [000711 22:53] wrote:
|
|
> Chris Bitmead <chrisb@nimrod.itg.telstra.com.au> writes:
|
|
> > Seems a lot trickier than you think. A backend can only be running
|
|
> > one transaction at a time, so you'd have to keep track of which backends
|
|
> > are in the middle of a transaction. I can imagine race conditions here.
|
|
>
|
|
> Aborting out of a transaction is no problem; we have code for that
|
|
> anyway. More serious problems:
|
|
>
|
|
> * We have no code for reassigning a backend to a different database,
|
|
> so the pooling would have to be per-database.
|
|
|
|
That would need to be fixed. How difficult would that be?
|
|
|
|
> * AFAIK there is no portable way to pass a socket connection from the
|
|
> postmaster to an already-existing backend process. If you do a
|
|
> fork() then the connection is inherited ... otherwise you've got a
|
|
> problem. (You could work around this if the postmaster relays
|
|
> every single byte in both directions between client and backend,
|
|
> but the performance problems with that should be obvious.)
|
|
|
|
no, see below.
|
|
|
|
> > And backends can have contexts that are set by various clients using
|
|
> > SET and friends.
|
|
>
|
|
> Resetting SET variables would be a problem, and there's also the
|
|
> assigned user name to be reset. This doesn't seem impossible, but
|
|
> it does seem tedious and error-prone. (OTOH, Peter E's recent work
|
|
> on guc.c might have unified option-handling enough to bring it
|
|
> within reason.)
|
|
|
|
What can be done is that each incomming connection can be assigned an
|
|
ID into a system table. As options are added the system would assign
|
|
them to key-value pairs in this table. Once someone detects that the
|
|
remote side has closed the connection the data can be destroyed, but
|
|
until then along with the descriptor passing the ID of the client
|
|
as an index into the table can be passed for the backend to fetch.
|
|
|
|
> The killer problem here is that you can't hand off a connection
|
|
> accepted by the postmaster to a backend except by fork() --- at least
|
|
> not with methods that work on a wide variety of Unixen. Unless someone
|
|
> has a way around that, I think the idea is dead in the water; the lesser
|
|
> issues don't matter.
|
|
|
|
The code has been around since 4.2BSD, it takes a bit of #ifdef to
|
|
get it right on all systems but it's not impossible, have a look at
|
|
http://www.fhttpd.org/ for a web server that does this in a portable
|
|
fashion.
|
|
|
|
I should have a library whipped up for you guys really soon now
|
|
to handle the descriptor and message passing.
|
|
|
|
--
|
|
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
|
|
"I have the heart of a child; I keep it in a jar on my desk."
|
|
|
|
From pgsql-hackers-owner+M4913@hub.org Wed Jul 12 03:06:54 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA11529
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 03:06:53 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C76ZW95615;
|
|
Wed, 12 Jul 2000 03:06:35 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C74gW93358
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 03:04:42 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id DAA22136;
|
|
Wed, 12 Jul 2000 03:04:13 -0400 (EDT)
|
|
To: Alfred Perlstein <bright@wintelcom.net>
|
|
cc: Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>, pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
In-reply-to: <20000711233049.A25571@fw.wintelcom.net>
|
|
References: <20000711185318.W25571@fw.wintelcom.net> <396BEA84.1A06F51F@nimrod.itg.telecom.com.au> <21892.963381176@sss.pgh.pa.us> <20000711233049.A25571@fw.wintelcom.net>
|
|
Comments: In-reply-to Alfred Perlstein <bright@wintelcom.net>
|
|
message dated "Tue, 11 Jul 2000 23:30:49 -0700"
|
|
Date: Wed, 12 Jul 2000 03:04:13 -0400
|
|
Message-ID: <22133.963385453@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
Alfred Perlstein <bright@wintelcom.net> writes:
|
|
> * Tom Lane <tgl@sss.pgh.pa.us> [000711 22:53] wrote:
|
|
>> The killer problem here is that you can't hand off a connection
|
|
>> accepted by the postmaster to a backend except by fork() --- at least
|
|
>> not with methods that work on a wide variety of Unixen.
|
|
|
|
> The code has been around since 4.2BSD, it takes a bit of #ifdef to
|
|
> get it right on all systems but it's not impossible, have a look at
|
|
> http://www.fhttpd.org/ for a web server that does this in a portable
|
|
> fashion.
|
|
|
|
I looked at this to see if it would teach me something I didn't know.
|
|
It doesn't. It depends on sendmsg() which is a BSD-ism and not very
|
|
portable.
|
|
|
|
regards, tom lane
|
|
|
|
From pgsql-hackers-owner+M4914@hub.org Wed Jul 12 03:12:40 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA11597
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 03:12:39 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C7CjW13459;
|
|
Wed, 12 Jul 2000 03:12:45 -0400 (EDT)
|
|
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C7B8W07036
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 03:11:08 -0400 (EDT)
|
|
Received: (from bright@localhost)
|
|
by fw.wintelcom.net (8.10.0/8.10.0) id e6C79lE02841;
|
|
Wed, 12 Jul 2000 00:09:47 -0700 (PDT)
|
|
Date: Wed, 12 Jul 2000 00:09:47 -0700
|
|
From: Alfred Perlstein <bright@wintelcom.net>
|
|
To: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Cc: Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>, pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
Message-ID: <20000712000947.D25571@fw.wintelcom.net>
|
|
References: <20000711185318.W25571@fw.wintelcom.net> <396BEA84.1A06F51F@nimrod.itg.telecom.com.au> <21892.963381176@sss.pgh.pa.us> <20000711233049.A25571@fw.wintelcom.net> <22133.963385453@sss.pgh.pa.us>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Disposition: inline
|
|
User-Agent: Mutt/1.2i
|
|
In-Reply-To: <22133.963385453@sss.pgh.pa.us>; from tgl@sss.pgh.pa.us on Wed, Jul 12, 2000 at 03:04:13AM -0400
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
* Tom Lane <tgl@sss.pgh.pa.us> [000712 00:04] wrote:
|
|
> Alfred Perlstein <bright@wintelcom.net> writes:
|
|
> > * Tom Lane <tgl@sss.pgh.pa.us> [000711 22:53] wrote:
|
|
> >> The killer problem here is that you can't hand off a connection
|
|
> >> accepted by the postmaster to a backend except by fork() --- at least
|
|
> >> not with methods that work on a wide variety of Unixen.
|
|
>
|
|
> > The code has been around since 4.2BSD, it takes a bit of #ifdef to
|
|
> > get it right on all systems but it's not impossible, have a look at
|
|
> > http://www.fhttpd.org/ for a web server that does this in a portable
|
|
> > fashion.
|
|
>
|
|
> I looked at this to see if it would teach me something I didn't know.
|
|
> It doesn't. It depends on sendmsg() which is a BSD-ism and not very
|
|
> portable.
|
|
|
|
It's also specified by Posix.1g if that means anything.
|
|
|
|
-Alfred
|
|
|
|
From pgsql-hackers-owner+M4916@hub.org Wed Jul 12 03:49:58 2000
|
|
Received: from hub.org (root@hub.org [216.126.84.1])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA11736
|
|
for <pgman@candle.pha.pa.us>; Wed, 12 Jul 2000 03:49:58 -0400 (EDT)
|
|
Received: from hub.org (majordom@localhost [127.0.0.1])
|
|
by hub.org (8.10.1/8.10.1) with SMTP id e6C7oBW95547;
|
|
Wed, 12 Jul 2000 03:50:11 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
|
|
by hub.org (8.10.1/8.10.1) with ESMTP id e6C7mPW93141
|
|
for <pgsql-hackers@hub.org>; Wed, 12 Jul 2000 03:48:25 -0400 (EDT)
|
|
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
|
|
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id DAA22297;
|
|
Wed, 12 Jul 2000 03:47:37 -0400 (EDT)
|
|
To: Philip Warner <pjw@rhyme.com.au>
|
|
cc: Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>,
|
|
Alfred Perlstein <bright@wintelcom.net>, pgsql-hackers@hub.org
|
|
Subject: Re: [HACKERS] Connection pooling.
|
|
In-reply-to: <3.0.5.32.20000712162210.0098fb00@mail.rhyme.com.au>
|
|
References: <396BEA84.1A06F51F@nimrod.itg.telecom.com.au> <20000711185318.W25571@fw.wintelcom.net> <396BEA84.1A06F51F@nimrod.itg.telecom.com.au> <3.0.5.32.20000712162210.0098fb00@mail.rhyme.com.au>
|
|
Comments: In-reply-to Philip Warner <pjw@rhyme.com.au>
|
|
message dated "Wed, 12 Jul 2000 16:22:10 +1000"
|
|
Date: Wed, 12 Jul 2000 03:47:37 -0400
|
|
Message-ID: <22294.963388057@sss.pgh.pa.us>
|
|
From: Tom Lane <tgl@sss.pgh.pa.us>
|
|
X-Mailing-List: pgsql-hackers@postgresql.org
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@hub.org
|
|
Status: OR
|
|
|
|
Philip Warner <pjw@rhyme.com.au> writes:
|
|
> What stops the interface library from using the host & port to talk to
|
|
> the postmaster, find the host & port the spare db server, then connect
|
|
> directly to the server?
|
|
|
|
You're assuming that we can change the on-the-wire protocol freely and
|
|
only the API presented by the client libraries matters. In a perfect
|
|
world that might be true, but reality is that we can't change the wire
|
|
protocol easily. If we do, it breaks all existing precompiled clients.
|
|
Updating clients can be an extremely painful experience in multiple-
|
|
machine installations.
|
|
|
|
Also, we don't have just one set of client libraries to fix. There are
|
|
at least three client-side implementations that don't depend on libpq.
|
|
|
|
We have done protocol updates in the past --- in fact I was responsible
|
|
for the last one. (And I'm still carrying the scars, which is why I'm
|
|
not too enthusiastic about another one.) It's not impossible, but it
|
|
needs more evidence than "this should speed up connections by
|
|
I-don't-know-how-much"...
|
|
|
|
It might also be worth pointing out that the goal was to speed up the
|
|
end-to-end connection time. Redirecting as you suggest is not free
|
|
(at minimum it would appear to require two TCP connection setups and two
|
|
authentication cycles). What evidence have you got that it'd be faster
|
|
than spawning a new backend?
|
|
|
|
I tend to agree with the opinion that connection-pooling on the client
|
|
side offers more bang for the buck than we could hope to get by doing
|
|
surgery on the postmaster/backend setup.
|
|
|
|
Also, to return to the original point, AFAIK we have not tried hard
|
|
to cut the backend startup time, other than the work that was done
|
|
a year or so back to eliminate exec() of a separate executable.
|
|
It'd be worth looking to see what could be done there with zero
|
|
impact on existing clients.
|
|
|
|
regards, tom lane
|
|
|
|
From pgsql-hackers-owner+M16940@postgresql.org Sun Dec 23 23:06:28 2001
|
|
Return-path: <pgsql-hackers-owner+M16940@postgresql.org>
|
|
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBO46R429655
|
|
for <pgman@candle.pha.pa.us>; Sun, 23 Dec 2001 23:06:27 -0500 (EST)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBO40oN57016;
|
|
Sun, 23 Dec 2001 22:00:50 -0600 (CST)
|
|
(envelope-from pgsql-hackers-owner+M16940@postgresql.org)
|
|
Received: from relay.pair.com (relay1.pair.com [209.68.1.20])
|
|
by postgresql.org (8.11.3/8.11.4) with SMTP id fBID0um78493
|
|
for <pgsql-hackers@postgresql.org>; Tue, 18 Dec 2001 08:00:56 -0500 (EST)
|
|
(envelope-from ml@augustz.com)
|
|
Received: (qmail 79914 invoked from network); 18 Dec 2001 13:00:58 -0000
|
|
Received: from acz01997-2.pomona.edu (HELO Microsoft) (134.173.91.3)
|
|
by relay1.pair.com with SMTP; 18 Dec 2001 13:00:58 -0000
|
|
X-pair-Authenticated: 134.173.91.3
|
|
From: "August Zajonc" <ml@augustz.com>
|
|
To: <pgsql-hackers@postgresql.org>
|
|
Subject: [HACKERS] Connection Pooling, a year later
|
|
Date: Tue, 18 Dec 2001 05:00:57 -0800
|
|
Message-ID: <OJEJIPPNGKHEBGFEHPLMAEPGCCAA.ml@augustz.com>
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Priority: 3 (Normal)
|
|
X-MSMail-Priority: Normal
|
|
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
|
|
Importance: Normal
|
|
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
I feel there was a reasonably nice client side attempt at this using a
|
|
worker pool model or something. Can't seem to track it down at this moment.
|
|
Also would spread queries in different ways to get a hot backup equivalent
|
|
etc. It was slick.
|
|
|
|
The key is that pgsql be able to support a very significant number of
|
|
transactions. Be neat to see some numbers on your attempt.
|
|
|
|
Site I used to run had 6 front end webservers running PHP apps. Each
|
|
persistent connection (a requirement to avoid overhead of set-up/teardowns)
|
|
lived as long as the httpd process lived, even if idle. That meant at 250
|
|
processes per server we had a good 1500 connections clicking over. Our
|
|
feeling was that rather than growing to 3,000 connections as the frontend
|
|
grew, why not pool those connections off each machine down to perhaps
|
|
75/machine worker threads that actually did the work.
|
|
|
|
Looks like that's not an issue if these backends suck up few resources.
|
|
Doing something similar with MySQL we'd experiance problems if we got into
|
|
the 2,000 connection range. (kernel/system limits bumped plenty high).
|
|
|
|
While we are on TODO's I would like to point out that some way to fully
|
|
vacume (ie recover deleted and changed) while a db is in full swing is
|
|
critical to larger installtions. We did 2 billion queries between reboots on
|
|
a quad zeon MySQL box, and those are real user based queries not data loads
|
|
or anything like that. At 750-1000 queries/second bringing the database down
|
|
or seriously degrading its performance is not a good option.
|
|
|
|
Enjoy playing with pgsql as always....
|
|
|
|
- AZ
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 4: Don't 'kill -9' the postmaster
|
|
|
|
From dhogaza@pacifier.com Tue Dec 18 11:15:06 2001
|
|
Return-path: <dhogaza@pacifier.com>
|
|
Received: from asteroid.pacifier.com ([199.2.117.154])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBIGF5419342
|
|
for <pgman@candle.pha.pa.us>; Tue, 18 Dec 2001 11:15:05 -0500 (EST)
|
|
Received: from pacifier.com (dsl-dhogaza.pacifier.net [207.202.226.68])
|
|
by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id fBIGEGe29925;
|
|
Tue, 18 Dec 2001 08:14:17 -0800 (PST)
|
|
Message-ID: <3C1F6B81.10500@pacifier.com>
|
|
Date: Tue, 18 Dec 2001 08:14:57 -0800
|
|
From: Don Baccus <dhogaza@pacifier.com>
|
|
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.6) Gecko/20011120
|
|
X-Accept-Language: en-us
|
|
MIME-Version: 1.0
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: mlw <markw@mohawksoft.com>, owensmk@earthlink.net,
|
|
pgsql-hackers@postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
References: <200112180342.fBI3g4s23880@candle.pha.pa.us>
|
|
Content-Type: text/plain; charset=us-ascii; format=flowed
|
|
Content-Transfer-Encoding: 7bit
|
|
Status: OR
|
|
|
|
Bruce Momjian wrote:
|
|
|
|
|
|
> It would just be nice to have it done internally rather than have all
|
|
> the clients do it, iff it can be done cleanly.
|
|
|
|
Serious client applications that need it already do it. Firing up an
|
|
Oracle or most other db's isn't that lightweight a deal, either, it's
|
|
not useful only for PG..
|
|
|
|
Personally I'd just view it as getting in the way, but then I use a
|
|
webserver that's provided connection pooling for client threads for the
|
|
last seven years ...
|
|
|
|
I agree with Tom that the client seems to be the best place to do this.
|
|
|
|
Among other things it isn't that difficult. If you know how to fire up
|
|
one connection, you know how to fire up N of them and adding logic to
|
|
pool them afterwards is easy enough.
|
|
--
|
|
Don Baccus
|
|
Portland, OR
|
|
http://donb.photo.net, http://birdnotes.net, http://openacs.org
|
|
|
|
|
|
From dhogaza@pacifier.com Tue Dec 18 11:24:33 2001
|
|
Return-path: <dhogaza@pacifier.com>
|
|
Received: from asteroid.pacifier.com ([199.2.117.154])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBIGOW421363
|
|
for <pgman@candle.pha.pa.us>; Tue, 18 Dec 2001 11:24:33 -0500 (EST)
|
|
Received: from pacifier.com (dsl-dhogaza.pacifier.net [207.202.226.68])
|
|
by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id fBIGNne00442;
|
|
Tue, 18 Dec 2001 08:23:49 -0800 (PST)
|
|
Message-ID: <3C1F6DBF.2040000@pacifier.com>
|
|
Date: Tue, 18 Dec 2001 08:24:31 -0800
|
|
From: Don Baccus <dhogaza@pacifier.com>
|
|
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.6) Gecko/20011120
|
|
X-Accept-Language: en-us
|
|
MIME-Version: 1.0
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: mlw <markw@mohawksoft.com>, owensmk@earthlink.net,
|
|
pgsql-hackers@postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
References: <200112180357.fBI3vBm24991@candle.pha.pa.us>
|
|
Content-Type: text/plain; charset=us-ascii; format=flowed
|
|
Content-Transfer-Encoding: 7bit
|
|
Status: OR
|
|
|
|
Bruce Momjian wrote:
|
|
|
|
|
|
> Yes, that is assuming you are using PHP. If you are using something
|
|
> else, you connection pooling in there too. All those client interfaces
|
|
> reimplementing connection pooling seems like a waste to me.
|
|
|
|
|
|
Effective pooling's pretty specific to your environment, though, so any
|
|
general mechanism would have to provide a wide-ranging suite of
|
|
parameters governing the number to pool, how long each handle should
|
|
live, what to do if a handle's released by a client while in the midst
|
|
of a transaction (AOLserver rolls back the transaction, other clients
|
|
might want to do something else, i.e. fire a callback or the like), etc etc.
|
|
|
|
I think it would be fairly complex and for those high-throughput
|
|
applications already written with client-side pooling no improvement.
|
|
|
|
And those are the only applications that need it.
|
|
|
|
--
|
|
Don Baccus
|
|
Portland, OR
|
|
http://donb.photo.net, http://birdnotes.net, http://openacs.org
|
|
|
|
|
|
From pgsql-hackers-owner+M16726@postgresql.org Tue Dec 18 11:48:16 2001
|
|
Return-path: <pgsql-hackers-owner+M16726@postgresql.org>
|
|
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBIGmG422658
|
|
for <pgman@candle.pha.pa.us>; Tue, 18 Dec 2001 11:48:16 -0500 (EST)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBIGkFN40986;
|
|
Tue, 18 Dec 2001 10:46:15 -0600 (CST)
|
|
(envelope-from pgsql-hackers-owner+M16726@postgresql.org)
|
|
Received: from comet.pacifier.com ([199.2.117.155])
|
|
by postgresql.org (8.11.3/8.11.4) with ESMTP id fBIGYZm93404
|
|
for <pgsql-hackers@postgresql.org>; Tue, 18 Dec 2001 11:34:35 -0500 (EST)
|
|
(envelope-from dhogaza@pacifier.com)
|
|
Received: from pacifier.com (dsl-dhogaza.pacifier.net [207.202.226.68])
|
|
by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id fBIGXCX29823;
|
|
Tue, 18 Dec 2001 08:33:12 -0800 (PST)
|
|
Message-ID: <3C1F6FF1.9030606@pacifier.com>
|
|
Date: Tue, 18 Dec 2001 08:33:53 -0800
|
|
From: Don Baccus <dhogaza@pacifier.com>
|
|
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.6) Gecko/20011120
|
|
X-Accept-Language: en-us
|
|
MIME-Version: 1.0
|
|
To: Mark Pritchard <mark@tangent.net.au>
|
|
cc: pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
References: <EGECIAPHKLJFDEJBGGOBGEIJFNAA.mark@tangent.net.au>
|
|
Content-Type: text/plain; charset=us-ascii; format=flowed
|
|
Content-Transfer-Encoding: 7bit
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Mark Pritchard wrote:
|
|
|
|
>>I think it is the startup cost that most people want to avoid, and our's
|
|
>>is higher than most db's that use threads; at least I think so.
|
|
>>
|
|
>>It would just be nice to have it done internally rather than have all
|
|
>>the clients do it, iff it can be done cleanly.
|
|
>>
|
|
>
|
|
> I'd add that client side connection pooling isn't effective in some cases
|
|
> anyway - one application we work with has 4 physical application servers
|
|
> running around 6 applications. Each of the applications was written by a
|
|
> different vendor, and thus a pool size of five gives you 120 open
|
|
> connections.
|
|
|
|
Tuning a central pooling mechanism to run well in this kind of situation
|
|
isn't going to be a trivial task, either. The next thing you'll want is
|
|
some way to prioritize the various clients so your more serious
|
|
applications have a better chance of getting a pool.
|
|
|
|
Or you'll want to set up subpools so they don't compete with each other,
|
|
in effect replicating what's done now, but adding more complexity to the
|
|
central service.
|
|
|
|
--
|
|
Don Baccus
|
|
Portland, OR
|
|
http://donb.photo.net, http://birdnotes.net, http://openacs.org
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 6: Have you searched our list archives?
|
|
|
|
http://archives.postgresql.org
|
|
|
|
From oleg@sai.msu.su Tue Dec 18 12:05:51 2001
|
|
Return-path: <oleg@sai.msu.su>
|
|
Received: from ra.sai.msu.su (ra.sai.msu.su [158.250.29.2])
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBIH5h423591
|
|
for <pgman@candle.pha.pa.us>; Tue, 18 Dec 2001 12:05:43 -0500 (EST)
|
|
Received: from ra (ra [158.250.29.2])
|
|
by ra.sai.msu.su (8.9.3/8.9.3) with ESMTP id UAA18592;
|
|
Tue, 18 Dec 2001 20:05:26 +0300 (GMT)
|
|
Date: Tue, 18 Dec 2001 20:05:26 +0300 (GMT)
|
|
From: Oleg Bartunov <oleg@sai.msu.su>
|
|
X-X-Sender: <megera@ra.sai.msu.su>
|
|
To: Don Baccus <dhogaza@pacifier.com>
|
|
cc: Bruce Momjian <pgman@candle.pha.pa.us>, mlw <markw@mohawksoft.com>,
|
|
<owensmk@earthlink.net>, <pgsql-hackers@postgresql.org>,
|
|
Tom Lane <tgl@sss.pgh.pa.us>
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
In-Reply-To: <3C1F6DBF.2040000@pacifier.com>
|
|
Message-ID: <Pine.GSO.4.33.0112182000400.12230-100000@ra.sai.msu.su>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Status: OR
|
|
|
|
Does schema support will resolve this discussion ?
|
|
If I understand correctly, initial arguments for connection pooling
|
|
was restriction in number of persistent connections. it's right in
|
|
current postgresql that if one wants keep connection for performance
|
|
reason to several databases the total number of connections will
|
|
doubled, trippled and so on. But if I understand schema support will
|
|
eventually put away these problem because we could keep only one
|
|
pool of connections to the *one* database.
|
|
|
|
Oleg
|
|
|
|
On Tue, 18 Dec 2001, Don Baccus wrote:
|
|
|
|
> Bruce Momjian wrote:
|
|
>
|
|
>
|
|
> > Yes, that is assuming you are using PHP. If you are using something
|
|
> > else, you connection pooling in there too. All those client interfaces
|
|
> > reimplementing connection pooling seems like a waste to me.
|
|
>
|
|
>
|
|
> Effective pooling's pretty specific to your environment, though, so any
|
|
> general mechanism would have to provide a wide-ranging suite of
|
|
> parameters governing the number to pool, how long each handle should
|
|
> live, what to do if a handle's released by a client while in the midst
|
|
> of a transaction (AOLserver rolls back the transaction, other clients
|
|
> might want to do something else, i.e. fire a callback or the like), etc etc.
|
|
>
|
|
> I think it would be fairly complex and for those high-throughput
|
|
> applications already written with client-side pooling no improvement.
|
|
>
|
|
> And those are the only applications that need it.
|
|
>
|
|
>
|
|
|
|
Regards,
|
|
Oleg
|
|
_____________________________________________________________
|
|
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
|
|
Sternberg Astronomical Institute, Moscow University (Russia)
|
|
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
|
|
phone: +007(095)939-16-83, +007(095)939-23-83
|
|
|
|
|
|
From pgsql-hackers-owner+M16748@postgresql.org Tue Dec 18 15:11:46 2001
|
|
Return-path: <pgsql-hackers-owner+M16748@postgresql.org>
|
|
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBIKBj405415
|
|
for <pgman@candle.pha.pa.us>; Tue, 18 Dec 2001 15:11:45 -0500 (EST)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBIKB0N47430;
|
|
Tue, 18 Dec 2001 14:11:00 -0600 (CST)
|
|
(envelope-from pgsql-hackers-owner+M16748@postgresql.org)
|
|
Received: from candle.pha.pa.us (216-55-132-35.dialup.tnt01.san-diego.abac.net [216.55.132.35])
|
|
by postgresql.org (8.11.3/8.11.4) with ESMTP id fBIJulm05030
|
|
for <pgsql-hackers@postgresql.org>; Tue, 18 Dec 2001 14:56:47 -0500 (EST)
|
|
(envelope-from pgman@candle.pha.pa.us)
|
|
Received: (from pgman@localhost)
|
|
by candle.pha.pa.us (8.11.6/8.10.1) id fBIJuVB04553;
|
|
Tue, 18 Dec 2001 14:56:31 -0500 (EST)
|
|
From: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
Message-ID: <200112181956.fBIJuVB04553@candle.pha.pa.us>
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
In-Reply-To: <3C1F6ED6.6080107@pacifier.com> "from Don Baccus at Dec 18, 2001
|
|
08:29:10 am"
|
|
To: Don Baccus <dhogaza@pacifier.com>
|
|
Date: Tue, 18 Dec 2001 14:56:31 -0500 (EST)
|
|
cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>,
|
|
mlw <markw@mohawksoft.com>, owensmk@earthlink.net,
|
|
pgsql-hackers@postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>
|
|
X-Mailer: ELM [version 2.4ME+ PL90 (25)]
|
|
MIME-Version: 1.0
|
|
Content-Transfer-Encoding: 7bit
|
|
Content-Type: text/plain; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
> Bruce Momjian wrote:
|
|
>
|
|
>
|
|
> >
|
|
> > The trick for that is to call COMMIT before you pass the backend to a
|
|
> > new person.
|
|
>
|
|
>
|
|
> The failure to COMMIT is a programmer error - ROLLBACK's much safer. At
|
|
> least that's what we decided in the AOLserver community, and that's
|
|
> what the drivers for Oracle and PG (the two I maintain) implement.
|
|
|
|
|
|
Then you can issue a "BEGIN;ROLLBACK;" when you pass the session to the
|
|
next user, and "RESET ALL;" of course.
|
|
|
|
> > Now, if you want to abort a left-over transaction, you can
|
|
> > do an ABORT but that is going to show up in the server logs because an
|
|
> > ABORT without a transaction causes an error message.
|
|
>
|
|
>
|
|
> The connection pooling mechanism needs to track the transaction state
|
|
> and only ROLLBACK a handle that's not in autocommit state or in the
|
|
> midst of a BEGIN/END transaction (again, Oracle vs. PG)..
|
|
|
|
Seems like a lot of work to keep track of transaction state in the
|
|
client; seems easier to just unconditionally issue the begin;rollback.
|
|
|
|
--
|
|
Bruce Momjian | http://candle.pha.pa.us
|
|
pgman@candle.pha.pa.us | (610) 853-3000
|
|
+ If your life is a hard drive, | 830 Blythe Avenue
|
|
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
|
|
|
|
From pgsql-hackers-owner+M16793@postgresql.org Wed Dec 19 00:46:50 2001
|
|
Return-path: <pgsql-hackers-owner+M16793@postgresql.org>
|
|
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBJ5kn426988
|
|
for <pgman@candle.pha.pa.us>; Wed, 19 Dec 2001 00:46:49 -0500 (EST)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBJ5gnN63439;
|
|
Tue, 18 Dec 2001 23:42:49 -0600 (CST)
|
|
(envelope-from pgsql-hackers-owner+M16793@postgresql.org)
|
|
Received: from deborah.paradise.net.nz (deborah.paradise.net.nz [203.96.152.32])
|
|
by postgresql.org (8.11.3/8.11.4) with ESMTP id fBJ5Uvm41224
|
|
for <pgsql-hackers@postgresql.org>; Wed, 19 Dec 2001 00:30:58 -0500 (EST)
|
|
(envelope-from andrew@catalyst.net.nz)
|
|
Received: from heidegger.catalyst.net.nz (203-96-145-108.adsl.paradise.net.nz [203.96.145.108])
|
|
by deborah.paradise.net.nz (Postfix) with ESMTP
|
|
id D1C7CD194D; Wed, 19 Dec 2001 18:31:01 +1300 (NZDT)
|
|
Received: from 127.0.0.1 (ident=unknown) by heidegger.catalyst.net.nz
|
|
with esmtp (MasqMail 0.1.15) id 16GZJK-5NU-00; Wed, 19 Dec 2001
|
|
18:30:34 +1300
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
From: Andrew McMillan <andrew@catalyst.net.nz>
|
|
To: owensmk@earthlink.net
|
|
cc: pgsql-hackers@postgresql.org
|
|
In-Reply-To: <200112180028.fBI0Sum06915@postgresql.org>
|
|
References: <200112180028.fBI0Sum06915@postgresql.org>
|
|
Content-Type: text/plain
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Mailer: Evolution/1.0 (Preview Release)
|
|
Date: 19 Dec 2001 18:30:34 +1300
|
|
Message-ID: <1008739834.25608.33.camel@kant.mcmillan.net.nz>
|
|
MIME-Version: 1.0
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
On Tue, 2001-12-18 at 13:46, Michael Owens wrote:
|
|
>
|
|
> By having the postmaster map multiple clients to a fixed number of backends,
|
|
> you achieve the happy medium: You never exceed the ideal number of active
|
|
> backends, and at the same time you are not limited to only accepting a fixed
|
|
> number of connections. Accepting connections can now be based on load
|
|
> (however you wish to define it), not number. You now make decisions based on
|
|
> utlization.
|
|
>
|
|
> If it were shown that even half of a backend's life consisted of idle time,
|
|
> leasing out that idle time to another active connection would potentially
|
|
> double the average number of simultaneous requests without (theoretically)
|
|
> incurring any significant degradation in performance.
|
|
>
|
|
|
|
Have you looked at the client-side connection pooling solutions out
|
|
there?
|
|
|
|
DBBalancer ( http://dbbalancer.sourceforge.net/ ) tries to sit very
|
|
transparently between your application and PostgreSQL, letting you
|
|
implement connection pooling with almost no application changes.
|
|
|
|
There was another one I came across too, but that one requires you to
|
|
make more wide-reaching changes to the application.
|
|
|
|
In my applications I have found DBBalancer to be roughly the same level
|
|
of performance as PHP persistent connections, but a lot fewer
|
|
connections are needed in the pool because they are only needed when
|
|
Apache is delivering dynamic content - not the associated static
|
|
stylesheets and images.
|
|
|
|
Regards,
|
|
Andrew.
|
|
--
|
|
--------------------------------------------------------------------
|
|
Andrew @ Catalyst .Net.NZ Ltd, PO Box 11-053, Manners St, Wellington
|
|
WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St
|
|
DDI: +64(4)916-7201 MOB: +64(21)635-694 OFFICE: +64(4)499-2267
|
|
Are you enrolled at http://schoolreunions.co.nz/ yet?
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 6: Have you searched our list archives?
|
|
|
|
http://archives.postgresql.org
|
|
|
|
From pgsql-hackers-owner+M16834@postgresql.org Wed Dec 19 14:17:47 2001
|
|
Return-path: <pgsql-hackers-owner+M16834@postgresql.org>
|
|
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBJJHk404096
|
|
for <pgman@candle.pha.pa.us>; Wed, 19 Dec 2001 14:17:46 -0500 (EST)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBJJENN87550;
|
|
Wed, 19 Dec 2001 13:14:23 -0600 (CST)
|
|
(envelope-from pgsql-hackers-owner+M16834@postgresql.org)
|
|
Received: from asteroid.pacifier.com ([199.2.117.154])
|
|
by postgresql.org (8.11.3/8.11.4) with ESMTP id fBJJ55m16181
|
|
for <pgsql-hackers@postgresql.org>; Wed, 19 Dec 2001 14:05:05 -0500 (EST)
|
|
(envelope-from dhogaza@pacifier.com)
|
|
Received: from pacifier.com (dsl-dhogaza.pacifier.net [207.202.226.68])
|
|
by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id fBJJ3fe20585;
|
|
Wed, 19 Dec 2001 11:03:41 -0800 (PST)
|
|
Message-ID: <3C20E4B9.8090200@pacifier.com>
|
|
Date: Wed, 19 Dec 2001 11:04:25 -0800
|
|
From: Don Baccus <dhogaza@pacifier.com>
|
|
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.6) Gecko/20011120
|
|
X-Accept-Language: en-us
|
|
MIME-Version: 1.0
|
|
To: owensmk@earthlink.net
|
|
cc: Andrew McMillan <andrew@catalyst.net.nz>, pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
References: <200112180028.fBI0Sum06915@postgresql.org> <1008739834.25608.33.camel@kant.mcmillan.net.nz> <E16Gl55-0005ug-00@swan.prod.itd.earthlink.net>
|
|
Content-Type: text/plain; charset=us-ascii; format=flowed
|
|
Content-Transfer-Encoding: 7bit
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
Michael Owens wrote:
|
|
|
|
> As long as each client's call is composed of a standalone transaction, there
|
|
> is no problem with external connection pools. But what about when a client's
|
|
> transactions spans two or more calls, such as SELECT FOR UPDATE? Then pooling
|
|
> is not safe: it offers no assurance of what may be interjected into an open
|
|
> transaction between calls. For example, each is a separate call to a shared
|
|
> connection:
|
|
>
|
|
> Client A: BEGIN WORK; SELECT last_name from customer for update where <X>;
|
|
>
|
|
> Client B: BEGIN WORK; SELECT street from customer for update where <Y>;
|
|
>
|
|
> Client A: update customer set lastname=<modified value> where <X>; COMMIT
|
|
> WORK;
|
|
>
|
|
>
|
|
> Now, isn't Client B's write lock gone with Client A's commit? Yet Client A's
|
|
> lock is still hanging around. While Client B's commit will close it, Client B
|
|
> has lost the assurance of its lock, defeating the purpose of SELECT FOR
|
|
> UPDATE.
|
|
>
|
|
> If this is corrent, then external connection pools limit what you can do with
|
|
> the database to a single call. Any transaction spanning more than one call is
|
|
> unsafe, because it is not isolated from other clients sharing the same
|
|
> connection.
|
|
|
|
|
|
The general idea is that you grab a handle and hold onto it until you're
|
|
done. This makes the above scenario impossible.
|
|
|
|
Forgetting to commit or rollback before relenquishing the handle is
|
|
another scenario that can lead to problems but that's already been
|
|
discussed in detail.
|
|
|
|
--
|
|
Don Baccus
|
|
Portland, OR
|
|
http://donb.photo.net, http://birdnotes.net, http://openacs.org
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 2: you can get off all lists at once with the unregister command
|
|
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
|
|
|
|
From pgsql-hackers-owner+M16838@postgresql.org Wed Dec 19 15:17:32 2001
|
|
Return-path: <pgsql-hackers-owner+M16838@postgresql.org>
|
|
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBJKHV408663
|
|
for <pgman@candle.pha.pa.us>; Wed, 19 Dec 2001 15:17:32 -0500 (EST)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBJKDNN89347;
|
|
Wed, 19 Dec 2001 14:13:23 -0600 (CST)
|
|
(envelope-from pgsql-hackers-owner+M16838@postgresql.org)
|
|
Received: from gull.prod.itd.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84])
|
|
by postgresql.org (8.11.3/8.11.4) with ESMTP id fBJKA2m62023
|
|
for <pgsql-hackers@postgresql.org>; Wed, 19 Dec 2001 15:10:03 -0500 (EST)
|
|
(envelope-from owensmk@earthlink.net)
|
|
Received: from sdn-ar-004txfworp179.dialsprint.net ([158.252.142.219] helo=there)
|
|
by gull.prod.itd.earthlink.net with smtp (Exim 3.33 #1)
|
|
id 16Gn2K-0005YP-00; Wed, 19 Dec 2001 12:09:57 -0800
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
From: Michael Owens <owensmk@earthlink.net>
|
|
Reply-To: owensmk@earthlink.net
|
|
To: Don Baccus <dhogaza@pacifier.com>
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
Date: Wed, 19 Dec 2001 14:28:14 -0600
|
|
X-Mailer: KMail [version 1.3.1]
|
|
cc: Andrew McMillan <andrew@catalyst.net.nz>, pgsql-hackers@postgresql.org
|
|
References: <200112180028.fBI0Sum06915@postgresql.org> <E16Gl55-0005ug-00@swan.prod.itd.earthlink.net> <3C20E4B9.8090200@pacifier.com>
|
|
In-Reply-To: <3C20E4B9.8090200@pacifier.com>
|
|
MIME-Version: 1.0
|
|
Content-Transfer-Encoding: 8bit
|
|
Message-ID: <E16Gn2K-0005YP-00@gull.prod.itd.earthlink.net>
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
On Wednesday 19 December 2001 01:04 pm, Don Baccus wrote:
|
|
|
|
|
|
> The general idea is that you grab a handle and hold onto it until you're
|
|
> done. This makes the above scenario impossible.
|
|
>
|
|
> Forgetting to commit or rollback before relenquishing the handle is
|
|
> another scenario that can lead to problems but that's already been
|
|
> discussed in detail.
|
|
|
|
But then the shared connection is unshared, sitting idle while the client
|
|
works in between calls, thus introducing idle time among a fixed number of
|
|
connections. The server is doing less than it could.
|
|
|
|
I agree that this connection pool has improved things in eliminating backend
|
|
startup time. But idle time still exists for the clients performing multiple
|
|
calls, proportional to the product of the number of multiple call clients and
|
|
the number of calls they make, plus the idle time between them.
|
|
|
|
However this probably only ever happens on update. Inserts and selects can be
|
|
done in one call. And, I suppose updates comprise only a small fraction of
|
|
the requests sent to the database. Even then, you can probably eliminate some
|
|
multiple calls by using things such as procedures.
|
|
|
|
Factoring all that in, you can probably do as well by optimizing your
|
|
particular database/application than by writing code.
|
|
|
|
I relent. Thanks for your thoughts.
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 3: if posting/reading through Usenet, please send an appropriate
|
|
subscribe-nomail command to majordomo@postgresql.org so that your
|
|
message can get through to the mailing list cleanly
|
|
|
|
From pgsql-hackers-owner+M16855@postgresql.org Thu Dec 20 01:02:51 2001
|
|
Return-path: <pgsql-hackers-owner+M16855@postgresql.org>
|
|
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
|
|
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBK62o404294
|
|
for <pgman@candle.pha.pa.us>; Thu, 20 Dec 2001 01:02:50 -0500 (EST)
|
|
Received: from postgresql.org (postgresql.org [64.49.215.8])
|
|
by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBK5xnN05417;
|
|
Wed, 19 Dec 2001 23:59:49 -0600 (CST)
|
|
(envelope-from pgsql-hackers-owner+M16855@postgresql.org)
|
|
Received: from deborah.paradise.net.nz (deborah.paradise.net.nz [203.96.152.32])
|
|
by postgresql.org (8.11.3/8.11.4) with ESMTP id fBK5mvm47263
|
|
for <pgsql-hackers@postgresql.org>; Thu, 20 Dec 2001 00:48:57 -0500 (EST)
|
|
(envelope-from andrew@catalyst.net.nz)
|
|
Received: from heidegger.catalyst.net.nz (203-96-145-94.adsl.paradise.net.nz [203.96.145.94])
|
|
by deborah.paradise.net.nz (Postfix) with ESMTP
|
|
id 7407FD2B76; Thu, 20 Dec 2001 18:49:01 +1300 (NZDT)
|
|
Received: from 127.0.0.1 (ident=unknown) by heidegger.catalyst.net.nz
|
|
with esmtp (MasqMail 0.1.15) id 16GrRk-2Ry-00; Thu, 20 Dec 2001
|
|
13:52:28 +1300
|
|
Subject: Re: [HACKERS] Connection Pooling, a year later
|
|
From: Andrew McMillan <andrew@catalyst.net.nz>
|
|
To: owensmk@earthlink.net
|
|
cc: pgsql-hackers@postgresql.org
|
|
In-Reply-To: <E16Gl55-0005ug-00@swan.prod.itd.earthlink.net>
|
|
References: <200112180028.fBI0Sum06915@postgresql.org>
|
|
<1008739834.25608.33.camel@kant.mcmillan.net.nz>
|
|
<E16Gl55-0005ug-00@swan.prod.itd.earthlink.net>
|
|
Content-Type: text/plain
|
|
Content-Transfer-Encoding: 7bit
|
|
X-Mailer: Evolution/1.0 (Preview Release)
|
|
Date: 20 Dec 2001 13:52:28 +1300
|
|
Message-ID: <1008809548.24470.48.camel@kant.mcmillan.net.nz>
|
|
MIME-Version: 1.0
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
On Thu, 2001-12-20 at 07:22, Michael Owens wrote:
|
|
> As long as each client's call is composed of a standalone transaction, there
|
|
> is no problem with external connection pools. But what about when a client's
|
|
> transactions spans two or more calls, such as SELECT FOR UPDATE? Then pooling
|
|
> is not safe: it offers no assurance of what may be interjected into an open
|
|
> transaction between calls. For example, each is a separate call to a shared
|
|
> connection:
|
|
>
|
|
> Client A: BEGIN WORK; SELECT last_name from customer for update where <X>;
|
|
>
|
|
> Client B: BEGIN WORK; SELECT street from customer for update where <Y>;
|
|
>
|
|
> Client A: update customer set lastname=<modified value> where <X>; COMMIT
|
|
> WORK;
|
|
>
|
|
>
|
|
> Now, isn't Client B's write lock gone with Client A's commit? Yet Client A's
|
|
> lock is still hanging around. While Client B's commit will close it, Client B
|
|
> has lost the assurance of its lock, defeating the purpose of SELECT FOR
|
|
> UPDATE.
|
|
>
|
|
> If this is corrent, then external connection pools limit what you can do with
|
|
> the database to a single call. Any transaction spanning more than one call is
|
|
> unsafe, because it is not isolated from other clients sharing the same
|
|
> connection.
|
|
|
|
Oh, I see. You are absolutely correct that client-side pooling wouldn't
|
|
work in that situation of course.
|
|
|
|
As an application developer nobody has forced me into such a corner yet,
|
|
however. Long running transactions are something I avoid like the
|
|
plague.
|
|
|
|
Cheers,
|
|
Andrew.
|
|
--
|
|
--------------------------------------------------------------------
|
|
Andrew @ Catalyst .Net.NZ Ltd, PO Box 11-053, Manners St, Wellington
|
|
WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St
|
|
DDI: +64(4)916-7201 MOB: +64(21)635-694 OFFICE: +64(4)499-2267
|
|
Are you enrolled at http://schoolreunions.co.nz/ yet?
|
|
|
|
|
|
---------------------------(end of broadcast)---------------------------
|
|
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
|
|
|