mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-08-18 14:03:26 +02:00
581 lines
22 KiB
Plaintext
581 lines
22 KiB
Plaintext
From mscott@sacadia.com Wed Nov 15 14:50:19 2000
|
|
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id OAA11583
|
|
for <pgman@candle.pha.pa.us>; Wed, 15 Nov 2000 14:50:13 -0500 (EST)
|
|
Received: from localhost (localhost [127.0.0.1])
|
|
by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id LAA09998;
|
|
Wed, 15 Nov 2000 11:35:33 -0800 (PST)
|
|
Date: Wed, 15 Nov 2000 11:35:33 -0800 (PST)
|
|
From: Myron Scott <mscott@sacadia.com>
|
|
X-Sender: mscott@goldengate.kojoworldwide.com.
|
|
To: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>,
|
|
Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
|
|
Subject: Please help with some advice
|
|
Message-ID: <Pine.GSO.4.10.10011151053260.9940-100000@goldengate.kojoworldwide.com.>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Status: ORr
|
|
|
|
Dear Sirs,
|
|
|
|
I have been lurking on the PostgreSQL hackers list for about 3 months now
|
|
and your names comes up more than any with helpful info about the project
|
|
so I was hoping you could help me.
|
|
|
|
Let me cut to the chase. I have been experimenting with 7.0.2 source to
|
|
see if I could create a mutlti-threaded version of the backend so
|
|
I could link directly from java ( I have a fe<->be protocol that I use for
|
|
my apps). Needless to say I got into much more than I bargained for. I
|
|
now have a version that works and it has some nice benefits that are very
|
|
helpful to a project that I am working on. What I gained was
|
|
|
|
prepared statements outside of spi
|
|
batched commits (fsync)
|
|
one connection per thread
|
|
multiple threads per process
|
|
multiple processes per installation
|
|
|
|
I never really intended for anyone else to see the work so I drifted
|
|
pretty far from the original code. I also ended up using Solaris threads
|
|
rather than pthreads, I did my own implementation of the bufmgr.c and
|
|
gram.y, and used Solaris implementation of mutex in place of S_LOCK and
|
|
TAS. I grabbed all global variables and put them in an environment
|
|
variable that is thread local. I also did some really stupid
|
|
things like making TransactionId uint64 and making all my inserts use the
|
|
same oid.
|
|
|
|
My question is this. I would like to get some critical feedback and
|
|
suggestions about the work from others. What is the best way to go about
|
|
this? I thought about trying to create a project on greatbridge.org
|
|
but I am rather new to open source and the code needs commented properly
|
|
and cleaned up before too many try and look at it.
|
|
|
|
Any suggestions would be greatly appreciated.
|
|
|
|
|
|
Thanks in advance,
|
|
|
|
Myron Scott
|
|
|
|
|
|
|
|
From mscott@sacadia.com Thu Nov 16 17:19:45 2000
|
|
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA04315
|
|
for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 17:19:43 -0500 (EST)
|
|
Received: from localhost (localhost [127.0.0.1])
|
|
by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id OAA11449;
|
|
Thu, 16 Nov 2000 14:05:15 -0800 (PST)
|
|
Date: Thu, 16 Nov 2000 14:05:15 -0800 (PST)
|
|
From: Myron Scott <mscott@sacadia.com>
|
|
X-Sender: mscott@goldengate.kojoworldwide.com.
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>, Tom Lane <tgl@sss.pgh.pa.us>
|
|
Subject: Re: Please help with some advice
|
|
In-Reply-To: <200011160533.AAA27886@candle.pha.pa.us>
|
|
Message-ID: <Pine.GSO.4.10.10011161401570.11441-100000@goldengate.kojoworldwide.com.>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Status: OR
|
|
|
|
Bruce Momjian wrote:
|
|
|
|
>I am curious how you isolated each thread. It seems we pretty much
|
|
>assume all our memory is controlled by a single query in the process.
|
|
|
|
|
|
I moved all global variables to a thread global variable which is accessed
|
|
by the method GetEnv(). Which looks like this
|
|
|
|
Env* GetEnv(void) {
|
|
Env* env;
|
|
thr_getspecific(*envkey,(void*)&env);
|
|
return env;
|
|
}
|
|
|
|
The Env struct includes the CurrentMemoryContext, TopMemoryContext,
|
|
PortalHeapMemory for each instance of a connection (one thread per
|
|
connection). So, for example,
|
|
EndPortalAllocMode uses GetEnv()->CurrentMemoryContext
|
|
|
|
void
|
|
EndPortalAllocMode()
|
|
{
|
|
PortalHeapMemory context;
|
|
|
|
AssertState(PortalManagerEnabled);
|
|
AssertState(IsA(GetEnv()->CurrentMemoryContext,
|
|
PortalHeapMemory));
|
|
|
|
context = (PortalHeapMemory) GetEnv()->CurrentMemoryContext;
|
|
AssertState(PointerIsValid(context->block)); /* XXX
|
|
Trap(...) */
|
|
|
|
/* free current mode */
|
|
AllocSetReset(&HEAPMEMBLOCK(context)->setData);
|
|
MemoryContextFree((MemoryContext)
|
|
PortalHeapMemoryGetVariableMemory(context),
|
|
context->block);
|
|
|
|
/* restore previous mode */
|
|
context->block = FixedStackPop(&context->stackData);
|
|
}
|
|
|
|
|
|
|
|
|
|
From vmikheev@SECTORBASE.COM Thu Nov 16 17:23:22 2000
|
|
Received: from sectorbase2.sectorbase.com ([208.48.122.131])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with SMTP id RAA04562
|
|
for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 17:23:21 -0500 (EST)
|
|
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2650.21)
|
|
id <V8XQB5RW>; Thu, 16 Nov 2000 14:05:24 -0800
|
|
Message-ID: <8F4C99C66D04D4118F580090272A7A234D318D@sectorbase1.sectorbase.com>
|
|
From: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
|
|
To: "'Myron Scott'" <mscott@sacadia.com>,
|
|
Bruce Momjian
|
|
<pgman@candle.pha.pa.us>
|
|
Cc: Tom Lane <tgl@sss.pgh.pa.us>
|
|
Subject: RE: Please help with some advice
|
|
Date: Thu, 16 Nov 2000 14:09:30 -0800
|
|
MIME-Version: 1.0
|
|
X-Mailer: Internet Mail Service (5.5.2650.21)
|
|
Content-Type: text/plain;
|
|
charset="iso-8859-1"
|
|
Status: ORr
|
|
|
|
I think the question do we want to make backend multy-threaded
|
|
should be discussed in hackers.
|
|
|
|
Vadim
|
|
|
|
> -----Original Message-----
|
|
> From: Myron Scott [mailto:mscott@sacadia.com]
|
|
> Sent: Thursday, November 16, 2000 2:05 PM
|
|
> To: Bruce Momjian
|
|
> Cc: Mikheev, Vadim; Tom Lane
|
|
> Subject: Re: Please help with some advice
|
|
>
|
|
>
|
|
> Bruce Momjian wrote:
|
|
>
|
|
> >I am curious how you isolated each thread. It seems we pretty much
|
|
> >assume all our memory is controlled by a single query in the process.
|
|
>
|
|
>
|
|
>
|
|
> I moved all global variables to a thread global variable
|
|
> which is accessed
|
|
> by the method GetEnv(). Which looks like this
|
|
>
|
|
> Env* GetEnv(void) {
|
|
> Env* env;
|
|
> thr_getspecific(*envkey,(void*)&env);
|
|
> return env;
|
|
> }
|
|
>
|
|
> The Env struct includes the CurrentMemoryContext, TopMemoryContext,
|
|
> PortalHeapMemory for each instance of a connection (one thread per
|
|
> connection). So, for example,
|
|
> EndPortalAllocMode uses GetEnv()->CurrentMemoryContext
|
|
>
|
|
> void
|
|
> EndPortalAllocMode()
|
|
> {
|
|
> PortalHeapMemory context;
|
|
>
|
|
> AssertState(PortalManagerEnabled);
|
|
> AssertState(IsA(GetEnv()->CurrentMemoryContext,
|
|
> PortalHeapMemory));
|
|
>
|
|
> context = (PortalHeapMemory) GetEnv()->CurrentMemoryContext;
|
|
> AssertState(PointerIsValid(context->block)); /* XXX
|
|
> Trap(...) */
|
|
>
|
|
> /* free current mode */
|
|
> AllocSetReset(&HEAPMEMBLOCK(context)->setData);
|
|
> MemoryContextFree((MemoryContext)
|
|
> PortalHeapMemoryGetVariableMemory(context),
|
|
> context->block);
|
|
>
|
|
> /* restore previous mode */
|
|
> context->block = FixedStackPop(&context->stackData);
|
|
> }
|
|
>
|
|
>
|
|
>
|
|
|
|
From mscott@sacadia.com Thu Nov 16 22:16:38 2000
|
|
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id WAA14638
|
|
for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 22:16:36 -0500 (EST)
|
|
Received: from localhost (localhost [127.0.0.1])
|
|
by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id TAA11874;
|
|
Thu, 16 Nov 2000 19:04:48 -0800 (PST)
|
|
Date: Thu, 16 Nov 2000 19:04:48 -0800 (PST)
|
|
From: Myron Scott <mscott@sacadia.com>
|
|
X-Sender: mscott@goldengate.kojoworldwide.com.
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
cc: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>, Tom Lane <tgl@sss.pgh.pa.us>
|
|
Subject: Re: Please help with some advice
|
|
In-Reply-To: <200011170156.UAA11438@candle.pha.pa.us>
|
|
Message-ID: <Pine.GSO.4.10.10011161904140.11870-100000@goldengate.kojoworldwide.com.>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Status: ORr
|
|
|
|
Thanks very much, I will post to hackers.
|
|
|
|
Myron
|
|
|
|
|
|
|
|
From pgsql-hackers-owner+M2691@postgresql.org Tue Jan 2 00:30:20 2001
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA08195
|
|
for <pgman@candle.pha.pa.us>; Tue, 2 Jan 2001 00:30:19 -0500 (EST)
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f025UjL33335;
|
|
Tue, 2 Jan 2001 00:30:45 -0500 (EST)
|
|
(envelope-from pgsql-hackers-owner+M2691@postgresql.org)
|
|
Received: from mailsys01.intnet.net (tmail.wwc.com [198.252.32.143] (may be forged))
|
|
by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f025UTL33232
|
|
for <pgsql-hackers@postgresql.org>; Tue, 2 Jan 2001 00:30:32 -0500 (EST)
|
|
(envelope-from mscott@sacadia.com)
|
|
Received: from [206.112.108.0] (HELO sacadia.com)
|
|
by mailsys01.intnet.net (CommuniGate Pro SMTP 3.3.2)
|
|
with ESMTP id 2214231; Tue, 02 Jan 2001 00:29:47 -0500
|
|
Message-ID: <3A5167DB.3050807@sacadia.com>
|
|
Date: Mon, 01 Jan 2001 21:32:11 -0800
|
|
From: Myron Scott <mscott@sacadia.com>
|
|
Reply-To: mscott@sacadia.com
|
|
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; m18) Gecko/20001108 Netscape6/6.0
|
|
X-Accept-Language: en
|
|
MIME-Version: 1.0
|
|
To: "Ross J. Reedstrom" <reedstrm@rice.edu>
|
|
CC: pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Using Threads?
|
|
References: <004401c058fd$fd498d40$f2356880@tracy> <Pine.GSO.4.10.10012032351040.28161-100000@goldengate.kojoworldwide.com.> <20001204113307.B5871@rice.edu>
|
|
Content-Type: text/plain; charset=us-ascii; format=flowed
|
|
Content-Transfer-Encoding: 7bit
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
For anyone interested,
|
|
|
|
I have posted my multi-threaded version of PostgreSQL here.
|
|
|
|
http://www.sacadia.com/mtpg.html
|
|
|
|
It is based on 7.0.2 and the TAO CORBA ORB which is here.
|
|
|
|
http://www.cs.wustl.edu/~schmidt/TAO.html
|
|
|
|
Myron Scott
|
|
mkscott@sacadia.com
|
|
|
|
|
|
|
|
From bright@fw.wintelcom.net Tue Jan 2 03:02:28 2001
|
|
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA16169
|
|
for <pgman@candle.pha.pa.us>; Tue, 2 Jan 2001 03:02:27 -0500 (EST)
|
|
Received: (from bright@localhost)
|
|
by fw.wintelcom.net (8.10.0/8.10.0) id f0282Vm10623;
|
|
Tue, 2 Jan 2001 00:02:31 -0800 (PST)
|
|
Date: Tue, 2 Jan 2001 00:02:31 -0800
|
|
From: Alfred Perlstein <bright@wintelcom.net>
|
|
To: Bruce Momjian <pgman@candle.pha.pa.us>
|
|
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Assuming that TAS() will succeed the first time is verboten
|
|
Message-ID: <20010102000230.C19572@fw.wintelcom.net>
|
|
References: <9850.978067943@sss.pgh.pa.us> <200101020759.CAA15836@candle.pha.pa.us>
|
|
Mime-Version: 1.0
|
|
Content-Type: text/plain; charset=us-ascii
|
|
Content-Disposition: inline
|
|
User-Agent: Mutt/1.2.5i
|
|
In-Reply-To: <200101020759.CAA15836@candle.pha.pa.us>; from pgman@candle.pha.pa.us on Tue, Jan 02, 2001 at 02:59:20AM -0500
|
|
Status: OR
|
|
|
|
* Bruce Momjian <pgman@candle.pha.pa.us> [010101 23:59] wrote:
|
|
> > Alfred Perlstein <bright@wintelcom.net> writes:
|
|
> > > One trick that may help is calling sched_yield(2) on a lock miss,
|
|
> > > it's a POSIX call and quite new so you'd need a 'configure' test
|
|
> > > for it.
|
|
> >
|
|
> > The author of the current s_lock code seems to have thought that
|
|
> > select() with a zero delay would do the equivalent of sched_yield().
|
|
> > I'm not sure if that's true on very many kernels, if indeed any...
|
|
> >
|
|
> > I doubt we could buy much by depending on sched_yield(); if you want
|
|
> > to assume POSIX facilities, ISTM you might as well go for user-space
|
|
> > semaphores and forget the whole TAS mechanism.
|
|
>
|
|
>
|
|
> Another issue is that sched_yield brings in the pthreads library/hooks
|
|
> on some OS's, which we certainly want to avoid.
|
|
|
|
I know it's a major undertaking, but since the work is sort of done,
|
|
have you guys considered the port to solaris threads and seeing about
|
|
making a pthreads port of that?
|
|
|
|
I know it would probably get you considerable gains under Windows
|
|
at the expense of dropping some really really legacy system.
|
|
|
|
Or you could do what apache (is rumored) does and have it do either
|
|
threads or processes or both...
|
|
|
|
--
|
|
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
|
|
"I have the heart of a child; I keep it in a jar on my desk."
|
|
|
|
From pgsql-hackers-owner+M4275@postgresql.org Mon Feb 5 21:45:00 2001
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA09262
|
|
for <pgman@candle.pha.pa.us>; Mon, 5 Feb 2001 21:44:59 -0500 (EST)
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f162ixx00920;
|
|
Mon, 5 Feb 2001 21:44:59 -0500 (EST)
|
|
(envelope-from pgsql-hackers-owner+M4275@postgresql.org)
|
|
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f162fSx00595
|
|
for <pgsql-hackers@postgresql.org>; Mon, 5 Feb 2001 21:41:29 -0500 (EST)
|
|
(envelope-from mscott@sacadia.com)
|
|
Received: from localhost (localhost [127.0.0.1])
|
|
by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id SAA03298
|
|
for <pgsql-hackers@postgresql.org>; Mon, 5 Feb 2001 18:25:05 -0800 (PST)
|
|
Date: Mon, 5 Feb 2001 18:25:05 -0800 (PST)
|
|
From: Myron Scott <mscott@sacadia.com>
|
|
X-Sender: mscott@goldengate.kojoworldwide.com.
|
|
To: pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Using Threads?
|
|
Message-ID: <Pine.GSO.4.10.10102051823210.3289-100000@goldengate.kojoworldwide.com.>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
I have put a new version of my multi-threaded
|
|
postgresql experiment at
|
|
|
|
http://www.sacadia.com/mtpg.html
|
|
|
|
This one actually works. I have added a server
|
|
based on omniORB, a CORBA 2.3 ORB from ATT. It
|
|
is much smaller than TAO and uses the thread per
|
|
connection model. I haven't added the java side
|
|
of the JNI interface yet but the C++ side is there.
|
|
|
|
It's still not stable but it is much better than
|
|
the last.
|
|
|
|
Myron Scott
|
|
mkscott@sacadia.com
|
|
|
|
|
|
|
|
|
|
|
|
From pgsql-hackers-owner+M4304@postgresql.org Tue Feb 6 10:24:21 2001
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA22027
|
|
for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 10:24:20 -0500 (EST)
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16FOBx97182;
|
|
Tue, 6 Feb 2001 10:24:11 -0500 (EST)
|
|
(envelope-from pgsql-hackers-owner+M4304@postgresql.org)
|
|
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16FLWx96814
|
|
for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 10:21:33 -0500 (EST)
|
|
(envelope-from mscott@sacadia.com)
|
|
Received: from localhost (localhost [127.0.0.1])
|
|
by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id HAA04170;
|
|
Tue, 6 Feb 2001 07:05:04 -0800 (PST)
|
|
Date: Tue, 6 Feb 2001 07:05:04 -0800 (PST)
|
|
From: Myron Scott <mscott@sacadia.com>
|
|
X-Sender: mscott@goldengate.kojoworldwide.com.
|
|
To: Karel Zak <zakkr@zf.jcu.cz>
|
|
cc: pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Using Threads
|
|
In-Reply-To: <Pine.LNX.3.96.1010206101030.20355B-100000@ara.zf.jcu.cz>
|
|
Message-ID: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
|
|
>
|
|
> Sorry I haven't time to see and test your experiment,
|
|
> but I have a question. How you solve memory management?
|
|
> The current mmgr is based on global variable
|
|
> CurrentMemoryContext that is very often changed and used.
|
|
> Use you for this locks? If yes it is probably problematic
|
|
> point for perfomance.
|
|
>
|
|
> Karel
|
|
>
|
|
|
|
There are many many globals I had to work around including all the memory
|
|
management stuff. I basically threw everything into and "environment"
|
|
variable which I stored in a thread specific using thr_setspecific.
|
|
|
|
Performance is acually very good for what I am doing. I was able to batch
|
|
commit transactions which cuts down on fsync calls, use prepared
|
|
statements from my client using CORBA, and the various locking calls for
|
|
the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast. I did
|
|
some performance tests for inserts
|
|
|
|
20 clients, 900 inserts per client, 1 insert per transaction, 4 different
|
|
tables.
|
|
|
|
7.0.2 About 10:52 average completion
|
|
multi-threaded 2:42 average completion
|
|
7.1beta3 1:13 average completion
|
|
|
|
If I increased the number of inserts per transaction, multi-threaded got
|
|
closer to 7.1 for inserts. I haven't tested other other types of
|
|
commands
|
|
yet.
|
|
|
|
|
|
Myron Scott
|
|
mkscott@sacadia.com
|
|
|
|
|
|
From pgsql-hackers-owner+M4313@postgresql.org Tue Feb 6 12:32:00 2001
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id MAA29163
|
|
for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 12:31:59 -0500 (EST)
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16HVox17454;
|
|
Tue, 6 Feb 2001 12:31:51 -0500 (EST)
|
|
(envelope-from pgsql-hackers-owner+M4313@postgresql.org)
|
|
Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16HV6x17323
|
|
for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 12:31:06 -0500 (EST)
|
|
(envelope-from zakkr@zf.jcu.cz)
|
|
Received: from localhost (zakkr@localhost)
|
|
by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id SAA03980;
|
|
Tue, 6 Feb 2001 18:31:02 +0100
|
|
Date: Tue, 6 Feb 2001 18:31:02 +0100 (CET)
|
|
From: Karel Zak <zakkr@zf.jcu.cz>
|
|
To: Myron Scott <mscott@sacadia.com>
|
|
cc: pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Using Threads
|
|
In-Reply-To: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
|
|
Message-ID: <Pine.LNX.3.96.1010206182112.3799B-100000@ara.zf.jcu.cz>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
|
|
On Tue, 6 Feb 2001, Myron Scott wrote:
|
|
|
|
> There are many many globals I had to work around including all the memory
|
|
> management stuff. I basically threw everything into and "environment"
|
|
> variable which I stored in a thread specific using thr_setspecific.
|
|
|
|
Yes, it's good. I working on multi-thread application server
|
|
(http://mape.jcu.cz) and I use for this project some things from PG (like
|
|
mmgr), I planning use same solution.
|
|
|
|
> Performance is acually very good for what I am doing. I was able to batch
|
|
> commit transactions which cuts down on fsync calls, use prepared
|
|
> statements from my client using CORBA, and the various locking calls for
|
|
> the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast. I did
|
|
> some performance tests for inserts
|
|
>
|
|
> 20 clients, 900 inserts per client, 1 insert per transaction, 4 different
|
|
> tables.
|
|
>
|
|
> 7.0.2 About 10:52 average completion
|
|
> multi-threaded 2:42 average completion
|
|
> 7.1beta3 1:13 average completion
|
|
|
|
It is very very good for time for 7.1, already look forward to 7.2! :-)
|
|
|
|
BTW, I not sure if you anytime in future will see threads in
|
|
official PostgreSQL and if you spending time on relevant things (IMHO).
|
|
|
|
Karel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
From pgsql-hackers-owner+M4304@postgresql.org Tue Feb 6 10:24:21 2001
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA22027
|
|
for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 10:24:20 -0500 (EST)
|
|
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16FOBx97182;
|
|
Tue, 6 Feb 2001 10:24:11 -0500 (EST)
|
|
(envelope-from pgsql-hackers-owner+M4304@postgresql.org)
|
|
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
|
|
by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16FLWx96814
|
|
for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 10:21:33 -0500 (EST)
|
|
(envelope-from mscott@sacadia.com)
|
|
Received: from localhost (localhost [127.0.0.1])
|
|
by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id HAA04170;
|
|
Tue, 6 Feb 2001 07:05:04 -0800 (PST)
|
|
Date: Tue, 6 Feb 2001 07:05:04 -0800 (PST)
|
|
From: Myron Scott <mscott@sacadia.com>
|
|
X-Sender: mscott@goldengate.kojoworldwide.com.
|
|
To: Karel Zak <zakkr@zf.jcu.cz>
|
|
cc: pgsql-hackers@postgresql.org
|
|
Subject: Re: [HACKERS] Using Threads
|
|
In-Reply-To: <Pine.LNX.3.96.1010206101030.20355B-100000@ara.zf.jcu.cz>
|
|
Message-ID: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
|
|
MIME-Version: 1.0
|
|
Content-Type: TEXT/PLAIN; charset=US-ASCII
|
|
Precedence: bulk
|
|
Sender: pgsql-hackers-owner@postgresql.org
|
|
Status: OR
|
|
|
|
|
|
>
|
|
> Sorry I haven't time to see and test your experiment,
|
|
> but I have a question. How you solve memory management?
|
|
> The current mmgr is based on global variable
|
|
> CurrentMemoryContext that is very often changed and used.
|
|
> Use you for this locks? If yes it is probably problematic
|
|
> point for perfomance.
|
|
>
|
|
> Karel
|
|
>
|
|
|
|
There are many many globals I had to work around including all the memory
|
|
management stuff. I basically threw everything into and "environment"
|
|
variable which I stored in a thread specific using thr_setspecific.
|
|
|
|
Performance is acually very good for what I am doing. I was able to batch
|
|
commit transactions which cuts down on fsync calls, use prepared
|
|
statements from my client using CORBA, and the various locking calls for
|
|
the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast. I did
|
|
some performance tests for inserts
|
|
|
|
20 clients, 900 inserts per client, 1 insert per transaction, 4 different
|
|
tables.
|
|
|
|
7.0.2 About 10:52 average completion
|
|
multi-threaded 2:42 average completion
|
|
7.1beta3 1:13 average completion
|
|
|
|
If I increased the number of inserts per transaction, multi-threaded got
|
|
closer to 7.1 for inserts. I haven't tested other other types of
|
|
commands
|
|
yet.
|
|
|
|
|
|
Myron Scott
|
|
mkscott@sacadia.com
|
|
|
|
|