From 69cd5efb23cc7c2dd179cf84be9cfda8ffb6945f Mon Sep 17 00:00:00 2001 From: Bruce Momjian Date: Thu, 18 Apr 2002 04:13:00 +0000 Subject: [PATCH] Add to PREPARE archive. --- doc/TODO.detail/prepare | 1250 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 1250 insertions(+) diff --git a/doc/TODO.detail/prepare b/doc/TODO.detail/prepare index 0ab2e4f256..5d66512690 100644 --- a/doc/TODO.detail/prepare +++ b/doc/TODO.detail/prepare @@ -445,3 +445,1253 @@ query cache in shared memory...etc. Too much queries, but less answers :-) TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) +From pgsql-hackers-owner+M21218@postgresql.org Fri Apr 12 04:52:19 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3C8qIS25666 + for ; Fri, 12 Apr 2002 04:52:18 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id AE2FA4769F1; Fri, 12 Apr 2002 03:54:34 -0400 (EDT) +Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4]) + by postgresql.org (Postfix) with ESMTP id A05A94769DC + for ; Fri, 12 Apr 2002 03:51:27 -0400 (EDT) +Received: from ara.zf.jcu.cz (LOCALHOST [127.0.0.1]) + by ara.zf.jcu.cz (8.12.1/8.12.1/Debian -5) with ESMTP id g3C7pHBK012031; + Fri, 12 Apr 2002 09:51:17 +0200 +Received: (from zakkr@localhost) + by ara.zf.jcu.cz (8.12.1/8.12.1/Debian -5) id g3C7pGum012030; + Fri, 12 Apr 2002 09:51:16 +0200 +Date: Fri, 12 Apr 2002 09:51:16 +0200 +From: Karel Zak +To: pgsql-hackers@postgresql.org +cc: Hiroshi Inoue +Subject: Re: [HACKERS] 7.3 schedule +Message-ID: <20020412095116.B6370@zf.jcu.cz> +References: <3CB52C54.4020507@freaky-namuh.com> <20020411115434.201ff92f.nconway@klamath.dyndns.org> <3CB61DAB.5010601@freaky-namuh.com> <24184.1018581907@sss.pgh.pa.us> <3CB65B49.93F2F790@tpf.co.jp> <20020412004134.5d35a2dd.nconway@klamath.dyndns.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +User-Agent: Mutt/1.2.5i +In-Reply-To: <20020412004134.5d35a2dd.nconway@klamath.dyndns.org>; from nconway@klamath.dyndns.org on Fri, Apr 12, 2002 at 12:41:34AM -0400 +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +On Fri, Apr 12, 2002 at 12:41:34AM -0400, Neil Conway wrote: +> On Fri, 12 Apr 2002 12:58:01 +0900 +> "Hiroshi Inoue" wrote: +> > +> > Just a confirmation. +> > Someone is working on PREPARE/EXECUTE ? +> > What about Karel's work ? + + Right question :-) + +> I am. My work is based on Karel's stuff -- at the moment I'm still +> basically working on getting Karel's patch to play nicely with +> current sources; once that's done I'll be addressing whatever +> issues are stopping the code from getting into CVS. + + My patch (qcache) for PostgreSQL 7.0 is available at + ftp://ftp2.zf.jcu.cz/users/zakkr/pg/. + + I very look forward to Neil's work on this. + + Notes: + + * It's experimental patch, but usable. All features below mentioned + works. + + * PREPARE/EXECUTE is not only SQL statements, I think good idea is + create something common and robus for query-plan caching, + beacuse there is for example SPI too. The RI triggers are based + on SPI_saveplan(). + + * My patch knows EXECUTE INTO feature: + + PREPARE foo AS SELECT * FROM pg_class WHERE relname ~~ $1 USING text; + + EXECUTE foo USING 'pg%'; <-- standard select + + EXECUTE foo INTO TEMP newtab USING 'pg%'; <-- select into + + + * The patch allows store query-planns to shared memory and is + possible EXECUTE it at more backends (over same DB) and planns + are persistent across connetions. For this feature I create special + memory context subsystem (like current aset.c, but it works with + IPC shared memory). + + This is maybe too complex solution and (maybe) sufficient is cache + query in one backend only. I know unbelief about this shared + memory solution (Tom?). + + + Karel + + + My experimental patch README (excuse my English): + + Implementation + ~~~~~~~~~~~~~~ + + The qCache allows save queryTree and queryPlan. There is available are + two space for data caching. + + LOCAL - data are cached in backend non-shared memory and data aren't + available in other backends. + + SHARE - data are cached in backend shared memory and data are + visible in all backends. + + Because size of share memory pool is limited and it is set during + postmaster start up, the qCache must remove all old planns if pool is + full. You can mark each entry as "REMOVEABLE" or "NOTREMOVEABLE". + + A removeable entry is removed if pool is full. + + A not-removeable entry must be removed via qCache_Remove() or + the other routines. The qCache not remove this entry itself. + + All records in qCache are cached (in the hash table) under some key. + The qCache knows two alternate of key --- "KEY_STRING" and "KEY_BINARY". + + The qCache API not allows access to shared memory, all cached planns that + API returns are copy to CurrentMemoryContext. All (qCache_ ) routines lock + shmem itself (exception is qCache_RemoveOldest_ShareRemoveAble()). + + - for locking is used spin lock. + + Memory management + ~~~~~~~~~~~~~~~~~ + The qCache use for qCache's shared pool its memory context independent on + standard aset/mcxt, but use compatible API --- it allows to use standard + palloc() (it is very needful for basic plan-tree operations, an example + for copyObject()). The qCache memory management is very simular to current + aset.c code. It is chunk-ed blocks too, but the block is smaller - 1024b. + + The number of blocks is available set in postmaster 'argv' via option + '-Z'. + + For plan storing is used separate MemoryContext for each plan, it + is good idea (Hiroshi's ?), bucause create new context is simple and + inexpensive and allows easy destroy (free) cached plan. This method is + used in my SPI overhaul instead TopMemoryContext feeding. + + Postmaster + ~~~~~~~~~~ + The query cache memory is init during potmaster startup. The size of + query cache pool is set via '-Z ' switch --- default + is 100 blocks where 1 block = 1024b, it is sufficient for 20-30 cached + planns. One query needs somewhere 3-10 blocks, for example query like + + PREPARE sel AS SELECT * FROM pg_class; + + needs 10Kb, because table pg_class has very much columns. + + Note: for development I add SQL function: "SELECT qcache_state();", + this routine show usage of qCache. + + SPI + ~~~ + I a little overwrite SPI save plan method and remove TopMemoryContext + "feeding". + + Standard SPI: + + SPI_saveplan() - save each plan to separate standard memory context. + + SPI_freeplan() - free plan. + + By key SPI: + + It is SPI interface for query cache and allows save planns to SHARED + or LOCAL cache 'by' arbitrary key (string or binary). Routines: + + SPI_saveplan_bykey() - save plan to query cache + + SPI_freeplan_bykey() - remove plan from query cache + + SPI_fetchplan_bykey() - fetch plan saved in query cache + + SPI_execp_bykey() - execute (via SPI) plan saved in query + cache + + - now, users can write functions that save planns to shared memory + and planns are visible in all backend and are persistent arcoss + connection. + + Example: + ~~~~~~~ + /* ---------- + * Save/exec query from shared cache via string key + * ---------- + */ + int keySize = 0; + flag = SPI_BYKEY_SHARE | SPI_BYKEY_STRING; + char *key = "my unique key"; + + res = SPI_execp_bykey(values, nulls, tcount, key, flag, keySize); + + if (res == SPI_ERROR_PLANNOTFOUND) + { + /* --- not plan in cache - must create it --- */ + + void *plan; + + plan = SPI_prepare(querystr, valnum, valtypes); + SPI_saveplan_bykey(plan, key, keySize, flag); + + res = SPI_execute(plan, values, Nulls, tcount); + } + + elog(NOTICE, "Processed: %d", SPI_processed); + + + PREPARE/EXECUTE + ~~~~~~~~~~~~~~~ + * Syntax: + + PREPARE AS + [ USING type, ... typeN ] + [ NOSHARE | SHARE | GLOBAL ] + + EXECUTE + [ INTO [ TEMPORARY | TEMP ] [ TABLE ] new_table ] + [ USING val, ... valN ] + [ NOSHARE | SHARE | GLOBAL ] + + DEALLOCATE PREPARE + [ [ NOSHARE | SHARE | GLOBAL ]] + [ ALL | ALL INTERNAL ] + + + I know that it is a little out of SQL92... (use CREATE/DROP PLAN instead + this?) --- what mean SQL standard guru? + + * Where: + + NOSHARE --- cached in local backend query cache - not accessable + from the others backends and not is persisten a across + conection. + + SHARE --- cached in shared query cache and accessable from + all backends which work over same database. + + GLOBAL --- cached in shared query cache and accessable from + all backends and all databases. + + - default is 'SHARE' + + Deallocate: + + ALL --- deallocate all users's plans + + ALL INTERNAL --- deallocate all internal plans, like planns + cached via SPI. It is needful if user + alter/drop table ...etc. + + * Parameters: + + "USING" part in the prepare statement is for datetype setting for + paremeters in the query. For example: + + PREPARE sel AS SELECT * FROM pg_class WHERE relname ~~ $1 USING text; + + EXECUTE sel USING 'pg%'; + + + * Limitation: + + - prepare/execute allow use full statement of SELECT/INSERT/DELETE/ + UPDATE. + - possible is use union, subselects, limit, ofset, select-into + + + Performance: + ~~~~~~~~~~~ + * the SPI + + - I for my tests a little change RI triggers to use SPI by_key API + and save planns to shared qCache instead to internal RI hash table. + + The RI use very simple (for parsing) queries and qCache interest is + not visible. It's better if backend very often startup and RI check + always same tables. In this situation speed go up --- 10-12%. + (This snapshot not include this RI change.) + + But all depend on how much complicate for parser is query in + trigger. + + * PREPARE/EXECUTE + + - For tests I use query that not use some table (the executor is + in boredom state), but is difficult for the parser. An example: + + SELECT 'a text ' || (10*10+(100^2))::text || ' next text ' || cast + (date_part('year', timestamp 'now') AS text ); + + - (10000 * this query): + + standard select: 54 sec + via prepare/execute: 4 sec (93% better) + + IMHO it is nod bad. + + - For standard query like: + + SELECT u.usename, r.relname FROM pg_class r, pg_user u WHERE + r.relowner = u.usesysid; + + it is with PREPARE/EXECUTE 10-20% faster. + +-- + Karel Zak + http://home.zf.jcu.cz/~zakkr/ + + C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz + +---------------------------(end of broadcast)--------------------------- +TIP 4: Don't 'kill -9' the postmaster + +From pgsql-hackers-owner+M21228@postgresql.org Fri Apr 12 10:15:34 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CEFXS29835 + for ; Fri, 12 Apr 2002 10:15:33 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 7BFE1475A55; Fri, 12 Apr 2002 10:15:27 -0400 (EDT) +Received: from sss.pgh.pa.us (unknown [192.204.191.242]) + by postgresql.org (Postfix) with ESMTP id 5659B474E71 + for ; Fri, 12 Apr 2002 10:14:31 -0400 (EDT) +Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) + by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3CEEQF27238; + Fri, 12 Apr 2002 10:14:26 -0400 (EDT) +To: Karel Zak +cc: pgsql-hackers@postgresql.org, Neil Conway +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <20020412095116.B6370@zf.jcu.cz> +References: <3CB52C54.4020507@freaky-namuh.com> <20020411115434.201ff92f.nconway@klamath.dyndns.org> <3CB61DAB.5010601@freaky-namuh.com> <24184.1018581907@sss.pgh.pa.us> <3CB65B49.93F2F790@tpf.co.jp> <20020412004134.5d35a2dd.nconway@klamath.dyndns.org> <20020412095116.B6370@zf.jcu.cz> +Comments: In-reply-to Karel Zak + message dated "Fri, 12 Apr 2002 09:51:16 +0200" +Date: Fri, 12 Apr 2002 10:14:26 -0400 +Message-ID: <27235.1018620866@sss.pgh.pa.us> +From: Tom Lane +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: ORr + +Karel Zak writes: +> * The patch allows store query-planns to shared memory and is +> possible EXECUTE it at more backends (over same DB) and planns +> are persistent across connetions. For this feature I create special +> memory context subsystem (like current aset.c, but it works with +> IPC shared memory). +> This is maybe too complex solution and (maybe) sufficient is cache +> query in one backend only. I know unbelief about this shared +> memory solution (Tom?). + +Yes, that is the part that was my sticking point last time around. +(1) Because shared memory cannot be extended on-the-fly, I think it is +a very bad idea to put data structures in there without some well +thought out way of predicting/limiting their size. (2) How the heck do +you get rid of obsoleted cached plans, if the things stick around in +shared memory even after you start a new backend? (3) A shared cache +requires locking; contention among multiple backends to access that +shared resource could negate whatever performance benefit you might hope +to realize from it. + +A per-backend cache kept in local memory avoids all of these problems, +and I have seen no numbers to make me think that a shared plan cache +would achieve significantly more performance benefit than a local one. + + regards, tom lane + +---------------------------(end of broadcast)--------------------------- +TIP 5: Have you checked our extensive FAQ? + +http://www.postgresql.org/users-lounge/docs/faq.html + +From pgsql-hackers-owner+M21233@postgresql.org Fri Apr 12 12:26:32 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CGQVS11018 + for ; Fri, 12 Apr 2002 12:26:31 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 38DBB475B20; Fri, 12 Apr 2002 12:22:08 -0400 (EDT) +Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) + by postgresql.org (Postfix) with ESMTP id 0DA70475B9E + for ; Fri, 12 Apr 2002 12:21:15 -0400 (EDT) +Received: (from pgman@localhost) + by candle.pha.pa.us (8.11.6/8.10.1) id g3CGL4310492; + Fri, 12 Apr 2002 12:21:04 -0400 (EDT) +From: Bruce Momjian +Message-ID: <200204121621.g3CGL4310492@candle.pha.pa.us> +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <27235.1018620866@sss.pgh.pa.us> +To: Tom Lane +Date: Fri, 12 Apr 2002 12:21:04 -0400 (EDT) +cc: Karel Zak , pgsql-hackers@postgresql.org, + Neil Conway +X-Mailer: ELM [version 2.4ME+ PL97 (25)] +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Type: text/plain; charset=US-ASCII +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +Tom Lane wrote: +> Karel Zak writes: +> > * The patch allows store query-planns to shared memory and is +> > possible EXECUTE it at more backends (over same DB) and planns +> > are persistent across connetions. For this feature I create special +> > memory context subsystem (like current aset.c, but it works with +> > IPC shared memory). +> > This is maybe too complex solution and (maybe) sufficient is cache +> > query in one backend only. I know unbelief about this shared +> > memory solution (Tom?). +> +> Yes, that is the part that was my sticking point last time around. +> (1) Because shared memory cannot be extended on-the-fly, I think it is +> a very bad idea to put data structures in there without some well +> thought out way of predicting/limiting their size. (2) How the heck do +> you get rid of obsoleted cached plans, if the things stick around in +> shared memory even after you start a new backend? (3) A shared cache +> requires locking; contention among multiple backends to access that +> shared resource could negate whatever performance benefit you might hope +> to realize from it. +> +> A per-backend cache kept in local memory avoids all of these problems, +> and I have seen no numbers to make me think that a shared plan cache +> would achieve significantly more performance benefit than a local one. + +Certainly a shared cache would be good for apps that connect to issue a +single query frequently. In such cases, there would be no local cache +to use. + +-- + Bruce Momjian | http://candle.pha.pa.us + pgman@candle.pha.pa.us | (610) 853-3000 + + If your life is a hard drive, | 830 Blythe Avenue + + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 + +---------------------------(end of broadcast)--------------------------- +TIP 4: Don't 'kill -9' the postmaster + +From pgsql-hackers-owner+M21234@postgresql.org Fri Apr 12 12:44:12 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CGiBS12385 + for ; Fri, 12 Apr 2002 12:44:12 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id AEAA7475C6C; Fri, 12 Apr 2002 12:43:17 -0400 (EDT) +Received: from barry.xythos.com (h-64-105-36-191.SNVACAID.covad.net [64.105.36.191]) + by postgresql.org (Postfix) with ESMTP id CE58C47598E + for ; Fri, 12 Apr 2002 12:42:48 -0400 (EDT) +Received: from xythos.com (localhost.localdomain [127.0.0.1]) + by barry.xythos.com (8.11.6/8.11.6) with ESMTP id g3CGgaI02920; + Fri, 12 Apr 2002 09:42:36 -0700 +Message-ID: <3CB70E7C.3090801@xythos.com> +Date: Fri, 12 Apr 2002 09:42:36 -0700 +From: Barry Lind +User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9) Gecko/20020310 +X-Accept-Language: en-us, en +MIME-Version: 1.0 +To: Tom Lane +cc: Karel Zak , pgsql-hackers@postgresql.org, + Neil Conway +Subject: Re: [HACKERS] 7.3 schedule +References: <3CB52C54.4020507@freaky-namuh.com> <20020411115434.201ff92f.nconway@klamath.dyndns.org> <3CB61DAB.5010601@freaky-namuh.com> <24184.1018581907@sss.pgh.pa.us> <3CB65B49.93F2F790@tpf.co.jp> <20020412004134.5d35a2dd.nconway@klamath.dyndns.org> <20020412095116.B6370@zf.jcu.cz> <27235.1018620866@sss.pgh.pa.us> +Content-Type: text/plain; charset=us-ascii; format=flowed +Content-Transfer-Encoding: 7bit +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: ORr + + + +Tom Lane wrote: +> Yes, that is the part that was my sticking point last time around. +> (1) Because shared memory cannot be extended on-the-fly, I think it is +> a very bad idea to put data structures in there without some well +> thought out way of predicting/limiting their size. (2) How the heck do +> you get rid of obsoleted cached plans, if the things stick around in +> shared memory even after you start a new backend? (3) A shared cache +> requires locking; contention among multiple backends to access that +> shared resource could negate whatever performance benefit you might hope +> to realize from it. +> +> A per-backend cache kept in local memory avoids all of these problems, +> and I have seen no numbers to make me think that a shared plan cache +> would achieve significantly more performance benefit than a local one. +> + +Oracle's implementation is a shared cache for all plans. This was +introduced in Oracle 6 or 7 (I don't remember which anymore). The net +effect was that in general there was a significant performance +improvement with the shared cache. However poorly written apps can now +bring the Oracle database to its knees because of the locking issues +associated with the shared cache. For example if the most frequently +run sql statements are coded poorly (i.e. they don't use bind variables, +eg. 'select bar from foo where foobar = $1' vs. 'select bar from foo +where foobar = || somevalue' (where somevalue is likely to be +different on every call)) the shared cache doesn't help and its overhead +becomes significant. + +thanks, +--Barry + + + +---------------------------(end of broadcast)--------------------------- +TIP 3: if posting/reading through Usenet, please send an appropriate +subscribe-nomail command to majordomo@postgresql.org so that your +message can get through to the mailing list cleanly + +From pgsql-hackers-owner+M21237@postgresql.org Fri Apr 12 12:50:28 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CGoRS13005 + for ; Fri, 12 Apr 2002 12:50:28 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 32A28475BA1; Fri, 12 Apr 2002 12:50:15 -0400 (EDT) +Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) + by postgresql.org (Postfix) with ESMTP id 07F1E475892 + for ; Fri, 12 Apr 2002 12:49:43 -0400 (EDT) +Received: (from pgman@localhost) + by candle.pha.pa.us (8.11.6/8.10.1) id g3CGnbw12950; + Fri, 12 Apr 2002 12:49:37 -0400 (EDT) +From: Bruce Momjian +Message-ID: <200204121649.g3CGnbw12950@candle.pha.pa.us> +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <3CB70E7C.3090801@xythos.com> +To: Barry Lind +Date: Fri, 12 Apr 2002 12:49:37 -0400 (EDT) +cc: Tom Lane , Karel Zak , + pgsql-hackers@postgresql.org, Neil Conway +X-Mailer: ELM [version 2.4ME+ PL97 (25)] +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Type: text/plain; charset=US-ASCII +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +Barry Lind wrote: +> Oracle's implementation is a shared cache for all plans. This was +> introduced in Oracle 6 or 7 (I don't remember which anymore). The net +> effect was that in general there was a significant performance +> improvement with the shared cache. However poorly written apps can now +> bring the Oracle database to its knees because of the locking issues +> associated with the shared cache. For example if the most frequently +> run sql statements are coded poorly (i.e. they don't use bind variables, +> eg. 'select bar from foo where foobar = $1' vs. 'select bar from foo +> where foobar = || somevalue' (where somevalue is likely to be +> different on every call)) the shared cache doesn't help and its overhead +> becomes significant. + +This is very interesting. We have always been concerned that shared +cache invalidation could cause more of a performance problem that the +shared cache gives benefit, and it sounds like you are saying exactly +that. + +-- + Bruce Momjian | http://candle.pha.pa.us + pgman@candle.pha.pa.us | (610) 853-3000 + + If your life is a hard drive, | 830 Blythe Avenue + + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 + +---------------------------(end of broadcast)--------------------------- +TIP 2: you can get off all lists at once with the unregister command + (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) + +From pgsql-hackers-owner+M21238@postgresql.org Fri Apr 12 12:51:55 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CGptS13119 + for ; Fri, 12 Apr 2002 12:51:55 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id C599D475BC6; Fri, 12 Apr 2002 12:51:47 -0400 (EDT) +Received: from sss.pgh.pa.us (unknown [192.204.191.242]) + by postgresql.org (Postfix) with ESMTP id C9F94475892 + for ; Fri, 12 Apr 2002 12:51:26 -0400 (EDT) +Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) + by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3CGpQF27967; + Fri, 12 Apr 2002 12:51:27 -0400 (EDT) +To: Bruce Momjian +cc: Karel Zak , pgsql-hackers@postgresql.org, + Neil Conway +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <200204121621.g3CGL4310492@candle.pha.pa.us> +References: <200204121621.g3CGL4310492@candle.pha.pa.us> +Comments: In-reply-to Bruce Momjian + message dated "Fri, 12 Apr 2002 12:21:04 -0400" +Date: Fri, 12 Apr 2002 12:51:26 -0400 +Message-ID: <27964.1018630286@sss.pgh.pa.us> +From: Tom Lane +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +Bruce Momjian writes: +> Certainly a shared cache would be good for apps that connect to issue a +> single query frequently. In such cases, there would be no local cache +> to use. + +We have enough other problems with the single-query-per-connection +scenario that I see no reason to believe that a shared plan cache will +help materially. The correct answer for those folks will *always* be +to find a way to reuse the connection. + + regards, tom lane + +---------------------------(end of broadcast)--------------------------- +TIP 4: Don't 'kill -9' the postmaster + +From pgsql-hackers-owner+M21241@postgresql.org Fri Apr 12 16:25:46 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CKPkS03078 + for ; Fri, 12 Apr 2002 16:25:46 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 9C3BD475CC6; Fri, 12 Apr 2002 16:25:42 -0400 (EDT) +Received: from klamath.dyndns.org (CPE002078144ae0.cpe.net.cable.rogers.com [24.102.202.35]) + by postgresql.org (Postfix) with ESMTP id B06D8475909 + for ; Fri, 12 Apr 2002 16:24:52 -0400 (EDT) +Received: from jiro (jiro [192.168.40.7]) + by klamath.dyndns.org (Postfix) with SMTP + id C05557013; Fri, 12 Apr 2002 16:24:53 -0400 (EDT) +Date: Fri, 12 Apr 2002 16:24:48 -0400 +From: Neil Conway +To: "Bruce Momjian" +cc: tgl@sss.pgh.pa.us, zakkr@zf.jcu.cz, pgsql-hackers@postgresql.org +Subject: Re: [HACKERS] 7.3 schedule +Message-ID: <20020412162448.4d46d747.nconway@klamath.dyndns.org> +In-Reply-To: <200204121621.g3CGL4310492@candle.pha.pa.us> +References: <27235.1018620866@sss.pgh.pa.us> + <200204121621.g3CGL4310492@candle.pha.pa.us> +X-Mailer: Sylpheed version 0.7.4 (GTK+ 1.2.10; i386-debian-linux-gnu) +MIME-Version: 1.0 +Content-Type: text/plain; charset=US-ASCII +Content-Transfer-Encoding: 7bit +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: ORr + +On Fri, 12 Apr 2002 12:21:04 -0400 (EDT) +"Bruce Momjian" wrote: +> Tom Lane wrote: +> > A per-backend cache kept in local memory avoids all of these problems, +> > and I have seen no numbers to make me think that a shared plan cache +> > would achieve significantly more performance benefit than a local one. +> +> Certainly a shared cache would be good for apps that connect to issue a +> single query frequently. In such cases, there would be no local cache +> to use. + +One problem with this kind of scenario is: what to do if the plan no +longer exists for some reason? (e.g. the code that was supposed to be +PREPARE-ing your statements failed to execute properly, or the cached +plan has been evicted from shared memory, or the database was restarted, +etc.) -- EXECUTE in and of itself won't have enough information to do +anything useful. We could perhaps provide a means for an application +to test for the existence of a cached plan (in which case the +application developer will need to add logic to their application +to re-prepare the query if necessary, which could get complicated). + +Cheers, + +Neil + +-- +Neil Conway +PGP Key ID: DB3C29FC + +---------------------------(end of broadcast)--------------------------- +TIP 5: Have you checked our extensive FAQ? + +http://www.postgresql.org/users-lounge/docs/faq.html + +From pgsql-hackers-owner+M21242@postgresql.org Fri Apr 12 17:27:24 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CLRNS14410 + for ; Fri, 12 Apr 2002 17:27:23 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id E05A1475D30; Fri, 12 Apr 2002 17:26:40 -0400 (EDT) +Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) + by postgresql.org (Postfix) with ESMTP id 36BBB475858 + for ; Fri, 12 Apr 2002 17:25:44 -0400 (EDT) +Received: (from pgman@localhost) + by candle.pha.pa.us (8.11.6/8.10.1) id g3CLPVa14231; + Fri, 12 Apr 2002 17:25:31 -0400 (EDT) +From: Bruce Momjian +Message-ID: <200204122125.g3CLPVa14231@candle.pha.pa.us> +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <20020412162448.4d46d747.nconway@klamath.dyndns.org> +To: Neil Conway +Date: Fri, 12 Apr 2002 17:25:31 -0400 (EDT) +cc: tgl@sss.pgh.pa.us, zakkr@zf.jcu.cz, pgsql-hackers@postgresql.org +X-Mailer: ELM [version 2.4ME+ PL97 (25)] +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Type: text/plain; charset=US-ASCII +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +Neil Conway wrote: +> On Fri, 12 Apr 2002 12:21:04 -0400 (EDT) +> "Bruce Momjian" wrote: +> > Tom Lane wrote: +> > > A per-backend cache kept in local memory avoids all of these problems, +> > > and I have seen no numbers to make me think that a shared plan cache +> > > would achieve significantly more performance benefit than a local one. +> > +> > Certainly a shared cache would be good for apps that connect to issue a +> > single query frequently. In such cases, there would be no local cache +> > to use. +> +> One problem with this kind of scenario is: what to do if the plan no +> longer exists for some reason? (e.g. the code that was supposed to be +> PREPARE-ing your statements failed to execute properly, or the cached +> plan has been evicted from shared memory, or the database was restarted, +> etc.) -- EXECUTE in and of itself won't have enough information to do +> anything useful. We could perhaps provide a means for an application +> to test for the existence of a cached plan (in which case the +> application developer will need to add logic to their application +> to re-prepare the query if necessary, which could get complicated). + +Oh, are you thinking that one backend would do the PREPARE and another +one the EXECUTE? I can't see that working at all. I thought there +would some way to quickly test if the submitted query was in the cache, +but maybe that is too much of a performance penalty to be worth it. + +-- + Bruce Momjian | http://candle.pha.pa.us + pgman@candle.pha.pa.us | (610) 853-3000 + + If your life is a hard drive, | 830 Blythe Avenue + + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 + +---------------------------(end of broadcast)--------------------------- +TIP 3: if posting/reading through Usenet, please send an appropriate +subscribe-nomail command to majordomo@postgresql.org so that your +message can get through to the mailing list cleanly + +From tgl@sss.pgh.pa.us Fri Apr 12 17:36:17 2002 +Return-path: +Received: from sss.pgh.pa.us (root@[192.204.191.242]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CLaGS16061 + for ; Fri, 12 Apr 2002 17:36:17 -0400 (EDT) +Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) + by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3CLaGF10813; + Fri, 12 Apr 2002 17:36:16 -0400 (EDT) +To: Bruce Momjian +cc: Neil Conway , zakkr@zf.jcu.cz, + pgsql-hackers@postgresql.org +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <200204122125.g3CLPVa14231@candle.pha.pa.us> +References: <200204122125.g3CLPVa14231@candle.pha.pa.us> +Comments: In-reply-to Bruce Momjian + message dated "Fri, 12 Apr 2002 17:25:31 -0400" +Date: Fri, 12 Apr 2002 17:36:16 -0400 +Message-ID: <10810.1018647376@sss.pgh.pa.us> +From: Tom Lane +Status: ORr + +Bruce Momjian writes: +> Oh, are you thinking that one backend would do the PREPARE and another +> one the EXECUTE? I can't see that working at all. + +Uh, why exactly were you advocating a shared cache then? Wouldn't that +be exactly the *point* of a shared cache? + + regards, tom lane + +From pgsql-hackers-owner+M21245@postgresql.org Fri Apr 12 17:39:13 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CLdCS16515 + for ; Fri, 12 Apr 2002 17:39:12 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id A904B475E15; Fri, 12 Apr 2002 17:39:09 -0400 (EDT) +Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) + by postgresql.org (Postfix) with ESMTP id B1A3F4758DE + for ; Fri, 12 Apr 2002 17:38:25 -0400 (EDT) +Received: (from pgman@localhost) + by candle.pha.pa.us (8.11.6/8.10.1) id g3CLcFX16347; + Fri, 12 Apr 2002 17:38:15 -0400 (EDT) +From: Bruce Momjian +Message-ID: <200204122138.g3CLcFX16347@candle.pha.pa.us> +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <10810.1018647376@sss.pgh.pa.us> +To: Tom Lane +Date: Fri, 12 Apr 2002 17:38:15 -0400 (EDT) +cc: Neil Conway , zakkr@zf.jcu.cz, + pgsql-hackers@postgresql.org +X-Mailer: ELM [version 2.4ME+ PL97 (25)] +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Type: text/plain; charset=US-ASCII +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +Tom Lane wrote: +> Bruce Momjian writes: +> > Oh, are you thinking that one backend would do the PREPARE and another +> > one the EXECUTE? I can't see that working at all. +> +> Uh, why exactly were you advocating a shared cache then? Wouldn't that +> be exactly the *point* of a shared cache? + +I thought it would somehow compare the SQL query string to the cached +plans and if it matched, it would use that plan rather than make a new +one. Any DDL statement would flush the cache. + +-- + Bruce Momjian | http://candle.pha.pa.us + pgman@candle.pha.pa.us | (610) 853-3000 + + If your life is a hard drive, | 830 Blythe Avenue + + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 + +---------------------------(end of broadcast)--------------------------- +TIP 5: Have you checked our extensive FAQ? + +http://www.postgresql.org/users-lounge/docs/faq.html + +From pgsql-hackers-owner+M21246@postgresql.org Fri Apr 12 17:56:58 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3CLuvS19021 + for ; Fri, 12 Apr 2002 17:56:58 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 1B4D6475E2C; Fri, 12 Apr 2002 17:56:55 -0400 (EDT) +Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) + by postgresql.org (Postfix) with ESMTP id 059F1475858 + for ; Fri, 12 Apr 2002 17:56:13 -0400 (EDT) +X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 +content-class: urn:content-classes:message +MIME-Version: 1.0 +Content-Type: text/plain; + charset="iso-8859-1" +Subject: Re: [HACKERS] 7.3 schedule +Date: Fri, 12 Apr 2002 14:59:15 -0700 +Message-ID: +Thread-Topic: [HACKERS] 7.3 schedule +Thread-Index: AcHia2aODSpgXEd4Tluz/N0jN5fJOQAAC//w +From: "Dann Corbit" +To: "Bruce Momjian" , "Tom Lane" +cc: "Neil Conway" , , + +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Content-Transfer-Encoding: 8bit +X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g3CLuvS19021 +Status: OR + +-----Original Message----- +From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] +Sent: Friday, April 12, 2002 2:38 PM +To: Tom Lane +Cc: Neil Conway; zakkr@zf.jcu.cz; pgsql-hackers@postgresql.org +Subject: Re: [HACKERS] 7.3 schedule + + +Tom Lane wrote: +> Bruce Momjian writes: +> > Oh, are you thinking that one backend would do the PREPARE and +another +> > one the EXECUTE? I can't see that working at all. +> +> Uh, why exactly were you advocating a shared cache then? Wouldn't +that +> be exactly the *point* of a shared cache? + +I thought it would somehow compare the SQL query string to the cached +plans and if it matched, it would use that plan rather than make a new +one. Any DDL statement would flush the cache. +>>------------------------------------------------------------------- +Many applications will have similar queries coming from lots of +different end-users. Imagine an order-entry program where people are +ordering parts. Many of the queries might look like this: + +SELECT part_number FROM parts WHERE part_id = 12324 AND part_cost +< 12.95 + +In order to cache this query, we first parse it to replace the data +fields with paramter markers. +Then it looks like this: +SELECT part_number FROM parts WHERE part_id = ? AND part_cost < ? +{in the case of a 'LIKE' query or some other query where you can use +key information, you might have a symbolic replacement like this: +WHERE field LIKE '{D}%' to indicate that the key can be used} +Then, we make sure that the case is consistent by either capitalizing +the whole query or changing it all into lower case: +select part_number from parts where part_id = ? and part_cost < ? +Then, we run a checksum on the parameterized string. +The checksum might be used as a hash table key, where we keep some +additional information like how stale the entry is, and a pointer to +the actual parameterized SQL (in case the hash key has a collision +it would be simply wrong to run an incorrect query for obvious enough +reasons). +Now, if there are a huge number of users of the same application, it +makes sense that the probabilities of reusing queries goes up with +the number of users of the same application. Therefore, I would +advocate that the cache be kept in shared memory. + +Consider a single application with 100 different queries. Now, add +one user, ten users, 100 users, ... 10,000 users and you can see +that the benefit would be greater and greater as we add users. +<<------------------------------------------------------------------- + +---------------------------(end of broadcast)--------------------------- +TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org + +From pgsql-hackers-owner+M21270@postgresql.org Sat Apr 13 02:30:47 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3D6UkS07169 + for ; Sat, 13 Apr 2002 02:30:46 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 23FEC475D1E; Sat, 13 Apr 2002 02:30:38 -0400 (EDT) +Received: from mail.iinet.net.au (symphony-01.iinet.net.au [203.59.3.33]) + by postgresql.org (Postfix) with SMTP id A08A4475C6C + for ; Sat, 13 Apr 2002 02:29:37 -0400 (EDT) +Received: (qmail 11594 invoked by uid 666); 13 Apr 2002 06:29:36 -0000 +Received: from unknown (HELO SOL) (203.59.103.193) + by mail.iinet.net.au with SMTP; 13 Apr 2002 06:29:36 -0000 +Message-ID: <002301c1e2b3$804bd000$0200a8c0@SOL> +From: "Christopher Kings-Lynne" +To: "Barry Lind" , "Tom Lane" +cc: "Karel Zak" , , + "Neil Conway" +References: <3CB52C54.4020507@freaky-namuh.com> <20020411115434.201ff92f.nconway@klamath.dyndns.org> <3CB61DAB.5010601@freaky-namuh.com> <24184.1018581907@sss.pgh.pa.us> <3CB65B49.93F2F790@tpf.co.jp> <20020412004134.5d35a2dd.nconway@klamath.dyndns.org> <20020412095116.B6370@zf.jcu.cz> <27235.1018620866@sss.pgh.pa.us> <3CB70E7C.3090801@xythos.com> +Subject: Re: [HACKERS] 7.3 schedule +Date: Sat, 13 Apr 2002 14:21:50 +0800 +MIME-Version: 1.0 +Content-Type: text/plain; + charset="iso-8859-1" +Content-Transfer-Encoding: 7bit +X-Priority: 3 +X-MSMail-Priority: Normal +X-Mailer: Microsoft Outlook Express 5.50.4522.1200 +X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +> > thought out way of predicting/limiting their size. (2) How the heck do +> > you get rid of obsoleted cached plans, if the things stick around in +> > shared memory even after you start a new backend? (3) A shared cache +> > requires locking; contention among multiple backends to access that +> > shared resource could negate whatever performance benefit you might hope +> > to realize from it. + +I don't understand all these locking problems? Surely the only lock a +transaction would need on a stored query is one that prevents the cache +invalidation mechanism from deleting it out from under it? Surely this +means that there would be tonnes of readers on the cache - none of them +blocking each other, and the odd invalidation event that needs a complete +lock? + +Also, as for invalidation, there probably could be just two reasons to +invalidate a query in the cache. (1) The cache is running out of space and +you use LRU or something to remove old queries, or (2) someone runs ANALYZE, +in which case all cached queries should just be flushed? If they specify an +actual table to analyze, then just drop all queries on the table. + +Could this cache mechanism be used to make views fast as well? You could +cache the queries that back views on first use, and then they can follow the +above rules for flushing... + +Chris + + + +---------------------------(end of broadcast)--------------------------- +TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org + +From pgsql-hackers-owner+M21276@postgresql.org Sat Apr 13 11:48:51 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3DFmoS27879 + for ; Sat, 13 Apr 2002 11:48:51 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 9EB81475C5C; Sat, 13 Apr 2002 11:46:52 -0400 (EDT) +Received: from sss.pgh.pa.us (unknown [192.204.191.242]) + by postgresql.org (Postfix) with ESMTP id 0FE0B474E78 + for ; Sat, 13 Apr 2002 11:46:09 -0400 (EDT) +Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) + by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g3DFk2F15743; + Sat, 13 Apr 2002 11:46:02 -0400 (EDT) +To: "Christopher Kings-Lynne" +cc: "Barry Lind" , "Karel Zak" , + pgsql-hackers@postgresql.org, "Neil Conway" +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <002301c1e2b3$804bd000$0200a8c0@SOL> +References: <3CB52C54.4020507@freaky-namuh.com> <20020411115434.201ff92f.nconway@klamath.dyndns.org> <3CB61DAB.5010601@freaky-namuh.com> <24184.1018581907@sss.pgh.pa.us> <3CB65B49.93F2F790@tpf.co.jp> <20020412004134.5d35a2dd.nconway@klamath.dyndns.org> <20020412095116.B6370@zf.jcu.cz> <27235.1018620866@sss.pgh.pa.us> <3CB70E7C.3090801@xythos.com> <002301c1e2b3$804bd000$0200a8c0@SOL> +Comments: In-reply-to "Christopher Kings-Lynne" + message dated "Sat, 13 Apr 2002 14:21:50 +0800" +Date: Sat, 13 Apr 2002 11:46:01 -0400 +Message-ID: <15740.1018712761@sss.pgh.pa.us> +From: Tom Lane +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +"Christopher Kings-Lynne" writes: +> thought out way of predicting/limiting their size. (2) How the heck do +> you get rid of obsoleted cached plans, if the things stick around in +> shared memory even after you start a new backend? (3) A shared cache +> requires locking; contention among multiple backends to access that +> shared resource could negate whatever performance benefit you might hope +> to realize from it. + +> I don't understand all these locking problems? + +Searching the cache and inserting/deleting entries in the cache probably +have to be mutually exclusive; concurrent insertions probably won't work +either (at least not without a remarkably intelligent data structure). +Unless the cache hit rate is remarkably high, there are going to be lots +of insertions --- and, at steady state, an equal rate of deletions --- +leading to lots of contention. + +This could possibly be avoided if the cache is not used for all query +plans but only for explicitly PREPAREd plans, so that only explicit +EXECUTEs would need to search it. But that approach also makes a +sizable dent in the usefulness of the cache to begin with. + + regards, tom lane + +---------------------------(end of broadcast)--------------------------- +TIP 2: you can get off all lists at once with the unregister command + (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) + +From pgsql-hackers-owner+M21280@postgresql.org Sat Apr 13 14:36:34 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3DIaYS10293 + for ; Sat, 13 Apr 2002 14:36:34 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id AA151475BB1; Sat, 13 Apr 2002 14:36:17 -0400 (EDT) +Received: from klamath.dyndns.org (CPE002078144ae0.cpe.net.cable.rogers.com [24.102.202.35]) + by postgresql.org (Postfix) with ESMTP id 42993475BCB + for ; Sat, 13 Apr 2002 14:35:42 -0400 (EDT) +Received: from jiro (jiro [192.168.40.7]) + by klamath.dyndns.org (Postfix) with SMTP + id 82B84700C; Sat, 13 Apr 2002 14:35:42 -0400 (EDT) +Date: Sat, 13 Apr 2002 14:35:39 -0400 +From: Neil Conway +To: "Christopher Kings-Lynne" +cc: barry@xythos.com, tgl@sss.pgh.pa.us, zakkr@zf.jcu.cz, + pgsql-hackers@postgresql.org +Subject: Re: [HACKERS] 7.3 schedule +Message-ID: <20020413143539.7818bf7d.nconway@klamath.dyndns.org> +In-Reply-To: <002301c1e2b3$804bd000$0200a8c0@SOL> +References: + <3CB52C54.4020507@freaky-namuh.com> + <20020411115434.201ff92f.nconway@klamath.dyndns.org> + <3CB61DAB.5010601@freaky-namuh.com> + <24184.1018581907@sss.pgh.pa.us> + <3CB65B49.93F2F790@tpf.co.jp> + <20020412004134.5d35a2dd.nconway@klamath.dyndns.org> + <20020412095116.B6370@zf.jcu.cz> + <27235.1018620866@sss.pgh.pa.us> + <3CB70E7C.3090801@xythos.com> + <002301c1e2b3$804bd000$0200a8c0@SOL> +X-Mailer: Sylpheed version 0.7.4 (GTK+ 1.2.10; i386-debian-linux-gnu) +MIME-Version: 1.0 +Content-Type: text/plain; charset=US-ASCII +Content-Transfer-Encoding: 7bit +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +On Sat, 13 Apr 2002 14:21:50 +0800 +"Christopher Kings-Lynne" wrote: +> Could this cache mechanism be used to make views fast as well? + +The current PREPARE/EXECUTE code will speed up queries that use +rules of any kind, including views: the query plan is cached after +it has been rewritten as necessary, so (AFAIK) this should mean +that rules will be evaluated once when the query is PREPAREd, and +then cached for subsequent EXECUTE commands. + +Cheers, + +Neil + +-- +Neil Conway +PGP Key ID: DB3C29FC + +---------------------------(end of broadcast)--------------------------- +TIP 2: you can get off all lists at once with the unregister command + (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) + +From pgsql-hackers-owner+M21309@postgresql.org Sun Apr 14 15:22:44 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3EJMiS24239 + for ; Sun, 14 Apr 2002 15:22:44 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 44BAC475E05; Sun, 14 Apr 2002 15:22:42 -0400 (EDT) +Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4]) + by postgresql.org (Postfix) with ESMTP id 3CD03475925 + for ; Sun, 14 Apr 2002 15:21:58 -0400 (EDT) +Received: from ara.zf.jcu.cz (LOCALHOST [127.0.0.1]) + by ara.zf.jcu.cz (8.12.1/8.12.1/Debian -5) with ESMTP id g3EJLiBK012612; + Sun, 14 Apr 2002 21:21:44 +0200 +Received: (from zakkr@localhost) + by ara.zf.jcu.cz (8.12.1/8.12.1/Debian -5) id g3EJLi3k012611; + Sun, 14 Apr 2002 21:21:44 +0200 +Date: Sun, 14 Apr 2002 21:21:44 +0200 +From: Karel Zak +To: Tom Lane +cc: Bruce Momjian , pgsql-hackers@postgresql.org, + Neil Conway +Subject: Re: [HACKERS] 7.3 schedule +Message-ID: <20020414212144.A12196@zf.jcu.cz> +References: <200204121621.g3CGL4310492@candle.pha.pa.us> <27964.1018630286@sss.pgh.pa.us> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +User-Agent: Mutt/1.2.5i +In-Reply-To: <27964.1018630286@sss.pgh.pa.us>; from tgl@sss.pgh.pa.us on Fri, Apr 12, 2002 at 12:51:26PM -0400 +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +On Fri, Apr 12, 2002 at 12:51:26PM -0400, Tom Lane wrote: +> Bruce Momjian writes: +> > Certainly a shared cache would be good for apps that connect to issue a +> > single query frequently. In such cases, there would be no local cache +> > to use. +> +> We have enough other problems with the single-query-per-connection +> scenario that I see no reason to believe that a shared plan cache will +> help materially. The correct answer for those folks will *always* be +> to find a way to reuse the connection. + + My query cache was write for 7.0. If some next release will use + pre-forked backend and after a client disconnection the backend will + still alives and waits for new client the shared cache is (maybe:-) not + needful. The current backend fork model is killer of all possible + caching. + + We have more caches. I hope persistent backend help will help to all + and I'm sure that speed will grow up with persistent backend and + persistent caches without shared memory usage. There I can agree with + Tom :-) + + Karel + +-- + Karel Zak + http://home.zf.jcu.cz/~zakkr/ + + C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz + +---------------------------(end of broadcast)--------------------------- +TIP 4: Don't 'kill -9' the postmaster + +From pgsql-hackers-owner+M21321@postgresql.org Sun Apr 14 20:40:08 2002 +Return-path: +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g3F0e7S29723 + for ; Sun, 14 Apr 2002 20:40:07 -0400 (EDT) +Received: from postgresql.org (postgresql.org [64.49.215.8]) + by postgresql.org (Postfix) with SMTP + id 3B5FB475DC5; Sun, 14 Apr 2002 20:40:03 -0400 (EDT) +Received: from localhost.localdomain (bgp01077650bgs.wanarb01.mi.comcast.net [68.40.135.112]) + by postgresql.org (Postfix) with ESMTP id 7B1D3474E71 + for ; Sun, 14 Apr 2002 20:39:18 -0400 (EDT) +Received: from localhost (camber@localhost) + by localhost.localdomain (8.11.6/8.11.6) with ESMTP id g3F0cmD10631; + Sun, 14 Apr 2002 20:38:48 -0400 +X-Authentication-Warning: localhost.localdomain: camber owned process doing -bs +Date: Sun, 14 Apr 2002 20:38:48 -0400 (EDT) +From: Brian Bruns +X-X-Sender: +To: Hannu Krosing +cc: +Subject: Re: [HACKERS] 7.3 schedule +In-Reply-To: <1018704763.1784.1.camel@taru.tm.ee> +Message-ID: +MIME-Version: 1.0 +Content-Type: TEXT/PLAIN; charset=US-ASCII +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +Status: OR + +On 13 Apr 2002, Hannu Krosing wrote: + +> On Fri, 2002-04-12 at 03:04, Brian Bruns wrote: +> > On 11 Apr 2002, Hannu Krosing wrote: +> > +> > > IIRC someone started work on modularising the network-related parts with +> > > a goal of supporting DRDA (DB2 protocol) and others in future. +> > +> > That was me, although I've been bogged down lately, and haven't been able +> > to get back to it. +> +> Has any of your modularisation work got into CVS yet ? + +No, Bruce didn't like the way I did certain things, and had some qualms +about the value of supporting multiple wire protocols IIRC. Plus the +patch was not really ready for primetime yet. + +I'm hoping to get back to it soon and sync it with the latest CVS, and +clean up the odds and ends. + +> > DRDA, btw, is not just a DB2 protocol but an opengroup +> > spec that hopefully will someday be *the* standard on the wire database +> > protocol. DRDA handles prepare/execute and is completely binary in +> > representation, among other advantages. +> +> What about extensibility - is there some predefined way of adding new +> types ? + +Not really, there is some ongoing standards activity adding some new +features. The list of supported types is pretty impressive, anything in +particular you are looking for? + +> Also, does it handle NOTIFY ? + +I don't know the answer to this. The spec is pretty huge, so it may, but +I haven't seen it. + +Even if it is supported as a secondary protocol, I believe there is alot +of value in having a single database protocol standard. (why else would I +be doing it!). I'm also looking into what it will take to do the same for +MySQL and Firebird. Hopefully they will be receptive to the idea as well. + +> ---------------- +> Hannu + +Cheers, + +Brian + + +---------------------------(end of broadcast)--------------------------- +TIP 5: Have you checked our extensive FAQ? + +http://www.postgresql.org/users-lounge/docs/faq.html +