From owner-pgsql-hackers@hub.org Wed Nov 18 14:40:49 1998 Received: from hub.org (majordom@hub.org [209.47.148.200]) by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id OAA29743 for ; Wed, 18 Nov 1998 14:40:36 -0500 (EST) Received: from localhost (majordom@localhost) by hub.org (8.9.1/8.9.1) with SMTP id OAA03716; Wed, 18 Nov 1998 14:37:04 -0500 (EST) (envelope-from owner-pgsql-hackers@hub.org) Received: by hub.org (TLB v0.10a (1.23 tibbs 1997/01/09 00:29:32)); Wed, 18 Nov 1998 14:34:39 +0000 (EST) Received: (from majordom@localhost) by hub.org (8.9.1/8.9.1) id OAA03395 for pgsql-hackers-outgoing; Wed, 18 Nov 1998 14:34:37 -0500 (EST) (envelope-from owner-pgsql-hackers@postgreSQL.org) Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de [53.2.131.8]) by hub.org (8.9.1/8.9.1) with SMTP id OAA03381 for ; Wed, 18 Nov 1998 14:34:31 -0500 (EST) (envelope-from wieck@sapserv.debis.de) Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@hub.org id m0zgDnj-000EBTC; Wed, 18 Nov 98 21:02 MET Message-Id: From: jwieck@debis.com (Jan Wieck) Subject: Re: [HACKERS] PREPARE To: meskes@usa.net (Michael Meskes) Date: Wed, 18 Nov 1998 21:02:06 +0100 (MET) Cc: pgsql-hackers@hub.org Reply-To: jwieck@debis.com (Jan Wieck) In-Reply-To: <19981118084843.B869@usa.net> from "Michael Meskes" at Nov 18, 98 08:48:43 am X-Mailer: ELM [version 2.4 PL25] Content-Type: text Sender: owner-pgsql-hackers@postgreSQL.org Precedence: bulk Status: RO Michael Meskes wrote: > > On Wed, Nov 18, 1998 at 03:23:30AM +0000, Thomas G. Lockhart wrote: > > > I didn't get this one completly. What input do you mean? > > > > Just the original string/query to be prepared... > > I see. But wouldn't it be more useful to preprocess the query and store the > resulting nodes instead? We don't want to parse the statement everytime a > variable binding comes in. Right. A real improvement would only be to have the prepared execution plan in the backend and just giving the parameter values. I can think of the following construct: PREPARE optimizable-statement; That one will run parser/rewrite/planner, create a new memory context with a unique identifier and saves the querytree's and plan's in it. Parameter values are identified by the usual $n notation. The command returns the identifier. EXECUTE QUERY identifier [value [, ...]]; then get's back the prepared plan and querytree by the id, creates an executor context with the given values in the parameter array and calls ExecutorRun() for them. The PREPARE needs to analyze the resulting parsetrees to get the datatypes (and maybe atttypmod's) of the parameters, so EXECUTE QUERY can convert the values into Datum's using the types input functions. And the EXECUTE has to be handled special in tcop (it's something between a regular query and an utility statement). But it's not too hard to implement. Finally a FORGET QUERY identifier; (don't remember how the others named it) will remove the prepared plan etc. simply by destroying the memory context and dropping the identifier from the id->mcontext+prepareinfo mapping. This all restricts the usage of PREPARE to optimizable statements. Is it required to be able to prepare utility statements (like CREATE TABLE or so) too? Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #======================================== jwieck@debis.com (Jan Wieck) # From pgsql-hackers-owner+M67@postgresql.org Tue Oct 31 19:18:16 2000 Received: from mail.postgresql.org ([216.126.85.28]) by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id TAA08916 for ; Tue, 31 Oct 2000 19:18:15 -0500 (EST) Received: from mail.postgresql.org ([216.126.85.28]) by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA10IOl60635; Tue, 31 Oct 2000 19:18:24 -0500 (EST) (envelope-from pgsql-hackers-owner+M67@postgresql.org) Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4]) by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA10H8l60400 for ; Tue, 31 Oct 2000 19:17:08 -0500 (EST) (envelope-from zakkr@zf.jcu.cz) Received: from localhost (zakkr@localhost) by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id BAA32036; Wed, 1 Nov 2000 01:16:42 +0100 Date: Wed, 1 Nov 2000 01:16:42 +0100 (CET) From: Karel Zak To: Alfred Perlstein cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Query cache import? In-Reply-To: <20001031151144.F22110@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR On Tue, 31 Oct 2000, Alfred Perlstein wrote: > I never saw much traffic regarding Karel's work on making stored > proceedures: > > http://people.freebsd.org/~alfred/karel-pgsql.txt > > What happened with this? It looked pretty interesting. :( It's probably a little about me :-) ... well, My query cache is in usable state and it's efficient for all things those motivate me to work on this. some basic features: - share parsed plans between backends in shared memory - store plans to private backend hash table - use parameters for stored queries - better design for SPI - memory usage for saved plans - save plans "by key" The current query cache code depend on 7.1 memory management. After official 7.1 release I prepare patch with query cache+SPI (if not hit me over head, please ..) All what will doing next time not depend on me, *it's on code developers*. For example Jan has interesting idea about caching all plans which processing backend. But it's far future and IMHO we must go by small steps to Oracle's funeral :-) If I need the query cache in the my work (typical for some web+pgsql) or will some public interest I will continue on this, if not I freeze it. (Exists more interesting work like http://mape.jcu.cz ... sorry of advertising :-) Karel From pgsql-hackers-owner+M312@postgresql.org Mon Nov 6 03:27:32 2000 Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28]) by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA28404 for ; Mon, 6 Nov 2000 03:27:32 -0500 (EST) Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28]) by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA68Pos51966; Mon, 6 Nov 2000 03:25:50 -0500 (EST) (envelope-from pgsql-hackers-owner+M312@postgresql.org) Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4]) by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA68Fes50414 for ; Mon, 6 Nov 2000 03:15:40 -0500 (EST) (envelope-from zakkr@zf.jcu.cz) Received: from localhost (zakkr@localhost) by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id JAA20862; Mon, 6 Nov 2000 09:15:04 +0100 Date: Mon, 6 Nov 2000 09:15:04 +0100 (CET) From: Karel Zak To: Christof Petig cc: Zeugswetter Andreas SB , The Hermit Hacker , pgsql-hackers@postgresql.org Subject: Re: AW: [HACKERS] Re: [GENERAL] Query caching In-Reply-To: <3A02DDFF.E8CBFCF3@wtal.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR On Fri, 3 Nov 2000, Christof Petig wrote: > Karel Zak wrote: > > > On Thu, 2 Nov 2000, Zeugswetter Andreas SB wrote: > > > > > > > > > Well I can re-write and resubmit this patch. Add it as a > > > > compile time option > > > > is not bad idea. Second possibility is distribute it as patch > > > > in the contrib > > > > tree. And if it until not good tested not dirty with this main tree... > > > > > > > > Ok, I next week prepare it... > > > > > > One thing that worries me though is, that it extends the sql language, > > > and there has been no discussion about the chosen syntax. > > > > > > Imho the standard embedded SQL syntax (prepare ...) could be a > > > starting point. > > > > Yes, you are right... my PREPARE/EXECUTE is not too much ready to SQL92, > > I some old letter I speculate about "SAVE/EXECUTE PLAN" instead > > PREPARE/EXECUTE. But don't forget, it will *experimental* patch... we can > > change it in future ..etc. > > > > Karel > > [Sorry, I didn't look into your patch, yet.] Please, read my old query cache and PREPARE/EXECUTE description... > What about parameters? Normally you can prepare a statement and execute it We have in PG parameters, see SPI, but now it's used inside backend only and not exist statement that allows to use this feature in be<->fe. > using different parameters. AFAIK postgres' frontend-backend protocol is not > designed to take parameters for statements (e.g. like result presents > results). A very long road to go. > By the way, I'm somewhat interested in getting this feature in. Perhaps it > should be part of a protocol redesign (e.g. binary parameters/results). > Handling endianness is one aspect, floats are harder (but float->ascii->float > sometimes fails as well). PREPARE AS [ USING type, ... typeN ] [ NOSHARE | SHARE | GLOBAL ] EXECUTE [ INTO [ TEMPORARY | TEMP ] [ TABLE ] new_table ] [ USING val, ... valN ] [ NOSHARE | SHARE | GLOBAL ] DEALLOCATE PREPARE [ [ NOSHARE | SHARE | GLOBAL ]] [ ALL | ALL INTERNAL ] An example: PREPARE chris_query AS SELECT * FROM pg_class WHERE relname = $1 USING text; EXECUTE chris_query USING 'pg_shadow'; Or mean you something other? Karel From pgsql-hackers-owner+M444@postgresql.org Thu Nov 9 03:32:10 2000 Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28]) by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA09953 for ; Thu, 9 Nov 2000 03:32:09 -0500 (EST) Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28]) by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA98RSs11426; Thu, 9 Nov 2000 03:27:28 -0500 (EST) (envelope-from pgsql-hackers-owner+M444@postgresql.org) Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4]) by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA98OPs11045; Thu, 9 Nov 2000 03:24:25 -0500 (EST) (envelope-from zakkr@zf.jcu.cz) Received: from localhost (zakkr@localhost) by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id JAA08951; Thu, 9 Nov 2000 09:23:41 +0100 Date: Thu, 9 Nov 2000 09:23:41 +0100 (CET) From: Karel Zak To: Christof Petig cc: PostgreSQL Hackers , Michael Meskes , Zeugswetter Andreas SB , The Hermit Hacker Subject: Re: AW: [HACKERS] Re: [GENERAL] Query caching In-Reply-To: <3A096BCE.F9887955@wtal.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR On Wed, 8 Nov 2000, Christof Petig wrote: > Karel Zak wrote: > > > > What about parameters? Normally you can prepare a statement and execute it > > > > We have in PG parameters, see SPI, but now it's used inside backend only > > and not exist statement that allows to use this feature in be<->fe. > > Sad. Since ecpg would certainly benefit from this. > > > > using different parameters. AFAIK postgres' frontend-backend protocol is not > > > designed to take parameters for statements (e.g. like result presents > > > results). A very long road to go. > > > By the way, I'm somewhat interested in getting this feature in. Perhaps it > > > should be part of a protocol redesign (e.g. binary parameters/results). > > > Handling endianness is one aspect, floats are harder (but float->ascii->float > > > sometimes fails as well). > > > > PREPARE AS > > [ USING type, ... typeN ] > > [ NOSHARE | SHARE | GLOBAL ] > > > > EXECUTE > > [ INTO [ TEMPORARY | TEMP ] [ TABLE ] new_table ] > > [ USING val, ... valN ] > > [ NOSHARE | SHARE | GLOBAL ] > > > > DEALLOCATE PREPARE > > [ [ NOSHARE | SHARE | GLOBAL ]] > > [ ALL | ALL INTERNAL ] > > > > An example: > > > > PREPARE chris_query AS SELECT * FROM pg_class WHERE relname = $1 USING text; > > I would prefer '?' as a parameter name, since this is in the embedded sql standard > (do you have a copy of the 94 draft? I can mail mine to you?) This not depend on query cache. The '$n' is PostgreSQL query parametr keyword and is defined in standard parser. The PREPARE statement not parsing query it's job for standard parser. > Also the standard says a whole lot about guessing the parameter's type. > > Also I vote for ?::type or type(?) or sql's cast(...) (don't know it's syntax) > instead of abusing the using keyword. The postgresql executor expect types of parametrs in separate input (array). I not sure how much expensive/executable is survey it from query. > > EXECUTE chris_query USING 'pg_shadow'; > > Great idea of yours to implement this! Since I was thinking about implementing a > more decent schema for ecpg but had no mind to touch the backend and be-fe > protocol (yet). > It would be desirable to do an 'execute immediate using', since using input > parameters would take a lot of code away from ecpg. By the way, PREPARE/EXECUTE is face only. More interesting in this period is query-cache-kernel. SQL92 is really a little unlike my PREPARE/EXECUTE. Karel From pgsql-hackers-owner+M9563@postgresql.org Thu May 31 16:31:59 2001 Return-path: Received: from postgresql.org (webmail.postgresql.org [216.126.85.28]) by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f4VKVxc26942 for ; Thu, 31 May 2001 16:31:59 -0400 (EDT) Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28]) by postgresql.org (8.11.3/8.11.1) with SMTP id f4VKVIE38645; Thu, 31 May 2001 16:31:18 -0400 (EDT) (envelope-from pgsql-hackers-owner+M9563@postgresql.org) Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4]) by postgresql.org (8.11.3/8.11.1) with ESMTP id f4VKNVE35356 for ; Thu, 31 May 2001 16:23:31 -0400 (EDT) (envelope-from zakkr@zf.jcu.cz) Received: (from zakkr@localhost) by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) id WAA19957; Thu, 31 May 2001 22:23:26 +0200 Date: Thu, 31 May 2001 22:23:26 +0200 From: Karel Zak To: Roberto Abalde cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Cache for query plans Message-ID: <20010531222326.B16862@ara.zf.jcu.cz> References: <000701c0e932$d17646c0$c6023dc8@ultra> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii User-Agent: Mutt/1.0.1i In-Reply-To: <000701c0e932$d17646c0$c6023dc8@ultra>; from roberto.abalde@galego21.org on Wed, May 30, 2001 at 03:00:53PM -0300 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ORr On Wed, May 30, 2001 at 03:00:53PM -0300, Roberto Abalde wrote: > Hi, > > I need to implement a cache for query plans as part of my BSc thesis. Does > anybody know what happened to Karel Zak's patch? > Hi, my patch is on my ftp and nobody works on it, but I mean it's good begin for some next work. I not sure with implement this experimental patch (but usable) to official sources. For example Jan has more complex idea about query plan cache ... but first time we must solve some sub-problems like memory management in shared memory that is transparently for starndard routines like copy query plan ... and Tom isn't sure with query cache in shared memory...etc. Too much queries, but less answers :-) Karel > > PS: Sorry for my english :( Do you anytime read any my mail :-) Karel -- Karel Zak http://home.zf.jcu.cz/~zakkr/ C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)