From pgsql-hackers-owner+M22549@postgresql.org Wed May 8 11:22:40 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48FMd408780 for ; Wed, 8 May 2002 11:22:40 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 631C147696B; Wed, 8 May 2002 10:49:42 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 5DF8047648C; Wed, 8 May 2002 10:32:21 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 9BB684762F2 for ; Wed, 8 May 2002 10:32:09 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id D2693475C4C for ; Wed, 8 May 2002 10:15:56 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g48CZKt22122; Wed, 8 May 2002 08:35:20 -0400 Message-ID: <3CD91B88.38CD6C3@mohawksoft.com> Date: Wed, 08 May 2002 08:35:20 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: PostgreSQL-development , Jan Wieck , Tom Lane , "Marc G. Fournier" , Dann Corbit Subject: [HACKERS] Path to PostgreSQL portabiliy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Do we want a Win32 native version of PostgreSQL? The only reasons *not* to use Cygwin is licensing, installation hassles, and maybe stability or performance. Therefore, there is no strong technical reason to defend its removal, only a philosophical one. The debates on licensing on this list go on for weeks and people feel passionately about the subject. It seems odd that no one speaks out about the GNU requirement of cygwin. If there is a desire to create a PostgreSQL that is "fork" free, then we should do it now. If now strong desire exists, then we should make an entry in the FAQ and move on. If we want to be "portable" (and this should help us with a threading model later on) we need to cleanup all of the global variables. PostgreSQL's postmaster should not touch any global variables that are defined outside something like a pg_global structure and should not touch any static variables at all. If postmaster initializes a variable that will get cloned on a fork(), conceptually it is a shared global variable and belongs in pg_globals. Going all the way and replacing all globals and statics with a struct should allow threading with TLS. (Thread Local Storage) Port lib. Regardless where it comes from, the porting code should be a self contained library, not a list of objects. On Windows, a .DLL can do some things easier than an application. Also, having a library allows more flexibility as to how a port is designed. We should spec out our port interface. This includes file, semaphores, shared memory, signals/events, process control, IPC, system resources, etc. This will grow as we re-port to other environments like Windows. any comments? ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M22551@postgresql.org Wed May 8 11:41:42 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48Fff409057 for ; Wed, 8 May 2002 11:41:41 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 3D2974763EC; Wed, 8 May 2002 11:07:27 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 9FB764765CA; Wed, 8 May 2002 10:43:21 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6F12D47627E for ; Wed, 8 May 2002 10:43:12 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 1CC71475F1B for ; Wed, 8 May 2002 10:21:24 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g48EGGt22584; Wed, 8 May 2002 10:16:16 -0400 Message-ID: <3CD93330.2F3541D2@mohawksoft.com> Date: Wed, 08 May 2002 10:16:16 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Tom Lane cc: PostgreSQL-development , Jan Wieck , "Marc G. Fournier" , Dann Corbit Subject: Re: [HACKERS] Path to PostgreSQL portabiliy References: <3CD91B88.38CD6C3@mohawksoft.com> <1782.1020866623@sss.pgh.pa.us> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Tom Lane wrote: > > mlw writes: > > Port lib. Regardless where it comes from, the porting code should be a > > self contained library, not a list of objects. On Windows, a .DLL can > > do some things easier than an application. Also, having a library > > allows more flexibility as to how a port is designed. > > That may be necessary on Windoze, but on any other platform breaking out > an essential part of the backend as a library strikes me as a dead loss. > You create extra risk of installation mistakes, can't-find-library > startup failures, version mismatch problems, etc, etc --- for zero gain > that I can see. It does not need, and probably should not be by default, a shared library under UNIX. A static library is fine. The issue is whether or not it makes sense to try and design all porting layers the same, or allow the port engineer the flexibility to create what they need the way they need to do it. A side note: The "Windoze" comment says a lot Tom. Believe me, I am currently no fan of Windows, but there is something to be said about doing a good job supporting such a popular platform, regardless of our personal opinions. When I was working at DMN, I had to make sure we could find country music and Brittany Spears. Distasteful, but certainly something that needed to be done. IMHO, I think a great PostgreSQL implementation for Win32 is a nail in the coffin for Windows. If we give them a great database, which runs well under Windows, for free, MSSQL will now have a serious competitor for the medium to small marketplace. Once MSSQL has viable cross-platform competition in this space, one less requirement for Windows will exist. Right now, if you implement on Windows, you are most likely going to use MSSQL and be stuck there. With a good Win32 PostgreSQL, an engineer can implement on PostgreSQL for Windows, and easily move it to a "real" environment for stability. I see it as an important step. ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22561@postgresql.org Wed May 8 15:02:45 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48J2i411999 for ; Wed, 8 May 2002 15:02:45 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 75F024766AF; Wed, 8 May 2002 14:15:35 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id E4F5F476768; Wed, 8 May 2002 13:25:20 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id EF975476774 for ; Wed, 8 May 2002 13:22:11 -0400 (EDT) Received: from sss.pgh.pa.us (unknown [192.204.191.242]) by postgresql.org (Postfix) with ESMTP id 275C2476365 for ; Wed, 8 May 2002 11:57:17 -0400 (EDT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g48FvBv02684; Wed, 8 May 2002 11:57:11 -0400 (EDT) To: Thomas Lockhart cc: mlw , PostgreSQL-development , Jan Wieck , "Marc G. Fournier" , Dann Corbit Subject: Re: [HACKERS] Path to PostgreSQL portabiliy In-Reply-To: <3CD9461E.1FDC6344@fourpalms.org> References: <3CD91B88.38CD6C3@mohawksoft.com> <3CD9461E.1FDC6344@fourpalms.org> Comments: In-reply-to Thomas Lockhart message dated "Wed, 08 May 2002 08:37:02 -0700" Date: Wed, 08 May 2002 11:57:11 -0400 Message-ID: <2681.1020873431@sss.pgh.pa.us> From: Tom Lane Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Thomas Lockhart writes: > 2) If (1) does not exempt the PostgreSQL app from GPL polution, then why > not distribute PostgreSQL on Windows using a GPL license? Given the cygwin licensing terms stated at http://cygwin.com/licensing.html it appears to me that we need not open that can of worms (and I'd much rather not muddy the licensing waters that way, regardless of any arguments about whether it would hurt or not...) As near as I can tell, we *could* develop a self-contained installation package for PG+cygwin without any licensing problem. So that set of problems could be solved with a reasonable amount of work. I'm still unclear on whether there are serious technical problems (performance, stability) with using cygwin. (Actually, even if there are performance or stability problems, an easily-installable package would still address the needs of people who want to "try it out" or "get their feet wet". And maybe that's all we need to do. We always have said that we recommend a Unix platform for production-grade PG installations, and IMNSHO that recommendation would not change one iota if there were a native rather than cygwin-based Windows port. So I'm unconvinced that we have a problem to solve anyway...) regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22572@postgresql.org Wed May 8 15:21:17 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48JLG412576 for ; Wed, 8 May 2002 15:21:16 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id BA544476CFE; Wed, 8 May 2002 14:58:31 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 516A14768DC; Wed, 8 May 2002 14:15:14 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5EFF147646E for ; Wed, 8 May 2002 13:41:38 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id EDE53475B42 for ; Wed, 8 May 2002 12:34:15 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g48GT7t23160; Wed, 8 May 2002 12:29:07 -0400 Message-ID: <3CD95253.45C01EC0@mohawksoft.com> Date: Wed, 08 May 2002 12:29:07 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Thomas Lockhart cc: Tom Lane , PostgreSQL-development , Jan Wieck , "Marc G. Fournier" , Dann Corbit Subject: Re: [HACKERS] Path to PostgreSQL portabiliy References: <3CD91B88.38CD6C3@mohawksoft.com> <3CD9461E.1FDC6344@fourpalms.org> <2681.1020873431@sss.pgh.pa.us> <3CD94E04.AF837261@fourpalms.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Thomas Lockhart wrote: > > ... > > As near as I can tell, we *could* develop a self-contained installation > > package for PG+cygwin without any licensing problem. > > Right. That was my opinion also. But istm that however the discussion > settles out, there is a path to success. These last couple days have really started me thinking about Windows again. I developed Windows software for over a decade, geez much longer than that, I wrote my first Windows program using the Windows 1.03 SDK. (I am in a 12 step program now, but you guys are causing a relapse!) Listen, here is purely my opinion on the matter, I am speaking from my experience as a Windows user, developer, and author (Tricks of the Windows 3.1 Masters). It is useless to spend serious time on a cygwin version. Yea, it is cool and all, but it won't be used. From the eyes of a Windows user cygwin is a hack and a mess. An IT guy that only knows Windows will never use it, and if presented with a program that forces a UNIX like directory tree on their hard drive and UNIX like tools to manage it, they will delete the program and curse the time spent installing it. Performance may also be an issue, I don't know for sure, but it is suspected. The cygwin fork troubles me as well. It may work, but I would not call it a "production" technique, how about you? Would you bet your business on cygwin and a hacked fork()? No matter what steps you take, cygwin will not be seen by Windows users as anything but a sloppy/messy/horrible hack. It is a fact of life. You are welcome to disagree, but I assure you it is true. >From a usefulness perspective, a cygwin version of PostgreSQL will be nothing more than a proof of concept, a test bed, or a demo. It will never be used as a serious database. How much work does that warrant? ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22576@postgresql.org Wed May 8 17:06:18 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48L6I414548 for ; Wed, 8 May 2002 17:06:18 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id BA4F7475E25; Wed, 8 May 2002 17:06:16 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 60B724766C8; Wed, 8 May 2002 16:04:42 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id CC22D4764C2 for ; Wed, 8 May 2002 15:57:12 -0400 (EDT) Received: from www.wgcr.org (www.wgcr.org [206.74.232.194]) by postgresql.org (Postfix) with ESMTP id E24B8476509 for ; Wed, 8 May 2002 14:58:13 -0400 (EDT) Received: from localhost.localdomain ([10.1.2.103]) by www.wgcr.org (8.9.3/8.9.3/WGCR) with ESMTP id OAA26136; Wed, 8 May 2002 14:58:09 -0400 Content-Type: text/plain; charset="iso-8859-1" From: Lamar Owen To: Thomas Lockhart , mlw Subject: Re: [HACKERS] Path to PostgreSQL portabiliy Date: Wed, 8 May 2002 14:49:39 -0400 User-Agent: KMail/1.4.1 cc: PostgreSQL-development , Jan Wieck , Tom Lane , "Marc G. Fournier" , Dann Corbit References: <3CD91B88.38CD6C3@mohawksoft.com> <3CD9461E.1FDC6344@fourpalms.org> In-Reply-To: <3CD9461E.1FDC6344@fourpalms.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID: <200205081449.39473.lamar.owen@wgcr.org> Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO On Wednesday 08 May 2002 11:37 am, Thomas Lockhart wrote: > 1) cygwin is licensed under GPL. So is GNU/Linux, which provides the > same APIs as cygwin does. Linux does not pollute application licenses, > presumably because Linux itself is not *required* to run the The Linux kernel is not under a pure GPL. COPYING in the kernel source says this, prepended to the GPL: NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does *not* fall under the heading of "derived work". Also note that the GPL below is copyrighted by the Free Software Foundation, but the instance of code that it refers to (the Linux kernel) is copyrighted by me and others who actually wrote it. Also note that the only valid version of the GPL as far as the kernel is concerned is _this_ particular version of the license (ie v2, not v2.2 or v3.x or whatever), unless explicitly otherwise stated. Linus Torvalds -------------------------------------------------------------------------- Does cygwin make the same statement? > 2) If (1) does not exempt the PostgreSQL app from GPL polution, then why > not distribute PostgreSQL on Windows using a GPL license? [snip] > 3) If (2) is the case, then development could continue under the BSD > license, since developers could use the BSD-original code for their > development work. So there is no risk of "backflow polution". Can PostgreSQL, Inc be the GPL distributor for these purposes, being a separate entity from the PostgreSQL Global Development Group? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M22580@postgresql.org Wed May 8 17:34:55 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48LYs414954 for ; Wed, 8 May 2002 17:34:54 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 389864764D9; Wed, 8 May 2002 17:25:43 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id D4E77476647; Wed, 8 May 2002 17:03:04 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 8860F4763EF for ; Wed, 8 May 2002 17:02:53 -0400 (EDT) Received: from pd2mo3so.prod.shaw.ca (h24-71-223-10.cg.shawcable.net [24.71.223.10]) by postgresql.org (Postfix) with ESMTP id 5A0764765E6 for ; Wed, 8 May 2002 15:53:59 -0400 (EDT) Received: from pd4mr2so.prod.shaw.ca (pd4mr2so-qfe3.prod.shaw.ca [10.0.141.213]) by l-daemon (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GVT009HA5Y0UG@l-daemon> for pgsql-hackers@postgresql.org; Wed, 08 May 2002 13:54:00 -0600 (MDT) Received: from pn2ml5so.prod.shaw.ca (pn2ml5so-qfe0.prod.shaw.ca [10.0.121.149]) by l-daemon (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GVT00GCO5Y0ZV@l-daemon> for pgsql-hackers@postgresql.org; Wed, 08 May 2002 13:54:00 -0600 (MDT) Received: from refractions.net (h24-65-178-198.gv.shawcable.net [24.65.178.198]) by l-daemon (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GVT00GCZ5XZ1F@l-daemon> for pgsql-hackers@postgresql.org; Wed, 08 May 2002 13:54:00 -0600 (MDT) Date: Wed, 08 May 2002 12:53:57 -0700 From: Paul Ramsey Subject: Re: [HACKERS] Path to PostgreSQL portabiliy To: mlw cc: PostgreSQL-development Message-ID: <3CD98255.4B52751@refractions.net> MIME-Version: 1.0 X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.17 i586) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7BIT X-Accept-Language: en References: <3CD91B88.38CD6C3@mohawksoft.com> <3CD9461E.1FDC6344@fourpalms.org> <2681.1020873431@sss.pgh.pa.us> <3CD94E04.AF837261@fourpalms.org> <3CD95253.45C01EC0@mohawksoft.com> Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO mlw wrote: > > No matter what steps you take, cygwin will not be seen by Windows users as > anything but a sloppy/messy/horrible hack. It is a fact of life. You are > welcome to disagree, but I assure you it is true. Just to clarify here: is it confirmed that having the complete cygwin distribution is a necessary condition to having a running PostgreSQL on windows? Is it not possible that, having built postgresql with the full cygwin, it would be possible to make a nice clean setup.exe package which bundles the postgresql executables, the required cygwin dlls and other niceties into an easy install package? Given that, I do not think your putative windows user would care at all about what was going on under the covers. As long as the install was clean, there were utilities (pgadmin?) to start working with the database right away, and things "just worked", the ugliness (or exquisite symmetry... I am not an expert) of the fork() implementation really would not be an issue :) Of course, an imaginary beautiful packaging regime hinges on the possibility of bundling the cygwin api libraries cleanly without bundling all the rest of the cygwin scruft (unix directory heirarchy, etc etc). Anyone have any light to shed on cygwin's "packagability"? P. ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22581@postgresql.org Wed May 8 17:58:25 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48LwP415342 for ; Wed, 8 May 2002 17:58:25 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id CC3F3476BD3; Wed, 8 May 2002 17:36:38 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id F00F5476833; Wed, 8 May 2002 17:09:20 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id D372D4764F1 for ; Wed, 8 May 2002 17:09:09 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id BC6DA4759FC for ; Wed, 8 May 2002 16:17:28 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g48KCHt24008; Wed, 8 May 2002 16:12:17 -0400 Message-ID: <3CD986A1.9DC78DD2@mohawksoft.com> Date: Wed, 08 May 2002 16:12:17 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Paul Ramsey cc: PostgreSQL-development Subject: Re: [HACKERS] Path to PostgreSQL portabiliy References: <3CD91B88.38CD6C3@mohawksoft.com> <3CD9461E.1FDC6344@fourpalms.org> <2681.1020873431@sss.pgh.pa.us> <3CD94E04.AF837261@fourpalms.org> <3CD95253.45C01EC0@mohawksoft.com> <3CD98255.4B52751@refractions.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Paul Ramsey wrote: > > mlw wrote: > > > > No matter what steps you take, cygwin will not be seen by Windows users as > > anything but a sloppy/messy/horrible hack. It is a fact of life. You are > > welcome to disagree, but I assure you it is true. > > Just to clarify here: is it confirmed that having the complete cygwin > distribution is a necessary condition to having a running PostgreSQL on > windows? Is it not possible that, having built postgresql with the full > cygwin, it would be possible to make a nice clean setup.exe package > which bundles the postgresql executables, the required cygwin dlls and > other niceties into an easy install package? Given that, I do not think > your putative windows user would care at all about what was going on > under the covers. As long as the install was clean, there were utilities > (pgadmin?) to start working with the database right away, and things > "just worked", the ugliness (or exquisite symmetry... I am not an > expert) of the fork() implementation really would not be an issue :) Windows users expect to have C:\my programs\postgres as the install location. A person who has used or looked at MSSQL would expect to deal with the real file system. The cygwin environment shields the UNIX program from Windows, the Windows user would expect the program to deal with the system as is. The Windows user that would install PostgreSQL would expect it to be a real windows program, but would be savvy enough (and prejudiced enough) to know if it weren't. ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22585@postgresql.org Wed May 8 19:23:09 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g48NN8416625 for ; Wed, 8 May 2002 19:23:08 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 7823A475A2E; Wed, 8 May 2002 19:22:50 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 19B3C4768DA; Wed, 8 May 2002 18:52:18 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 260C14767D9 for ; Wed, 8 May 2002 18:52:05 -0400 (EDT) Received: from temp.joelburton.com (unknown [209.190.238.101]) by postgresql.org (Postfix) with ESMTP id EF620475C14 for ; Wed, 8 May 2002 18:09:39 -0400 (EDT) Received: from joeltf0d13cyyp (unknown [209.190.238.100]) by temp.joelburton.com (Postfix) with SMTP id F16DF2B95F; Wed, 8 May 2002 18:14:05 -0400 (EDT) From: "Joel Burton" To: "Paul Ramsey" , "mlw" cc: "PostgreSQL-development" Subject: Re: [HACKERS] Path to PostgreSQL portabiliy Date: Wed, 8 May 2002 18:09:41 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3CD98255.4B52751@refractions.net> Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO > -----Original Message----- > From: pgsql-hackers-owner@postgresql.org > [mailto:pgsql-hackers-owner@postgresql.org]On Behalf Of Paul Ramsey > Sent: Wednesday, May 08, 2002 3:54 PM > To: mlw > Cc: PostgreSQL-development > Subject: Re: [HACKERS] Path to PostgreSQL portabiliy > > > mlw wrote: > > > > No matter what steps you take, cygwin will not be seen by > Windows users as > > anything but a sloppy/messy/horrible hack. It is a fact of life. You are > > welcome to disagree, but I assure you it is true. > > Just to clarify here: is it confirmed that having the complete cygwin > distribution is a necessary condition to having a running PostgreSQL on > windows? Is it not possible that, having built postgresql with the full > cygwin, it would be possible to make a nice clean setup.exe package > which bundles the postgresql executables, the required cygwin dlls and > other niceties into an easy install package? Given that, I do not think > your putative windows user would care at all about what was going on > under the covers. As long as the install was clean, there were utilities > (pgadmin?) to start working with the database right away, and things > "just worked", the ugliness (or exquisite symmetry... I am not an > expert) of the fork() implementation really would not be an issue :) > > Of course, an imaginary beautiful packaging regime hinges on the > possibility of bundling the cygwin api libraries cleanly without > bundling all the rest of the cygwin scruft (unix directory heirarchy, > etc etc). Anyone have any light to shed on cygwin's "packagability"? Certainly, we don't need all of cygwin (eg bison, gcc, perl, et al). We'd need the dll, sh, rm, and few other things. I'm not sure if it would need to be in the standard cygwin file structure; I know that you can reconfigure this when you use cygwin (I used to). In any event, instead of having to have a novice pick & guess which of >100 packages they need, we could put together the 5 or 6 they need. I'm not sure I agree entirely with mlw: some Windows admins will be afraid of cygwin, but, I'll bet more than a few won't even notice that its being used (especially if we can change the dir names, provide windows shortcuts to the commands like initdb, create database, pg_ctl, etc., which would be trivial to do). Still unanswered is real data on whether cygwin would be good for serious production use by real people. However, for the test/play/try-out model, I think cygwin would be a fine solution, and wouldn't (shouldn't?) require too much work. - J. ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22603@postgresql.org Thu May 9 03:24:18 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g497OG421663 for ; Thu, 9 May 2002 03:24:17 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id BD8C94762B1; Thu, 9 May 2002 03:24:11 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id A799C475F77; Thu, 9 May 2002 03:23:37 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 82716476077 for ; Thu, 9 May 2002 03:23:11 -0400 (EDT) Received: from smtp6.jaring.my (smtp6.jaring.my [61.6.32.56]) by postgresql.org (Postfix) with ESMTP id AB0DD475A35 for ; Thu, 9 May 2002 03:21:53 -0400 (EDT) Received: from pc.mecomb.com (j202.crc20.jaring.my [61.6.157.216]) by smtp6.jaring.my (8.11.4/8.11.4) with ESMTP id g497LYl12649; Thu, 9 May 2002 15:21:35 +0800 (MYT) Message-ID: <5.1.0.14.1.20020509143654.036387b0@192.228.128.13> X-Sender: lyeoh@192.228.128.13 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 09 May 2002 15:34:06 +0800 To: mlw , Lee Kindness From: Lincoln Yeoh Subject: Re: [HACKERS] Path to PostgreSQL portabiliy cc: PostgreSQL-development In-Reply-To: <3CD949FA.86BD95D2@mohawksoft.com> References: <3CD91B88.38CD6C3@mohawksoft.com> <15577.17903.180863.251963@kelvin.csl.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Who really is your target "market" on the windows platform? Microsoft Access users (many)? MySQL users(insignificant?)? MSSQL (many)? Assuming that the postgresql team isn't getting lots of money or resources to do it. I don't see why you would want to invest a lot to support windows from a long term point of view. Windows can be a costly platform to support. Because if you become a serious threat, Microsoft can rip the rug from beneath you any chance they get. Also Microsoft WILL always change their APIs. They're not stupid. If Microsoft freezes their APIs they will end up like "yet another BIOS manufacturer", and bye bye profit margins. Microsoft will strive to keep it a proprietary AND changing API. Windows is rather different operationally. Automating vacuum etc on windows is going to be different. Starting postgresql as a service is going to be different as well. Same for uninstalling. So support requests are going to be different. If your target market is consumer - Windows consumer users also have different expectations. Most will want nicer GUIs (those that don't care won't mind running Postgresql elsewhere). BTW if your target market is a bit higher end - typically those that "must use" windows also "must use" MSSQL/Oracle/etc. You will thus have to build brand recognition for Postgresql on Windows. All this will cost you. That said, is it easier to support only Windows NT/2000 and forget about Win9x? The bigger dbs don't support win9x either (how does Oracle/DB2 support NT? They seem to work ok). Leave MySQL to the Win9x people ;). BTW does MySQL really perform OK on Win9x? Forget the Cygwin approach. Is there really a market for that? Unless things have got a lot easier, installing Cygwin is like installing a new O/S just to install your app. And installing and learning a new system has got to be one of the major barriers, otherwise people will either buy a new USD500 1.5+ GHz pc or use VMware+BSD/Linux+Postgresql ;). Cheerio, Link. At 11:53 AM 5/8/02 -0400, mlw wrote: >writing software for over 20 years now, and sometimes you just have to hold >your nose. It would be nice if we could code what we want, the way we want, in >the language we want, on the platforms we want. > >Windows represents a HUGE user base, it also represents a platform for which a >real good native PostgreSQL should do well. There are, to my knowledge, no >good >and free databases available for Windows. > >PostgreSQL on Windows could be very cool as a serious poster child for why >open-source is the way to go. ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22605@postgresql.org Thu May 9 03:42:59 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g497gw422047 for ; Thu, 9 May 2002 03:42:58 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id A01394762EE; Thu, 9 May 2002 03:42:53 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id E51C14762AD; Thu, 9 May 2002 03:42:47 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6547E475A24 for ; Thu, 9 May 2002 03:42:19 -0400 (EDT) Received: from salem.vale-housing.co.uk (mailgate.vale-housing.co.uk [193.195.77.162]) by postgresql.org (Postfix) with SMTP id 2D2A5475A19 for ; Thu, 9 May 2002 03:42:18 -0400 (EDT) Subject: Re: [HACKERS] Path to PostgreSQL portabiliy Date: Thu, 9 May 2002 08:42:18 +0100 Message-ID: <214E9C0A75426D47A876A2FD8A07426E9747@salem.vale-housing.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Thread-Topic: Path to PostgreSQL portabiliy Thread-Index: AcH251A9jOrpk8IETTmDgJDVmp55WQAAClTQABEg5nA= From: "Dave Page" To: "Dann Corbit" , "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g497gw422047 Status: RO > -----Original Message----- > From: Dann Corbit [mailto:DCorbit@connx.com] > Sent: 09 May 2002 00:31 > To: PostgreSQL-development > Subject: Re: Path to PostgreSQL portabiliy > > > If you have a Win32 workstation... > Look here: > http://sources.redhat.com/cygwin/ > > Then click on the thing that says "Install Now" (Looks like a > black "C" with a green tongue). > > after a small boatload of clicks, you will see a Window > labeled "Cygwin Setup". Under +All you will find... > +Admin > +Archive > +Base > +Database > > Click on the plus sign next to the Database category. > > You will see: > 7.2.1-1 [options] [Bin] [Src] [Package] posgresql: > PostgreSQL Data Base Management System > > In other words, they already have an automated installation > procedure for PostgreSQL if you are using Cygwin. The last time I tried that (coupla months ago) it listed the versions of the packages in reverse order, so I spent about 15 very tedious minutes making sure that I have the latest version of all the packages I wanted selected. Then I spent an hour or 2 battling with ntsec and initdb on my laptop (logged onto, but disconnected from the domain). After that I gave up and went back to my very old release that works fine. The point I'm trying to make is that if I, as a not inexperienced sysadmin of both Windows and Unix systems (not to mention PostgreSQL which I like to think I'm fairly familiar with) has this trouble, what impression is that going to give the first time user, who's probably going to go elsewhere at the first sign of trouble? Regards, Dave. ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M22628@postgresql.org Thu May 9 10:26:16 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49EQF406536 for ; Thu, 9 May 2002 10:26:16 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id A60614764BC; Thu, 9 May 2002 10:26:09 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 5A7754763DA; Thu, 9 May 2002 10:13:35 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 31877476414 for ; Thu, 9 May 2002 10:13:23 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 2BD714764C9 for ; Thu, 9 May 2002 10:10:25 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g49E53t28604; Thu, 9 May 2002 10:05:03 -0400 Message-ID: <3CDA820F.136D65F5@mohawksoft.com> Date: Thu, 09 May 2002 10:05:03 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Jan Wieck cc: Tom Lane , "Marc G. Fournier" , PostgreSQL-development Subject: Re: [HACKERS] How much work is a native Windows application? References: <200205091344.g49DiBp01273@saturn.janwieck.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Jan Wieck wrote: > > Tom Lane wrote: > > "Marc G. Fournier" writes: > > > On Tue, 7 May 2002, Tom Lane wrote: > > >> It'd be worth trying to understand cygwin issues in detail before we > > >> sign up to do and support a native Windows port. > > > > > Actually, there are licensing issues involved ... we could never put a > > > 'windows binary' up for anon-ftp, since to distribute it would require the > > > cygwin.dll to be distributed, and to do that, there is a licensing cost > > > ... of course, I guess we could require ppl to download cygwin seperately, > > > install that, then install the binary over top of that ... > > > > <> And how much development time are we supposed to expend to > > avoid that? > > > > Give me a technical case for avoiding Cygwin, and maybe I can get > > excited about it. I'm not planning to lift a finger on the basis > > of licensing though... after all, Windows users are accustomed to > > paying for software, no? > > Nobody asked you to lift any of your fingers. A few people > (including me) just see value in a native Windows port, > kicking out the Cygwin requirement. > > I have the impression you never did use Cygwin. I did, thanks > but no thanks. I have used the cygwin version too. It is a waste of time. No Windows user will ever accept it. No windows-only user is going to use the cygwin tools. From a production stand point, would anyone reading this trust their data to PostgreSQL running on cygwin? Think about it, if you wouldn't, why would anyone else. I think, and I know people are probably sick of me spouting opinions, that if you want a Windows presence for PostgreSQL, then we should write a real Win32 version. If the global/static variables which are initialized by the postmaster are moved to a structure, we can should be able to remove the fork() requirement and port to a Win32 native system. ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22630@postgresql.org Thu May 9 10:37:12 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49EbB409822 for ; Thu, 9 May 2002 10:37:11 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 8ADB347661B; Thu, 9 May 2002 10:37:02 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 834AC4765D2; Thu, 9 May 2002 10:32:37 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id B04C04763ED for ; Thu, 9 May 2002 10:32:26 -0400 (EDT) Received: from sss.pgh.pa.us (unknown [192.204.191.242]) by postgresql.org (Postfix) with ESMTP id 858BD47644B for ; Thu, 9 May 2002 10:25:44 -0400 (EDT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g49EPhW19333; Thu, 9 May 2002 10:25:43 -0400 (EDT) To: mlw cc: Jan Wieck , "Marc G. Fournier" , PostgreSQL-development Subject: Re: [HACKERS] How much work is a native Windows application? In-Reply-To: <3CDA820F.136D65F5@mohawksoft.com> References: <200205091344.g49DiBp01273@saturn.janwieck.net> <3CDA820F.136D65F5@mohawksoft.com> Comments: In-reply-to mlw message dated "Thu, 09 May 2002 10:05:03 -0400" Date: Thu, 09 May 2002 10:25:43 -0400 Message-ID: <19330.1020954343@sss.pgh.pa.us> From: Tom Lane Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO mlw writes: > I have used the cygwin version too. It is a waste of time. No Windows user will > ever accept it. No windows-only user is going to use the cygwin tools. With decent packaging, no windows-only user would even know we have cygwin in there. The above argument is just plain irrelevant. The real point is that we need a nice clean friendly GUI for both installation and administration --- and AFAICS that will take about the same amount of work to write whether the server requires cygwin internally or not. Rather than expending largely-pointless work on internal rewrites of the server, people who care about this issue ought to be thinking about the GUI problems. > From a production stand point, would anyone reading this trust their > data to PostgreSQL running on cygwin? I wouldn't trust my data to *any* database running on a Microsoft OS. Period. The above argument thus doesn't impress me at all, especially when it's being made without offering a shred of evidence that cygwin contributes any major degree of instability. I am especially unhappy about the prospect of major code revisions and development time spent on chasing this rather than improving our performance and stability on Unix-type OSes. I agree with the comment someone else made: that's just playing Microsoft's game. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22634@postgresql.org Thu May 9 11:12:12 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49FCB419783 for ; Thu, 9 May 2002 11:12:11 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id DCF27475C4C; Thu, 9 May 2002 11:12:06 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 5CAC8476567; Thu, 9 May 2002 11:08:52 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 582F94762CF for ; Thu, 9 May 2002 11:08:41 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 807C3475C8E for ; Thu, 9 May 2002 11:00:17 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g49Et1t28775; Thu, 9 May 2002 10:55:01 -0400 Message-ID: <3CDA8DC5.A76330A1@mohawksoft.com> Date: Thu, 09 May 2002 10:55:01 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Tom Lane cc: Jan Wieck , "Marc G. Fournier" , PostgreSQL-development Subject: Re: [HACKERS] How much work is a native Windows application? References: <200205091344.g49DiBp01273@saturn.janwieck.net> <3CDA820F.136D65F5@mohawksoft.com> <19330.1020954343@sss.pgh.pa.us> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Tom Lane wrote: > With decent packaging, no windows-only user would even know we have > cygwin in there. The above argument is just plain irrelevant. The real > point is that we need a nice clean friendly GUI for both installation > and administration --- and AFAICS that will take about the same amount of > work to write whether the server requires cygwin internally or not. Can a cygwin version of PostgreSQL see the native file system, like: C:\My Database, D:\postgres? > > From a production stand point, would anyone reading this trust their > > data to PostgreSQL running on cygwin? > > I wouldn't trust my data to *any* database running on a Microsoft OS. That is a prejudice that is affecting your judgment. Many people do trust Windows, do I? No, but a lot of people do. People have trusted their businesses on Windows NT/2K/XP, many are still doing so. We want these people to use PostgreSQL, so when they see the error in their ways, they have a way out. > The above argument thus doesn't impress me at all, especially > when it's being made without offering a shred of evidence that cygwin > contributes any major degree of instability. >From a software development standpoint, I am VERY uncomfortable with the technique of a user space program copying its writeable memory to another process's. It may work until Microsoft changes something with the next version of IE. What about anti-virus software, cygwin has problems with them, and you have to have anti-virus software on Windows. On top of that, the time spent copying the whole process is too long, and it forces real memory to be allocated and initialized at process startup. So, the cygwin fork() will cause PostgreSQL to be slower and use more memory than a native version, and will not co-exist well with anti-virus software. > > I am especially unhappy about the prospect of major code revisions > and development time spent on chasing this rather than improving our > performance and stability on Unix-type OSes. I agree with the comment > someone else made: that's just playing Microsoft's game. Maybe is is playing "Microsoft's Game" but the end result will be a program that can seriously compete with MSSQL on Windows, and provide a REAL migration path to UNIX. Many developers use MSSQL because they "have it" in MSDN, so to them, it is free. Once they develop something using it, they are tied to Windows. When it comes time to deploy their pet project, the company has to cough up the price of the server. A native, friendly, Win32 PostgreSQL that works the same on Windows as it does on FreeBSD, Linux, Solaris, etc. Will offer the developer real options away from Windows. Also: I don't think it needs to be a major rewrite, no strategy needs to change, it is basically renaming variables, i.e. my_global_var becomes pg_globals.my_global_var. Once that is done, a port writer can do what ever they need to do to get that structure to the child correctly. As an exercise, I bet if we did this, we would find bugs which are lurking, as yet unfound. Besides, the discipline of using a globals structure will improve the code base. Don't you agree? ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M22635@postgresql.org Thu May 9 11:26:08 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49FQ7422749 for ; Thu, 9 May 2002 11:26:07 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 2AD70475C8E; Thu, 9 May 2002 11:26:03 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 73B2E4764F7; Thu, 9 May 2002 11:23:55 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 53F84475FE2 for ; Thu, 9 May 2002 11:23:46 -0400 (EDT) Received: from myst.fourpalms.org (www.fourpalms.org [64.3.68.148]) by postgresql.org (Postfix) with ESMTP id 5269D47626D for ; Thu, 9 May 2002 11:13:16 -0400 (EDT) Received: from fourpalms.org (localhost.localdomain [127.0.0.1]) by myst.fourpalms.org (Postfix) with ESMTP id 254F4201196; Thu, 9 May 2002 08:13:18 -0700 (PDT) Message-ID: <3CDA920D.9AF8F9CF@fourpalms.org> Date: Thu, 09 May 2002 08:13:17 -0700 From: Thomas Lockhart Organization: Yes X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.8-34.1mdk i686) X-Accept-Language: en MIME-Version: 1.0 To: mlw cc: Lee Kindness , Jan Wieck , Tom Lane , "Marc G. Fournier" , PostgreSQL-development Subject: Re: [HACKERS] How much work is a native Windows application? References: <29206.1020833367@sss.pgh.pa.us> <200205091344.g49DiBp01273@saturn.janwieck.net> <15578.33799.700751.945380@kelvin.csl.co.uk> <3CDA866D.25E7FADF@mohawksoft.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO ... > PostgreSQL's feature set and price ($0), with a good installer, would do VERY > well. That may be (I'd like to think so!). We've identified at least a couple of barriers to folks running PostgreSQL on Windows. The installer and GUI issue needs to be solved no matter what, and we *could* have a version running on Windows with just those things in place. imho if we are going down the path, we need to take the first steps. And those do *not* require code rewrites to do so (or at least don't appear to). If we had a package available for Windows -- with some developers such as yourself supporting it -- then we could talk about putting more resources into supporting that platform better. But the perception of at least some of the key developers (including myself) is that *if* we did the code rewrite, and *if* we spent the effort to end up as a native on Windows, then we *very well might* be an unreliable database on an unreliable platform. istm that getting a well packaged system running now, then being able to identify *only cygwin* as the barrier to better reliability would get more support for changes in the backend code. And if we were working toward some ability to do threading anyway (I don't see that in the near future, but we've talked in the past about structuring the query engine around "tuple sources" which could then be distributed across threads or across machines) then maybe the next step is easier. My 2c... - Thomas ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M22644@postgresql.org Thu May 9 12:57:33 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49GvX424180 for ; Thu, 9 May 2002 12:57:33 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id B60CA47643B; Thu, 9 May 2002 12:57:27 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 841634766DB; Thu, 9 May 2002 12:50:50 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5B98D476528 for ; Thu, 9 May 2002 12:50:36 -0400 (EDT) Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by postgresql.org (Postfix) with SMTP id D4DB2475AD5 for ; Thu, 9 May 2002 12:41:12 -0400 (EDT) Received: from psc.progress.com (HELO saturn.janwieck.net) (janwieck@192.233.92.200 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 9 May 2002 16:41:17 -0000 Received: (from wieck@localhost) by saturn.janwieck.net (8.11.2/8.11.2) id g49GbGx01621; Thu, 9 May 2002 12:37:16 -0400 From: Jan Wieck Message-ID: <200205091637.g49GbGx01621@saturn.janwieck.net> Subject: Re: [HACKERS] How much work is a native Windows application? In-Reply-To: <3CDA820F.136D65F5@mohawksoft.com> from mlw at "May 9, 2002 10:05:03 am" To: mlw Date: Thu, 9 May 2002 12:37:16 -0400 (EDT) cc: Jan Wieck , Tom Lane , "Marc G. Fournier" , PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO mlw wrote: > I think, and I know people are probably sick of me spouting opinions, that if > you want a Windows presence for PostgreSQL, then we should write a real Win32 > version. > > If the global/static variables which are initialized by the postmaster are > moved to a structure, we can should be able to remove the fork() requirement > and port to a Win32 native system. My opinion here is that until May 1998 Postgres did exec(), so it was clean and okay for CreateProcess() up to then. Just because we optimized it for the copy-on-write behaviour, modern Unix kernels do with fork() only, is NO reason to accept sloppy coding. The Postmaster and the backend have different responsibilities. In fact, I still consider them beeing different programs even if they reside in one executable. Mixing global variables of one with the other is wrong. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com # ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22662@postgresql.org Thu May 9 14:47:23 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49IlM425877 for ; Thu, 9 May 2002 14:47:22 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id C6581475F6B; Thu, 9 May 2002 14:47:21 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id D1B6D476703; Thu, 9 May 2002 14:42:52 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id DB6884766AF for ; Thu, 9 May 2002 14:42:40 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 3726B47649E for ; Thu, 9 May 2002 14:28:24 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g49IN2t29643; Thu, 9 May 2002 14:23:02 -0400 Message-ID: <3CDABE86.D38370F4@mohawksoft.com> Date: Thu, 09 May 2002 14:23:02 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: cbbrowne@cbbrowne.com cc: PostgreSQL-development Subject: Re: [HACKERS] How much work is a native Windows application? References: <200205091344.g49DiBp01273@saturn.janwieck.net> <3CDA820F.136D65F5@mohawksoft.com> <20020509170233.2DC5A38CB3@cbbrowne.com> <3CDAB0B3.C7375A12@mohawksoft.com> <20020509174604.89E5938D1A@cbbrowne.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO cbbrowne@cbbrowne.com wrote: > > > cbbrowne@cbbrowne.com wrote: > >>> I think, and I know people are probably sick of me spouting > >>> opinions, that if you want a Windows presence for PostgreSQL, then > >>> we should write a real Win32 version. > >> > >> The crucial wrong word is the word "we." > > >> If _you_ want a Windows presence, then _you_ should write a real > >> Win32 version. That clearly attaches responsibility to someone who > >> is interested. > > > I have already said that I am willing to write the pieces for a > > Windows port. The issue is changes in PostgreSQL required to do it. > > No, I don't think you understand. > > If you're planning to do a port, then _all_ changes are your > responsibility. Nobody ought to need to change PostgreSQL in order for > you to write a Windows port; that, in fact, would be a waste of time, > having several people working on something that should probably be done > by one person. Without buy-in from the group, there is no point in me wasting my time doing all the work necessary. I'm not interested in making Mark's special version of PostgreSQL. If we can agree on a strategy and a course, then it is worth doing. If all the changes made fall on the floor because the group does not like them, then I wasted my time. Got it? Also, doing the Windows portions of the code will represent a significant investment of my time. I'm not interested in doing a lot of work on a shoddy project. If you ask the core group to put out a crappy version of PostgreSQL for a UNIX, they would fight long and hard against it. Why should we be willing to produce a crappy version for Windows, just because the people here don't like Windows. I don't care about Solaris, but I understand WHY it is important to make PostgreSQL work well on it. I don't understand why the people in this group don't see the same purpose for a Windows port. To be honest, I think a good Windows port will do wonders for PostgreSQL's acceptance. ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M22680@postgresql.org Thu May 9 17:57:11 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49LvA428333 for ; Thu, 9 May 2002 17:57:10 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 348D7476690; Thu, 9 May 2002 17:57:01 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 64C1747675C; Thu, 9 May 2002 17:56:24 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id CF8E1476162 for ; Thu, 9 May 2002 17:56:13 -0400 (EDT) Received: from rh72.home.ee (adsl1030.estpak.ee [213.168.29.11]) by postgresql.org (Postfix) with ESMTP id F0DEF475F1A for ; Thu, 9 May 2002 17:55:09 -0400 (EDT) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by rh72.home.ee (8.11.6/8.11.6) with ESMTP id g49JrRc02592; Fri, 10 May 2002 00:53:28 +0500 Subject: Re: [HACKERS] How much work is a native Windows application? From: Hannu Krosing To: Tom Lane cc: mlw , Jan Wieck , "Marc G. Fournier" , PostgreSQL-development In-Reply-To: <19330.1020954343@sss.pgh.pa.us> References: <200205091344.g49DiBp01273@saturn.janwieck.net> <3CDA820F.136D65F5@mohawksoft.com> <19330.1020954343@sss.pgh.pa.us> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.4 Date: 10 May 2002 00:53:27 +0500 Message-ID: <1020974008.2080.65.camel@rh72.home.ee> MIME-Version: 1.0 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO On Thu, 2002-05-09 at 19:25, Tom Lane wrote: > mlw writes: > > I have used the cygwin version too. It is a waste of time. No Windows user will > > ever accept it. No windows-only user is going to use the cygwin tools. > > With decent packaging, no windows-only user would even know we have > cygwin in there. The above argument is just plain irrelevant. The real > point is that we need a nice clean friendly GUI for both installation > and administration --- and AFAICS that will take about the same amount of > work to write whether the server requires cygwin internally or not. We can go the Oracle way and write a 200MB cross-platform java installer requiring and exact version of java runtime > Rather than expending largely-pointless work on internal rewrites of > the server, people who care about this issue ought to be thinking about > the GUI problems. pgAccess is quite nice (Disclaimer: I'm not a windows weenie, I run it inside vmware/win98 IE browser test environment on my Linux workstation ;). Why not just bundle what we've got ? > > From a production stand point, would anyone reading this trust their > > data to PostgreSQL running on cygwin? > > I wouldn't trust my data to *any* database running on a Microsoft OS. > Period. Do we support Xenix and SCO ? > The above argument thus doesn't impress me at all, especially > when it's being made without offering a shred of evidence that cygwin > contributes any major degree of instability. >From the comments here it seems to be either cygwin or more likely cygipc > I am especially unhappy about the prospect of major code revisions > and development time spent on chasing this rather than improving our > performance and stability on Unix-type OSes. I agree with the comment > someone else made: that's just playing Microsoft's game. Not! I think that this thread is mostly about coordinating code and interface cleanups that are likely beneficial for both *NIX and non-*NIX platforms mainly * cleaner support for semaphores * separating shared and per-process data * process creation * (file operations) * (init and service scripts) if done properly none of these will degrade code quality nor performance. Also, having a clean interface for those will not only enable any interested party to make windows/BeOS/OSX/QNX binaries with less effort, it will most likely make it easier make use of advances in *NIX world like AIO, multiprocessor systems, NUMA and distributed systems, and just make things more robust and reliable by making code inspection easier. --------------- Hannu ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M22694@postgresql.org Thu May 9 21:02:16 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4A12F400739 for ; Thu, 9 May 2002 21:02:15 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id F206D47693C; Thu, 9 May 2002 21:01:55 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 7D8704768C8; Thu, 9 May 2002 21:01:21 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 932C8475D9D for ; Thu, 9 May 2002 21:01:10 -0400 (EDT) Received: from smtp001.bizmail.yahoo.com (smtp001.bizmail.yahoo.com [216.136.172.125]) by postgresql.org (Postfix) with SMTP id 3159C475E90 for ; Thu, 9 May 2002 20:59:55 -0400 (EDT) Received: from unknown (HELO egutierrez) (eg@eldergriffon.org@205.216.186.2 with login) by smtp1.bm.vip.sc5.yahoo.com with SMTP; 10 May 2002 00:59:59 -0000 Message-ID: <00f701c1f7bd$ffdeddc0$8500005a@egutierrez> Reply-To: "Ernesto Gutierrez" From: "Ernesto Gutierrez" To: "mlw" , "Tom Lane" cc: "Jan Wieck" , "Marc G. Fournier" , "PostgreSQL-development" References: <200205091344.g49DiBp01273@saturn.janwieck.net> <3CDA820F.136D65F5@mohawksoft.com> <19330.1020954343@sss.pgh.pa.us> Subject: Re: [HACKERS] How much work is a native Windows application? Date: Thu, 9 May 2002 17:59:55 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Tom Lane writes: > mlw writes: > > I have used the cygwin version too. It is a waste of time. No Windows user will > > ever accept it. No windows-only user is going to use the cygwin tools. > > With decent packaging, no windows-only user would even know we have > cygwin in there. The above argument is just plain irrelevant. The real > point is that we need a nice clean friendly GUI for both installation > and administration --- and AFAICS that will take about the same amount of > work to write whether the server requires cygwin internally or not. I'm afraid I agree with mlw, Tom. I don't think the problem ends at the GUI, although for many people it would. The issue extends at least also to support and troubleshooting. In a production environment, I have a better chance of figuring out what's going wrong with an application written natively for an operating system dealing directly with that operating system. I would take a dim view of using PostgreSQL running on cygwin unless I had extensive experience doing it, or if there were no other alternative. > > From a production stand point, would anyone reading this trust their > > data to PostgreSQL running on cygwin? > > I wouldn't trust my data to *any* database running on a Microsoft OS. > Period. The above argument thus doesn't impress me at all, especially > when it's being made without offering a shred of evidence that cygwin > contributes any major degree of instability. If you could prove to me that cygwin doesn't contribute *any* instability, I'd still be pretty worried, probably for the same reasons that you don't trust any Microsoft OS. There are increased chances that something could go critically wrong, particularly in an environment fundamentally different. I think mlw's basic point is quite valid, that PG+cygwin will not ever find favor with decision-makers who are used to Windows systems. Suspicion of the other environment's foibles is common, and goes both ways. > I am especially unhappy about the prospect of major code revisions > and development time spent on chasing this rather than improving our > performance and stability on Unix-type OSes. I agree with the comment > someone else made: that's just playing Microsoft's game. There I don't deny you may be right. Ernie Gutierrez Walnut Creek, CA ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22642@postgresql.org Thu May 9 12:49:06 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49Gn5424077 for ; Thu, 9 May 2002 12:49:05 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id D130647648A; Thu, 9 May 2002 12:48:55 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 4AB694766F0; Thu, 9 May 2002 12:37:35 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id D272E4765B6 for ; Thu, 9 May 2002 12:37:19 -0400 (EDT) Received: from barry.xythos.com (sdsl-216-36-77-241.dsl.sjc.megapath.net [216.36.77.241]) by postgresql.org (Postfix) with ESMTP id 1C87E476482 for ; Thu, 9 May 2002 12:34:59 -0400 (EDT) Received: from xythos.com (localhost.localdomain [127.0.0.1]) by barry.xythos.com (8.11.6/8.11.6) with ESMTP id g49GGZg09296; Thu, 9 May 2002 09:16:35 -0700 Message-ID: <3CDAA0E3.5060706@xythos.com> Date: Thu, 09 May 2002 09:16:35 -0700 From: Barry Lind User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020417 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Tom Lane cc: "Henshall, Stuart - WCP" , Thomas Lockhart , mlw , PostgreSQL-development , Jan Wieck , "Marc G. Fournier" , Dann Corbit , Joel Burton Subject: Re: [HACKERS] PG+Cygwin Production Experience (was RE: Path to PostgreSQL References: <19035.1020952312@sss.pgh.pa.us> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO I have found this whole thread very interesting (I'm still not sure where it is going though :-). But let me throw in some of my thoughts. A windows version of postgres (whether native of cygwin based) is important. I have many developers with windows as their desktop OS and they have a postgres db installed to do development work. Postgres on cygwin is fine for this need. While I may not trust it in a production environment it is certainly good enough for development. A second use we have for postgres on windows is in evals of our product. We provide an eval version of our software as an InstallShield installed .exe that includes our code, postgres and the necessary cygwin parts. People doing evals just want to install the eval on their everyday machine (most likely running windows) and it needs to be dead simple to install. This can be done with postgres and cygwin. In this example again the current postgres+cygwin works well enough for our evals. Again I wouldn't run the production version in this environment, but it is good enough for an eval. Our eval does show that it is possible to repackage postgres plus the parts of cygwin it needs into a nice installer and have it work. (It is a lot of work but is certainly possible). In fact in our eval install we even use cygrunsrv to install postgres as a windows service. The biggest problem we have had is the fact that the utility scripts (like pg_ctl, createdb, etc) are all shell scripts that call a whole host of other utilities. It is pretty straight forward to package up the postgres executable and the libraries it needs from cygwin. It is a whole different problem making sure you have a standard unix style shell environment with all the utilities installed so that you can run the shell scripts. thanks, --Barry Tom Lane wrote: > "Henshall, Stuart - WCP" writes: > >>Cygwin is not the only additon needed, cygipc will also be needed (GPL) >>(see: http://www.neuro.gatech.edu/users/cwilson/cygutils/cygipc/index.html ) > > > Good point, but is this a requirement that we could get rid of, now that > we have the SysV IPC stuff somewhat isolated? AFAICT cygipc provides > the SysV IPC API (shmget, semget, etc) --- but if there are usable > equivalents in the basic Cygwin environment, we could probably use them > now. > > Considering how often we see the forgot-to-start-cygipc mistake, > removing this requirement would be a clear win. > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly > ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22665@postgresql.org Thu May 9 15:30:11 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49JUA426599 for ; Thu, 9 May 2002 15:30:10 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 2FCB8475B68; Thu, 9 May 2002 15:29:59 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 0739B47669C; Thu, 9 May 2002 15:28:49 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 4074C47642C for ; Thu, 9 May 2002 15:28:37 -0400 (EDT) Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by postgresql.org (Postfix) with SMTP id 48B34476481 for ; Thu, 9 May 2002 15:24:32 -0400 (EDT) Received: from psc.progress.com (HELO saturn.janwieck.net) (janwieck@192.233.92.200 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 9 May 2002 19:24:32 -0000 Received: (from wieck@localhost) by saturn.janwieck.net (8.11.2/8.11.2) id g49JKN402145; Thu, 9 May 2002 15:20:23 -0400 From: Jan Wieck Message-ID: <200205091920.g49JKN402145@saturn.janwieck.net> Subject: Re: [HACKERS] PG+Cygwin Production Experience (was RE: Path to PostgreSQL In-Reply-To: <3CDAA0E3.5060706@xythos.com> from Barry Lind at "May 9, 2002 09:16:35 am" To: Barry Lind Date: Thu, 9 May 2002 15:20:23 -0400 (EDT) cc: Tom Lane , "Henshall, Stuart - WCP" , Thomas Lockhart , mlw , PostgreSQL-development , Jan Wieck , "Marc G. Fournier" , Dann Corbit , Joel Burton X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Barry Lind wrote: > I have found this whole thread very interesting (I'm still not sure > where it is going though :-). But let me throw in some of my thoughts. > > A windows version of postgres (whether native of cygwin based) is > important. I have many developers with windows as their desktop OS and > they have a postgres db installed to do development work. Postgres on > cygwin is fine for this need. While I may not trust it in a production > environment it is certainly good enough for development. > > A second use we have for postgres on windows is in evals of our product. > We provide an eval version of our software as an InstallShield > installed .exe that includes our code, postgres and the necessary cygwin > parts. People doing evals just want to install the eval on their > everyday machine (most likely running windows) and it needs to be dead > simple to install. This can be done with postgres and cygwin. In this > example again the current postgres+cygwin works well enough for our > evals. Again I wouldn't run the production version in this environment, > but it is good enough for an eval. > > Our eval does show that it is possible to repackage postgres plus the > parts of cygwin it needs into a nice installer and have it work. (It is > a lot of work but is certainly possible). In fact in our eval install > we even use cygrunsrv to install postgres as a windows service. > > The biggest problem we have had is the fact that the utility scripts > (like pg_ctl, createdb, etc) are all shell scripts that call a whole > host of other utilities. It is pretty straight forward to package up > the postgres executable and the libraries it needs from cygwin. It is a > whole different problem making sure you have a standard unix style shell > environment with all the utilities installed so that you can run the > shell scripts. Do I read this right? You wrap the binary eval version of your product, the binary PostgreSQL and CygWin including some of the utility programs into one InstallShield .exe and ship that? Hmmm, PostgreSQL's license is totally fine with that. And your program is your program. But as far as I know bundling with CygWin like this costs money. So you pay license fees to RedHat for shipping eval copies of your product and don't see any value in a native Windows port? Nobel, nobel, or does your product have such an outstanding eval/sell ratio that it doesn't matter? Jan > > thanks, > --Barry > > Tom Lane wrote: > > "Henshall, Stuart - WCP" writes: > > > >>Cygwin is not the only additon needed, cygipc will also be needed (GPL) > >>(see: http://www.neuro.gatech.edu/users/cwilson/cygutils/cygipc/index.html ) > > > > > > Good point, but is this a requirement that we could get rid of, now that > > we have the SysV IPC stuff somewhat isolated? AFAICT cygipc provides > > the SysV IPC API (shmget, semget, etc) --- but if there are usable > > equivalents in the basic Cygwin environment, we could probably use them > > now. > > > > Considering how often we see the forgot-to-start-cygipc mistake, > > removing this requirement would be a clear win. > > > > regards, tom lane > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 3: if posting/reading through Usenet, please send an appropriate > > subscribe-nomail command to majordomo@postgresql.org so that your > > message can get through to the mailing list cleanly > > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com # ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M22643@postgresql.org Thu May 9 12:55:39 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49Gtc424128 for ; Thu, 9 May 2002 12:55:38 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 8604C4759A4; Thu, 9 May 2002 12:55:11 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id E12EE4766BF; Thu, 9 May 2002 12:50:48 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 69ED347650F for ; Thu, 9 May 2002 12:50:35 -0400 (EDT) Received: from mail1.ihs.com (unknown [170.207.70.222]) by postgresql.org (Postfix) with ESMTP id CA565475A6C for ; Thu, 9 May 2002 12:40:18 -0400 (EDT) Received: from css120.ihs.com (css120.ihs.com [170.207.105.120]) by mail1.ihs.com (8.12.3/8.12.3) with ESMTP id g49GdO1j016176 for ; Thu, 9 May 2002 10:39:24 -0600 (MDT) Date: Thu, 9 May 2002 10:34:58 -0600 (MDT) From: Scott Marlowe To: PostgreSQL-development Subject: [HACKERS] Issues tangential to win32 support In-Reply-To: <3CDA820F.136D65F5@mohawksoft.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-MailScanner: Found to be clean Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO There are some issues that the whole idea of a win32 port should bring up. One of them is whether or not postgresql should be rewritten as a multi-threaded app. If postgresql will never be rewritten as a multi-threaded app, then performance under Windows is likely to ALWAYS be slow, since that multi-thread is the preferred model for good performance on W32. note that many Unixes prefer multi-threaded models as well (Solaris comes to mind) so there's the possibility that a multi-threaded postgresql could enjoy better performance on more than just windows. If postgresql IS going to eventually be multi-threaded, then the whole win32 port should probably be delayed until then, since it would solve many of the issues of fork() versus createprocess(). Just my thoughts on it. Scott ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M22660@postgresql.org Thu May 9 14:31:54 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49IVs425698 for ; Thu, 9 May 2002 14:31:54 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 3CD18476584; Thu, 9 May 2002 14:31:53 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id C48014767E8; Thu, 9 May 2002 14:24:13 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5C1B34762A6 for ; Thu, 9 May 2002 14:24:00 -0400 (EDT) Received: from mail1.ihs.com (unknown [170.207.70.222]) by postgresql.org (Postfix) with ESMTP id 0BF97476700 for ; Thu, 9 May 2002 14:19:32 -0400 (EDT) Received: from css120.ihs.com (css120.ihs.com [170.207.105.120]) by mail1.ihs.com (8.12.3/8.12.3) with ESMTP id g49IIQsp027731; Thu, 9 May 2002 12:18:26 -0600 (MDT) Date: Thu, 9 May 2002 12:13:59 -0600 (MDT) From: Scott Marlowe To: Jan Wieck cc: PostgreSQL-development Subject: Re: [HACKERS] Issues tangential to win32 support In-Reply-To: <200205091751.g49HpTi01846@saturn.janwieck.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-MailScanner: Found to be clean Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO On Thu, 9 May 2002, Jan Wieck wrote: > > If postgresql IS going to eventually be multi-threaded, then the whole > > win32 port should probably be delayed until then, since it would solve > > many of the issues of fork() versus createprocess(). > > If multi-threading is the plan, then there is light at the > end of the tunnel ... the upcoming train... That's a bit extreme don't you think? I'm not fan of multi-threading as the one true way, and since I use linux as my server for postgresql, there is no gain for me in a multi-threaded model. In fact, I'd prefer postgresql stay multi-process for robustness. BUT, if there are plans to go multi-thread, or could be plans, then those should take priority over how to port to windows, since making postgresql multi-threaded will change it so much as to make the current "how do we port to windows" thread meaningless. One of the primary reasons given for avoiding cygwin is that postgresql runs so slowly under it on windows, but I submit that it probably won't run a heck of a lot faster if it was written as a native app as long as it's running as a multi-process design. Since there's probably no great gain to be had from moving it out from under cygwin, why bother? My vote would be to stay multi-process and Unix compatible. I have no real use for windows as a server, and do NOT want to sacrifice the performance / reliability I have with postgresql under Linux for a windows port. ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M22683@postgresql.org Thu May 9 18:20:10 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49MK9428615 for ; Thu, 9 May 2002 18:20:10 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 723B04767BA; Thu, 9 May 2002 18:19:59 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id B32AF476783; Thu, 9 May 2002 18:19:19 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 1ED46475EFF for ; Thu, 9 May 2002 18:19:06 -0400 (EDT) Received: from rh72.home.ee (adsl1030.estpak.ee [213.168.29.11]) by postgresql.org (Postfix) with ESMTP id 1234E475B2D for ; Thu, 9 May 2002 18:16:40 -0400 (EDT) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by rh72.home.ee (8.11.6/8.11.6) with ESMTP id g49KF2c02624; Fri, 10 May 2002 01:15:06 +0500 Subject: Re: [HACKERS] Issues tangential to win32 support From: Hannu Krosing To: Dann Corbit cc: PostgreSQL-development Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.4 Date: 10 May 2002 01:15:02 +0500 Message-ID: <1020975306.2451.76.camel@rh72.home.ee> MIME-Version: 1.0 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO On Fri, 2002-05-10 at 02:33, Dann Corbit wrote: > > It took a few hundred man hours to do it. About 2-8 weeks for one full time programmer ? > I see the whole Win32 port as > a non issue. Several parties have already completed it (including the > place where I work -- CONNX Solutions Inc.). If we did not do it or all > parties who already did it were hit by a comet or something, someone > else would accomplish it. It isn't trivial but it isn't impossible > either. If a need is large enough, someone will manage it. The need is > large enough. Ergo... Do you know which of these run ((reasonably) well) on win9x ? > Here are some other things related: > > A ready to go Win32 PosgreSQL package: > http://www.dbexperts.net/postgresql Perhaps we should back up and let dbexperts et.al. recover their costs and after that repent and commit changes back to main tree ;) ## insert a little ad-hominem attack to everyone objecting a native ## win32 port as owning stock in some win32-pg-selling company BTW, do they have an evaluation version or do they think that people would in that case evaluate on win32 and then buy a cheap linux box for $495.- :) -------------------- Hannu ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22676@postgresql.org Thu May 9 17:36:50 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49Lan428029 for ; Thu, 9 May 2002 17:36:49 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 2AB3947673E; Thu, 9 May 2002 17:36:47 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id BA90247671E; Thu, 9 May 2002 17:35:47 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id AA57D47660B for ; Thu, 9 May 2002 17:35:37 -0400 (EDT) Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) by postgresql.org (Postfix) with ESMTP id 87D3F475A41 for ; Thu, 9 May 2002 17:34:39 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [HACKERS] Issues tangential to win32 support content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 Date: Thu, 9 May 2002 14:33:40 -0700 Message-ID: Thread-Topic: [HACKERS] Issues tangential to win32 support Thread-Index: AcH3nsEyepyPj/gRTNiZtJugpq+PAQAAduvQ From: "Dann Corbit" To: "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g49Lan428029 Status: RO > -----Original Message----- > From: Hannu Krosing [mailto:hannu@tm.ee] > Sent: Thursday, May 09, 2002 12:10 PM > To: Jan Wieck > Cc: Scott Marlowe; PostgreSQL-development > Subject: Re: [HACKERS] Issues tangential to win32 support > > > On Thu, 2002-05-09 at 22:51, Jan Wieck wrote: > > Scott Marlowe wrote: > > > There are some issues that the whole idea of a win32 port > should bring up. > > > One of them is whether or not postgresql should be rewritten as a > > > multi-threaded app. > > > > Please, don't add this one to it. > > > > I'm all for the native Windows port, yes, but I've discussed > > the multi-thread questions for days at Great Bridge, then > > again with my new employer, with people on shows and whatnot. > > > > Anything in the whole backend is designed with a multi- > > process model in mind. You'll not do that in any reasonable > > amount of time. > > IIRC you are replying to the man who _has_ actually don this ? > > Perhaps using an unreasonable amount of time but still ... :) It took a few hundred man hours to do it. I see the whole Win32 port as a non issue. Several parties have already completed it (including the place where I work -- CONNX Solutions Inc.). If we did not do it or all parties who already did it were hit by a comet or something, someone else would accomplish it. It isn't trivial but it isn't impossible either. If a need is large enough, someone will manage it. The need is large enough. Ergo... Here are some other things related: A ready to go Win32 PosgreSQL package: http://www.dbexperts.net/postgresql An open source project to productize PostgreSQL for Windows (has gone nowhere so far): http://gborg.postgresql.org/project/winpackage/projdisplay.php A native Win32 port accomplished by a Japanese Group: http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html If you look under the "Gists for Patch" it contains exactly the same tasks that CONNX Solutions Inc. had to accomplish in every case. ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M22677@postgresql.org Thu May 9 17:48:03 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49Lm3428149 for ; Thu, 9 May 2002 17:48:03 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 0F0CF475A41; Thu, 9 May 2002 17:48:00 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id B847047670D; Thu, 9 May 2002 17:47:08 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id EAE1E475A41 for ; Thu, 9 May 2002 17:46:57 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 527374766C2 for ; Thu, 9 May 2002 17:46:55 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g49LfTt30402; Thu, 9 May 2002 17:41:37 -0400 Message-ID: <3CDAED09.E890975B@mohawksoft.com> Date: Thu, 09 May 2002 17:41:29 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Dann Corbit cc: PostgreSQL-development Subject: Re: [HACKERS] Issues tangential to win32 support References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Dann Corbit wrote: > It took a few hundred man hours to do it. I see the whole Win32 port as > a non issue. Several parties have already completed it (including the > place where I work -- CONNX Solutions Inc.). If we did not do it or all > parties who already did it were hit by a comet or something, someone > else would accomplish it. It isn't trivial but it isn't impossible > either. If a need is large enough, someone will manage it. The need is > large enough. Ergo... > > Here are some other things related: > > A ready to go Win32 PosgreSQL package: > http://www.dbexperts.net/postgresql > > An open source project to productize PostgreSQL for Windows (has gone > nowhere so far): > http://gborg.postgresql.org/project/winpackage/projdisplay.php > > A native Win32 port accomplished by a Japanese Group: > http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html > If you look under the "Gists for Patch" it contains exactly the same > tasks that CONNX Solutions Inc. had to accomplish in every case. These packages are based upon cygwin. Problems with cygwin: (1) GNU license issues. (2) Does not work well with anti-virus software (3) Since OS level copy-on-write is negated, process creation is much slower. (4) Since OS level copy on write is negated, memory that otherwise would not be allocated to the process is forced to ba allocated when the parent process data is copied. ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M22679@postgresql.org Thu May 9 17:53:53 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49Lrq428243 for ; Thu, 9 May 2002 17:53:53 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id AF15A4766A3; Thu, 9 May 2002 17:53:48 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 9C26F4766EC; Thu, 9 May 2002 17:53:08 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 8F95D475D60 for ; Thu, 9 May 2002 17:52:57 -0400 (EDT) Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) by postgresql.org (Postfix) with ESMTP id A2590475C8C for ; Thu, 9 May 2002 17:52:56 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [HACKERS] Issues tangential to win32 support content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 Date: Thu, 9 May 2002 14:51:50 -0700 Message-ID: Thread-Topic: Issues tangential to win32 support Thread-Index: AcH3oxZJFXJ0zN07QyKdZT4UsSj7LQAAB7dA From: "Dann Corbit" To: "mlw" cc: "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g49Lrq428243 Status: RO > -----Original Message----- > From: mlw [mailto:markw@mohawksoft.com] > Sent: Thursday, May 09, 2002 2:41 PM > To: Dann Corbit > Cc: PostgreSQL-development > Subject: Re: Issues tangential to win32 support > > > Dann Corbit wrote: > > It took a few hundred man hours to do it. I see the whole > Win32 port as > > a non issue. Several parties have already completed it > (including the > > place where I work -- CONNX Solutions Inc.). If we did not > do it or all > > parties who already did it were hit by a comet or something, someone > > else would accomplish it. It isn't trivial but it isn't impossible > > either. If a need is large enough, someone will manage it. > The need is > > large enough. Ergo... > > > > Here are some other things related: > > > > A ready to go Win32 PosgreSQL package: > > http://www.dbexperts.net/postgresql > > > > An open source project to productize PostgreSQL for Windows > (has gone > > nowhere so far): > > http://gborg.postgresql.org/project/winpackage/projdisplay.php > > > > A native Win32 port accomplished by a Japanese Group: > > http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html > > If you look under the "Gists for Patch" it contains exactly the same > > tasks that CONNX Solutions Inc. had to accomplish in every case. > > These packages are based upon cygwin. Problems with cygwin: > > (1) GNU license issues. > (2) Does not work well with anti-virus software > (3) Since OS level copy-on-write is negated, process creation > is much slower. > (4) Since OS level copy on write is negated, memory that > otherwise would not be > allocated to the process is forced to ba allocated when the > parent process data > is copied. Our package avoids Cygwin altogether. We wrote our own POSIX layer from scratch, and we junked fork() for CreateProcess() {and inserted copious: #ifdef ICKY_WIN32_KLUDGE /* our code goes here */ #else /* Standard UNIX code goes here */ #endif It's complete, and it performs like the burning blue blazes. We have run the full PostgreSQL test suite to completion with success. However, before we release any SQL tool, we have our own test suite with tens of thousands of tests to perform. Hence, we won't have a release until June at the earliest. I think the Japanese one also does not use Cygwin (but I have not tried installing it yet). ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22681@postgresql.org Thu May 9 18:02:06 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49M26428426 for ; Thu, 9 May 2002 18:02:06 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6E81C476693; Thu, 9 May 2002 18:02:03 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 69A68476733; Thu, 9 May 2002 18:01:30 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id A3AC1475A7B for ; Thu, 9 May 2002 18:01:16 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 739E74759A5 for ; Thu, 9 May 2002 18:01:15 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g49Lu1t30454; Thu, 9 May 2002 17:56:01 -0400 Message-ID: <3CDAF071.CF3DAB58@mohawksoft.com> Date: Thu, 09 May 2002 17:56:01 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Dann Corbit cc: PostgreSQL-development Subject: Re: [HACKERS] Issues tangential to win32 support References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Dann Corbit wrote: > Our package avoids Cygwin altogether. We wrote our own POSIX layer from > scratch, and we junked fork() for CreateProcess() {and inserted copious: > #ifdef ICKY_WIN32_KLUDGE > /* our code goes here */ > #else > /* Standard UNIX code goes here */ > #endif OK, what sorts of things did you do in your ICKY_WIN32_KLUDGE? Were they ever migrated back into the main tree? Did you simulate fork() or a stand-alone? I know Windows very well, but I have thus far remained ignorant of PostgreSQL internals. > > It's complete, and it performs like the burning blue blazes. We have > run the full PostgreSQL test suite to completion with success. However, > before we release any SQL tool, we have our own test suite with tens of > thousands of tests to perform. Hence, we won't have a release until > June at the earliest. > > I think the Japanese one also does not use Cygwin (but I have not tried > installing it yet). The japanese site claims cygwin. ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M22682@postgresql.org Thu May 9 18:17:03 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49MH3428574 for ; Thu, 9 May 2002 18:17:03 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 745F04765DF; Thu, 9 May 2002 18:16:59 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id C342F47678A; Thu, 9 May 2002 18:13:59 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5AF8D475FBD for ; Thu, 9 May 2002 18:13:42 -0400 (EDT) Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) by postgresql.org (Postfix) with ESMTP id 1612E47621B for ; Thu, 9 May 2002 18:11:46 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [HACKERS] Issues tangential to win32 support content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 Date: Thu, 9 May 2002 15:10:43 -0700 Message-ID: Thread-Topic: Issues tangential to win32 support Thread-Index: AcH3pRcvhdfkS6Z+SXCu6lfzMZH4qAAAK5Cw From: "Dann Corbit" To: "mlw" cc: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g49MH3428574 Status: RO > -----Original Message----- > From: mlw [mailto:markw@mohawksoft.com] > Sent: Thursday, May 09, 2002 2:56 PM > To: Dann Corbit > Cc: PostgreSQL-development > Subject: Re: Issues tangential to win32 support > > > Dann Corbit wrote: > > Our package avoids Cygwin altogether. We wrote our own > POSIX layer from > > scratch, and we junked fork() for CreateProcess() {and > inserted copious: > > #ifdef ICKY_WIN32_KLUDGE > > /* our code goes here */ > > #else > > /* Standard UNIX code goes here */ > > #endif > > OK, what sorts of things did you do in your > ICKY_WIN32_KLUDGE? Were they ever > migrated back into the main tree? Did you simulate fork() or > a stand-alone? I explained it in another mail. We had quite a few changes we had to make (several hundred man-hours, about half of which was the POSIX layer and the precise time routines). No sense trying to simulate fork() -- it stinks on Win32. The Cygwin and PW32 implementations of fork() are dogs. Smarter folks than us tried it and failed miserably. Why reinvent a broken wheel? We use create process and our own startup code. Our version is competitive with fork() on Linux for spawning tasks and in general the queries run considerably faster. > I know Windows very well, but I have thus far remained > ignorant of PostgreSQL > internals. ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M22684@postgresql.org Thu May 9 18:34:44 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49MYi428719 for ; Thu, 9 May 2002 18:34:44 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id BC4F84759C3; Thu, 9 May 2002 18:34:41 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 6AD90476737; Thu, 9 May 2002 18:33:51 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6926D475B2D for ; Thu, 9 May 2002 18:33:41 -0400 (EDT) Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) by postgresql.org (Postfix) with ESMTP id 7563E4759C3 for ; Thu, 9 May 2002 18:33:40 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [HACKERS] Issues tangential to win32 support content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 Date: Thu, 9 May 2002 15:31:14 -0700 Message-ID: Thread-Topic: Issues tangential to win32 support Thread-Index: AcH3pRcvhdfkS6Z+SXCu6lfzMZH4qAAA0jKA From: "Dann Corbit" To: "mlw" cc: "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g49MYi428719 Status: RO > -----Original Message----- > From: mlw [mailto:markw@mohawksoft.com] > Sent: Thursday, May 09, 2002 2:56 PM > To: Dann Corbit > Cc: PostgreSQL-development > Subject: Re: Issues tangential to win32 support > > > Dann Corbit wrote: > > Our package avoids Cygwin altogether. We wrote our own > POSIX layer from > > scratch, and we junked fork() for CreateProcess() {and > inserted copious: > > #ifdef ICKY_WIN32_KLUDGE > > /* our code goes here */ > > #else > > /* Standard UNIX code goes here */ > > #endif > > OK, what sorts of things did you do in your > ICKY_WIN32_KLUDGE? Were they ever > migrated back into the main tree? Did you simulate fork() or > a stand-alone? > > I know Windows very well, but I have thus far remained > ignorant of PostgreSQL > internals. > > > > > It's complete, and it performs like the burning blue > blazes. We have > > run the full PostgreSQL test suite to completion with > success. However, > > before we release any SQL tool, we have our own test suite > with tens of > > thousands of tests to perform. Hence, we won't have a release until > > June at the earliest. > > > > I think the Japanese one also does not use Cygwin (but I > have not tried > > installing it yet). > > The japanese site claims cygwin. This is not correct. (Fortunately, we have someone here who reads and writes Japanese). At any rate, it is a complete, native implementation of PostgreSQL without any need for Cygwin. Just to be sure, I did a "depends" on each of the binaries and none of them use Cywin. So the Japanese site did exactly the same thing that we did. Here are bitmaps showing the complete dependency trees of both the Japanese efforts and ours as well: Us: ftp://cap.connx.com/pub/chess-engines/new-approach/connx.bmp Japanese: ftp://cap.connx.com/pub/chess-engines/new-approach/japanese.bmp No Cygwin in sight in either case. ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22685@postgresql.org Thu May 9 18:39:40 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49Mde428784 for ; Thu, 9 May 2002 18:39:40 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6F0264767C5; Thu, 9 May 2002 18:39:37 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 9916F476790; Thu, 9 May 2002 18:39:19 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id E3B4C475F09 for ; Thu, 9 May 2002 18:39:05 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id C52C3475B2D for ; Thu, 9 May 2002 18:39:04 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g49MXot30610; Thu, 9 May 2002 18:33:50 -0400 Message-ID: <3CDAF94E.2F60382E@mohawksoft.com> Date: Thu, 09 May 2002 18:33:50 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Dann Corbit cc: PostgreSQL-development Subject: Re: [HACKERS] Issues tangential to win32 support References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Dann Corbit wrote: http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html Mentions cygwin, am I misunderstanding? Does not matter, the issue is that you guys said you did it. OK, have you been able to bring the changed back into the main source tree? (Are you not trying?) ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M22686@postgresql.org Thu May 9 18:46:24 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49MkN428851 for ; Thu, 9 May 2002 18:46:23 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 32CEF476649; Thu, 9 May 2002 18:46:00 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 232D6476760; Thu, 9 May 2002 18:45:39 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id A983647642B for ; Thu, 9 May 2002 18:45:28 -0400 (EDT) Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) by postgresql.org (Postfix) with ESMTP id 782BD475F09 for ; Thu, 9 May 2002 18:44:49 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [HACKERS] Issues tangential to win32 support content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 Date: Thu, 9 May 2002 15:43:47 -0700 Message-ID: Thread-Topic: Issues tangential to win32 support Thread-Index: AcH3ql/GnGAJD4NhSHug0+KSRuGWRAAAAz7Q From: "Dann Corbit" To: "mlw" cc: "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g49MkN428851 Status: RO > -----Original Message----- > From: mlw [mailto:markw@mohawksoft.com] > Sent: Thursday, May 09, 2002 3:34 PM > To: Dann Corbit > Cc: PostgreSQL-development > Subject: Re: Issues tangential to win32 support > > > Dann Corbit wrote: > http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html > > Mentions cygwin, am I misunderstanding? > > Does not matter, the issue is that you guys said you did it. > OK, have you been > able to bring the changed back into the main source tree? > (Are you not trying?) I am not enrolled in the CVS project, and don't even know how to use it. We use "Visual Source Safe" here -- really an icky tool but at least everyone here knows it. There is some debate here as to whether to keep the changes private or to turn them back to the project. Not sure how it will turn out. I am not sure that the project would want them anyway, since the represent some pretty major surgery and impact the readability of the code in a quite adverse way. At any rate, the Japanese version appears to be released. In fact, I have downloaded the whole project and gave it a spin. It is actually very nice. If you just need to use something for right now, why not go with that version? In any case, there is simply no way possible that anything will ever escape from here before June at the absolute earliest (full regression test is company policy). ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22687@postgresql.org Thu May 9 18:57:40 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g49Mve428946 for ; Thu, 9 May 2002 18:57:40 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 48E914767D3; Thu, 9 May 2002 18:57:28 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 91564476774; Thu, 9 May 2002 18:57:04 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id B1145475F1A for ; Thu, 9 May 2002 18:56:51 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 77DAA475C93 for ; Thu, 9 May 2002 18:56:50 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g49MpWt30673; Thu, 9 May 2002 18:51:36 -0400 Message-ID: <3CDAFD74.177491F2@mohawksoft.com> Date: Thu, 09 May 2002 18:51:32 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Dann Corbit cc: PostgreSQL-development Subject: Re: [HACKERS] Issues tangential to win32 support References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Dann Corbit wrote: > I am not enrolled in the CVS project, and don't even know how to use it. > We use "Visual Source Safe" here -- really an icky tool but at least > everyone here knows it. Source Safe? Yikes. I haven't used that in a long time. > > There is some debate here as to whether to keep the changes private or > to turn them back to the project. Not sure how it will turn out. > > I am not sure that the project would want them anyway, since the > represent some pretty major surgery and impact the readability of the > code in a quite adverse way. I hear you on that. I have tons of code that has #ifdef GCC and #ifdef WIN32 in lots of places. Obviously you wrap what you can in macros and/or functions, but you can't do that 100% the time. Some people REALLY hate #ifdef/#endif and view them as a bad coding practice. Others, like myself, view them as a proper usage of the language constructs and judicious use of them actually help the developer understand the code better. > > At any rate, the Japanese version appears to be released. In fact, I > have downloaded the whole project and gave it a spin. It is actually > very nice. If you just need to use something for right now, why not go > with that version? I have no desire for a Windows version for myself, but I see the need for it. > > In any case, there is simply no way possible that anything will ever > escape from here before June at the absolute earliest (full regression > test is company policy). ok ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M22697@postgresql.org Thu May 9 21:51:20 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4A1pJ401192 for ; Thu, 9 May 2002 21:51:19 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 02BA5476885; Thu, 9 May 2002 21:51:12 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 1003F4768DF; Thu, 9 May 2002 21:50:45 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 9D095476366 for ; Thu, 9 May 2002 21:50:33 -0400 (EDT) Received: from sraigw.sra.co.jp (sraigw.sra.co.jp [202.32.10.2]) by postgresql.org (Postfix) with ESMTP id 29196475F41 for ; Thu, 9 May 2002 21:50:31 -0400 (EDT) Received: from srascb.sra.co.jp (srascb [133.137.8.65]) by sraigw.sra.co.jp (8.9.3/3.7W-sraigw) with ESMTP id KAA91464; Fri, 10 May 2002 10:50:26 +0900 (JST) Received: (from root@localhost) by srascb.sra.co.jp (8.11.6/8.11.6) id g4A1nxJ71113; Fri, 10 May 2002 10:49:59 +0900 (JST) (envelope-from t-ishii@sra.co.jp) Received: from sranhm.sra.co.jp (sranhm [133.137.170.62]) by srascb.sra.co.jp (8.11.6/8.11.6av) with ESMTP id g4A1nwV71105; Fri, 10 May 2002 10:49:58 +0900 (JST) (envelope-from t-ishii@sra.co.jp) Received: from localhost (IDENT:t-ishii@srapc1474.sra.co.jp [133.137.170.59]) by sranhm.sra.co.jp (8.9.3+3.2W/3.7W-srambox) with ESMTP id KAA19819; Fri, 10 May 2002 10:50:25 +0900 To: markw@mohawksoft.com cc: DCorbit@connx.com, pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Issues tangential to win32 support In-Reply-To: <3CDAF94E.2F60382E@mohawksoft.com> References: <3CDAF94E.2F60382E@mohawksoft.com> X-Mailer: Mew version 1.94.2 on Emacs 20.7 / Mule 4.1 =?iso-2022-jp?B?KBskQjAqGyhCKQ==?= MIME-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20020510104904P.t-ishii@sra.co.jp> Date: Fri, 10 May 2002 10:49:04 +0900 From: Tatsuo Ishii X-Dispatcher: imput version 20000228(IM140) Lines: 16 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO > Dann Corbit wrote: > http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html > > Mentions cygwin, am I misunderstanding? Are you talking about following in the page? ---------------------------------------------------------------- * Notice: Based upon the GNU-cygwin, there is a version that works similar to the Unix-compatible operability. Tanida-san Web site is supporting this environment in Japanese. ---------------------------------------------------------------- It cleary refers to another work. -- Tatsuo Ishii ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22721@postgresql.org Fri May 10 10:04:20 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4AE4J417221 for ; Fri, 10 May 2002 10:04:19 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id DFC5B47635A; Fri, 10 May 2002 10:04:05 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 59D6347691C; Fri, 10 May 2002 09:59:39 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id CC721476554 for ; Fri, 10 May 2002 09:59:26 -0400 (EDT) Received: from salem.vale-housing.co.uk (mailgate.vale-housing.co.uk [193.195.77.162]) by postgresql.org (Postfix) with ESMTP id DDEE5475F77 for ; Fri, 10 May 2002 09:55:50 -0400 (EDT) Subject: [HACKERS] FW: Cygwin PostgreSQL Information and Suggestions Date: Fri, 10 May 2002 14:55:53 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Message-ID: <214E9C0A75426D47A876A2FD8A07426E66B4@salem.vale-housing.co.uk> content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Thread-Topic: Cygwin PostgreSQL Information and Suggestions Thread-Index: AcH4KhqAuSFbFlyoSAuLblrF7gO4DQAACK+w From: "Dave Page" To: cc: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g4AE4J417221 Status: RO Some comments from Jason Tishler the Cygwin-PostgreSQL maintainer... > -----Original Message----- > From: Jason Tishler [mailto:jason@tishler.net] > Sent: 10 May 2002 15:00 > To: Dave Page > Cc: pgsql-cygwin@postgresql.org > Subject: Cygwin PostgreSQL Information and Suggestions > > > Dave, > > Would you forward this to pgsql-hackers since I'm not subscribed? > > On Thu, May 09, 2002 at 10:45:42PM +0100, Dave Page wrote: > > > -----Original Message----- > > > From: Jason Tishler [mailto:jason@tishler.net] > > > Sent: 09 May 2002 21:52 > > > To: Dave Page > > > > > > On Thu, May 09, 2002 at 07:51:33PM +0100, Dave Page wrote: > > > > BTW Are you aware there is currently a rather busy thread > > > > about native Windows/Beos ports on -hackers... > > > > > > No, I'm not subscribed, but I just read all that I could find > > > in the archives. > > > [snip] > > > > > > > ...which is currently drifting towards a cutdown Cygwin version? > > > > > > Maybe I'll be out of (another) job soon? :,) > > > > [snip] > > > > Personnally, I think (from a 'good for PostgreSQL' rather > than 'good > > for Cygwin' perspective) that the way forward is a Cygwin > based system > > but using a tailored downloader/installer that installs the system > > 'like a Windows app' (and quickly & easily etc.) rather than the > > current way which is Windows 'being' *nix. I think that's very > > offputting for many potential users (as others have said on the > > -hackers thread). > > I agree with the above, but more can be done with Cygwin and > its setup.exe that can give a fair amount of bang for the > buck for some good short time gains too. I will give some > details below. > > I also wanted to dispel some misinformation (IMO) that I > perceived from the above mentioned posts and/or elaborate on > some of the items: > > 1. Cygwin's setup.exe supports categories and dependencies. > Hence, there is no reason to install all Cygwin packages in > order to ensure properly PostgreSQL operation. Someone just > has to determine what is the minimal set of packages > necessary for PostgreSQL and I will update the setup.hint > accordingly. The current setup.hint is as follows: > > sdesc: "PostgreSQL Data Base Management System" > category: Database > requires: ash cygwin readline zlib libreadline5 > > Sorry, but since I install all Cygwin packages plus about 30 > additional ones I haven't desire to determine what are the > minimal requirements. > > 2. Cygwin's setup.exe is customizable. There is a tool > called "upset" that generates the setup.ini file that drives > setup.exe. PostgreSQL could offer a customized setup. For > example, this is what the XEmacs folks are doing. > > 3. Cygwin's setup.exe can run package specific postinstall > scripts during the installation. Hence, someone could > automate the steps enumerated (e.g., postmaster NT service > installation, initdb, etc.) in my README: > http://www.tishler.net/jason/software/postgresql/postgresql-7.2.1.README to ease the installation burden. 4. Cygwin PostgreSQL is perceived to have poor performance. I have never done any benchmarks regarding this issue, but apparently Terry Carlin (from the defunct Great Bridge) did: http://archives.postgresql.org/pgsql-cygwin/2001-08/msg00029.php Specifically, he indicates the following: BTW, Up through 40 users, PostgreSQL under CYGWIN using the TPC-C benchmark performed very much the same as Linux PostgreSQL on the exact hardware. 5. Cygwin PostgreSQL is perceived to have poor reliability. Unfortunately, I have not been able to gather data to concur or refute this perception due a sudden job "change" last summer. :,) However, there are reports such as the following on the pgsql-cygwin list: http://archives.postgresql.org/pgsql-cygwin/2002-04/msg00021.php IMO, the biggest reliability issue with Cygwin PostgreSQL is it's dependency on cygipc. There is some very recent work to create a Cygwin daemon to support features such as System V IPC. So soon the cygipc dependency and its "problems" will be going way. Those interested in a "Windows" PostgreSQL should possibly consider contributing in this area or other "hard edges" (due to Windows-isms) that would improve the reliability of Cygwin PostgreSQL. BTW, I have found the Cygwin core developers very responsive to PostgreSQL problems because it drives the Cygwin DLL harder than most other applications. 6. Satisfying the Cygwin license for binary distribution is very simple. Just include the source for the Cygwin DLL and all executables that are linked with it in your distribution package. It is really that easy. Jason ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22731@postgresql.org Fri May 10 12:36:48 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4AGal420683 for ; Fri, 10 May 2002 12:36:47 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 48C10476917; Fri, 10 May 2002 12:36:38 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 749E94767FA; Fri, 10 May 2002 12:32:53 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 393C4476885 for ; Fri, 10 May 2002 12:32:42 -0400 (EDT) Received: from sss.pgh.pa.us (unknown [192.204.191.242]) by postgresql.org (Postfix) with ESMTP id 4EE7C47696B for ; Fri, 10 May 2002 12:31:28 -0400 (EDT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g4AGV6W28998; Fri, 10 May 2002 12:31:06 -0400 (EDT) To: "Dave Page" cc: pgsql-hackers@postgresql.org, Jason@tishler.net Subject: Re: [HACKERS] FW: Cygwin PostgreSQL Information and Suggestions In-Reply-To: <214E9C0A75426D47A876A2FD8A07426E66B4@salem.vale-housing.co.uk> References: <214E9C0A75426D47A876A2FD8A07426E66B4@salem.vale-housing.co.uk> Comments: In-reply-to "Dave Page" message dated "Fri, 10 May 2002 14:55:53 +0100" Date: Fri, 10 May 2002 12:31:06 -0400 Message-ID: <28995.1021048266@sss.pgh.pa.us> From: Tom Lane Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO "Dave Page" forwards: > 4. Cygwin PostgreSQL is perceived to have poor performance. I have > never done any benchmarks regarding this issue, but apparently Terry > Carlin (from the defunct Great Bridge) did: > http://archives.postgresql.org/pgsql-cygwin/2001-08/msg00029.php > Specifically, he indicates the following: > BTW, Up through 40 users, PostgreSQL under CYGWIN using the TPC-C > benchmark performed very much the same as Linux PostgreSQL on the > exact hardware. It should be noted that the benchmark Terry is describing fires up N concurrent backends and then measures the runtime for a specific query workload. So it's not measuring connection startup time, which is alleged by some to be Cygwin's weak spot. Nonetheless, I invite the Postgres-on-Cygwin-isn't-worth-our-time camp to produce some benchmarks supporting their position. I'm getting tired of reading unsubstantiated assertions. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M22748@postgresql.org Fri May 10 17:51:28 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4ALpR425360 for ; Fri, 10 May 2002 17:51:27 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id B8E63476930; Fri, 10 May 2002 17:51:25 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id D9736476A51; Fri, 10 May 2002 17:50:11 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 055B3476966; Fri, 10 May 2002 17:49:59 -0400 (EDT) Received: from rh72.home.ee (adsl1030.estpak.ee [213.168.29.11]) by postgresql.org (Postfix) with ESMTP id E7336476919; Fri, 10 May 2002 17:49:49 -0400 (EDT) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by rh72.home.ee (8.11.6/8.11.6) with ESMTP id g4AJmC601988; Sat, 11 May 2002 00:48:15 +0500 Subject: Re: [HACKERS] Native Win32, How about this? From: Hannu Krosing To: mlw cc: Jan Wieck , Jean-Michel POURE , Justin Clift , "Marc G. Fournier" , Iavor Raytchev , PostgreSQL-development In-Reply-To: <3CDC3AAE.E39FF877@mohawksoft.com> References: <3CDC3AAE.E39FF877@mohawksoft.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.4 Date: 11 May 2002 00:48:12 +0500 Message-ID: <1021060097.1891.6.camel@rh72.home.ee> MIME-Version: 1.0 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO On Sat, 2002-05-11 at 02:25, mlw wrote: > A binary version of PostgreSQL for Windows should not use the cygwin dll. I > know and understand there is some disagreement with this position, but in this > I'm sure about this. ... > I believe we can use the cygwin development environment, and direct gcc not to > link with the cygwin dll. Last time I looked it was a command line option. This > will produce a native windows application. No emulation, just a standard C > runtime. It seems that mingw (http://www.mingw.org/) does exactly this and provides needed headers/libs. And they have also non-cycwin minimal build environment (MSYS) that supplies make,sh and other stuff we might use for running initdb and other shell scripts. > Some of the hits will be file path manipulation, '/' vs '\', the notion of > drive letters, and case insensitivity in file names. > > Unicode may be an issue, I haven't looked at that yet. Is that a must for the > initial release? Probably not. >> > A couple simple programs can be written using msvc to monitor, start and stop > PostgreSQL. The programs will be simple using the application wizard, just make > a small dialog box application. dev-c++ has also wizards for easy making of trivial user interfaces http://sourceforge.net/projects/dev-cpp/ -------------- Hannu ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M22745@postgresql.org Fri May 10 17:31:05 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4ALV5424831 for ; Fri, 10 May 2002 17:31:05 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 49E56476A11; Fri, 10 May 2002 17:31:03 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 7536F4769BC; Fri, 10 May 2002 17:30:42 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 05493475AC1; Fri, 10 May 2002 17:30:31 -0400 (EDT) Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by postgresql.org (Postfix) with ESMTP id 9F61B475985; Fri, 10 May 2002 17:30:29 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g4ALP2t04627; Fri, 10 May 2002 17:25:06 -0400 Message-ID: <3CDC3AAE.E39FF877@mohawksoft.com> Date: Fri, 10 May 2002 17:25:02 -0400 From: mlw X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.18-SMP-020426 i686) X-Accept-Language: en MIME-Version: 1.0 To: Jan Wieck , Jean-Michel POURE , Justin Clift , "Marc G. Fournier" , Iavor Raytchev , PostgreSQL-development Subject: [HACKERS] Native Win32, How about this? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO A binary version of PostgreSQL for Windows should not use the cygwin dll. I know and understand there is some disagreement with this position, but in this I'm sure about this. The tools used to create the binary need not be Microsoft, many venders have used Borland or Watcom, the run of the mill user/developer does not care. The developers who do care won't mind the cygwin development environment as long as it produces a native Windows binary that does not play tricks such as fork(). Windows developers don't care too much about source code. The build environment will not be a problem. The issue is that the system must perform well and must be stable. I do not believe that cygwin can meet this requirement. Having done some research for these discussions, I think we know it has startup performance issues and unknown operational issues. FYI: My PHP project msession, can produce both a Windows version and a Cygwin version. It is threaded C++, but I have measured a performance improvements using the native Windows version over the cygwin version. I have since abandoned the cygwin version. I believe we can use the cygwin development environment, and direct gcc not to link with the cygwin dll. Last time I looked it was a command line option. This will produce a native windows application. No emulation, just a standard C runtime. Some of the hits will be file path manipulation, '/' vs '\', the notion of drive letters, and case insensitivity in file names. Unicode may be an issue, I haven't looked at that yet. Is that a must for the initial release? There will be a need for some emulation/api specification of things like semaphores, shared memory, file API (I would like to use Windows native CreateFile routines, as these should be pretty fast.), and so on. We will also have to breakup postgres and postmaster, and for the Windows version use CreateProcess. There are a number of ways to attack this, globals in a structure based in shared memory, globals in a .DLL exported to processes and shared, and so on. I think a huge time savings can be had by avoiding rewriting everything for the Microsoft build environment. As far as I know, and please correct me if I'm wrong, code produced by the cygwin gcc is freely distributable and need not be GPL. Once we have it working without fork() using the cygwin build environment, we will have a native Windows application, we can then further evaluate whether or not we want to expend the work to make a MSC version. Once the backend and most of the tools are built without requiring the cygwin.dll, installation is a breeze. Just dump it somewhere and run it. A couple simple programs can be written using msvc to monitor, start and stop PostgreSQL. The programs will be simple using the application wizard, just make a small dialog box application. Pgaccess will provide all the GUI stuff, and we may even be able to wrap the monitor code into pgaccess. The server install can be done with install shield. There is code that will run any program as an NT service. We can use that for server installations. We can use the MSVC wizard application to pop-up in the tool bar. Have I missed anything? Is this a realistic and attainable plan? ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M22762@postgresql.org Sat May 11 11:30:25 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4BFUP414425 for ; Sat, 11 May 2002 11:30:25 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5492D475933; Sat, 11 May 2002 11:30:19 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 27EFF475D4A; Sat, 11 May 2002 11:29:38 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 84B5C475954 for ; Sat, 11 May 2002 11:29:25 -0400 (EDT) Received: from tomts19-srv.bellnexxia.net (tomts19.bellnexxia.net [209.226.175.73]) by postgresql.org (Postfix) with ESMTP id D9DFB475933 for ; Sat, 11 May 2002 11:29:23 -0400 (EDT) Received: from cbbrowne.com ([64.229.208.28]) by tomts19-srv.bellnexxia.net (InterMail vM.5.01.04.05 201-253-122-122-105-20011231) with ESMTP id <20020511152928.RCBU27276.tomts19-srv.bellnexxia.net@cbbrowne.com> for ; Sat, 11 May 2002 11:29:28 -0400 Received: from cbbrowne.com (localhost [127.0.0.1]) by cbbrowne.com (Postfix) with ESMTP id CE43E36B5A for ; Sat, 11 May 2002 11:26:18 -0400 (EDT) X-Mailer: exmh version 2.5 07/13/2001 (debian 2.5-1) with nmh-1.0.4+dev To: PostgreSQL-development From: cbbrowne@cbbrowne.com Subject: Re: [HACKERS] Native Win32, How about this? In-Reply-To: Message from mlw of "Fri, 10 May 2002 17:25:02 EDT." <3CDC3AAE.E39FF877@mohawksoft.com> References: <3CDC3AAE.E39FF877@mohawksoft.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==_Exmh_-82808586P"; micalg=pgp-sha1; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit Date: Sat, 11 May 2002 11:26:18 -0400 Message-ID: <20020511152618.CE43E36B5A@cbbrowne.com> Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO --==_Exmh_-82808586P Content-Type: text/plain; charset=us-ascii > A binary version of PostgreSQL for Windows should not use the cygwin > dll. I know and understand there is some disagreement with this > position, but in this I'm sure about this. That may ultimately be desirable. In the short term, it is likely preferable to use cygwin. It is only necessary to point at MySQL for an example. Cygwin is used there. It is being used widely, "crap" or not. Cygwin may not ultimately be the ideal thing to use; we don't yet live in Pangloss' "Best of All Possible Worlds," and thus have to live with some things not being ideal. If having the installer install Cygwin as well as the DBMS makes it easy to have something usable soon, and this allows 100,000 WinFolk to try out PostgreSQL, then that's a Big Win. Out of 100K users, surely two or three may be attracted into working on a more Panglossian solution. It may be fair to say that none of those 100K folk would be using PostgreSQL to support HA applications involving hundreds of GB of data. That's _fine_. If there are new 100K folk using PostgreSQL/cygwin, _some_ of them will outgrow its capabilities, and come looking for improvements. And as they're Windows users, accustomed to having to pay hefty amounts to Microsoft to get support no better than that provided by the Psychic Friends Network (see ), they'll doubtless be prepared to have to pay _something_ in order for "PostgreSQL/Win3K-Enterprise Edition" to become available. That seems a not too unreasonable path towards the "Best of All Possible Worlds." There may be a bit of hyperbole in the above, but any time Voltaire gets quoted, that's likely to happen :-). -- (reverse (concatenate 'string "gro.gultn@" "enworbbc")) http://www.cbbrowne.com/info/wp.html Eagles may soar, but weasels don't get sucked into jet engines. -- (concatenate 'string "cbbrowne" "@ntlug.org") http://www.cbbrowne.com/info/multiplexor.html It's a little known fact that the Dark Ages were caused by the Y1K problem. --==_Exmh_-82808586P Content-Type: application/pgp-signature -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-Type: text/plain; charset=us-ascii > A binary version of PostgreSQL for Windows should not use the cygwin > dll. I know and understand there is some disagreement with this > position, but in this I'm sure about this. That may ultimately be desirable. In the short term, it is likely preferable to use cygwin. It is only necessary to point at MySQL for an example. Cygwin is used there. It is being used widely, "crap" or not. Cygwin may not ultimately be the ideal thing to use; we don't yet live in Pangloss' "Best of All Possible Worlds," and thus have to live with some things not being ideal. If having the installer install Cygwin as well as the DBMS makes it easy to have something usable soon, and this allows 100,000 WinFolk to try out PostgreSQL, then that's a Big Win. Out of 100K users, surely two or three may be attracted into working on a more Panglossian solution. It may be fair to say that none of those 100K folk would be using PostgreSQL to support HA applications involving hundreds of GB of data. That's _fine_. If there are new 100K folk using PostgreSQL/cygwin, _some_ of them will outgrow its capabilities, and come looking for improvements. And as they're Windows users, accustomed to having to pay hefty amounts to Microsoft to get support no better than that provided by the Psychic Friends Network (see ), they'll doubtless be prepared to have to pay _something_ in order for "PostgreSQL/Win3K-Enterprise Edition" to become available. That seems a not too unreasonable path towards the "Best of All Possible Worlds." There may be a bit of hyperbole in the above, but any time Voltaire gets quoted, that's likely to happen :-). - -- (reverse (concatenate 'string "gro.gultn@" "enworbbc")) http://www.cbbrowne.com/info/wp.html Eagles may soar, but weasels don't get sucked into jet engines. - -- (concatenate 'string "cbbrowne" "@ntlug.org") http://www.cbbrowne.com/info/multiplexor.html It's a little known fact that the Dark Ages were caused by the Y1K problem. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: Exmh version 2.3.1 01/18/2001 (debian 2.3.1-1) iD8DBQE83TgaN7hZUGqmpxMRAqSeAJ9jkunhAG72NLz7rcPMVcWXbHWY+gCgxu+D wumPJyqj0/9k5bc+v7NVvFI= =pKbi -----END PGP SIGNATURE----- --==_Exmh_-82808586P-- From pgsql-hackers-owner+M22812@postgresql.org Mon May 13 11:30:34 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4DFUY429652 for ; Mon, 13 May 2002 11:30:34 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5ADB2476B07; Mon, 13 May 2002 11:30:26 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id C126F47616C; Mon, 13 May 2002 10:57:45 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 0592C475D9A; Mon, 13 May 2002 10:57:34 -0400 (EDT) Received: from smtp.comcast.net (smtp.comcast.net [24.153.64.2]) by postgresql.org (Postfix) with ESMTP id 7C929476103; Mon, 13 May 2002 09:51:48 -0400 (EDT) Received: from althea.tishler.net (bgp572446bgs.eatntn01.nj.comcast.net [68.39.6.197]) by mtaout03.icomcast.net (iPlanet Messaging Server 5.1 HotFix 0.6 (built Apr 26 2002)) with SMTP id <0GW100GMXYIEET@mtaout03.icomcast.net>; Mon, 13 May 2002 09:51:51 -0400 (EDT) Received: by althea.tishler.net (sSMTP sendmail emulation); Mon, 13 May 2002 09:59:04 -0400 Date: Mon, 13 May 2002 09:59:04 -0400 From: Jason Tishler Subject: Re: [HACKERS] Native Win32, How about this? In-Reply-To: <3CDC3AAE.E39FF877@mohawksoft.com> To: mlw cc: Jan Wieck , Jean-Michel POURE , Justin Clift , "Marc G. Fournier" , Iavor Raytchev , PostgreSQL-development Mail-Followup-To: mlw , Jan Wieck , Jean-Michel POURE , Justin Clift , "Marc G. Fournier" , Iavor Raytchev , PostgreSQL-development Message-ID: <20020513135857.GB2224@tishler.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7BIT Content-Disposition: inline User-Agent: Mutt/1.3.24i References: <3CDC3AAE.E39FF877@mohawksoft.com> Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO mlw, On Fri, May 10, 2002 at 05:25:02PM -0400, mlw wrote: > A binary version of PostgreSQL for Windows should not use the cygwin > dll. I know and understand there is some disagreement with this position, > but in this I'm sure about this. Sorry, but I'm not going to touch the above -- even with a ten foot pole. Or, at least try not to... :,) > I believe we can use the cygwin development environment, and direct gcc > not to link with the cygwin dll. Last time I looked it was a command line > option. This will produce a native windows application. No emulation, > just a standard C runtime. Yes, the above mentioned option is "-mno-cygwin". > Some of the hits will be file path manipulation, '/' vs '\', the notion of > drive letters, and case insensitivity in file names. Case insensitivity is typically "enabled" regardless. Unless you are referring to CYGWIN=check_case:strict, but almost no one uses this setting. Just to be explicit, another hit will be the loss of Posix. > [snip] > > I think a huge time savings can be had by avoiding rewriting everything > for the Microsoft build environment. Yes, you should use Cygwin and gcc -mno-cygwin or MSYS and Mingw. > As far as I know, and please correct me if I'm wrong, code produced by > the cygwin gcc is freely distributable and need not be GPL. The above is true only with gcc -mno-cygwin or Mingw code. Any code produced by the normal Cygwin gcc (and hence, linked against cygwin1.dll) is effectively GPL'd or at least required to be open source. > [snip] Jason ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M22887@postgresql.org Thu May 16 07:48:56 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4GBmtB15522 for ; Thu, 16 May 2002 07:48:55 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 0271647632A; Thu, 16 May 2002 07:48:50 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id BAB2B475985; Thu, 16 May 2002 07:48:23 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id CCF92475961 for ; Thu, 16 May 2002 07:48:11 -0400 (EDT) Received: from n05.sp.kp.dlr.de (gw.sp.kp.dlr.de [129.247.97.187]) by postgresql.org (Postfix) with ESMTP id 241A9475958 for ; Thu, 16 May 2002 07:48:10 -0400 (EDT) Received: from localhost.localdomain (IDENT:root@yyy.kp.dlr.de [129.247.113.223]) by n05.sp.kp.dlr.de (8.11.2/8.11.2) with ESMTP id g4GBmCa79404 for ; Thu, 16 May 2002 13:48:12 +0200 Received: from there (Joelap.sea.com [192.168.0.21]) by localhost.localdomain (8.9.3/8.9.3) with SMTP id NAA14722 for ; Thu, 16 May 2002 13:48:10 +0200 Message-ID: <200205161148.NAA14722@localhost.localdomain> Content-Type: text/plain; charset="iso-8859-1" From: Joerg Hessdoerfer Organization: S.E.A Datentechnik GmbH To: pgsql-hackers@postgresql.org Subject: [HACKERS] WIN32 native ... lets start?!? Date: Thu, 16 May 2002 13:47:45 +0200 X-Mailer: KMail [version 1.3.2] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Hi all, I followed the various threads regarding this for some time now. My current situation is: I'm working at a company which does industrial automation, and does it's own custom products. We try to be cross-platform, but it's a windoze world, as far as most measurement devices or PLCs are concerned. We also employ databases for various tasks (including simple ones as holding configuration data, but also hammering production data into it at a rate of several hundred records/sec.) Well, we would *love* to use PostgreSQL in most our projects and products, (and we do already use it in some), because it has proven to be very reliable and quite fast. So, I'm faced with using PostgreSQL on windows also (you can't always put a Linux box besides). We do this using cygwin, but it's a bit painful ;-) (although it works!). Thinking about the hreads I read, it seems there are 2 obstacles to native PG on W: 1.) no fork, 2.) no SYSV IPC Ok, 1.) is an issue, but there's a fork() in MinGW, so it's 'just' going to be a bit slow on new connections to the DB, right?? But this could be sorted out once we *have* a native WIN32 build. The second one's a bit harder, but... I'm currently trying to find time to do a minimal implementation of SYSV IPC on WIN32 calls, just enough to get PG up (doesn't need msg*() for example, right?). As far as I understand it, we would not need to have IPC items around *after* all backends and postmaster have gone away, or? Then there's no need for a 'daemon' process like in cygwin. So, my route would be to get it to run *somehow* without paying attention to speed and not to change much of the existing code, THEN see how we could get rid of fork() on windows. What do you guys think? Anyone up to join efforts? (I'll start the IPC thingy anyway, as an exercise, and see where I'll end). Greetings, Joerg P.s.: thanks for a great database system!! -- Leading SW developer - S.E.A GmbH Mail: joerg.hessdoerfer@sea-gmbh.com WWW: http://www.sea-gmbh.com ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M22888@postgresql.org Thu May 16 09:39:14 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g4GDdDB16601 for ; Thu, 16 May 2002 09:39:13 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 3B67F476017; Thu, 16 May 2002 09:39:07 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 2F5C2476361; Thu, 16 May 2002 09:38:37 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 79F99475E20 for ; Thu, 16 May 2002 09:38:25 -0400 (EDT) Received: from earth.hub.org (earth.hub.org [64.49.215.11]) by postgresql.org (Postfix) with ESMTP id B5EB6475D63 for ; Thu, 16 May 2002 09:38:24 -0400 (EDT) Received: from localhost.localdomain (earth.hub.org [64.49.215.11]) by localhost (Postfix) with ESMTP id AB6651035F3; Thu, 16 May 2002 10:38:26 -0300 (ADT) Received: from earth.hub.org (earth.hub.org [64.49.215.11]) by earth.hub.org (Postfix) with ESMTP id 92DBB1035CA; Thu, 16 May 2002 10:38:25 -0300 (ADT) Date: Thu, 16 May 2002 10:38:25 -0300 (ADT) From: "Marc G. Fournier" To: Joerg Hessdoerfer cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] WIN32 native ... lets start?!? In-Reply-To: <200205161148.NAA14722@localhost.localdomain> Message-ID: <20020516103501.Q6260-100000@mail1.hub.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr Actually, take a look at the thread starting at: http://archives.postgresql.org/pgsql-hackers/2002-05/msg00665.php Right now, IMHO, the big show stopper is passing global variables to the child processes in Windows ... the above thread talks about a method of pulling together the global variables *cleanly* that Tom seems to feel wouldn't add much in the way of long term maintenance headaches ... *and*, as I understand it, would provide us with a means to use threading in future developments if deemed appropriate ... >From what I read by those 'in the know' about Windows programming, if we could centralize the global variables somewhat, using CreateProcess in Windows shouldn't be a big deal, eliminiating the whole fork() headache ... On Thu, 16 May 2002, Joerg Hessdoerfer wrote: > Hi all, > > I followed the various threads regarding this for some time now. My current > situation is: > > I'm working at a company which does industrial automation, and does it's own > custom products. We try to be cross-platform, but it's a windoze world, as > far as most measurement devices or PLCs are concerned. We also employ > databases for various tasks (including simple ones as holding configuration > data, but also hammering production data into it at a rate of several hundred > records/sec.) > Well, we would *love* to use PostgreSQL in most our projects and products, > (and we do already use it in some), because it has proven to be very reliable > and quite fast. > > So, I'm faced with using PostgreSQL on windows also (you can't always put a > Linux box besides). We do this using cygwin, but it's a bit painful ;-) > (although it works!). > > Thinking about the hreads I read, it seems there are 2 obstacles to native PG > on W: > > 1.) no fork, > 2.) no SYSV IPC > > Ok, 1.) is an issue, but there's a fork() in MinGW, so it's 'just' going to > be a bit slow on new connections to the DB, right?? But this could be sorted > out once we *have* a native WIN32 build. > > The second one's a bit harder, but... I'm currently trying to find time to do > a minimal implementation of SYSV IPC on WIN32 calls, just enough to get PG up > (doesn't need msg*() for example, right?). > As far as I understand it, we would not need to have IPC items around *after* > all backends and postmaster have gone away, or? Then there's no need for a > 'daemon' process like in cygwin. > > So, my route would be to get it to run *somehow* without paying attention to > speed and not to change much of the existing code, THEN see how we could get > rid of fork() on windows. > > What do you guys think? Anyone up to join efforts? (I'll start the IPC thingy > anyway, as an exercise, and see where I'll end). > > Greetings, > Joerg > > P.s.: thanks for a great database system!! > -- > Leading SW developer - S.E.A GmbH > Mail: joerg.hessdoerfer@sea-gmbh.com > WWW: http://www.sea-gmbh.com > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M23371@postgresql.org Mon Jun 3 07:19:20 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g53BJJB21871 for ; Mon, 3 Jun 2002 07:19:19 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 304D947586C; Mon, 3 Jun 2002 07:19:09 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id DE7D6475FD8; Sun, 2 Jun 2002 21:16:18 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id A3E99475AEC for ; Sun, 2 Jun 2002 21:16:05 -0400 (EDT) Received: from mta03.fuse.net (mx3.fuse.net [216.68.1.123]) by postgresql.org (Postfix) with ESMTP id DCDF147590F for ; Sun, 2 Jun 2002 21:14:20 -0400 (EDT) Received: from void ([216.196.153.74]) by mta03.fuse.net (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20020603011508.MQKV21986.mta03.fuse.net@void> for ; Sun, 2 Jun 2002 21:15:08 -0400 Message-ID: <010201c20a9b$8779a4a0$b401a8c0@vrlas> Reply-To: "coventry" From: "coventry" To: "pg hackers" Subject: Re: [HACKERS] HEADS UP: Win32/OS2/BeOS native ports Date: Sun, 2 Jun 2002 21:11:02 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2615.200 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO I think its already been determined that the cygwin option is too low performing. However, the apache stuff could be quite useful - but if that effort were to be undertaken, it would make more sense to move all versions of the code the the apache runtime, for all platforms. Are there any other runtime libraries out there that are cross platform, open/free and high performance? I know the mozilla XPCOM libraries work quite nicely, but are geared more towards multithreaded apps - and the COM-alike infrastructure is something we wouldn't need. ~Jon ----- Original Message ----- From: Bruce Momjian To: mlw Cc: Tom Lane ; Marc G. Fournier ; Sent: Sunday, June 02, 2002 8:49 PM Subject: Re: [HACKERS] HEADS UP: Win32/OS2/BeOS native ports > mlw wrote: > > Like I told Marc, I don't care. You spec out what you want and I'll write it > > for Windows. > > > > That being said, a SysV IPC interface for native Windows would be kind of cool > > to have. > > I am wondering why we don't just use the Cygwin shm/sem code in our > project, or maybe the Apache stuff; why bother reinventing the wheel. > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 853-3000 > + If your life is a hard drive, | 830 Blythe Avenue > + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From jason@tishler.net Mon Jun 3 09:19:17 2002 Return-path: Received: from smtp.comcast.net (smtp.comcast.net [24.153.64.2]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g53DJFB23357 for ; Mon, 3 Jun 2002 09:19:16 -0400 (EDT) Received: from althea.tishler.net (bgp572446bgs.eatntn01.nj.comcast.net [68.39.6.197]) by mtaout03.icomcast.net (iPlanet Messaging Server 5.1 HotFix 0.8 (built May 13 2002)) with SMTP id <0GX40012FSX60N@mtaout03.icomcast.net> for pgman@candle.pha.pa.us; Mon, 03 Jun 2002 09:17:30 -0400 (EDT) Received: by althea.tishler.net (sSMTP sendmail emulation); Mon, 03 Jun 2002 09:18:41 -0400 Date: Mon, 03 Jun 2002 09:18:40 -0400 From: Jason Tishler Subject: Re: [HACKERS] HEADS UP: Win32/OS2/BeOS native ports In-Reply-To: <200206030049.g530nL822000@candle.pha.pa.us> To: Bruce Momjian cc: mlw , Tom Lane , "Marc G. Fournier" , pgsql-hackers@postgresql.org Mail-Followup-To: Bruce Momjian , mlw , Tom Lane , "Marc G. Fournier" , pgsql-hackers@postgresql.org Message-ID: <20020603131840.GA1020@tishler.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7BIT Content-Disposition: inline User-Agent: Mutt/1.3.24i References: <3CD318C1.FF21DFDB@mohawksoft.com> <200206030049.g530nL822000@candle.pha.pa.us> Status: RO Bruce, On Sun, Jun 02, 2002 at 08:49:21PM -0400, Bruce Momjian wrote: > mlw wrote: > > Like I told Marc, I don't care. You spec out what you want and I'll write it > > for Windows. > > > > That being said, a SysV IPC interface for native Windows would be kind of > > cool to have. > > I am wondering why we don't just use the Cygwin shm/sem code in our > project, or maybe the Apache stuff; why bother reinventing the wheel. Are you referring to cygipc above? If so, they even one of the original cygipc authors would discourage this: http://sources.redhat.com/ml/cygwin-apps/2001-09/msg00017.html Specifically, Ludovic Lange states the following: > I really think the solution would be to start again from scratch > another implementation, as was suggested. The way we did it was > quick and dirty, the goals weren't to have production systems > running on it but only to run prototypes. So the internal design > (if there is any) may not be adequate for the cygwin project. However, Rob Collins has contributed a MinGW daemon to Cygwin to support switching users, System V IPC, etc. So, this code base may be a more suitable starting point to satisfy PostgreSQL's native Win32 System V IPC needs. Jason From pgsql-hackers-owner+M23378@postgresql.org Mon Jun 3 09:29:13 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g53DTCB24153 for ; Mon, 3 Jun 2002 09:29:12 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id E30C2475E32; Mon, 3 Jun 2002 09:29:03 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 026FB475D81; Mon, 3 Jun 2002 09:28:50 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 56123475BFD for ; Mon, 3 Jun 2002 09:28:32 -0400 (EDT) Received: from smtp.comcast.net (smtp.comcast.net [24.153.64.2]) by postgresql.org (Postfix) with ESMTP id 5B1E2475A3E for ; Mon, 3 Jun 2002 09:27:36 -0400 (EDT) Received: from althea.tishler.net (bgp572446bgs.eatntn01.nj.comcast.net [68.39.6.197]) by mtaout02.icomcast.net (iPlanet Messaging Server 5.1 HotFix 0.8 (built May 13 2002)) with SMTP id <0GX4003JUTE16I@mtaout02.icomcast.net> for pgsql-hackers@postgresql.org; Mon, 03 Jun 2002 09:27:38 -0400 (EDT) Received: by althea.tishler.net (sSMTP sendmail emulation); Mon, 03 Jun 2002 09:28:48 -0400 Date: Mon, 03 Jun 2002 09:28:48 -0400 From: Jason Tishler Subject: Re: [HACKERS] HEADS UP: Win32/OS2/BeOS native ports In-Reply-To: <3CFAC785.4AAF9F79@mohawksoft.com> To: mlw cc: Bruce Momjian , Tom Lane , "Marc G. Fournier" , pgsql-hackers@postgresql.org Mail-Followup-To: mlw , Bruce Momjian , Tom Lane , "Marc G. Fournier" , pgsql-hackers@postgresql.org Message-ID: <20020603132848.GB1020@tishler.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7BIT Content-Disposition: inline User-Agent: Mutt/1.3.24i References: <200206030049.g530nL822000@candle.pha.pa.us> <3CFAC785.4AAF9F79@mohawksoft.com> Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO On Sun, Jun 02, 2002 at 09:33:57PM -0400, mlw wrote: > Bruce Momjian wrote: > > mlw wrote: > > > Like I told Marc, I don't care. You spec out what you want and I'll write > > > it for Windows. > > > > > > That being said, a SysV IPC interface for native Windows would be kind of > > > cool to have. > > > > I am wondering why we don't just use the Cygwin shm/sem code in our > > project, or maybe the Apache stuff; why bother reinventing the wheel. > > but! in the course of testing some code, I managed to gain some experience > with cygwin. I have seen fork() problems with a large number of processes. Since Cygwin's fork() is implemented with WaitForMultipleObjects(), it has a limitation of only 63 children per parent. Also, there can be DLL base address conflicts (causing Cygwin fork() to fail) that are avoidable by rebasing the appropriate DLLs. AFAICT, Cygwin PostgreSQL is currently *not* affected by this issue where as other Cygwin applications such as Python and Apache are. Jason ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M23379@postgresql.org Mon Jun 3 10:08:15 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g53E8EB27402 for ; Mon, 3 Jun 2002 10:08:15 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 2B4DA475F6D; Mon, 3 Jun 2002 10:08:06 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 3D70D475DB5; Mon, 3 Jun 2002 09:54:41 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id CEDC6475C95 for ; Mon, 3 Jun 2002 09:54:25 -0400 (EDT) Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by postgresql.org (Postfix) with SMTP id D5BB0475DA3 for ; Mon, 3 Jun 2002 09:46:48 -0400 (EDT) Received: from unknown (HELO saturn.janwieck.net) (janwieck@192.233.92.205 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 3 Jun 2002 13:46:50 -0000 Received: (from wieck@localhost) by saturn.janwieck.net (8.11.2/8.11.2) id g53Didi02442; Mon, 3 Jun 2002 09:44:39 -0400 From: Jan Wieck Message-ID: <200206031344.g53Didi02442@saturn.janwieck.net> Subject: Re: [HACKERS] HEADS UP: Win32/OS2/BeOS native ports In-Reply-To: <20020603132848.GB1020@tishler.net> from Jason Tishler at "Jun 3, 2002 09:28:48 am" To: Jason Tishler Date: Mon, 3 Jun 2002 09:44:38 -0400 (EDT) cc: mlw , Bruce Momjian , Tom Lane , "Marc G. Fournier" , pgsql-hackers@postgresql.org X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Jason Tishler wrote: > On Sun, Jun 02, 2002 at 09:33:57PM -0400, mlw wrote: > > Bruce Momjian wrote: > > > mlw wrote: > > > > Like I told Marc, I don't care. You spec out what you want and I'll write > > > > it for Windows. > > > > > > > > That being said, a SysV IPC interface for native Windows would be kind of > > > > cool to have. > > > > > > I am wondering why we don't just use the Cygwin shm/sem code in our > > > project, or maybe the Apache stuff; why bother reinventing the wheel. > > > > but! in the course of testing some code, I managed to gain some experience > > with cygwin. I have seen fork() problems with a large number of processes. > > Since Cygwin's fork() is implemented with WaitForMultipleObjects(), > it has a limitation of only 63 children per parent. Also, there can > be DLL base address conflicts (causing Cygwin fork() to fail) that are > avoidable by rebasing the appropriate DLLs. AFAICT, Cygwin PostgreSQL is > currently *not* affected by this issue where as other Cygwin applications > such as Python and Apache are. Whatever technical problems there are, we can debate on and on if it's worth working around them in PostgreSQL or fixing them in CygWIN or whatever. The main problem will remain. That using PostgreSQL under CygWIN requires some UNIX know how. So a pure Windows user/shop needs UNIX knowledge to run our "Windows port" of PostgreSQL? Interesting definition of "port". Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com # ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From markw@mohawksoft.com Mon Jun 3 17:40:50 2002 Return-path: Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.client2.attbi.com [24.147.138.78]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g53LemB04653 for ; Mon, 3 Jun 2002 17:40:49 -0400 (EDT) Received: from mohawksoft.com (localhost [127.0.0.1]) by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g53LcIG12116; Mon, 3 Jun 2002 17:38:18 -0400 Sender: markw@snoopy.mohawksoft.com Message-ID: <3CFBE1C9.A0B83716@mohawksoft.com> Date: Mon, 03 Jun 2002 17:38:17 -0400 From: mlw X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.18-3smp i686) X-Accept-Language: en MIME-Version: 1.0 To: Jason Tishler cc: Bruce Momjian , Tom Lane , "Marc G. Fournier" , pgsql-hackers@postgresql.org Subject: Re: [HACKERS] HEADS UP: Win32/OS2/BeOS native ports References: <200206030049.g530nL822000@candle.pha.pa.us> <3CFAC785.4AAF9F79@mohawksoft.com> <20020603132848.GB1020@tishler.net> <3CFB70F3.63AE630B@mohawksoft.com> <20020603142957.GA952@tishler.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Status: RO Jason Tishler wrote: > > On Mon, Jun 03, 2002 at 09:36:51AM -0400, mlw wrote: > > Jason Tishler wrote: > > > > > > On Sun, Jun 02, 2002 at 09:33:57PM -0400, mlw wrote: > > > > Bruce Momjian wrote: > > > > > mlw wrote: > > > > > > Like I told Marc, I don't care. You spec out what you want and I'll > > > > > > write it for Windows. > > > > > > > > > > > > That being said, a SysV IPC interface for native Windows would be > > > > > > kind of cool to have. > > > > > > > > > > I am wondering why we don't just use the Cygwin shm/sem code in our > > > > > project, or maybe the Apache stuff; why bother reinventing the wheel. > > > > > > > > but! in the course of testing some code, I managed to gain some experience > > > > with cygwin. I have seen fork() problems with a large number of processes. > > > > > > Since Cygwin's fork() is implemented with WaitForMultipleObjects(), > > > it has a limitation of only 63 children per parent. Also, there can > > > be DLL base address conflicts (causing Cygwin fork() to fail) that are > > > avoidable by rebasing the appropriate DLLs. AFAICT, Cygwin PostgreSQL is > > > currently *not* affected by this issue where as other Cygwin applications > > > such as Python and Apache are. > > > > Why would not PostgreSQL be affected by this? > > Sorry, if I was unclear -- I should have used two paragraphs above and > maybe a few more words... :,) > > Cygwin PostgreSQL *is* affected by the Cygwin 63 children per parent > fork limitation. > > PostgreSQL *can* be affected by the Cygwin DLL base address conflict > fork issue, but in my experience (both personal and by monitoring the > Cygwin and pgsql-cygwin lists), no one has been affected yet. The DLL > base address conflict is a "probability" thing. The more DLLs loaded > the greater the chance of a conflict (and fork() failing). Since, Cygwin > PostgreSQL loads only a few DLLs, this has not become an issue (yet). I'm not sure the DLL load address is a big issue for PostgreSQL, AFAIK no option DLLs will be loaded by Postmaster. So, with fork() it will be a simple process. A PostgreSQL child will die upon completion, and never execute fork(). My concern would be the limit on the number of child processes allowed. 63 is far below what would be considered a usable number in production, and as long as that is an issue, I don't think anyone would take PostgreSQL seriously. A Windows version of PostgreSQL must run within the confines of the Windows OS. The reason, IMHO, that no one has found any serious bugs in the cygwin version, is because no one is seriously using it. Anyone who *would* seriously use it, knows better. From robert.schrem@WiredMinds.de Mon Jun 3 10:08:28 2002 Return-path: Received: from mail.wiredminds.de (mail.wiredminds.de [212.9.164.91]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g53E8QB27442 for ; Mon, 3 Jun 2002 10:08:27 -0400 (EDT) Received: from pc-robert (roberts@pc-robert.wiredminds.de [192.168.111.59]) by mail.wiredminds.de (8.11.3/8.11.4) with ESMTP id g53E8lC04991; Mon, 3 Jun 2002 16:08:47 +0200 Content-Type: text/plain; charset="iso-8859-1" From: Robert Schrem Reply-To: robert.schrem@WiredMinds.de Organization: WiredMinds GmbH To: Jason Tishler , Bruce Momjian Subject: Re: HEADS UP: Win32/OS2/BeOS native ports - the 'BEST OPEN SOURCE database backend' Date: Mon, 3 Jun 2002 16:08:14 +0200 X-Mailer: KMail [version 1.4] cc: mlw , Tom Lane , "Marc G. Fournier" , pgsql-hackers@postgresql.org References: <3CD318C1.FF21DFDB@mohawksoft.com> <200206030049.g530nL822000@candle.pha.pa.us> <20020603131840.GA1020@tishler.net> In-Reply-To: <20020603131840.GA1020@tishler.net> MIME-Version: 1.0 Message-ID: <200206031608.14726.robert.schrem@wiredminds.de> Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g53E8QB27442 Status: RO Hi, You may want to have a look at: http://www.garret.ru/~knizhnik/ You find there code for a 'Fast synchronized access to shared memory for Windows and for i86 Unix-es". kind regards, Robert > Bruce, > > On Sun, Jun 02, 2002 at 08:49:21PM -0400, Bruce Momjian wrote: > > mlw wrote: > > > Like I told Marc, I don't care. You spec out what you want and I'll > > > write it for Windows. > > > > > > That being said, a SysV IPC interface for native Windows would be kind > > > of cool to have. > > > > I am wondering why we don't just use the Cygwin shm/sem code in our > > project, or maybe the Apache stuff; why bother reinventing the wheel. > > Are you referring to cygipc above? If so, they even one of the original > cygipc authors would discourage this: > > http://sources.redhat.com/ml/cygwin-apps/2001-09/msg00017.html > > Specifically, Ludovic Lange states the following: > > I really think the solution would be to start again from scratch > > another implementation, as was suggested. The way we did it was > > quick and dirty, the goals weren't to have production systems > > running on it but only to run prototypes. So the internal design > > (if there is any) may not be adequate for the cygwin project. > > However, Rob Collins has contributed a MinGW daemon to Cygwin to support > switching users, System V IPC, etc. So, this code base may be a more > suitable starting point to satisfy PostgreSQL's native Win32 System V > IPC needs. > > Jason > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M23404@postgresql.org Wed Jun 5 00:34:21 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g554YKs05329 for ; Wed, 5 Jun 2002 00:34:20 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id F1588475998; Wed, 5 Jun 2002 00:34:15 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 6300547627A; Wed, 5 Jun 2002 00:34:02 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 2B9BA475A5B for ; Wed, 5 Jun 2002 00:33:49 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id AD19E475E84 for ; Wed, 5 Jun 2002 00:33:43 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g554XiN05245 for pgsql-hackers@postgreSQL.org; Wed, 5 Jun 2002 00:33:44 -0400 (EDT) From: Bruce Momjian Message-ID: <200206050433.g554XiN05245@candle.pha.pa.us> Subject: [HACKERS] Roadmap for a Win32 port In-Reply-To: <20020516103501.Q6260-100000@mail1.hub.org> To: PostgreSQL-development Date: Wed, 5 Jun 2002 00:33:44 -0400 (EDT) X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr OK, I think I am now caught up on the Win32/cygwin discussion, and would like to make some remarks. First, are we doing enough to support the Win32 platform? I think the answer is clearly "no". There are 3-5 groups/companies working on Win32 ports of PostgreSQL. We always said there would not be PostgreSQL forks if we were doing our job to meet user needs. Well, obviously, a number of groups see a need for a better Win32 port and we aren't meeting that need, so they are. I believe this is one of the few cases where groups are going out on their own because we are falling behind. So, there is no question in my mind we need to do more to encourage Win32 ports. Now, on to the details. INSTALLER --------- We clearly need an installer that is zero-hassle for users. We need to decide on a direction for this. GUI --- We need a slick GUI. pgadmin2 seems to be everyone's favorite, with pgaccess on Win32 also an option. What else do we need here? BINARY ------ This is the big daddy. It is broken down into several sections: FORK() How do we handle fork()? Do we use the cygwin method that copies the whole data segment, or put the global data in shared memory and copy that small part manually after we create a new process? THREADING Related to fork(), do we implement an optionally threaded postmaster, which eliminates CreateProcess() entirely? I don't think we will have superior performance on Win32 without it. (This would greatly help Solaris as well.) IPC We can use Cygwin, MinGW, Apache, or our own code for this. Are there other options? ENVIRONMENT Lots of our code requires a unix shell and utilities. Will we continue using cygwin for this? --------------------------------------------------------------------------- As a roadmap, it would be good to get consensus on as many of these items as possible so people can start working in these areas. We can keep a web page of decisions we have made to help rally developers to the project. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M23405@postgresql.org Wed Jun 5 01:02:39 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5552cs07802 for ; Wed, 5 Jun 2002 01:02:38 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id C97A6475986; Wed, 5 Jun 2002 01:02:39 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 1F2124761F2; Wed, 5 Jun 2002 01:02:13 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 3866A475A3B for ; Wed, 5 Jun 2002 01:01:59 -0400 (EDT) Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) by postgresql.org (Postfix) with ESMTP id D4F2B475986 for ; Wed, 5 Jun 2002 01:01:57 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [HACKERS] Roadmap for a Win32 port X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 content-class: urn:content-classes:message Date: Tue, 4 Jun 2002 22:02:14 -0700 Message-ID: Thread-Topic: [HACKERS] Roadmap for a Win32 port Thread-Index: AcIMSk/7WQpVSQqTSsiRN1SUew56HgAAWkLg From: "Dann Corbit" To: "Bruce Momjian" , "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g5552cs07802 Status: RO > -----Original Message----- > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] > Sent: Tuesday, June 04, 2002 9:34 PM > To: PostgreSQL-development > Subject: [HACKERS] Roadmap for a Win32 port > > > OK, I think I am now caught up on the Win32/cygwin > discussion, and would > like to make some remarks. > > First, are we doing enough to support the Win32 platform? I think the > answer is clearly "no". There are 3-5 groups/companies > working on Win32 > ports of PostgreSQL. We always said there would not be > PostgreSQL forks > if we were doing our job to meet user needs. Well, > obviously, a number > of groups see a need for a better Win32 port and we aren't > meeting that > need, so they are. I believe this is one of the few cases > where groups > are going out on their own because we are falling behind. > > So, there is no question in my mind we need to do more to encourage > Win32 ports. Now, on to the details. > > INSTALLER > --------- > > We clearly need an installer that is zero-hassle for users. > We need to > decide on a direction for this. > > GUI > --- > > We need a slick GUI. pgadmin2 seems to be everyone's favorite, with > pgaccess on Win32 also an option. What else do we need here? Nothing else. It is better than any commercial tools in current use. An excellent piece of work. > BINARY > ------ > > This is the big daddy. It is broken down into several sections: > > FORK() > > How do we handle fork()? Do we use the cygwin method that copies the > whole data segment, or put the global data in shared memory and copy > that small part manually after we create a new process? Do not try to do a fork() on Win32. The one at PW32 is better, but still awful. Win32 just does not have fascilities for fork(). If you use Cygwin, it will kill the project for commercial use (at least for many institutions). That's fine, but it will become an academic exercise instead of a viable commercial tool. If they are comfortable in that [Cygwin] environment, it makes no sense to use Cygwin instead of Redhat. The Redhat version will fork() 100 times faster. After all, if they are going to use unix tools in a unix interface with Unix scripts you might as well use UNIX. And Cygwin requires a license for commercial use. http://cygwin.com/licensing.html > THREADING > > Related to fork(), do we implement an optionally threaded postmaster, > which eliminates CreateProcess() entirely? I don't think we will have > superior performance on Win32 without it. (This would greatly help > Solaris as well.) CreateProcess() works well for Win32. That is the approach that we used and also the approach used by the Japanese team. It is very simple. Simply do a create process call and then perform the same operations that were done up to that point. It isn't difficult. Threading is another possibility. I think create process is better, because you can clone the rights of the one who attaches for the spawned server (if you want to do that). > > IPC > > We can use Cygwin, MinGW, Apache, or our own code for this. Are there > other options? We wrote our own from scratch. Cygwin will kill it. If there is a MinGW version it might be OK, but if MinGW is GPL, that will kill it. Have a look at ACE: http://www.cs.wustl.edu/~schmidt/ACE.html Their license is on the same level as a BSD license. Now, they use C++, but you can always write: extern "C" { } wrappers for stuff and keep PostgreSQL itself in pure, vanilla C. GCC does come with a C++ compiler, so it isn't going to cut anyone off. > ENVIRONMENT > > Lots of our code requires a Unix shell and utilities. Will > we continue > using cygwin for this? We wrote our own utilities from scratch (e.g. initdb). The Japanese group that did the port did the same thing. > -------------------------------------------------------------- > ------------- > > As a roadmap, it would be good to get consensus on as many of these > items as possible so people can start working in these areas. We can > keep a web page of decisions we have made to help rally developers to > the project. If you want a roadmap, the Japanese group laid it out for you. They did the exact same steps as we did. Now, I don't know if we will be able to contribute or not (it is very much up in the air). And we had to do a lot of hacking of the source, so you might not want it if we volunteered. Suggestion: Ask the Japanese group if they would like to post their changes back or expose them so that the programming team can get ideas form it. I actually like what they did better than what we did (A giant DLL and all the binaries are microscopic -- it was how I suggested to do it here but it was vetoed). Anyway, here is a roadmap laid out for you exactly. Just do what it says and you will be fine: http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html Look at where it says "Gists for patch" and do that. ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From markir@slingshot.co.nz Wed Jun 5 03:44:54 2002 Return-path: Received: from smtp-1.attica.net.nz (pop3-1.tranzpeer.net [202.180.66.199]) by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g557iqs18525 for ; Wed, 5 Jun 2002 03:44:53 -0400 (EDT) Received: (qmail 93765 invoked from network); 5 Jun 2002 07:44:08 -0000 Received: from p175.nas1.wlg.callplus.net.nz (202.180.103.175) by pop3-1.tranzpeer.net with SMTP; 5 Jun 2002 07:44:08 -0000 Subject: Re: [HACKERS] Roadmap for a Win32 port From: Mark kirkwood To: Bruce Momjian cc: PostgreSQL-development In-Reply-To: <200206050433.g554XiN05245@candle.pha.pa.us> References: <200206050433.g554XiN05245@candle.pha.pa.us> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/1.0.2-5mdk Date: 05 Jun 2002 19:38:52 +1200 Message-ID: <1023262733.1314.7.camel@spikey.slithery.org> MIME-Version: 1.0 Status: ROr On Wed, 2002-06-05 at 16:33, Bruce Momjian wrote: > OK, I think I am now caught up on the Win32/cygwin discussion, and would > like to make some remarks. > > First, are we doing enough to support the Win32 platform? I think the > answer is clearly "no". There are 3-5 groups/companies working on Win32 > ports of PostgreSQL. We always said there would not be PostgreSQL forks > if we were doing our job to meet user needs. Well, obviously, a number > of groups see a need for a better Win32 port and we aren't meeting that > need, so they are. I believe this is one of the few cases where groups > are going out on their own because we are falling behind. > > So, there is no question in my mind we need to do more to encourage > Win32 ports. Now, on to the details. > > INSTALLER > --------- > > We clearly need an installer that is zero-hassle for users. We need to > decide on a direction for this. > > GUI > --- > > We need a slick GUI. pgadmin2 seems to be everyone's favorite, with > pgaccess on Win32 also an option. What else do we need here? > > BINARY > ------ > > This is the big daddy. It is broken down into several sections: > > FORK() > > How do we handle fork()? Do we use the cygwin method that copies the > whole data segment, or put the global data in shared memory and copy > that small part manually after we create a new process? > > THREADING > > Related to fork(), do we implement an optionally threaded postmaster, > which eliminates CreateProcess() entirely? I don't think we will have > superior performance on Win32 without it. (This would greatly help > Solaris as well.) > > IPC > > We can use Cygwin, MinGW, Apache, or our own code for this. Are there > other options? > > ENVIRONMENT > > Lots of our code requires a unix shell and utilities. Will we continue > using cygwin for this? > > --------------------------------------------------------------------------- > > As a roadmap, it would be good to get consensus on as many of these > items as possible so people can start working in these areas. We can > keep a web page of decisions we have made to help rally developers to > the project. > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 853-3000 > + If your life is a hard drive, | 830 Blythe Avenue > + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > Is it worth looking at how the mysql crowd did their win32 port - (or is that intrinsically a _bad_thing_ to do..) ? (I am guessing that is why their sources requires c++ ....) regards Mark From pgsql-hackers-owner+M23408@postgresql.org Wed Jun 5 08:16:29 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g55CGSs07449 for ; Wed, 5 Jun 2002 08:16:28 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6FA76475997; Wed, 5 Jun 2002 08:16:22 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id AECD347622F; Wed, 5 Jun 2002 08:14:38 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id AD0E7476110 for ; Wed, 5 Jun 2002 08:14:27 -0400 (EDT) Received: from smtp.comcast.net (smtp.comcast.net [24.153.64.2]) by postgresql.org (Postfix) with ESMTP id D067D475FE1 for ; Wed, 5 Jun 2002 08:05:47 -0400 (EDT) Received: from althea.tishler.net (bgp572446bgs.eatntn01.nj.comcast.net [68.39.6.197]) by mtaout06.icomcast.net (iPlanet Messaging Server 5.1 HotFix 0.8 (built May 13 2002)) with SMTP id <0GX800E3YEXHHV@mtaout06.icomcast.net> for pgsql-hackers@postgreSQL.org; Wed, 05 Jun 2002 08:05:41 -0400 (EDT) Received: by althea.tishler.net (sSMTP sendmail emulation); Wed, 05 Jun 2002 08:07:06 -0400 Date: Wed, 05 Jun 2002 08:07:06 -0400 From: Jason Tishler Subject: Re: [HACKERS] Roadmap for a Win32 port To: Dann Corbit cc: Bruce Momjian , PostgreSQL-development Mail-Followup-To: Dann Corbit , Bruce Momjian , PostgreSQL-development Message-ID: <20020605120705.GA1544@tishler.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7BIT Content-Disposition: inline User-Agent: Mutt/1.3.24i References: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr Dan, The following is to help keep the archives accurate and should not be construed as an argument against the native Win32 port. On Tue, Jun 04, 2002 at 10:02:14PM -0700, Dann Corbit wrote: > And Cygwin requires a license for commercial use. > http://cygwin.com/licensing.html The above is not necessarily true: Red Hat sells a special Cygwin License for customers who are unable to provide their application in open source code form. Note that the above only comes into play if your application links with the Cygwin DLL. This is easily avoidable by using JDBC, ODBC, Win32 libpq, etc. Hence, most people will not be required to purchase this license from Red Hat. Jason ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From Igor.Kovalenko@motorola.com Wed Jun 5 15:31:26 2002 Return-path: Received: from ftpbox.mot.com (ftpbox.mot.com [129.188.136.101]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g55JVPs09926 for ; Wed, 5 Jun 2002 15:31:26 -0400 (EDT) Received: [from pobox3.mot.com (pobox3.mot.com [10.64.251.242]) by ftpbox.mot.com (ftpbox 2.1) with ESMTP id MAA21713; Wed, 5 Jun 2002 12:31:25 -0700 (MST)] Received: [from pronto1.comm.mot.com (pronto1.comm.mot.com [173.6.1.22]) by pobox3.mot.com (MOT-pobox3 2.0) with ESMTP id MAA08146; Wed, 5 Jun 2002 12:31:08 -0700 (MST)] Received: from kovalenkoigor (idennt19534 [145.1.195.34]) by pronto1.comm.mot.com (8.9.3/8.9.3) with SMTP id OAA09587; Wed, 5 Jun 2002 14:31:24 -0500 (CDT) Message-ID: <11f301c20cc7$b1f73a70$22c30191@comm.mot.com> From: "Igor Kovalenko" To: "Bruce Momjian" , "PostgreSQL-development" References: <200206050433.g554XiN05245@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port Date: Wed, 5 Jun 2002 14:32:13 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Status: ROr I might be naive here, but would not proper threading model remove the need for fork() altogether? On both Unix and Win32? Should not be too hard to come up with abstraction which encapsulates POSIX, BeOS and Win32 threads... I am not sure how universal POSIX threads are by now. Any important Unix platforms which don't support them yet? This has downside of letting any bug to kill the whole thing. On the bright side, performance should be better on some platforms (note however, Apache group still can't come up with implementation of threaded model which would provide better performance than forked or other models). The need to deal with possibility of 'alien' postmaster running along with orphaned backends would also be removed since there would be only one process. Issue of thread safety of code will come up undoubtedly and some things will probably have to be revamped. But in long term this is probably best way if you want to have efficient and uniform Unix AND Win32 implementations. I am not too familiar with Win32. Speaking about POSIX threads, it would be something like a thread pool with low & high watermarks. Main thread would handle thread pool and hand over requests to worker threads (blocked on condvar). How does that sound? -- igor ----- Original Message ----- From: "Bruce Momjian" To: "PostgreSQL-development" Sent: Tuesday, June 04, 2002 11:33 PM Subject: [HACKERS] Roadmap for a Win32 port > OK, I think I am now caught up on the Win32/cygwin discussion, and would > like to make some remarks. > > First, are we doing enough to support the Win32 platform? I think the > answer is clearly "no". There are 3-5 groups/companies working on Win32 > ports of PostgreSQL. We always said there would not be PostgreSQL forks > if we were doing our job to meet user needs. Well, obviously, a number > of groups see a need for a better Win32 port and we aren't meeting that > need, so they are. I believe this is one of the few cases where groups > are going out on their own because we are falling behind. > > So, there is no question in my mind we need to do more to encourage > Win32 ports. Now, on to the details. > > INSTALLER > --------- > > We clearly need an installer that is zero-hassle for users. We need to > decide on a direction for this. > > GUI > --- > > We need a slick GUI. pgadmin2 seems to be everyone's favorite, with > pgaccess on Win32 also an option. What else do we need here? > > BINARY > ------ > > This is the big daddy. It is broken down into several sections: > > FORK() > > How do we handle fork()? Do we use the cygwin method that copies the > whole data segment, or put the global data in shared memory and copy > that small part manually after we create a new process? > > THREADING > > Related to fork(), do we implement an optionally threaded postmaster, > which eliminates CreateProcess() entirely? I don't think we will have > superior performance on Win32 without it. (This would greatly help > Solaris as well.) > > IPC > > We can use Cygwin, MinGW, Apache, or our own code for this. Are there > other options? > > ENVIRONMENT > > Lots of our code requires a unix shell and utilities. Will we continue > using cygwin for this? > > -------------------------------------------------------------------------- - > > As a roadmap, it would be good to get consensus on as many of these > items as possible so people can start working in these areas. We can > keep a web page of decisions we have made to help rally developers to > the project. > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 853-3000 > + If your life is a hard drive, | 830 Blythe Avenue > + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > From pgsql-hackers-owner+M23416@postgresql.org Wed Jun 5 16:05:36 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g55K5Zs15637 for ; Wed, 5 Jun 2002 16:05:36 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id D84B3476449; Wed, 5 Jun 2002 16:05:31 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id CD4754763D0; Wed, 5 Jun 2002 16:05:19 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6CD45476001 for ; Wed, 5 Jun 2002 16:05:07 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id 09559475FC9 for ; Wed, 5 Jun 2002 16:05:06 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g55K52615577; Wed, 5 Jun 2002 16:05:02 -0400 (EDT) From: Bruce Momjian Message-ID: <200206052005.g55K52615577@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <11f301c20cc7$b1f73a70$22c30191@comm.mot.com> To: Igor Kovalenko Date: Wed, 5 Jun 2002 16:05:02 -0400 (EDT) cc: PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Igor Kovalenko wrote: > I might be naive here, but would not proper threading model remove the need > for fork() altogether? On both Unix and Win32? Should not be too hard to > come up with abstraction which encapsulates POSIX, BeOS and Win32 threads... > I am not sure how universal POSIX threads are by now. Any important Unix > platforms which don't support them yet? > > This has downside of letting any bug to kill the whole thing. On the bright > side, performance should be better on some platforms (note however, Apache > group still can't come up with implementation of threaded model which would > provide better performance than forked or other models). The need to deal > with possibility of 'alien' postmaster running along with orphaned backends > would also be removed since there would be only one process. > > Issue of thread safety of code will come up undoubtedly and some things will > probably have to be revamped. But in long term this is probably best way if > you want to have efficient and uniform Unix AND Win32 implementations. > > I am not too familiar with Win32. Speaking about POSIX threads, it would be > something like a thread pool with low & high watermarks. Main thread would > handle thread pool and hand over requests to worker threads (blocked on > condvar). How does that sound? Good summary. I think we would support both threaded and fork() operation, and users can control which they prefer. For a web backend where many sessions are a single query, people may want to give up the stability of fork() and go with threads, even on Unix. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From thomas@fourpalms.org Wed Jun 5 18:02:46 2002 Return-path: Received: from myst.fourpalms.org (fwuser@sentry.sonalysts.com [198.6.208.103]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g55M2js26493 for ; Wed, 5 Jun 2002 18:02:45 -0400 (EDT) Message-ID: <3CFE8A79.84738F2F@fourpalms.org> Date: Wed, 05 Jun 2002 15:02:33 -0700 From: Thomas Lockhart X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.8-34.1mdk i686) MIME-Version: 1.0 To: Bruce Momjian cc: Igor Kovalenko , PostgreSQL-development Subject: Re: [HACKERS] Roadmap for a Win32 port References: <200206052005.g55K52615577@candle.pha.pa.us> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Status: RO ... > Good summary. I think we would support both threaded and fork() > operation, and users can control which they prefer. For a web backend > where many sessions are a single query, people may want to give up the > stability of fork() and go with threads, even on Unix. I would think that we would build on our strengths of having a fork/exec model for separate clients. A threaded model *could* benefit individual clients who are doing queries on multiprocessor servers, and I would be supportive of efforts to enable that. But the requirements for that may be less severe than for managing multiple clients within the same process, and imho there is not strong requirement to enable the latter for our current crop of well supported targets. If it came for free then great, but if it came with a high cost then the choice is not as obvious. It is also not a *requirement* if we were instead able to do the multiple threads for a single client scenerio first. - Thomas From pgsql-hackers-owner+M23421@postgresql.org Wed Jun 5 18:42:44 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g55Mghs29966 for ; Wed, 5 Jun 2002 18:42:44 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 96863475D45; Wed, 5 Jun 2002 18:42:41 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 8F68F47645B; Wed, 5 Jun 2002 18:36:44 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id D8C2B47630D for ; Wed, 5 Jun 2002 18:36:22 -0400 (EDT) Received: from myst.fourpalms.org (sentry.sonalysts.com [198.6.208.103]) by postgresql.org (Postfix) with ESMTP id 35C5647639F for ; Wed, 5 Jun 2002 18:21:31 -0400 (EDT) Message-ID: <3CFE8EE2.46020ACC@fourpalms.org> Date: Wed, 05 Jun 2002 15:21:22 -0700 From: Thomas Lockhart X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.8-34.1mdk i686) MIME-Version: 1.0 To: Dann Corbit cc: Bruce Momjian , Igor Kovalenko , PostgreSQL-development Subject: Re: [HACKERS] Roadmap for a Win32 port References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO ... > Notion: > Have one version do both. Your server can fork(), and your sever can > thread. It can fork() and thread, it can fork() or thread. > That gives the best of all worlds. One client who has his attachments > to a database all setup might want to do a bunch of similar queries. > Hence a threaded model is nice. > A server may be set up to clone the rights of the attaching process for > security reasons. Then you launch a new server with fork(). Right. If/when that is possible then let's do it, as long as the cost is not too high. But the intermediate steps are a possibility also, and are not precluded from discussion. This will all work out as a *convergence* of interests imho. And there is no great identifiable benefit for our current crop of platforms for going to a threaded model *unless* that enables queries for a single client to execute in parallel (all imho of course ;). So our convergence of interests for all platforms is in enabling threading for these two purposes, and focusing on enabling the multithreaded single client *first* means that the current crop of clients don't have to accept all negatives while we start on the road to better support of Win32 machines. - Thomas ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M23422@postgresql.org Wed Jun 5 19:23:03 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g55NN2s03496 for ; Wed, 5 Jun 2002 19:23:02 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 66BF747611A; Wed, 5 Jun 2002 19:22:57 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 9D9824765FF; Wed, 5 Jun 2002 19:05:26 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 7939A476591 for ; Wed, 5 Jun 2002 19:05:08 -0400 (EDT) Received: from mail.celerityonline.com (unknown [65.90.8.22]) by postgresql.org (Postfix) with SMTP id 52DB8476304 for ; Wed, 5 Jun 2002 18:49:42 -0400 (EDT) Received: (qmail 11143 invoked from network); 5 Jun 2002 23:09:05 -0000 Received: from unknown (HELO jmf1) (199.3.237.161) by 0 with SMTP; 5 Jun 2002 23:09:05 -0000 Message-ID: <001201c20ce3$6e21bed0$a1ed03c7@dev.ngcn> From: "Jon Franz" To: References: <200206052005.g55K52615577@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port Date: Wed, 5 Jun 2002 18:50:46 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO One note: SGI developers discovered they could get amazing performance using as hybrid threaded and forked-process model with apache - we might want to look into this. They even have a library for network-communication utilizing thier 'state threads' model. Please see: http://state-threads.sourceforge.net/docs/st.html Thus, on platforms where it can be supported, we should keep in mind that a hybrid multiprocess/multithreaded postgresql might be the fastest solution... ----- Original Message ----- From: "Bruce Momjian" To: "Igor Kovalenko" Cc: "PostgreSQL-development" Sent: Wednesday, June 05, 2002 4:05 PM Subject: Re: [HACKERS] Roadmap for a Win32 port > Igor Kovalenko wrote: > > I might be naive here, but would not proper threading model remove the need > > for fork() altogether? On both Unix and Win32? Should not be too hard to > > come up with abstraction which encapsulates POSIX, BeOS and Win32 threads... > > I am not sure how universal POSIX threads are by now. Any important Unix > > platforms which don't support them yet? > > > > This has downside of letting any bug to kill the whole thing. On the bright > > side, performance should be better on some platforms (note however, Apache > > group still can't come up with implementation of threaded model which would > > provide better performance than forked or other models). The need to deal > > with possibility of 'alien' postmaster running along with orphaned backends > > would also be removed since there would be only one process. > > > > Issue of thread safety of code will come up undoubtedly and some things will > > probably have to be revamped. But in long term this is probably best way if > > you want to have efficient and uniform Unix AND Win32 implementations. > > > > I am not too familiar with Win32. Speaking about POSIX threads, it would be > > something like a thread pool with low & high watermarks. Main thread would > > handle thread pool and hand over requests to worker threads (blocked on > > condvar). How does that sound? > > Good summary. I think we would support both threaded and fork() > operation, and users can control which they prefer. For a web backend > where many sessions are a single query, people may want to give up the > stability of fork() and go with threads, even on Unix. > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 853-3000 > + If your life is a hard drive, | 830 Blythe Avenue > + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M23423@postgresql.org Wed Jun 5 20:10:50 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g560Aos07172 for ; Wed, 5 Jun 2002 20:10:50 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id D99C3475A0B; Wed, 5 Jun 2002 20:10:46 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id A128A476354; Wed, 5 Jun 2002 20:08:02 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id B15E44759BA for ; Wed, 5 Jun 2002 20:07:52 -0400 (EDT) Received: from klamath.dyndns.org (CPE002078144ae0.cpe.net.cable.rogers.com [24.102.202.35]) by postgresql.org (Postfix) with ESMTP id ADAE6475999 for ; Wed, 5 Jun 2002 20:07:45 -0400 (EDT) Received: from boston (unknown [192.168.40.12]) by klamath.dyndns.org (Postfix) with SMTP id 13A207010; Wed, 5 Jun 2002 20:07:48 -0400 (EDT) Date: Wed, 5 Jun 2002 20:05:44 -0400 From: Neil Conway To: "Jon Franz" cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Roadmap for a Win32 port Message-ID: <20020605200544.4a486fe4.nconway@klamath.dyndns.org> In-Reply-To: <001201c20ce3$6e21bed0$a1ed03c7@dev.ngcn> References: <200206052005.g55K52615577@candle.pha.pa.us> <001201c20ce3$6e21bed0$a1ed03c7@dev.ngcn> X-Mailer: Sylpheed version 0.7.6 (GTK+ 1.2.10; i386-debian-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr On Wed, 5 Jun 2002 18:50:46 -0400 "Jon Franz" wrote: > One note: SGI developers discovered they could get amazing performance using > as hybrid threaded and forked-process model with apache - we might want to > look into this. They even have a library for network-communication > utilizing thier 'state threads' model. I think ST is designed for network I/O-bound apps -- last I checked, disk I/O will still block an entire ST process. While you can get around that by using another process to do disk I/O, it sounds like ST won't be that useful. However, Chris KL. (I believe) raised the idea of using POSIX AIO for PostgreSQL. Without having looked into it extensively, this technique sounds promising. Perhaps someone who has looked into this further (e.g. someone from Redhat) can comment? Cheers, Neil -- Neil Conway PGP Key ID: DB3C29FC ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M23424@postgresql.org Wed Jun 5 20:56:59 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g560uws10602 for ; Wed, 5 Jun 2002 20:56:58 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 33E48475FC6; Wed, 5 Jun 2002 20:56:55 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id C7F5C476365; Wed, 5 Jun 2002 20:52:49 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 7BF93476263 for ; Wed, 5 Jun 2002 20:52:34 -0400 (EDT) Received: from ingenico.com.au (unknown [202.167.40.101]) by postgresql.org (Postfix) with ESMTP id 2E2064764A7 for ; Wed, 5 Jun 2002 20:50:40 -0400 (EDT) Message-ID: <004301c20cf4$1b8d8c10$660d090a@software.ingenico.com.au> From: "Nicolas Bazin" To: "Jon Franz" , References: <200206052005.g55K52615577@candle.pha.pa.us> <001201c20ce3$6e21bed0$a1ed03c7@dev.ngcn> Subject: Re: [HACKERS] Roadmap for a Win32 port Date: Thu, 6 Jun 2002 10:50:09 +1000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Yes I proposed to use the GNU Pth library instead. It's an event demultiplexer just like the sgi library, but has a posix thread interface. This architecture is actually the more robust and also the more scalable. On a single processor server, you don't have the multi-thread synchronization and context switching overhead and you also take full advantage of multi-processor servers when you create several processes. Plus you have much less concern about global variables. Also for those concerned about the licence of this library here is an abstract of it: "The author places this library under the LGPL to make sure that it can be used both commercially and non-commercially provided that modifications to the code base are always donated back to the official code base under the same license conditions. Please keep in mind that especially using this library in code not staying under the GPL or the LGPL _is_ allowed and that any taint or license creap into code that uses the library is not the authors intention. It is just the case that _including_ this library into the source tree of other applications is a little bit more inconvinient because of the LGPL. But it has to be this way for good reasons. And keep in mind that inconvinient doesn't mean not allowed or even impossible." So it can be used in both commercial and non commercial project. ----- Original Message ----- From: "Jon Franz" To: Sent: Thursday, June 06, 2002 8:50 AM Subject: Re: [HACKERS] Roadmap for a Win32 port > One note: SGI developers discovered they could get amazing performance using > as hybrid threaded and forked-process model with apache - we might want to > look into this. They even have a library for network-communication > utilizing thier 'state threads' model. Please see: > > http://state-threads.sourceforge.net/docs/st.html > > Thus, on platforms where it can be supported, we should keep in mind that a > hybrid multiprocess/multithreaded postgresql might be the fastest > solution... > > > ----- Original Message ----- > From: "Bruce Momjian" > To: "Igor Kovalenko" > Cc: "PostgreSQL-development" > Sent: Wednesday, June 05, 2002 4:05 PM > Subject: Re: [HACKERS] Roadmap for a Win32 port > > > > Igor Kovalenko wrote: > > > I might be naive here, but would not proper threading model remove the > need > > > for fork() altogether? On both Unix and Win32? Should not be too hard to > > > come up with abstraction which encapsulates POSIX, BeOS and Win32 > threads... > > > I am not sure how universal POSIX threads are by now. Any important Unix > > > platforms which don't support them yet? > > > > > > This has downside of letting any bug to kill the whole thing. On the > bright > > > side, performance should be better on some platforms (note however, > Apache > > > group still can't come up with implementation of threaded model which > would > > > provide better performance than forked or other models). The need to > deal > > > with possibility of 'alien' postmaster running along with orphaned > backends > > > would also be removed since there would be only one process. > > > > > > Issue of thread safety of code will come up undoubtedly and some things > will > > > probably have to be revamped. But in long term this is probably best way > if > > > you want to have efficient and uniform Unix AND Win32 implementations. > > > > > > I am not too familiar with Win32. Speaking about POSIX threads, it would > be > > > something like a thread pool with low & high watermarks. Main thread > would > > > handle thread pool and hand over requests to worker threads (blocked on > > > condvar). How does that sound? > > > > Good summary. I think we would support both threaded and fork() > > operation, and users can control which they prefer. For a web backend > > where many sessions are a single query, people may want to give up the > > stability of fork() and go with threads, even on Unix. > > > > -- > > Bruce Momjian | http://candle.pha.pa.us > > pgman@candle.pha.pa.us | (610) 853-3000 > > + If your life is a hard drive, | 830 Blythe Avenue > > + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M23425@postgresql.org Wed Jun 5 21:01:51 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5611ps10999 for ; Wed, 5 Jun 2002 21:01:51 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id E700E476412; Wed, 5 Jun 2002 21:01:30 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id CE484476430; Wed, 5 Jun 2002 21:00:21 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 37F39475A7F for ; Wed, 5 Jun 2002 21:00:10 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id EA1694763B1 for ; Wed, 5 Jun 2002 20:53:19 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g560rF010370; Wed, 5 Jun 2002 20:53:15 -0400 (EDT) From: Bruce Momjian Message-ID: <200206060053.g560rF010370@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <20020605200544.4a486fe4.nconway@klamath.dyndns.org> To: Neil Conway Date: Wed, 5 Jun 2002 20:53:15 -0400 (EDT) cc: Jon Franz , pgsql-hackers@postgresql.org X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Neil Conway wrote: > On Wed, 5 Jun 2002 18:50:46 -0400 > "Jon Franz" wrote: > > One note: SGI developers discovered they could get amazing performance using > > as hybrid threaded and forked-process model with apache - we might want to > > look into this. They even have a library for network-communication > > utilizing thier 'state threads' model. > > I think ST is designed for network I/O-bound apps -- last I checked, > disk I/O will still block an entire ST process. While you can get around > that by using another process to do disk I/O, it sounds like ST won't be > that useful. > > However, Chris KL. (I believe) raised the idea of using POSIX AIO for > PostgreSQL. Without having looked into it extensively, this technique > sounds promising. Perhaps someone who has looked into this further > (e.g. someone from Redhat) can comment? I know Red Hat is interested in AIO. Only a few OS's support it so it was hard to get exited about it at the time, but with threading, a AIO-specific module could be attempted. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M23426@postgresql.org Wed Jun 5 21:07:19 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5617Js11634 for ; Wed, 5 Jun 2002 21:07:19 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6F5E14761D1; Wed, 5 Jun 2002 21:07:16 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id B3CC8476464; Wed, 5 Jun 2002 21:03:39 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id F20694762DC for ; Wed, 5 Jun 2002 21:03:27 -0400 (EDT) Received: from candeias.terra.com.br (candeias.terra.com.br [200.176.3.18]) by postgresql.org (Postfix) with ESMTP id 62EE047594F for ; Wed, 5 Jun 2002 21:01:57 -0400 (EDT) Received: from smtp4-poa.terra.com.br (smtp4-poa.terra.com.br [200.176.3.35]) by candeias.terra.com.br (Postfix) with ESMTP id B442E43DB2; Wed, 5 Jun 2002 22:01:59 -0300 (EST) Received: from 200.176.55.246 (cm-net-cwb-C8B037F6.brdterra.com.br [200.176.55.246]) (authenticated user howe) by smtp4-poa.terra.com.br (Postfix) with ESMTP id 0F7F3AC5A2; Wed, 5 Jun 2002 22:01:57 -0300 (EST) Date: Wed, 5 Jun 2002 22:05:11 -0300 From: Steve Howe X-Mailer: The Bat! (v1.60i) Reply-To: Steve Howe Organization: ACME X-Priority: 3 (Normal) Message-ID: <527177268.20020605220511@carcass.dhs.org> To: Thomas Lockhart cc: Bruce Momjian , Igor Kovalenko , PostgreSQL-development Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <3CFE8A79.84738F2F@fourpalms.org> References: <200206052005.g55K52615577@candle.pha.pa.us> <3CFE8A79.84738F2F@fourpalms.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Hello Thomas, Wednesday, June 5, 2002, 7:02:33 PM, you wrote: TL> ... >> Good summary. I think we would support both threaded and fork() >> operation, and users can control which they prefer. For a web backend >> where many sessions are a single query, people may want to give up the >> stability of fork() and go with threads, even on Unix. TL> I would think that we would build on our strengths of having a fork/exec TL> model for separate clients. A threaded model *could* benefit individual TL> clients who are doing queries on multiprocessor servers, and I would be TL> supportive of efforts to enable that. Just a note - this is also the solution adopted by Interbase/Firebird and it seems interesting. They already had the same problems PostgreSQL has been under today. Those interested in read about Interbase's architeture, please refer to http://community.borland.com/article/0,1410,23217,00.html. "Classic" is the fork() model, and the "SuperServer" is the threaded model. ------------- Best regards, Steve Howe mailto:howe@carcass.dhs.org ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M23427@postgresql.org Wed Jun 5 21:24:58 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g561Ovs13529 for ; Wed, 5 Jun 2002 21:24:57 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 503F8476458; Wed, 5 Jun 2002 21:24:53 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 75959475FC6; Wed, 5 Jun 2002 21:24:26 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 87A5F475C05 for ; Wed, 5 Jun 2002 21:23:59 -0400 (EDT) Received: from ingenico.com.au (unknown [202.167.40.101]) by postgresql.org (Postfix) with ESMTP id 0DE4B475BC7 for ; Wed, 5 Jun 2002 21:23:57 -0400 (EDT) Message-ID: <002001c20cf8$a6b86400$660d090a@software.ingenico.com.au> From: "Nicolas Bazin" To: "Nicolas Bazin" , "Jon Franz" , References: <200206052005.g55K52615577@candle.pha.pa.us> <001201c20ce3$6e21bed0$a1ed03c7@dev.ngcn> <004301c20cf4$1b8d8c10$660d090a@software.ingenico.com.au> Subject: Re: [HACKERS] Roadmap for a Win32 port Date: Thu, 6 Jun 2002 11:22:40 +1000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Gnu Pth also supports AIO ----- Original Message ----- From: "Nicolas Bazin" To: "Jon Franz" ; Sent: Thursday, June 06, 2002 10:50 AM Subject: Re: [HACKERS] Roadmap for a Win32 port > Yes I proposed to use the GNU Pth library instead. It's an event > demultiplexer just like the sgi library, but has a posix thread interface. > This architecture is actually the more robust and also the more scalable. On > a single processor server, you don't have the multi-thread synchronization > and context switching overhead and you also take full advantage of > multi-processor servers when you create several processes. Plus you have > much less concern about global variables. > > Also for those concerned about the licence of this library here is an > abstract of it: > "The author places this library under the LGPL to make sure that it > can be used both commercially and non-commercially provided that > modifications to the code base are always donated back to the official > code base under the same license conditions. Please keep in mind that > especially using this library in code not staying under the GPL or > the LGPL _is_ allowed and that any taint or license creap into code > that uses the library is not the authors intention. It is just the > case that _including_ this library into the source tree of other > applications is a little bit more inconvinient because of the LGPL. > But it has to be this way for good reasons. And keep in mind that > inconvinient doesn't mean not allowed or even impossible." > > So it can be used in both commercial and non commercial project. > > > ----- Original Message ----- > From: "Jon Franz" > To: > Sent: Thursday, June 06, 2002 8:50 AM > Subject: Re: [HACKERS] Roadmap for a Win32 port > > > > One note: SGI developers discovered they could get amazing performance > using > > as hybrid threaded and forked-process model with apache - we might want to > > look into this. They even have a library for network-communication > > utilizing thier 'state threads' model. Please see: > > > > http://state-threads.sourceforge.net/docs/st.html > > > > Thus, on platforms where it can be supported, we should keep in mind that > a > > hybrid multiprocess/multithreaded postgresql might be the fastest > > solution... > > > > > > ----- Original Message ----- > > From: "Bruce Momjian" > > To: "Igor Kovalenko" > > Cc: "PostgreSQL-development" > > Sent: Wednesday, June 05, 2002 4:05 PM > > Subject: Re: [HACKERS] Roadmap for a Win32 port > > > > > > > Igor Kovalenko wrote: > > > > I might be naive here, but would not proper threading model remove the > > need > > > > for fork() altogether? On both Unix and Win32? Should not be too hard > to > > > > come up with abstraction which encapsulates POSIX, BeOS and Win32 > > threads... > > > > I am not sure how universal POSIX threads are by now. Any important > Unix > > > > platforms which don't support them yet? > > > > > > > > This has downside of letting any bug to kill the whole thing. On the > > bright > > > > side, performance should be better on some platforms (note however, > > Apache > > > > group still can't come up with implementation of threaded model which > > would > > > > provide better performance than forked or other models). The need to > > deal > > > > with possibility of 'alien' postmaster running along with orphaned > > backends > > > > would also be removed since there would be only one process. > > > > > > > > Issue of thread safety of code will come up undoubtedly and some > things > > will > > > > probably have to be revamped. But in long term this is probably best > way > > if > > > > you want to have efficient and uniform Unix AND Win32 implementations. > > > > > > > > I am not too familiar with Win32. Speaking about POSIX threads, it > would > > be > > > > something like a thread pool with low & high watermarks. Main thread > > would > > > > handle thread pool and hand over requests to worker threads (blocked > on > > > > condvar). How does that sound? > > > > > > Good summary. I think we would support both threaded and fork() > > > operation, and users can control which they prefer. For a web backend > > > where many sessions are a single query, people may want to give up the > > > stability of fork() and go with threads, even on Unix. > > > > > > -- > > > Bruce Momjian | http://candle.pha.pa.us > > > pgman@candle.pha.pa.us | (610) 853-3000 > > > + If your life is a hard drive, | 830 Blythe Avenue > > > + Christ can be your backup. | Drexel Hill, Pennsylvania > 19026 > > > > > > ---------------------------(end of broadcast)--------------------------- > > > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > > > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster > > ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M23428@postgresql.org Wed Jun 5 21:36:56 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g561aus16058 for ; Wed, 5 Jun 2002 21:36:56 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id D2A20475DCB; Wed, 5 Jun 2002 21:36:52 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 04705476424; Wed, 5 Jun 2002 21:34:17 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 761234761D1 for ; Wed, 5 Jun 2002 21:34:06 -0400 (EDT) Received: from matinhos.terra.com.br (matinhos.terra.com.br [200.176.3.21]) by postgresql.org (Postfix) with ESMTP id C366E4760C3 for ; Wed, 5 Jun 2002 21:33:59 -0400 (EDT) Received: from mucuri.terra.com.br (mucuri.terra.com.br [200.176.3.39]) by matinhos.terra.com.br (Postfix) with ESMTP id 2AF4346F9A; Wed, 5 Jun 2002 22:34:03 -0300 (EST) Received: from 200.176.55.246 (cm-net-cwb-C8B037F6.brdterra.com.br [200.176.55.246]) (authenticated user howe) by mucuri.terra.com.br (Postfix) with ESMTP id BDDA2BE8C8; Wed, 5 Jun 2002 22:34:02 -0300 (EST) Date: Wed, 5 Jun 2002 22:37:17 -0300 From: Steve Howe X-Mailer: The Bat! (v1.60i) Reply-To: Steve Howe Organization: ACME X-Priority: 3 (Normal) Message-ID: <17029102737.20020605223717@carcass.dhs.org> To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <200206050433.g554XiN05245@candle.pha.pa.us> References: <200206050433.g554XiN05245@candle.pha.pa.us> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Hello Bruce, Wednesday, June 5, 2002, 1:33:44 AM, you wrote: BM> INSTALLER BM> --------- BM> We clearly need an installer that is zero-hassle for users. We need to BM> decide on a direction for this. I suggest Nullsoft install system (http://www.nullsoft.com/free/nsis/). It's real good and very simple to use. I can help on this if you want. BM> ENVIRONMENT BM> Lots of our code requires a unix shell and utilities. Will we continue BM> using cygwin for this? There are other ports ( http://unxutils.sourceforge.net/ ) that won't require Cygwin but they won't provide an environment so complete as Cygwin does. I also would like to empathize that probably a small GUI for controlling the PostgreSQL service/application would be nice. I think about something sitting in the system tray like MSSQL, Oracle, Interbase, etc. does. I could code this in Delphi if you like. I don't have experience in writing GUI apps in C. There is an open source versions of Delphi so it won't be a problem compiling it. Also coming with this, a code for starting the PostgreSQL as a service would be really nice. For those from UNIX world that don't know what a service is, think about it as a daemon for Windows. A service can be automatically started when the machine boots up. ------------- Best regards, Steve Howe mailto:howe@carcass.dhs.org ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M23429@postgresql.org Wed Jun 5 22:54:37 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g562sas28774 for ; Wed, 5 Jun 2002 22:54:36 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5F28C475956; Wed, 5 Jun 2002 22:54:32 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id B91B8476423; Wed, 5 Jun 2002 22:54:05 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 1A253475999 for ; Wed, 5 Jun 2002 22:53:53 -0400 (EDT) Received: from ftpbox.mot.com (ftpbox.mot.com [129.188.136.101]) by postgresql.org (Postfix) with ESMTP id ED3F8475956 for ; Wed, 5 Jun 2002 22:53:47 -0400 (EDT) Received: [from pobox4.mot.com (pobox4.mot.com [10.64.251.243]) by ftpbox.mot.com (ftpbox 2.1) with ESMTP id TAA19790; Wed, 5 Jun 2002 19:53:41 -0700 (MST)] Received: [from pronto1.comm.mot.com (pronto1.comm.mot.com [173.6.1.22]) by pobox4.mot.com (MOT-pobox4 2.0) with ESMTP id TAA00693; Wed, 5 Jun 2002 19:53:40 -0700 (MST)] Received: from kovalenkoigor (idennt19534 [145.1.195.34]) by pronto1.comm.mot.com (8.9.3/8.9.3) with SMTP id VAA03882; Wed, 5 Jun 2002 21:53:39 -0500 (CDT) Message-ID: <129501c20d05$7a671a10$22c30191@comm.mot.com> From: "Igor Kovalenko" To: "Neil Conway" , "Jon Franz" cc: References: <200206052005.g55K52615577@candle.pha.pa.us><001201c20ce3$6e21bed0$a1ed03c7@dev.ngcn> <20020605200544.4a486fe4.nconway@klamath.dyndns.org> Subject: Re: [HACKERS] Roadmap for a Win32 port Date: Wed, 5 Jun 2002 21:54:29 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr I think SGI gets amazing performance because they have very good (efficient) synchronisation primitives on SGI. Some proprietary light-weight mutexes. Using threaded or mixed model just by itself is not going to do a miracle. Threads will save you some context switch time, but that will probably translate into lower CPU usage rather than performance boost. And if your mutexes are not fast or awkwardly implemented (say Linux), it might be even worse. Apache is not all that fast on Linux as on SGI, whatever model you chose. I also doubt that purely threaded model would be slower than mixed one. Now about the AIO model. It is useful when you need to do something else while I/O requests are being processed as long as platform does it in some useful way. If all you can do is to submit requests and keep sitting in select/poll then AIO does not buy you anything you can't get by just using threaded model. However, if you can tag the requests and set up notifications, then few I/O threads could handle relatively large number of requests from different clients. Note, this means you don't have any association between clients and servers at all, there is pool of generic I/O threads which serve requests from whoever they come. It saves system resources and scales very well. It also provides interesting possibilities for fault recovery - since handlers are generic all the state information would have to be kept in some kind of global context area. That area can be saved into persistent memory or dumped onto disk and *recovered* after a forced restart. Server and library could be designed in such a way that clients may continue where they left with a recoverable error. In POSIX AIO model you can tag requests and set up notifications via synchronous signals. You wait for them *synchronously* in 'waiter' thread via sigwaitinfo() and avoid the headache of asynchronous signals hitting you any time... Unfortunately on some platforms (Solaris) the depth of synchronous signal queue is fixed at magic value 32 (and not adjustable). This may not be a problem if you're sure that waiting thread will be able to drain the queue faster than it gets filled with notifications... but I'm not sure there is a portable way to guarantee that, so you need to check for overloads and handle them... that complicates things. On Solaris you also need a mile of compiler/linker switches to even get this scheme to work and I am afraid other platforms may not support it at all (but then again, they may not support AIO to begin with). And speaking about getting best of all worlds. Note how Apache spent nearly 3 years developing their portable Multi-Processing Modules scheme. What they got for that is handful of models neither of which perform noticeably better than original pre-fork() model. Trying to swallow all possible ways to handle things on all possible platforms usually does not produce very fast code. It tends to produce very complex code with mediocre performance and introduces extra complexity into configuration process. If you consider all that was done mostly to support Win32, one might doubt if it was worth the while. What I am trying to say is, extra complexity in model to squeeze few percent of performance is not a wise investment of time and efforts. On Win32 you don't really compete in terms of performance. You compete in terms of easyness and features. Spend 3 years trying to support Windows and Unix in most optimal way including all subvariants of Unix ... meanwhile MSFT will come up with some bundled SQL server. It probably will have more features since they will spend time doing features rather than inventing a model to support gazillion of platforms. Chances are, it will be faster too - due to better integration with OS and better compiler. I am not in position to tell you what to do guys. But if I was asked, I'd say supporting Win32 is only worth it if it comes as a natural result of a simple, coherent and uniform model applied to Unix. Threaded model may not have as much inherent stability as forked/mixed, but it has inherent simplicity and better Unix/Windows/BeOS portability. It can be done faster and simpler code will make work on features easier. Regards, - Igor "There are 2 ways to design an efficient system - first is to design it so complex that there are no obvious deficiencies, second is to design it so simple that there are obviously no deficiencies. Second way is much harder" (author unknown to me) ----- Original Message ----- From: "Neil Conway" To: "Jon Franz" Cc: Sent: Wednesday, June 05, 2002 7:05 PM Subject: Re: [HACKERS] Roadmap for a Win32 port > On Wed, 5 Jun 2002 18:50:46 -0400 > "Jon Franz" wrote: > > One note: SGI developers discovered they could get amazing performance using > > as hybrid threaded and forked-process model with apache - we might want to > > look into this. They even have a library for network-communication > > utilizing thier 'state threads' model. > > I think ST is designed for network I/O-bound apps -- last I checked, > disk I/O will still block an entire ST process. While you can get around > that by using another process to do disk I/O, it sounds like ST won't be > that useful. > > However, Chris KL. (I believe) raised the idea of using POSIX AIO for > PostgreSQL. Without having looked into it extensively, this technique > sounds promising. Perhaps someone who has looked into this further > (e.g. someone from Redhat) can comment? > > Cheers, > > Neil > > -- > Neil Conway > PGP Key ID: DB3C29FC > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster > ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M23430@postgresql.org Wed Jun 5 22:58:30 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g562wUs29151 for ; Wed, 5 Jun 2002 22:58:30 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 04083476398; Wed, 5 Jun 2002 22:58:23 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id D1058476456; Wed, 5 Jun 2002 22:57:50 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5C9D447627F for ; Wed, 5 Jun 2002 22:57:40 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id 2E0344760D1 for ; Wed, 5 Jun 2002 22:56:59 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g562v0q28990 for pgsql-hackers@postgreSQL.org; Wed, 5 Jun 2002 22:57:00 -0400 (EDT) From: Bruce Momjian Message-ID: <200206060257.g562v0q28990@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <200206050433.g554XiN05245@candle.pha.pa.us> To: PostgreSQL-development Date: Wed, 5 Jun 2002 22:57:00 -0400 (EDT) X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Here is a summary of the responses to my Win32 roadmap. I hope this will allow further discussion. --------------------------------------------------------------------------- INSTALLER --------- Cygwin Setup.exe http://cygwin.com Nullsoft http://www.nullsoft.com/free/nsis/ GUI --- pgAdmin2 http://pgadmin.postgresql.org/pgadmin2.php?ContentID=1 pgaccess http://pgaccess.org/ Java admin (to be written) Dev-C++ admin (to be written) http://sourceforge.net/projects/dev-cpp/ BINARY ------ FORK() cygwin fork() http://cygwin.com CreateProcess() and copy global area THREADING Posix threads Gnu pth http://www.gnu.org/software/pth/ ST http://state-threads.sourceforge.net/docs/st.html (single-session multi-threading possible) (Posix AIO is possible) IPC Cygwin http://cygwin.com MinGW http://www.mingw.org/ ACE http://www.cs.wustl.edu/~schmidt/ACE.html APR http://apr.apache.org/ Our own ENVIRONMENT Cygwin http://cygwin.com UnxUtils http://unxutils.sourceforge.net/ Write own initdb IMPLEMENTATIONS --------------- PostgreSQLe http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html Dbexperts http://www.dbexperts.net/postgresql Connx http://www.connx.com/ gborg http://gborg.postgresql.org/project/winpackage/projdisplay.php Interbase http://community.borland.com/article/0,1410,23217,00.html -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M23432@postgresql.org Thu Jun 6 01:01:09 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g56518s09229 for ; Thu, 6 Jun 2002 01:01:08 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 4E125476364; Thu, 6 Jun 2002 01:01:10 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id C658E47644F; Thu, 6 Jun 2002 01:00:02 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id BB1CF475ADA for ; Thu, 6 Jun 2002 00:59:53 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id 85238475AFB for ; Thu, 6 Jun 2002 00:59:51 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g564xaS09100; Thu, 6 Jun 2002 00:59:36 -0400 (EDT) From: Bruce Momjian Message-ID: <200206060459.g564xaS09100@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <129501c20d05$7a671a10$22c30191@comm.mot.com> To: Igor Kovalenko Date: Thu, 6 Jun 2002 00:59:36 -0400 (EDT) cc: Neil Conway , Jon Franz , pgsql-hackers@postgresql.org X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Igor Kovalenko wrote: > I think SGI gets amazing performance because they have very good (efficient) > synchronization primitives on SGI. Some proprietary light-weight mutexes. > Using threaded or mixed model just by itself is not going to do a miracle. > Threads will save you some context switch time, but that will probably > translate into lower CPU usage rather than performance boost. And if your > mutexes are not fast or awkwardly implemented (say Linux), it might be even > worse. Apache is not all that fast on Linux as on SGI, whatever model you > chose. I also doubt that purely threaded model would be slower than mixed > one. Let me throw out an idea. I have been mentioning full fork, light fork(copy globals only), and threading as possible solutions. Another idea uses neither threading nor copying. It is the old system we used before I removed exec() from our code. We used to pass the database name as an argument to an exec'ed postgres binary that continued with the database connection. We removed the exec, then started moving what we could into the postmaster so each backend didn't need to do the initialization. One solution is to return to that for Win32 only, so instead of doing: initialization() want for connection() fork backend() we do for Win32: want for connection() exec backend() initialization() It wouldn't be hard to do. We would still do CreateProcess rather than CreateThread, but it eliminates the fork/threading issues. We don't know the database before the connection arrives, so we don't do a whole lot of initialization. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From janwieck@yahoo.com Thu Jun 6 09:40:20 2002 Return-path: Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g56DeJs00804 for ; Thu, 6 Jun 2002 09:40:19 -0400 (EDT) Received: from unknown (HELO saturn.janwieck.net) (janwieck@192.233.92.205 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 6 Jun 2002 13:40:18 -0000 Received: (from wieck@localhost) by saturn.janwieck.net (8.11.2/8.11.2) id g56DZFt26403; Thu, 6 Jun 2002 09:35:15 -0400 From: Jan Wieck Message-ID: <200206061335.g56DZFt26403@saturn.janwieck.net> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <200206060459.g564xaS09100@candle.pha.pa.us> from Bruce Momjian at "Jun 6, 2002 00:59:36 am" To: Bruce Momjian Date: Thu, 6 Jun 2002 09:35:14 -0400 (EDT) cc: Igor Kovalenko , Neil Conway , Jon Franz , pgsql-hackers@postgresql.org X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Status: RO Bruce Momjian wrote: > > Let me throw out an idea. I have been mentioning full fork, light > fork(copy globals only), and threading as possible solutions. > > Another idea uses neither threading nor copying. It is the old system > we used before I removed exec() from our code. We used to pass the > database name as an argument to an exec'ed postgres binary that > continued with the database connection. > > We removed the exec, then started moving what we could into the > postmaster so each backend didn't need to do the initialization. > > One solution is to return to that for Win32 only, so instead of doing: > > initialization() > want for connection() > fork backend() > > we do for Win32: > > want for connection() > exec backend() > initialization() Summarizes pretty much what we discussed Monday on the phone. Except that the postmaster still has to initialize the shared memory and other stuff. It's just that the backends and helper processes need to reinitialize themself (attach). > It wouldn't be hard to do. We would still do CreateProcess rather than > CreateThread, but it eliminates the fork/threading issues. We don't > know the database before the connection arrives, so we don't do a whole > lot of initialization. All I see so far is the reading of the postgresql.conf, the pg_hba.conf and the password files. Nothing fancy and the postmaster could easily write out a binary content only file that the backends then read, eliminating the parsing overhead. The bad news is that Tom is right. We did a terrible job in using the new side effect, that the shared memory segment is at the same address in all forked processes, after removing the need to reattach. In detail the XLog code, the FreeSpaceMap code and the "shared memory" hashtable code now use pointers, located in shared memory. For the XLog and FreeSpace code this is understandable, because they where developed under the fork() only model. But the dynahash code used offsets only until v7.1! All three (no claim that that's all) make it impossible to ever have someone attaching to the shared memory from the outside. So with these moves we made the shared memory a "Postmaster and children" only thing. Raises the question, why we need an IPC key at all any more. Anyhow, looks as if I can get that fork() vs. fork()+exec() feature done pretty soon. It'll be controlled by another Postmaster commandline switch. After cleaning up the mess I did to get it working quick, I'll provide a patch for discussion. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com # From pgsql-hackers-owner+M23442@postgresql.org Thu Jun 6 11:14:35 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g56FEZs11270 for ; Thu, 6 Jun 2002 11:14:35 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 9055E4765D8; Thu, 6 Jun 2002 11:14:24 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id F0E4C476584; Thu, 6 Jun 2002 11:11:44 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5646247635D for ; Thu, 6 Jun 2002 11:11:33 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id DF1964765F7 for ; Thu, 6 Jun 2002 11:06:24 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g56F6IW10267; Thu, 6 Jun 2002 11:06:18 -0400 (EDT) From: Bruce Momjian Message-ID: <200206061506.g56F6IW10267@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <200206061335.g56DZFt26403@saturn.janwieck.net> To: Jan Wieck Date: Thu, 6 Jun 2002 11:06:18 -0400 (EDT) cc: Igor Kovalenko , Neil Conway , Jon Franz , pgsql-hackers@postgresql.org X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Jan Wieck wrote: > > One solution is to return to that for Win32 only, so instead of doing: > > > > initialization() > > want for connection() > > fork backend() > > > > we do for Win32: > > > > want for connection() > > exec backend() > > initialization() > > Summarizes pretty much what we discussed Monday on the phone. > Except that the postmaster still has to initialize the shared > memory and other stuff. It's just that the backends and > helper processes need to reinitialize themself (attach). Yes, obviously I simplified, and I do believe our optimizations are helping on Unix. It is just that I think for Win32 the fork is more harmful than removing those optimizations. One thing that may not have been clear is that we don't need to play with globals at all. We just pass whatever info we want to the child via command-line arguments, rather than shared memory. > > It wouldn't be hard to do. We would still do CreateProcess rather than > > CreateThread, but it eliminates the fork/threading issues. We don't > > know the database before the connection arrives, so we don't do a whole > > lot of initialization. > > All I see so far is the reading of the postgresql.conf, the > pg_hba.conf and the password files. Nothing fancy and the > postmaster could easily write out a binary content only file > that the backends then read, eliminating the parsing > overhead. Yes, that is clearly possible. Another option is to just write out a no-comments, no-whitespace version of each file and just have the backends read those. The advantage is that we can use the same code to read them, and I don't think it would be any slower than a binary file. > The bad news is that Tom is right. We did a terrible job in > using the new side effect, that the shared memory segment is > at the same address in all forked processes, after removing > the need to reattach. > > In detail the XLog code, the FreeSpaceMap code and the > "shared memory" hashtable code now use pointers, located in > shared memory. For the XLog and FreeSpace code this is > understandable, because they where developed under the fork() > only model. But the dynahash code used offsets only until > v7.1! > > All three (no claim that that's all) make it impossible to > ever have someone attaching to the shared memory from the > outside. So with these moves we made the shared memory a > "Postmaster and children" only thing. Raises the question, > why we need an IPC key at all any more. Well, we could force shmat() to bind to the same address, but I suspect that might fail in some cases. > Anyhow, looks as if I can get that fork() vs. fork()+exec() > feature done pretty soon. It'll be controlled by another > Postmaster commandline switch. After cleaning up the mess I > did to get it working quick, I'll provide a patch for > discussion. Yes, very little impact. We then need someone to do some Win32 timings to see if things have improved. As Tom mentioned, we need some hard numbers for these things. In fact, I would like a Win32 test that takes our code and compares fork(), then exit(), with CreateProcess(), exit(). It doesn't have create a db session, but I would like to see some timings to know what we are gaining. Heck, time CreateThread too and let's see what that shows. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M23447@postgresql.org Thu Jun 6 13:18:53 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g56HIqs22989 for ; Thu, 6 Jun 2002 13:18:52 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 099C0476682; Thu, 6 Jun 2002 13:18:51 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 43A26476705; Thu, 6 Jun 2002 13:15:06 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 1D3D447667B for ; Thu, 6 Jun 2002 13:14:55 -0400 (EDT) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by postgresql.org (Postfix) with SMTP id 52DB64765A1 for ; Thu, 6 Jun 2002 13:10:32 -0400 (EDT) Received: (qmail 15366 invoked by uid 0); 6 Jun 2002 17:10:31 -0000 Received: from pd902f0fa.dip0.t-ipconnect.de (217.2.240.250) by mail.gmx.net (mp007-rz3) with SMTP; 6 Jun 2002 17:10:31 -0000 Date: Thu, 6 Jun 2002 19:11:12 +0200 (CEST) From: Peter Eisentraut X-X-Sender: peter@localhost.localdomain To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <200206050433.g554XiN05245@candle.pha.pa.us> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr Bruce Momjian writes: > Lots of our code requires a unix shell and utilities. Will we continue > using cygwin for this? We should probably get rid of using shell scripts for application programs altogether, for a number of reasons besides this one, such as the inability to properly handle input values with spaces, commas, etc. (we probably don't handle very long values either on some platforms), the inability to maintain open database connections so that createlang needs to prompt for the same password thrice, general portable scripting headaches, and the lack of internationalization facilities. I'd even volunteer to do this. Comments? -- Peter Eisentraut peter_e@gmx.net ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M23453@postgresql.org Thu Jun 6 15:28:52 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g56JSqs06231 for ; Thu, 6 Jun 2002 15:28:52 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 3F4C2476828; Thu, 6 Jun 2002 15:28:50 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 96B12476900; Thu, 6 Jun 2002 14:52:04 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 4313B476855 for ; Thu, 6 Jun 2002 14:51:53 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id 88ED6476852 for ; Thu, 6 Jun 2002 13:57:10 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g56Hv5D27106; Thu, 6 Jun 2002 13:57:05 -0400 (EDT) From: Bruce Momjian Message-ID: <200206061757.g56Hv5D27106@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: To: Peter Eisentraut Date: Thu, 6 Jun 2002 13:57:05 -0400 (EDT) cc: PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Peter Eisentraut wrote: > Bruce Momjian writes: > > > Lots of our code requires a unix shell and utilities. Will we continue > > using cygwin for this? > > We should probably get rid of using shell scripts for application programs > altogether, for a number of reasons besides this one, such as the > inability to properly handle input values with spaces, commas, etc. (we > probably don't handle very long values either on some platforms), the > inability to maintain open database connections so that createlang needs > to prompt for the same password thrice, general portable scripting > headaches, and the lack of internationalization facilities. > > I'd even volunteer to do this. Comments? I know I have discouraged it because I think shell script language has a good toolset for those applications. I have fixed all the spacing issues. What language where you thinking of using? C? Also, it seems Win32 doesn't need these scripts, except initdb. PostgreSQLe didn't use the, it just did initdb, and the rest were done using a GUI. However, initdb would remain a problem. PostgreSQLe wrote its own. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M23466@postgresql.org Fri Jun 7 03:43:24 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g577hNs29367 for ; Fri, 7 Jun 2002 03:43:23 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 0CDD6475AF5; Fri, 7 Jun 2002 03:43:20 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id E5D12476626; Fri, 7 Jun 2002 03:42:51 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id EB46E4758D8 for ; Fri, 7 Jun 2002 03:42:33 -0400 (EDT) Received: from anchor-post-30.mail.demon.net (anchor-post-30.mail.demon.net [194.217.242.88]) by postgresql.org (Postfix) with ESMTP id 276C247517F for ; Fri, 7 Jun 2002 03:42:33 -0400 (EDT) Received: from mailgate.vale-housing.co.uk ([193.195.77.162] helo=dogbert.vale-housing.co.uk) by anchor-post-30.mail.demon.net with esmtp (Exim 3.35 #1) id 17GEOH-000ApS-0U; Fri, 07 Jun 2002 08:42:33 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: Re: [HACKERS] Roadmap for a Win32 port X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 7 Jun 2002 08:42:33 +0100 Message-ID: Thread-Topic: Roadmap for a Win32 port Thread-Index: AcINqHe1e6l54BF0TYqaBR+Lc+A3PwATWOng From: "Dave Page" To: "Steve Howe" , "Bruce Momjian" cc: "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g577hNs29367 Status: RO > -----Original Message----- > From: Steve Howe [mailto:howe@carcass.dhs.org] > Sent: 06 June 2002 02:37 > To: Bruce Momjian > Cc: PostgreSQL-development > Subject: Re: Roadmap for a Win32 port > > > Hello Bruce, > > Wednesday, June 5, 2002, 1:33:44 AM, you wrote: > > BM> INSTALLER > BM> --------- > > BM> We clearly need an installer that is zero-hassle for > users. We need > BM> to decide on a direction for this. > I suggest Nullsoft install system > (http://www.nullsoft.com/free/nsis/). It's > real good and very > simple to use. I can help on this if you want. I think that a Windows Installer compatible package would be better as it would allow us to build the package as a merge module which others could use in their installers for their PostgreSQL based apps, allowing one installation to install everything they require easily and (more importantly) correctly. An example of this can be found in the psqlODBC installer. I can handle this if required. > BM> ENVIRONMENT > > I also would like to empathize that probably a small GUI for > controlling the PostgreSQL service/application would be nice. I'm happy to add such code to pgAdmin - seems like the natural thing to do (to me at least!). Regards, Dave. ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M23560@postgresql.org Mon Jun 10 10:09:49 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5AE9ms04329 for ; Mon, 10 Jun 2002 10:09:48 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 6D20B4762F3; Mon, 10 Jun 2002 09:46:34 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id A1731476422; Mon, 10 Jun 2002 09:30:48 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 134104763B9 for ; Fri, 7 Jun 2002 18:08:40 -0400 (EDT) Mailbox-Line: From ss@technicalpursuit.com Fri Jun 7 18:08:40 2002 Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by postgresql.org (Postfix) with ESMTP id 54385476379 for ; Fri, 7 Jun 2002 18:08:38 -0400 (EDT) Received: from idearatxp ([12.253.59.186]) by sccrmhc01.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with SMTP id <20020607220840.TUBC1024.sccrmhc01.attbi.com@idearatxp> for ; Fri, 7 Jun 2002 22:08:40 +0000 Message-ID: <032201c20e6f$815a1b40$80c310ac@idearatxp> From: "Scott Shattuck" To: "PostgreSQL-development" References: Subject: Re: [HACKERS] Roadmap for a Win32 port Date: Fri, 7 Jun 2002 16:05:58 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-Spam-Status: Yes, hits=5.0 required=5.0 tests=RCVD_IN_OSIRUSOFT_COM,X_OSIRU_SPAM_SRC version=2.20 X-Spam-Flag: YES X-Spam-Level: ***** X-Spam-Checker-Version: SpamAssassin 2.20 (devel $Id: win32,v 1.5 2003/05/23 16:34:35 momjian Exp $) X-Spam-Report: Detailed Report SPAM: -------------------- Start SpamAssassin results ---------------------- SPAM: This mail is probably spam. The original message has been altered SPAM: so you can recognise or block similar unwanted mail in future. SPAM: See http://spamassassin.org/tag/ for more details. SPAM: SPAM: Content analysis details: (5 hits, 5 required) SPAM: Hit! (2.0 points) Received via a relay in relays.osirusoft.com SPAM: [RBL check: found 61.202.127.204.relays.osirusoft.com., type: 127.0.0.4] SPAM: Hit! (3.0 points) DNSBL: sender is Confirmed Spam Source SPAM: SPAM: -------------------- End of SpamAssassin results --------------------- Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO How about a SOAP interface and a web-based front end that provides the cross platform support? My company's TIBET framework would provide a solid foundation for this kind of admin suite. In fact, we're already in the planning stages on doing just that. ss Scott Shattuck Technical Pursuit Inc. ----- Original Message ----- From: "Peter Eisentraut" To: "Bruce Momjian" Cc: "PostgreSQL-development" Sent: Friday, June 07, 2002 11:42 AM Subject: Re: [HACKERS] Roadmap for a Win32 port > Bruce Momjian writes: > > > GUI > > --- > > pgAdmin2 http://pgadmin.postgresql.org/pgadmin2.php?ContentID=1 > > pgaccess http://pgaccess.org/ > > Java admin (to be written) > > Dev-C++ admin (to be written) http://sourceforge.net/projects/dev-cpp/ > > Surely Unix folks would like a GUI as well? > > -- > Peter Eisentraut peter_e@gmx.net > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M23489@postgresql.org Fri Jun 7 18:31:35 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g57MVYs01005 for ; Fri, 7 Jun 2002 18:31:34 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id CFFC547671F; Fri, 7 Jun 2002 18:31:28 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 682B44767FC; Fri, 7 Jun 2002 18:27:47 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 63436476915 for ; Fri, 7 Jun 2002 18:27:34 -0400 (EDT) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by postgresql.org (Postfix) with SMTP id 1A6AB47666E for ; Fri, 7 Jun 2002 18:27:09 -0400 (EDT) Received: (qmail 15805 invoked by uid 0); 7 Jun 2002 22:27:10 -0000 Received: from pd902f0ad.dip0.t-ipconnect.de (217.2.240.173) by mail.gmx.net (mp002-rz3) with SMTP; 7 Jun 2002 22:27:10 -0000 Date: Sat, 8 Jun 2002 00:27:59 +0200 (CEST) From: Peter Eisentraut X-X-Sender: peter@localhost.localdomain To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: <200206061757.g56Hv5D27106@candle.pha.pa.us> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr Bruce Momjian writes: > I know I have discouraged it because I think shell script language has a > good toolset for those applications. I have fixed all the spacing > issues. My point is that it is not, for the reasons that I listed. Handling spaces is a small part of one of the several problems, there are problems with newlines, tabs, commas, slashes, quotes -- everytime you call sed or read you lose one character. > What language where you thinking of using? C? Yes, that way we can share code (pg_dumpall<->pg_dump, initdb<->postgres), use the established internationalization facilities, and use libpq directly in create* and drop*. > Also, it seems Win32 doesn't need these scripts, except initdb. The utility of these programs is independent of the platform. If we think pg_dumpall is not useful, then let's remove it. -- Peter Eisentraut peter_e@gmx.net ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From tgl@sss.pgh.pa.us Sat Jun 8 11:48:49 2002 Return-path: Received: from sss.pgh.pa.us (root@[192.204.191.242]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g58Fmns11593 for ; Sat, 8 Jun 2002 11:48:49 -0400 (EDT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id g58FmKb13969; Sat, 8 Jun 2002 11:48:21 -0400 (EDT) To: Peter Eisentraut cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: References: Comments: In-reply-to Peter Eisentraut message dated "Sat, 08 Jun 2002 00:27:59 +0200" Date: Sat, 08 Jun 2002 11:48:20 -0400 Message-ID: <13966.1023551300@sss.pgh.pa.us> From: Tom Lane Status: ROr Peter Eisentraut writes: >> Also, it seems Win32 doesn't need these scripts, except initdb. > The utility of these programs is independent of the platform. If we think > pg_dumpall is not useful, then let's remove it. I have been seriously considering converting pg_dumpall to C anyway, because it's already *very* messy, and I don't see any reasonable way to make it support dumping per-database and per-user config settings. (Do you really want to try to parse array values in a shell script?) (I'd actually consider making pg_dumpall a part of the pg_dump executable; then it could invoke pg_dump as a subroutine call...) If Peter's got the time/energy to convert 'em all, I'm for it. regards, tom lane From pgsql-hackers-owner+M23524@postgresql.org Sat Jun 8 17:48:57 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g58Lmus13022 for ; Sat, 8 Jun 2002 17:48:56 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 1C2704758FB; Sat, 8 Jun 2002 17:48:54 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 27ACF475DD1; Sat, 8 Jun 2002 17:48:29 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id E92D64759A6 for ; Sat, 8 Jun 2002 17:48:17 -0400 (EDT) Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35]) by postgresql.org (Postfix) with ESMTP id 0CDA4475951 for ; Sat, 8 Jun 2002 17:48:16 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g58LmCa13018; Sat, 8 Jun 2002 17:48:12 -0400 (EDT) From: Bruce Momjian Message-ID: <200206082148.g58LmCa13018@candle.pha.pa.us> Subject: Re: [HACKERS] Roadmap for a Win32 port In-Reply-To: To: Peter Eisentraut Date: Sat, 8 Jun 2002 17:48:12 -0400 (EDT) cc: PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL97 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: ROr Peter Eisentraut wrote: > Bruce Momjian writes: > > > I know I have discouraged it because I think shell script language has a > > good toolset for those applications. I have fixed all the spacing > > issues. > > My point is that it is not, for the reasons that I listed. Handling > spaces is a small part of one of the several problems, there are problems > with newlines, tabs, commas, slashes, quotes -- everytime you call sed or > read you lose one character. > > > What language where you thinking of using? C? > > Yes, that way we can share code (pg_dumpall<->pg_dump, initdb<->postgres), > use the established internationalization facilities, and use libpq > directly in create* and drop*. > > > Also, it seems Win32 doesn't need these scripts, except initdb. > > The utility of these programs is independent of the platform. If we think > pg_dumpall is not useful, then let's remove it. I think the first two targets for C-ification would be pg_dumpall and initdb. The others have SQL equivalents. Maybe pg_ctl too. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M23541@postgresql.org Sun Jun 9 07:10:14 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g59BAEs18595 for ; Sun, 9 Jun 2002 07:10:14 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 78848475BCE; Sun, 9 Jun 2002 07:09:39 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 34984475908; Sun, 9 Jun 2002 06:41:03 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 8A3F84758E6 for ; Sun, 9 Jun 2002 06:40:01 -0400 (EDT) Received: from anchor-post-30.mail.demon.net (anchor-post-30.mail.demon.net [194.217.242.88]) by postgresql.org (Postfix) with ESMTP id 1ACAB4758E6 for ; Sun, 9 Jun 2002 06:38:35 -0400 (EDT) Received: from mailgate.vale-housing.co.uk ([193.195.77.162] helo=dogbert.vale-housing.co.uk) by anchor-post-30.mail.demon.net with esmtp (Exim 3.35 #1) id 17H05a-000G8M-0U; Sun, 09 Jun 2002 11:38:27 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: Re: [HACKERS] Roadmap for a Win32 port X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Sun, 9 Jun 2002 11:38:26 +0100 Message-ID: Thread-Topic: Roadmap for a Win32 port Thread-Index: AcIPOqo6zyhIgICZRXi+W7OR5HfNigAZldCg From: "Dave Page" To: "Bruce Momjian" , "Peter Eisentraut" cc: "PostgreSQL-development" Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g59BAEs18595 Status: RO > -----Original Message----- > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] > Sent: 08 June 2002 22:48 > To: Peter Eisentraut > Cc: PostgreSQL-development > Subject: Re: Roadmap for a Win32 port > > > > > > > Also, it seems Win32 doesn't need these scripts, except initdb. > > > > The utility of these programs is independent of the > platform. If we > > think pg_dumpall is not useful, then let's remove it. > > I think the first two targets for C-ification would be > pg_dumpall and initdb. The others have SQL equivalents. > Maybe pg_ctl too. I looked at this issue some time ago & came to the conclusion that the only scripts that Win32 really needed were pg_dumpall, initdb & initlocation. The others have SQL equivalents as you say, apart from pg_ctl which under Windows should probably (and generally is) be replaced by the SCM (Service Control Manager). The only thing that comes to mind that the SCM can't do is a reload. Regards, Dave. ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M23406@postgresql.org Wed Jun 5 01:06:31 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5556Vs08055 for ; Wed, 5 Jun 2002 01:06:31 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 169794762A7; Wed, 5 Jun 2002 01:06:33 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id C60EB47625B; Wed, 5 Jun 2002 01:06:25 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id EB789475C75 for ; Wed, 5 Jun 2002 01:06:14 -0400 (EDT) Received: from voyager.corporate.connx.com (unknown [209.20.248.131]) by postgresql.org (Postfix) with ESMTP id 4B950475A3B for ; Wed, 5 Jun 2002 01:06:14 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: [HACKERS] Cooperation X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 content-class: urn:content-classes:message Date: Tue, 4 Jun 2002 22:06:31 -0700 Message-ID: Thread-Topic: Cooperation Thread-Index: AcIMTsGaRnYrKZMWTBSGbLPlP1vlAQ== From: "Dann Corbit" To: , , cc: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g5556Vs08055 Status: RO I apologize for my English language message. I am unable to speak Japanese. We do have a native Japanese speaker here, who could be called upon if necessary. The PostgreSQL team is planning to do a native Win32 port. Perhaps you would like to help with the effort. In that way, your changes will get propagated back up the source code tree and you can gain the benefits from future development efforts without performing any work. We did a port to Win32 also, but your approach seems much better. We have very fat executables and you have a marvelous DLL approach. Probably, the way that you perform the operations is much better. ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M23451@postgresql.org Thu Jun 6 15:09:15 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g56J9Es04710 for ; Thu, 6 Jun 2002 15:09:14 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 233E7476798; Thu, 6 Jun 2002 15:09:12 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 53BA8476C21; Thu, 6 Jun 2002 13:48:28 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 5FF6E475B2B for ; Thu, 6 Jun 2002 03:39:06 -0400 (EDT) Received: from ns.astrodesign.co.jp (ns.astrodesign.co.jp [210.169.228.106]) by postgresql.org (Postfix) with ESMTP id A7083475AD1 for ; Thu, 6 Jun 2002 03:39:00 -0400 (EDT) Date: Thu, 6 Jun 2002 16:38:55 +0900 From: ISHIKAWA Toshiyuki To: "Dann Corbit" cc: sogapj@fb.freeserve.ne.jp, ichiro@ichiro.org, pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Cooperation Message-ID: <20020606163855.4d8f2be5.t-ishikawa@astrodesign.co.jp> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Dann Corbit wrote: > I apologize for my English language message. I am unable to speak > Japanese. We do have a native Japanese speaker here, who could be > called upon if necessary. There is no need to aplogize writing an e-mail in English. It's global standards, but some portion is a bit difficult to understand. Anyhow, we must firstly express our thanks for your interest in our project, though we are facing also hard obstacles as listed on the Web site. > The PostgreSQL team is planning to do a native Win32 port. Perhaps you > would like to help with the effort. In that way, your changes will get > propagated back up the source code tree and you can gain the benefits > from future development efforts without performing any work. It is nice to hear that the PostgreSQL development team has also working on this subject. Will you please illustrate the procedure more clearly how to we contribute our effort to your project. The last four words in the above clause mean that once we supply you with the changed source, then everything afterwords could be handled by the team? How the copy right will be dealt with? The development has been continued by the volunteer developers here, however, we have to admit that businesses (companies) are also involved to support those people providing time to work on the development, not to commercialization purpose but expecting some return, e.g. earning company's prestige. So, we have to regulate those backgrouds first based upon your proposal. We are positive to help you with our effort anyway, if things goes well. > We did a port to Win32 also, but your approach seems much better. We > have very fat executables and you have a marvelous DLL approach. > Probably, the way that you perform the operations is much better. Thanks. Toshi ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From dpage@vale-housing.co.uk Sun Jun 9 06:38:29 2002 Return-path: Received: from anchor-post-30.mail.demon.net (anchor-post-30.mail.demon.net [194.217.242.88]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g59AcSs05017 for ; Sun, 9 Jun 2002 06:38:29 -0400 (EDT) Received: from mailgate.vale-housing.co.uk ([193.195.77.162] helo=dogbert.vale-housing.co.uk) by anchor-post-30.mail.demon.net with esmtp (Exim 3.35 #1) id 17H05a-000G8M-0U; Sun, 09 Jun 2002 11:38:27 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: RE: Roadmap for a Win32 port X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Sun, 9 Jun 2002 11:38:26 +0100 Message-ID: Thread-Topic: Roadmap for a Win32 port Thread-Index: AcIPOqo6zyhIgICZRXi+W7OR5HfNigAZldCg From: "Dave Page" To: "Bruce Momjian" , "Peter Eisentraut" cc: "PostgreSQL-development" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g59AcSs05017 Status: RO > -----Original Message----- > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] > Sent: 08 June 2002 22:48 > To: Peter Eisentraut > Cc: PostgreSQL-development > Subject: Re: Roadmap for a Win32 port > > > > > > > Also, it seems Win32 doesn't need these scripts, except initdb. > > > > The utility of these programs is independent of the > platform. If we > > think pg_dumpall is not useful, then let's remove it. > > I think the first two targets for C-ification would be > pg_dumpall and initdb. The others have SQL equivalents. > Maybe pg_ctl too. I looked at this issue some time ago & came to the conclusion that the only scripts that Win32 really needed were pg_dumpall, initdb & initlocation. The others have SQL equivalents as you say, apart from pg_ctl which under Windows should probably (and generally is) be replaced by the SCM (Service Control Manager). The only thing that comes to mind that the SCM can't do is a reload. Regards, Dave. From pgsql-hackers-owner+M23620@postgresql.org Tue Jun 11 10:20:54 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5BEKrs26980 for ; Tue, 11 Jun 2002 10:20:53 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id E71DB476475; Tue, 11 Jun 2002 10:20:47 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id E021E47641C; Tue, 11 Jun 2002 10:20:29 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 78C77475905 for ; Tue, 11 Jun 2002 10:20:17 -0400 (EDT) Received: from mail.gne.de (mail.gne.de [213.83.0.2]) by postgresql.org (Postfix) with ESMTP id C02B0475864 for ; Tue, 11 Jun 2002 10:20:15 -0400 (EDT) Received: from DO5GNE-MTA by mail.gne.de with Novell_GroupWise; Tue, 11 Jun 2002 16:19:39 +0200 Message-ID: X-Mailer: Novell GroupWise Internet Agent 6.0.1 Date: Tue, 11 Jun 2002 16:19:21 +0200 From: "Ulrich Neumann" To: Subject: [HACKERS] Native Win32/OS2/BeOS/NetWare ports MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Guinevere: 1.1.14 ; GNE Grebe Neumann Gl Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Hello together i've seen a lot of discussion about a native win32/OS2/BEOS port of PostgreSQL. During the last months i've ported PostgreSQL over to Novell NetWare and i've changed the code that I use pthreads instead of fork() now. I had a lot of work with the variables and cleanup but mayor parts are done. I would appreciate if we could combine this work. My plan was to finish this port, discuss the port with other people and offer all the work to the PostgreSQL source tree, but now i'm jumping in here because of all the discussions. What i've done in detail: - i've defined #USE_PTHREADS in pg_config.h to differentiate between the forked and the threaded backend. - I've added several parts in postmaster.c so all functions are based on pthreads now. - I've changed the signal handling because signals are process based - I've changed code in ipc.c to have a clean shutdown of threads - I've written some functions to switch the global variables. The globals are controled with POSIX semaphores. - I've written a new implementation of shared memory and semaphores- With pthreads I don't need real shared memory any more and i'm using POSIX semaphores now - Several minor changes. There is still some more work to do like fixing memory leaks or handling bad situations, but in general it's functional on NetWare. BTW: Is it possible to add some lines on the PostgreSQL webpage that there is a first beta of PostgreSQL for NetWare available and to offer a binary download for the NetWare version? Ulrich Neumann ---------------------------------- This e-mail is virus scanned Diese e-mail ist virusgeprueft ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M23641@postgresql.org Tue Jun 11 15:01:17 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5BJ1Gs23405 for ; Tue, 11 Jun 2002 15:01:17 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 950F7476CC5; Tue, 11 Jun 2002 15:01:14 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 113114768E2; Tue, 11 Jun 2002 14:48:15 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 83CF5475B98 for ; Tue, 11 Jun 2002 14:48:02 -0400 (EDT) Received: from ftpbox.mot.com (ftpbox.mot.com [129.188.136.101]) by postgresql.org (Postfix) with ESMTP id BC57F4760D1 for ; Tue, 11 Jun 2002 14:14:13 -0400 (EDT) Received: [from pobox.mot.com (pobox.mot.com [129.188.137.100]) by ftpbox.mot.com (ftpbox 2.1) with ESMTP id LAA03778; Tue, 11 Jun 2002 11:14:07 -0700 (MST)] Received: [from pronto1.comm.mot.com (pronto1.comm.mot.com [173.6.1.22]) by pobox.mot.com (MOT-pobox 2.0) with ESMTP id LAA14457; Tue, 11 Jun 2002 11:14:07 -0700 (MST)] Received: from kovalenkoigor (idennt19534 [145.1.195.34]) by pronto1.comm.mot.com (8.9.3/8.9.3) with SMTP id NAA23438; Tue, 11 Jun 2002 13:14:06 -0500 (CDT) Message-ID: <00c901c21173$e5f95870$22c30191@comm.mot.com> From: "Igor Kovalenko" To: "Ulrich Neumann" , References: Subject: Re: [HACKERS] Native Win32/OS2/BeOS/NetWare ports Date: Tue, 11 Jun 2002 13:14:58 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO > Hello together > > i've seen a lot of discussion about a native win32/OS2/BEOS port of > PostgreSQL. > > During the last months i've ported PostgreSQL over to Novell NetWare > and i've > changed the code that I use pthreads instead of fork() now. > > I had a lot of work with the variables and cleanup but mayor parts are > done. > > I would appreciate if we could combine this work. Very nice... I have patches for QNX6 which also involved redoing shared memory and sempahores stuff. It would make very good sense to intergate, especially since you managed to do something very close to what I wanted :) > My plan was to finish this port, discuss the port with other people and > offer all the work > to the PostgreSQL source tree, but now i'm jumping in here because of > all the discussions. > > What i've done in detail: > - i've defined #USE_PTHREADS in pg_config.h to differentiate between > the forked and the > threaded backend. > - I've added several parts in postmaster.c so all functions are based > on pthreads now. > - I've changed the signal handling because signals are process based Careful here. On certain systems (on many, I suspect) POSIX semantics for signals is NOT default. Enforcing POSIX semantics requires certain compile time switches which will also change behavior of various functions. > - I've changed code in ipc.c to have a clean shutdown of threads > - I've written some functions to switch the global variables. The > globals are controled with > POSIX semaphores. > - I've written a new implementation of shared memory and semaphores- > With pthreads I don't > need real shared memory any more and i'm using POSIX semaphores now POSIX semaphores for what? I assume by the conext that you're talking about replacing SysV semaphores which are used to control access to shared memory. If that is the case, POSIX semaphores are not the best choice really. POSIX mutexes would be okay, but on SMP systems spinlocks (hardware TAS based macros or POSIX spinlocks) would probably be better anyway. Note that on most platforms spinlocks are used for that and SysV semaphores were just a 'last resort' which had unacceptable performance and so I guess it was not used at all. Do you have your patch somewhere online? -- igor ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M23662@postgresql.org Wed Jun 12 04:38:26 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g5C8cQs16000 for ; Wed, 12 Jun 2002 04:38:26 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id BFF694758D8; Wed, 12 Jun 2002 04:38:16 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 627664767B0; Wed, 12 Jun 2002 04:37:56 -0400 (EDT) Received: from localhost.localdomain (postgresql.org [64.49.215.8]) by localhost (Postfix) with ESMTP id 26D69475961 for ; Wed, 12 Jun 2002 04:37:42 -0400 (EDT) Received: from mail.gne.de (mail.gne.de [213.83.0.2]) by postgresql.org (Postfix) with ESMTP id BE69B475D95 for ; Wed, 12 Jun 2002 04:36:40 -0400 (EDT) Received: from DO5GNE-MTA by mail.gne.de with Novell_GroupWise; Wed, 12 Jun 2002 10:35:57 +0200 Message-ID: X-Mailer: Novell GroupWise Internet Agent 6.0.1 Date: Wed, 12 Jun 2002 10:35:24 +0200 From: "Ulrich Neumann" To: Subject: Antw: Re: [HACKERS] Native Win32/OS2/BeOS/NetWare ports MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Guinevere: 1.1.14 ; GNE Grebe Neumann Gl Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: RO Hi Igor, Thanks for your information. I was aware of the "signal" problems and i've done it with thread based signals This part is functional on my platform but it isn't fully cooked. Another problem is to make this part portable. Your assumption to replace SysV semaphores with POSIX semaphores is correct. My first guess was to use mutexes instead of semaphores at all because the way semaphores are used in Postgres is more something like a "mutex", but only semaphores worked for me at this time because the underlying C Library had some problems with mutexes and spinlocks. (I'm also working on a new C Library for a future OS). Actually I don't have my code downloadable somewhere because the code doesn't look very nice in some parts. There is also temporary debug code in it right now. The best I think is to send it to you via email. If this is OK please give me a short notice or send an email to me and I'll send you a copy. Ulrich >>> "Igor Kovalenko" 11.06.2002 20:14:58 >>> > Hello together > > i've seen a lot of discussion about a native win32/OS2/BEOS port of > PostgreSQL. > > During the last months i've ported PostgreSQL over to Novell NetWare > and i've > changed the code that I use pthreads instead of fork() now. > > I had a lot of work with the variables and cleanup but mayor parts are > done. > > I would appreciate if we could combine this work. Very nice... I have patches for QNX6 which also involved redoing shared memory and sempahores stuff. It would make very good sense to intergate, especially since you managed to do something very close to what I wanted :) > My plan was to finish this port, discuss the port with other people and > offer all the work > to the PostgreSQL source tree, but now i'm jumping in here because of > all the discussions. > > What i've done in detail: > - i've defined #USE_PTHREADS in pg_config.h to differentiate between > the forked and the > threaded backend. > - I've added several parts in postmaster.c so all functions are based > on pthreads now. > - I've changed the signal handling because signals are process based Careful here. On certain systems (on many, I suspect) POSIX semantics for signals is NOT default. Enforcing POSIX semantics requires certain compile time switches which will also change behavior of various functions. > - I've changed code in ipc.c to have a clean shutdown of threads > - I've written some functions to switch the global variables. The > globals are controled with > POSIX semaphores. > - I've written a new implementation of shared memory and semaphores- > With pthreads I don't > need real shared memory any more and i'm using POSIX semaphores now POSIX semaphores for what? I assume by the conext that you're talking about replacing SysV semaphores which are used to control access to shared memory. If that is the case, POSIX semaphores are not the best choice really. POSIX mutexes would be okay, but on SMP systems spinlocks (hardware TAS based macros or POSIX spinlocks) would probably be better anyway. Note that on most platforms spinlocks are used for that and SysV semaphores were just a 'last resort' which had unacceptable performance and so I guess it was not used at all. Do you have your patch somewhere online? -- igor ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html ---------------------------------- This e-mail is virus scanned Diese e-mail ist virusgeprueft ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M28769@postgresql.org Thu Sep 12 13:30:45 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8CHUhE10966 for ; Thu, 12 Sep 2002 13:30:44 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 40E144762BC; Thu, 12 Sep 2002 13:29:53 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 74F634761BD; Thu, 12 Sep 2002 13:29:51 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id B0228475BEC; Thu, 12 Sep 2002 13:29:47 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id 54553475AE5; Thu, 12 Sep 2002 13:29:46 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id NAA10985; Thu, 12 Sep 2002 13:30:07 -0400 Message-ID: <3D80CEF0.1010900@mascari.com> Date: Thu, 12 Sep 2002 13:29:20 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mike Mascari cc: "scott.marlowe" , Justin Clift , Dave Page , Bruce Momjian , PostgreSQL Hackers Mailing List Subject: Re: [HACKERS] PGXLOG variable worthwhile? References: <3D80C847.1070000@mascari.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO I wrote: > scott.marlowe wrote: >> >> I wouldn't assume that. It's been years since I tested it, but back >> then, the command line and all program I used could see the link >> created by ln that came with the resource kit. They were distinctly >> different from the shortcut type of links, in that they seems >> transparent like short cuts in unix generally are. >> >> Do you have the resource kit or the gnu utils from it? > > > The situation appears to be this: > > 1. Soft links are available on NTFS 5 (2K/XP) as Reparse Points via the > DeviceIoControl() function for any application using the standard C > library routines. > > 2. Soft links are available on any filesystem under 95/98/ME/NT4/2K/XP > as OLE streams (.lnk files) for Shell-aware applications. > > 3. Hard links are available on NTFS 5 (2K/XP) via the CreateHardLink() API. > 4. Hard links are available on NTFS (NT3.1/NT4) via the BackupWrite() > API by writing a special stream to the NTFS. I also believe (I could be wrong) that for directories, the only two methods of links are the Soft link methods above. So PGXLOG cannot use soft links on a non-XP/2K machine unless it is "Shell-Aware". For example, in a cygwin bash command window: mkdir dir1 ln dir1 dir2 <- Error using Cygwin implementation ln -s dir1 dir2 <- Creates a Shell short-cut (NT4) echo "Hello" > dir1/test.txt cat dir2/test.txt "Hello" <- Cygwin's cat(bash?) is shell short-cut aware Now, in a Windows NT command prompt: notepad dir2\test.txt <- Notepad can't find file notepad dir2.lnk <- Displays link contents That means for a native port with a different PGXLOG directory running on NT4, the only choice *using links* is to make the native port shell short-cut aware. I could be wrong but I don't think so. Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From mascarm@mascari.com Thu Sep 12 13:01:55 2002 Return-path: Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8CH1oE07922 for ; Thu, 12 Sep 2002 13:01:53 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id NAA10893; Thu, 12 Sep 2002 13:01:42 -0400 Message-ID: <3D80C847.1070000@mascari.com> Date: Thu, 12 Sep 2002 13:00:55 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "scott.marlowe" cc: Justin Clift , Dave Page , Bruce Momjian , PostgreSQL Hackers Mailing List Subject: Re: [HACKERS] PGXLOG variable worthwhile? References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean Status: RO scott.marlowe wrote: > On Fri, 13 Sep 2002, Justin Clift wrote: > >>Would it be correct to say that the 'ln' command in the MS Resource Kit >>creates this kind of shortcut too, as the Reparse Points feature doesn't >>seem to be possible under NT4? > > > I wouldn't assume that. It's been years since I tested it, but back then, > the command line and all program I used could see the link created by ln > that came with the resource kit. They were distinctly different from the > shortcut type of links, in that they seems transparent like short cuts in > unix generally are. > > Do you have the resource kit or the gnu utils from it? The situation appears to be this: 1. Soft links are available on NTFS 5 (2K/XP) as Reparse Points via the DeviceIoControl() function for any application using the standard C library routines. 2. Soft links are available on any filesystem under 95/98/ME/NT4/2K/XP as OLE streams (.lnk files) for Shell-aware applications. 3. Hard links are available on NTFS 5 (2K/XP) via the CreateHardLink() API. See: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/createhardlink.asp 4. Hard links are available on NTFS (NT3.1/NT4) via the BackupWrite() API by writing a special stream to the NTFS. Example: http://www.mvps.org/win32/ntfs/lnw.cpp The cygwin implementation of link(): http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/winsup/cygwin/syscalls.cc?rev=1.149.2.23&content-type=text/x-cvsweb-markup&cvsroot=src 1. Will use CreateHardLink() if on 2K/XP 2. Will try to use the BackupWrite() method 3. Failing #2 will just copy the file See how fun Microsoft makes things? Mike Mascari mascarm@mascari.com From pgsql-hackers-owner+M28774@postgresql.org Thu Sep 12 15:12:54 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8CJCqE21744 for ; Thu, 12 Sep 2002 15:12:52 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 77755476CA2; Thu, 12 Sep 2002 15:12:48 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 98016476ABD; Thu, 12 Sep 2002 15:12:31 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 3A604476A7F; Thu, 12 Sep 2002 15:12:28 -0400 (EDT) Received: from sss.pgh.pa.us (unknown [192.204.191.242]) by postgresql.org (Postfix) with ESMTP id B93FD475D87; Thu, 12 Sep 2002 15:12:20 -0400 (EDT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.12.5/8.12.5) with ESMTP id g8CJC9Ko009064; Thu, 12 Sep 2002 15:12:09 -0400 (EDT) To: Curt Sampson cc: Justin Clift , PostgreSQL Hackers Mailing List Subject: Re: [HACKERS] PGXLOG variable worthwhile? In-Reply-To: References: Comments: In-reply-to Curt Sampson message dated "Fri, 13 Sep 2002 01:28:39 +0900" Date: Thu, 12 Sep 2002 15:12:09 -0400 Message-ID: <9063.1031857929@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: ROr Curt Sampson writes: > On Thu, 12 Sep 2002, Justin Clift wrote: >> Am just wondering if we've ever considered adding a PGXLOG environment >> variable that would point to the pg_xlog directory? > IMHO, a much better way to support this is to put this information into > the config file. That way it can't easily change when you happen to, say, > start postgres in the wrong window. Yes. We rejected environment-variable-based xlog location for reasons that apply equally well to Windows. The xlog location *must* be stored in a physical file in the data directory; anything else is too unsafe. The current technology for that is a symlink. While it doesn't have to be a symlink as opposed to some sort of config file, I don't have the slightest problem with saying that we don't support relocation of xlog on older Windoid platforms. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M29089@postgresql.org Thu Sep 19 01:07:35 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8J57XE17033 for ; Thu, 19 Sep 2002 01:07:34 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id C4A62476A3C; Thu, 19 Sep 2002 01:07:34 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id A818A4760B0; Thu, 19 Sep 2002 01:07:32 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 4E7414769C9 for ; Thu, 19 Sep 2002 01:07:26 -0400 (EDT) Received: from ece.rice.edu (ece.rice.edu [128.42.4.34]) by postgresql.org (Postfix) with ESMTP id B6B5447590C for ; Thu, 19 Sep 2002 01:07:24 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by ece.rice.edu (Postfix) with ESMTP id 337C368A68; Thu, 19 Sep 2002 00:07:24 -0500 (CDT) Received: from wallace.ece.rice.edu (wallace.ece.rice.edu [128.42.12.154]) by ece.rice.edu (Postfix) with ESMTP id 1EF0D68A65; Thu, 19 Sep 2002 00:07:23 -0500 (CDT) Received: from reedstrm by wallace.ece.rice.edu with local (Exim 3.34 #1 (Debian)) id 17rtX8-00042E-00; Thu, 19 Sep 2002 00:07:22 -0500 Date: Thu, 19 Sep 2002 00:07:22 -0500 From: "Ross J. Reedstrom" To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions Message-ID: <20020919050722.GC15352@rice.edu> Mail-Followup-To: "Ross J. Reedstrom" , Bruce Momjian , PostgreSQL-development References: <200209190001.g8J01gG13849@candle.pha.pa.us> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200209190001.g8J01gG13849@candle.pha.pa.us> User-Agent: Mutt/1.3.27i X-Virus-Scanned: by AMaViS snapshot-20020300 X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO On Wed, Sep 18, 2002 at 08:01:42PM -0400, Bruce Momjian wrote: > Second, when you unlink() a file on Win32, do applications continue > accessing the old file contents if they had the file open before the > unlink? I'm pretty sure it errors with 'file in use'. Pretty ugly, huh? Ross ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M29091@postgresql.org Thu Sep 19 01:24:54 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8J5OqE18478 for ; Thu, 19 Sep 2002 01:24:53 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id A5031476A31; Thu, 19 Sep 2002 01:24:52 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id F2ED047698F; Thu, 19 Sep 2002 01:24:50 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id BA69E476086 for ; Thu, 19 Sep 2002 01:24:47 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id 56A2D475FE3 for ; Thu, 19 Sep 2002 01:24:46 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id BAA15293; Thu, 19 Sep 2002 01:23:54 -0400 Message-ID: <3D895F60.4010902@mascari.com> Date: Thu, 19 Sep 2002 01:23:45 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <200209190001.g8J01gG13849@candle.pha.pa.us> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: ROr Bruce Momjian wrote: > I am working with several groups getting the Win32 port ready for 7.4 > and I have a few questions: > > What is the standard workaround for the fact that rename() isn't atomic > on Win32? Do we need to create our own locking around the > reading/writing of files that are normally updated in place using > rename()? Visual C++ comes with the source to Microsoft's C library: rename() calls MoveFile() which will error if: 1. The target file exists 2. The source file is in use MoveFileEx() (not available on 95/98) can overwrite the target file if it exists. The Apache APR portability library uses MoveFileEx() to rename files if under NT/XP/2K vs. a sequence of : 1. CreateFile() to test for target file existence 2. DeleteFile() to remove the target file 3. MoveFile() to rename the old file to new under Windows 95/98. Of course, some other process could create the target file between 2 and 3, so their rename() would just error out in that situation. I haven't tested it, but I recall reading somewhere that MoveFileEx() has the ability to rename an opened file. I'm 99% sure MoveFile() will fail if the source file is open. > > Second, when you unlink() a file on Win32, do applications continue > accessing the old file contents if they had the file open before the > unlink? > unlink() just calls DeleteFile() which will error if: 1. The target file is in use CreateFile() has the option: FILE_FLAG_DELETE_ON_CLOSE which might be able to be used to simulate traditional unlink() behavior. Hope that helps, Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M29090@postgresql.org Thu Sep 19 01:23:40 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8J5NcE18313 for ; Thu, 19 Sep 2002 01:23:38 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 8B685476A38; Thu, 19 Sep 2002 01:23:39 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 54C6E4769AA; Thu, 19 Sep 2002 01:23:37 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 47A9B4762E6 for ; Thu, 19 Sep 2002 01:23:33 -0400 (EDT) Received: from houston.familyhealth.com.au (unknown [203.59.48.253]) by postgresql.org (Postfix) with ESMTP id 9D4C6476171 for ; Thu, 19 Sep 2002 01:23:31 -0400 (EDT) Received: (from root@localhost) by houston.familyhealth.com.au (8.11.6/8.11.6) id g8J5NUJ42439 for pgsql-hackers@postgresql.org; Thu, 19 Sep 2002 13:23:30 +0800 (WST) (envelope-from chriskl@familyhealth.com.au) Received: from mariner (mariner.internal [192.168.0.101]) by houston.familyhealth.com.au (8.11.6/8.9.3) with SMTP id g8J5NSk42348; Thu, 19 Sep 2002 13:23:28 +0800 (WST) From: "Christopher Kings-Lynne" To: "Ross J. Reedstrom" cc: "PostgreSQL-development" Subject: Re: [HACKERS] Win32 rename()/unlink() questions Date: Thu, 19 Sep 2002 13:24:01 +0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <20020919050722.GC15352@rice.edu> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal X-scanner: scanned by Inflex 0.1.5c - (http://www.inflex.co.za/) X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO > On Wed, Sep 18, 2002 at 08:01:42PM -0400, Bruce Momjian wrote: > > > Second, when you unlink() a file on Win32, do applications continue > > accessing the old file contents if they had the file open before the > > unlink? > > I'm pretty sure it errors with 'file in use'. Pretty ugly, huh? Yeah - the windows filesystem is pretty poor when it comes to multiuser access. That's why even as administrator I cannot delete borked files and people's profiles and stuff off our NT server - the files are always 'in use'. Even if you kick all users off, reboot the machine, do whatever. It's terrible. Chris ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M29092@postgresql.org Thu Sep 19 01:32:51 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8J5WnE19160 for ; Thu, 19 Sep 2002 01:32:50 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id A34DA476A7D; Thu, 19 Sep 2002 01:32:50 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 80422476A62; Thu, 19 Sep 2002 01:32:48 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 496D1476A44 for ; Thu, 19 Sep 2002 01:32:44 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id 228C74769F6 for ; Thu, 19 Sep 2002 01:32:43 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id BAA15313; Thu, 19 Sep 2002 01:31:28 -0400 Message-ID: <3D896127.2070103@mascari.com> Date: Thu, 19 Sep 2002 01:31:19 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Christopher Kings-Lynne cc: "Ross J. Reedstrom" , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Christopher Kings-Lynne wrote: >>On Wed, Sep 18, 2002 at 08:01:42PM -0400, Bruce Momjian wrote: >> >> >>>Second, when you unlink() a file on Win32, do applications continue >>>accessing the old file contents if they had the file open before the >>>unlink? >> >>I'm pretty sure it errors with 'file in use'. Pretty ugly, huh? > > > Yeah - the windows filesystem is pretty poor when it comes to multiuser > access. That's why even as administrator I cannot delete borked files and > people's profiles and stuff off our NT server - the files are always 'in > use'. Even if you kick all users off, reboot the machine, do whatever. > It's terrible. > > Chris > Yep. That's why often it requires rebooting to uninstall software. How can the installer remove itself? Under Windows 95/98/ME, you have to manually add entries to WININIT.INI. With Windows NT/XP/2K, MoveFileEx() with a NULL target and the MOVEFILE_DELAY_UNTIL_REBOOT flag will add the appropriate entries into the system registry so that the next time the machine reboots it will remove the files specified. Its a real pain and a real hack of an OS. Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M29135@postgresql.org Thu Sep 19 16:26:02 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8JKPvE10469 for ; Thu, 19 Sep 2002 16:25:57 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 55024476101; Thu, 19 Sep 2002 16:25:40 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id B87A6476A3F; Thu, 19 Sep 2002 16:25:34 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id CC7F5476101 for ; Thu, 19 Sep 2002 16:24:38 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id E04B8475AFF for ; Thu, 19 Sep 2002 16:24:35 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8JKO1g10337; Thu, 19 Sep 2002 16:24:01 -0400 (EDT) From: Bruce Momjian Message-ID: <200209192024.g8JKO1g10337@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D895F60.4010902@mascari.com> To: Mike Mascari Date: Thu, 19 Sep 2002 16:24:01 -0400 (EDT) cc: PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: ROr Mike Mascari wrote: > Bruce Momjian wrote: > > I am working with several groups getting the Win32 port ready for 7.4 > > and I have a few questions: > > > > What is the standard workaround for the fact that rename() isn't atomic > > on Win32? Do we need to create our own locking around the > > reading/writing of files that are normally updated in place using > > rename()? > > Visual C++ comes with the source to Microsoft's C library: > > rename() calls MoveFile() which will error if: > > 1. The target file exists > 2. The source file is in use > > MoveFileEx() (not available on 95/98) can overwrite the target > file if it exists. The Apache APR portability library uses > MoveFileEx() to rename files if under NT/XP/2K vs. a sequence of : > > 1. CreateFile() to test for target file existence > 2. DeleteFile() to remove the target file > 3. MoveFile() to rename the old file to new > > under Windows 95/98. Of course, some other process could create > the target file between 2 and 3, so their rename() would just > error out in that situation. I haven't tested it, but I recall > reading somewhere that MoveFileEx() has the ability to rename an > opened file. I'm 99% sure MoveFile() will fail if the source > file is open. OK, I downloaded APR and see in apr_file_rename(): if (MoveFileEx(frompath, topath, MOVEFILE_REPLACE_EXISTING | MOVEFILE_COPY_ALLOWED)) Looking at the entire APR function, they have lots of tests so it works on Win9X and wide characters. I think we will just use the APR as a guide in implementing the things we need. I think MoveFileEx() is the proper way to go; any other solution requires loop tests for rename. I see the MoveFileEx manual page at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/movefile.asp > > Second, when you unlink() a file on Win32, do applications continue > > accessing the old file contents if they had the file open before the > > unlink? > > > > unlink() just calls DeleteFile() which will error if: > > 1. The target file is in use > > CreateFile() has the option: > > FILE_FLAG_DELETE_ON_CLOSE > > which might be able to be used to simulate traditional unlink() > behavior. No, that flag isn't going to help us. I wonder what MoveFileEx does if the target file exists _and_ is open by another user? I don't see any loop in that Win32 rename() routine, and I looked at the Unix version of apr_file_rename and its just a straight rename() call. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgman Thu Sep 19 22:50:41 2002 Return-path: Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8K2ofr29042; Thu, 19 Sep 2002 22:50:41 -0400 (EDT) From: Bruce Momjian Message-ID: <200209200250.g8K2ofr29042@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <200209192024.g8JKO1g10337@candle.pha.pa.us> To: Bruce Momjian Date: Thu, 19 Sep 2002 22:50:41 -0400 (EDT) cc: Mike Mascari , PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Status: RO Bruce Momjian wrote: > > > Second, when you unlink() a file on Win32, do applications continue > > > accessing the old file contents if they had the file open before the > > > unlink? > > > > > > > unlink() just calls DeleteFile() which will error if: > > > > 1. The target file is in use > > > > CreateFile() has the option: > > > > FILE_FLAG_DELETE_ON_CLOSE > > > > which might be able to be used to simulate traditional unlink() > > behavior. > > No, that flag isn't going to help us. I wonder what MoveFileEx does if > the target file exists _and_ is open by another user? I don't see any > loop in that Win32 rename() routine, and I looked at the Unix version of > apr_file_rename and its just a straight rename() call. This says that if the target is in use, it is overwritten: http://support.microsoft.com/default.aspx?scid=KB;EN-US;q140570& While I think that is good news, does it open the problem of other readers reading partial updates to the file and therefore seeing garbage. Not sure how to handle that, nor am I even sure how I would test it. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 From mascarm@mascari.com Fri Sep 20 00:02:33 2002 Return-path: Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K42SE12294 for ; Fri, 20 Sep 2002 00:02:31 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id AAA18322; Fri, 20 Sep 2002 00:01:34 -0400 Message-ID: <3D8A9DAD.5040500@mascari.com> Date: Fri, 20 Sep 2002 00:01:49 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <200209200250.g8K2ofr29042@candle.pha.pa.us> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean Status: ROr Bruce Momjian wrote: > Bruce Momjian wrote: >>> >>>unlink() just calls DeleteFile() which will error if: >>> >>>1. The target file is in use >>> >>>CreateFile() has the option: >>> >>>FILE_FLAG_DELETE_ON_CLOSE >>> >>>which might be able to be used to simulate traditional unlink() >>>behavior. >> >>No, that flag isn't going to help us. I wonder what MoveFileEx does if >>the target file exists _and_ is open by another user? I don't see any >>loop in that Win32 rename() routine, and I looked at the Unix version of >>apr_file_rename and its just a straight rename() call. > > > This says that if the target is in use, it is overwritten: > > http://support.microsoft.com/default.aspx?scid=KB;EN-US;q140570& I read the article and did not come away with that conclusion. The article describes using the MOVEFILE_DELAY_UNTIL_REBOOT flag, which was created for the express purpose of allowing a SETUP.EXE to remove itself, or rather tell Windows to remove it on the next reboot. Also, if you want the Win32 port to run in 95/98/ME, you can't rely on MoveFileEx(), you have to use MoveFile(). I will do some testing with concurrency and let you know. But don't get your hopes up. This is one of the many advantages that TABLESPACEs have when more than one relation is stored in a single DATAFILE. There was Oracle for MS-DOS, after all.. Mike Mascari mascarm@mascari.com From pgsql-hackers-owner+M29177@postgresql.org Fri Sep 20 00:06:31 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K46TE12770 for ; Fri, 20 Sep 2002 00:06:29 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 44057476CF8; Fri, 20 Sep 2002 00:06:24 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id A0C78476C00; Fri, 20 Sep 2002 00:06:22 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 5AB4E476D1C for ; Fri, 20 Sep 2002 00:05:57 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id 990B9476D38 for ; Fri, 20 Sep 2002 00:05:55 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8K45RV12655; Fri, 20 Sep 2002 00:05:27 -0400 (EDT) From: Bruce Momjian Message-ID: <200209200405.g8K45RV12655@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8A9DAD.5040500@mascari.com> To: Mike Mascari Date: Fri, 20 Sep 2002 00:05:27 -0400 (EDT) cc: PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Mike Mascari wrote: > I read the article and did not come away with that conclusion. > The article describes using the MOVEFILE_DELAY_UNTIL_REBOOT > flag, which was created for the express purpose of allowing a > SETUP.EXE to remove itself, or rather tell Windows to remove it > on the next reboot. Also, if you want the Win32 port to run in > 95/98/ME, you can't rely on MoveFileEx(), you have to use > MoveFile(). > > I will do some testing with concurrency and let you know. But > don't get your hopes up. This is one of the many advantages that > TABLESPACEs have when more than one relation is stored in a > single DATAFILE. There was Oracle for MS-DOS, after all.. I was focusing on handling of pg_pwd and other config file that are written by various backend while other backends are reading them. The actual data files should be OK because we have an exclusive lock when we are adding/removing them. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M29179@postgresql.org Fri Sep 20 00:33:22 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K4XKE14843 for ; Fri, 20 Sep 2002 00:33:20 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id EB433476D5B; Fri, 20 Sep 2002 00:32:25 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 2ED3A476D4E; Fri, 20 Sep 2002 00:32:24 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 43723476994 for ; Fri, 20 Sep 2002 00:32:20 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id A66C6476883 for ; Fri, 20 Sep 2002 00:32:18 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id AAA18391; Fri, 20 Sep 2002 00:31:31 -0400 Message-ID: <3D8AA4B2.8090507@mascari.com> Date: Fri, 20 Sep 2002 00:31:46 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <200209200405.g8K45RV12655@candle.pha.pa.us> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Bruce Momjian wrote: > Mike Mascari wrote: > >>I will do some testing with concurrency and let you know. But >>don't get your hopes up. This is one of the many advantages that >>TABLESPACEs have when more than one relation is stored in a >>single DATAFILE. There was Oracle for MS-DOS, after all.. > > > I was focusing on handling of pg_pwd and other config file that are > written by various backend while other backends are reading them. The > actual data files should be OK because we have an exclusive lock when we > are adding/removing them. > OK. So you want to test: 1. Process 1 opens "foo" 2. Process 2 opens "foo" 3. Process 1 renames "foo" to "bar" 4. Process 2 can safely read from its open file handle Is that what you want tested? I have a small Win32 app ready to test. Just let me know the scenarios... Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M29180@postgresql.org Fri Sep 20 01:02:47 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K52kE17470 for ; Fri, 20 Sep 2002 01:02:46 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id A4AB5476D73; Fri, 20 Sep 2002 01:02:29 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 35B61476D76; Fri, 20 Sep 2002 01:02:21 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 4D3D14760AB for ; Fri, 20 Sep 2002 01:01:55 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id D259E475EAA for ; Fri, 20 Sep 2002 01:01:53 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id BAA18603; Fri, 20 Sep 2002 01:00:49 -0400 Message-ID: <3D8AAB8F.8010001@mascari.com> Date: Fri, 20 Sep 2002 01:01:03 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mike Mascari cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <200209200405.g8K45RV12655@candle.pha.pa.us> <3D8AA4B2.8090507@mascari.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Mike Mascari wrote: > Bruce Momjian wrote: > >> Mike Mascari wrote: >> >>> I will do some testing with concurrency and let you know. But don't >>> get your hopes up. This is one of the many advantages that >>> TABLESPACEs have when more than one relation is stored in a single >>> DATAFILE. There was Oracle for MS-DOS, after all.. >> >> >> >> I was focusing on handling of pg_pwd and other config file that are >> written by various backend while other backends are reading them. The >> actual data files should be OK because we have an exclusive lock when we >> are adding/removing them. >> > > OK. So you want to test: > > 1. Process 1 opens "foo" > 2. Process 2 opens "foo" > 3. Process 1 renames "foo" to "bar" > 4. Process 2 can safely read from its open file handle Actually, looking at the pg_pwd code, you want to determine a way for: 1. Process 1 opens "foo" 2. Process 2 opens "foo" 3. Process 1 creates "bar" 4. Process 1 renames "bar" to "foo" 5. Process 2 can continue to read data from the open file handle and get the original "foo" data. Is that correct? Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M29181@postgresql.org Fri Sep 20 01:30:05 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K5U2E20514 for ; Fri, 20 Sep 2002 01:30:03 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id CDCF2476D1D; Fri, 20 Sep 2002 01:29:59 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id CB1CB476C00; Fri, 20 Sep 2002 01:29:57 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id D9372475FC6 for ; Fri, 20 Sep 2002 01:29:52 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id 21835475BF9 for ; Fri, 20 Sep 2002 01:29:51 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8K5TYr20440; Fri, 20 Sep 2002 01:29:34 -0400 (EDT) From: Bruce Momjian Message-ID: <200209200529.g8K5TYr20440@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8AAB8F.8010001@mascari.com> To: Mike Mascari Date: Fri, 20 Sep 2002 01:29:33 -0400 (EDT) cc: PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Mike Mascari wrote: > Actually, looking at the pg_pwd code, you want to determine a > way for: > > 1. Process 1 opens "foo" > 2. Process 2 opens "foo" > 3. Process 1 creates "bar" > 4. Process 1 renames "bar" to "foo" > 5. Process 2 can continue to read data from the open file handle > and get the original "foo" data. Yep, that's it. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M29182@postgresql.org Fri Sep 20 01:36:21 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K5aJE21658 for ; Fri, 20 Sep 2002 01:36:20 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 926CA4764F2; Fri, 20 Sep 2002 01:36:20 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 8F0E4475EEE; Fri, 20 Sep 2002 01:36:18 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 3A84C4769DA for ; Fri, 20 Sep 2002 01:36:14 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id E8DAA476413 for ; Fri, 20 Sep 2002 01:36:12 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id BAA18679; Fri, 20 Sep 2002 01:35:09 -0400 Message-ID: <3D8AB39B.80708@mascari.com> Date: Fri, 20 Sep 2002 01:35:23 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bruce Momjian cc: PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <200209200529.g8K5TYr20440@candle.pha.pa.us> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Bruce Momjian wrote: > Mike Mascari wrote: > >>Actually, looking at the pg_pwd code, you want to determine a >>way for: >> >>1. Process 1 opens "foo" >>2. Process 2 opens "foo" >>3. Process 1 creates "bar" >>4. Process 1 renames "bar" to "foo" >>5. Process 2 can continue to read data from the open file handle >>and get the original "foo" data. > > > Yep, that's it. > So far, MoveFileEx("foo", "bar", MOVEFILE_REPLACE_EXISTING) returns "Access Denied" when Process 1 attempts the rename. But I'm continuing to investigate the possibilities... Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M29183@postgresql.org Fri Sep 20 01:50:47 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K5ojE22893 for ; Fri, 20 Sep 2002 01:50:46 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id A8A20476D3B; Fri, 20 Sep 2002 01:50:46 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 70EBC476B1F; Fri, 20 Sep 2002 01:50:44 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 8B227475EEE for ; Fri, 20 Sep 2002 01:50:40 -0400 (EDT) Received: from megazone.bigpanda.com (megazone.bigpanda.com [63.150.15.178]) by postgresql.org (Postfix) with ESMTP id AE58D476D2C for ; Fri, 20 Sep 2002 01:50:36 -0400 (EDT) Received: by megazone.bigpanda.com (Postfix, from userid 1001) id 709D1D61E; Thu, 19 Sep 2002 22:50:36 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by megazone.bigpanda.com (Postfix) with ESMTP id 624665C02; Thu, 19 Sep 2002 22:50:36 -0700 (PDT) Date: Thu, 19 Sep 2002 22:50:36 -0700 (PDT) From: Stephan Szabo To: Mike Mascari cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8AB39B.80708@mascari.com> Message-ID: <20020919224718.H36366-100000@megazone23.bigpanda.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO On Fri, 20 Sep 2002, Mike Mascari wrote: > Bruce Momjian wrote: > > Mike Mascari wrote: > > > >>Actually, looking at the pg_pwd code, you want to determine a > >>way for: > >> > >>1. Process 1 opens "foo" > >>2. Process 2 opens "foo" > >>3. Process 1 creates "bar" > >>4. Process 1 renames "bar" to "foo" > >>5. Process 2 can continue to read data from the open file handle > >>and get the original "foo" data. > > > > > > Yep, that's it. > > > > So far, MoveFileEx("foo", "bar", MOVEFILE_REPLACE_EXISTING) > returns "Access Denied" when Process 1 attempts the rename. But > I'm continuing to investigate the possibilities... Does a sequence like Process 1 opens "foo" Process 2 opens "foo" Process 1 creates "bar" Process 1 renames "foo" to - where something is generated to not overlap an existing file Process 1 renames "bar" to "foo" Process 2 continues reading let you do the replace and keep reading (at the penalty that you've now got to have a way to know when to remove the various s) ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From mascarm@mascari.com Fri Sep 20 02:04:41 2002 Return-path: Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K64WE24578 for ; Fri, 20 Sep 2002 02:04:38 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id CAA18737; Fri, 20 Sep 2002 02:03:29 -0400 Message-ID: <3D8ABA3F.6030002@mascari.com> Date: Fri, 20 Sep 2002 02:03:43 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stephan Szabo cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <20020919224718.H36366-100000@megazone23.bigpanda.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean Status: RO Stephan Szabo wrote: > On Fri, 20 Sep 2002, Mike Mascari wrote: >>Bruce Momjian wrote: >>>Mike Mascari wrote: >>>>Actually, looking at the pg_pwd code, you want to determine a >>>>way for: >>>> >>>>1. Process 1 opens "foo" >>>>2. Process 2 opens "foo" >>>>3. Process 1 creates "bar" >>>>4. Process 1 renames "bar" to "foo" >>>>5. Process 2 can continue to read data from the open file handle >>>>and get the original "foo" data. >>> >>> >>>Yep, that's it. >>> >> >>So far, MoveFileEx("foo", "bar", MOVEFILE_REPLACE_EXISTING) >>returns "Access Denied" when Process 1 attempts the rename. But >>I'm continuing to investigate the possibilities... > > > Does a sequence like > Process 1 opens "foo" > Process 2 opens "foo" > Process 1 creates "bar" > Process 1 renames "foo" to > - where something is generated to not overlap an existing file > Process 1 renames "bar" to "foo" > Process 2 continues reading > let you do the replace and keep reading (at the penalty that > you've now got to have a way to know when to remove the > various s) Yes! Indeed that does work. Mike Mascari mascarm@mascari.com From pgsql-hackers-owner+M29185@postgresql.org Fri Sep 20 02:14:29 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K6ERE25614 for ; Fri, 20 Sep 2002 02:14:27 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id B0200476D0C; Fri, 20 Sep 2002 02:14:26 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id D323F47655C; Fri, 20 Sep 2002 02:14:24 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 9EEBF476D02 for ; Fri, 20 Sep 2002 02:14:20 -0400 (EDT) Received: from megazone.bigpanda.com (megazone.bigpanda.com [63.150.15.178]) by postgresql.org (Postfix) with ESMTP id 695B1475EAA for ; Fri, 20 Sep 2002 02:14:16 -0400 (EDT) Received: by megazone.bigpanda.com (Postfix, from userid 1001) id B10E9D61E; Thu, 19 Sep 2002 23:14:14 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by megazone.bigpanda.com (Postfix) with ESMTP id A6B475C03; Thu, 19 Sep 2002 23:14:14 -0700 (PDT) Date: Thu, 19 Sep 2002 23:14:14 -0700 (PDT) From: Stephan Szabo To: Mike Mascari cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8ABA3F.6030002@mascari.com> Message-ID: <20020919230827.A36505-100000@megazone23.bigpanda.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO On Fri, 20 Sep 2002, Mike Mascari wrote: > Stephan Szabo wrote: > > On Fri, 20 Sep 2002, Mike Mascari wrote: > >>So far, MoveFileEx("foo", "bar", MOVEFILE_REPLACE_EXISTING) > >>returns "Access Denied" when Process 1 attempts the rename. But > >>I'm continuing to investigate the possibilities... > > > > > > Does a sequence like > > Process 1 opens "foo" > > Process 2 opens "foo" > > Process 1 creates "bar" > > Process 1 renames "foo" to > > - where something is generated to not overlap an existing file > > Process 1 renames "bar" to "foo" > > Process 2 continues reading > > let you do the replace and keep reading (at the penalty that > > you've now got to have a way to know when to remove the > > various s) > > Yes! Indeed that does work. Thinking back, I think that may still fail on Win95 (using MoveFile). Once in the past I had to work on (un)installers for Win* and I vaguely remember Win95 being more strict than Win98 but that may just have been with moving the executable you're currently running. ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From mascarm@mascari.com Fri Sep 20 03:14:03 2002 Return-path: Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8K7DwE00605 for ; Fri, 20 Sep 2002 03:14:01 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id DAA18894; Fri, 20 Sep 2002 03:13:06 -0400 Message-ID: <3D8ACA96.80504@mascari.com> Date: Fri, 20 Sep 2002 03:13:26 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stephan Szabo cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <20020919230827.A36505-100000@megazone23.bigpanda.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean Status: RO Stephan Szabo wrote: > On Fri, 20 Sep 2002, Mike Mascari wrote: >> >>Yes! Indeed that does work. > > > Thinking back, I think that may still fail on Win95 (using MoveFile). > Once in the past I had to work on (un)installers for Win* and I > vaguely remember Win95 being more strict than Win98 but that may just > have been with moving the executable you're currently running. Well, here's the test: foo.txt contains "This is FOO!" bar.txt contains "This is BAR!" Process 1 opens foo.txt Process 2 opens foo.txt Process 1 sleeps 7.5 seconds Process 2 sleeps 15 seconds Process 1 uses MoveFile() to rename "foo.txt" to "foo2.txt" Process 1 uses MoveFile() to rename "bar.txt" to "foo.txt" Process 1 uses DeleteFile() to remove "foo2.txt" Process 2 awakens and displays "This is FOO!" On the filesystem, we then have: foo.txt containing "This is BAR!" The good news is that this works fine under NT 4 using just MoveFile(). The bad news is that it requires the files be opened using CreateFile() with the FILE_SHARE_DELETE flag set. The C library which ships with Visual C++ 6 ultimately calls CreateFile() via fopen() but with no opportunity through the standard C library routines to use the FILE_SHARE_DELETE flag. And the FILE_SHARE_DELETE flag cannot be used under Windows 95/98 (Bad Parameter). Which means, on those platforms, there still doesn't appear to be a solution. Under NT/XP/2K, AllocateFile() will have to modified to call CreateFile() instead of fopen(). I'm not sure about ME, but I suspect it behaves similarly to 95/98. Mike Mascari mascarm@mascari.com From pgsql-hackers-owner+M29205@postgresql.org Fri Sep 20 10:36:48 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KEalE13770 for ; Fri, 20 Sep 2002 10:36:47 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id BBC23476E72; Fri, 20 Sep 2002 10:35:40 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 00890476D13; Fri, 20 Sep 2002 10:35:31 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 5D2BC476D6D for ; Fri, 20 Sep 2002 10:30:17 -0400 (EDT) Received: from sss.pgh.pa.us (unknown [192.204.191.242]) by postgresql.org (Postfix) with ESMTP id B17FE476CD8 for ; Fri, 20 Sep 2002 10:30:15 -0400 (EDT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.12.5/8.12.5) with ESMTP id g8KERq5D016068; Fri, 20 Sep 2002 10:27:52 -0400 (EDT) To: Stephan Szabo cc: Mike Mascari , Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <20020919224718.H36366-100000@megazone23.bigpanda.com> References: <20020919224718.H36366-100000@megazone23.bigpanda.com> Comments: In-reply-to Stephan Szabo message dated "Thu, 19 Sep 2002 22:50:36 -0700" Date: Fri, 20 Sep 2002 10:27:52 -0400 Message-ID: <16067.1032532072@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Stephan Szabo writes: > ... let you do the replace and keep reading (at the penalty that > you've now got to have a way to know when to remove the > various s) That is the hard part. Mike's description omitted one crucial step: 6. The old "foo" goes away when the last open file handle for it is closed. I doubt there is any practical way for Postgres to cause that to happen if the OS itself does not have any support for it. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M29206@postgresql.org Fri Sep 20 10:37:53 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KEbpE13898 for ; Fri, 20 Sep 2002 10:37:51 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 891F9476DC6; Fri, 20 Sep 2002 10:36:31 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 2207D476F07; Fri, 20 Sep 2002 10:36:22 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 90195476DD2 for ; Fri, 20 Sep 2002 10:33:48 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id 5ECE6476DB5 for ; Fri, 20 Sep 2002 10:33:46 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8KEVMg13344; Fri, 20 Sep 2002 10:31:22 -0400 (EDT) From: Bruce Momjian Message-ID: <200209201431.g8KEVMg13344@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8ACA96.80504@mascari.com> To: Mike Mascari Date: Fri, 20 Sep 2002 10:31:22 -0400 (EDT) cc: Stephan Szabo , PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO I don't think we are not going to be supporting Win9X so there isn't an issue there. We will be supporting Win2000/NT/XP. I don't understand FILE_SHARE_DELETE. I read the description at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/createfile.asp but I don't understand it: FILE_SHARE_DELETE - Windows NT/2000/XP: Subsequent open operations on the object will succeed only if delete access is requested. --------------------------------------------------------------------------- Mike Mascari wrote: > Stephan Szabo wrote: > > On Fri, 20 Sep 2002, Mike Mascari wrote: > >> > >>Yes! Indeed that does work. > > > > > > Thinking back, I think that may still fail on Win95 (using MoveFile). > > Once in the past I had to work on (un)installers for Win* and I > > vaguely remember Win95 being more strict than Win98 but that may just > > have been with moving the executable you're currently running. > > Well, here's the test: > > foo.txt contains "This is FOO!" > bar.txt contains "This is BAR!" > > Process 1 opens foo.txt > Process 2 opens foo.txt > Process 1 sleeps 7.5 seconds > Process 2 sleeps 15 seconds > Process 1 uses MoveFile() to rename "foo.txt" to "foo2.txt" > Process 1 uses MoveFile() to rename "bar.txt" to "foo.txt" > Process 1 uses DeleteFile() to remove "foo2.txt" > Process 2 awakens and displays "This is FOO!" > > On the filesystem, we then have: > > foo.txt containing "This is BAR!" > > The good news is that this works fine under NT 4 using just > MoveFile(). The bad news is that it requires the files be opened > using CreateFile() with the FILE_SHARE_DELETE flag set. The C > library which ships with Visual C++ 6 ultimately calls > CreateFile() via fopen() but with no opportunity through the > standard C library routines to use the FILE_SHARE_DELETE flag. > And the FILE_SHARE_DELETE flag cannot be used under Windows > 95/98 (Bad Parameter). Which means, on those platforms, there > still doesn't appear to be a solution. Under NT/XP/2K, > AllocateFile() will have to modified to call CreateFile() > instead of fopen(). I'm not sure about ME, but I suspect it > behaves similarly to 95/98. > > Mike Mascari > mascarm@mascari.com > > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M29208@postgresql.org Fri Sep 20 11:30:26 2002 Return-path: Received: from west.navpoint.com (west.navpoint.com [207.106.42.13]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KFUOE24840 for ; Fri, 20 Sep 2002 11:30:25 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by west.navpoint.com (8.11.6/8.10.1) with ESMTP id g8KFBP217840 for ; Fri, 20 Sep 2002 11:11:25 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id BD0D2476E5B; Fri, 20 Sep 2002 11:09:49 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 4691C476D5D; Fri, 20 Sep 2002 11:09:47 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 8341F475CB4 for ; Fri, 20 Sep 2002 10:58:12 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id B1E1A474E5C for ; Fri, 20 Sep 2002 10:58:10 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id KAA20019; Fri, 20 Sep 2002 10:56:39 -0400 Message-ID: <3D8B373C.7060102@mascari.com> Date: Fri, 20 Sep 2002 10:57:00 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bruce Momjian cc: Stephan Szabo , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <200209201431.g8KEVMg13344@candle.pha.pa.us> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Bruce Momjian wrote: > I don't think we are not going to be supporting Win9X so there isn't an > issue there. We will be supporting Win2000/NT/XP. > > I don't understand FILE_SHARE_DELETE. I read the description at: > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/createfile.asp > > but I don't understand it: > > FILE_SHARE_DELETE - Windows NT/2000/XP: Subsequent open operations on > the object will succeed only if delete access is requested. I think that's a rather poor description. I think it just means that if the file is opened once via CreateFile() with FILE_SHARE_DELETE, then any subsequent CreateFile() calls will fail unless they too have FILE_SHARE_DELETE. In other words, if one of us can delete this file while its open, any of us can. Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M29213@postgresql.org Fri Sep 20 11:30:47 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KFUkE24923 for ; Fri, 20 Sep 2002 11:30:46 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id DFE91476EA6; Fri, 20 Sep 2002 11:28:47 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 124F8476E95; Fri, 20 Sep 2002 11:28:46 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id C8ADF476F70 for ; Fri, 20 Sep 2002 11:26:25 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id DC5BD476F53 for ; Fri, 20 Sep 2002 11:26:19 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8KF5ch17250; Fri, 20 Sep 2002 11:05:38 -0400 (EDT) From: Bruce Momjian Message-ID: <200209201505.g8KF5ch17250@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8B373C.7060102@mascari.com> To: Mike Mascari Date: Fri, 20 Sep 2002 11:05:38 -0400 (EDT) cc: Stephan Szabo , PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Mike Mascari wrote: > Bruce Momjian wrote: > > I don't think we are not going to be supporting Win9X so there isn't an > > issue there. We will be supporting Win2000/NT/XP. > > > > I don't understand FILE_SHARE_DELETE. I read the description at: > > > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/createfile.asp > > > > but I don't understand it: > > > > FILE_SHARE_DELETE - Windows NT/2000/XP: Subsequent open operations on > > the object will succeed only if delete access is requested. > > I think that's a rather poor description. I think it just means > that if the file is opened once via CreateFile() with > FILE_SHARE_DELETE, then any subsequent CreateFile() calls will > fail unless they too have FILE_SHARE_DELETE. In other words, if > one of us can delete this file while its open, any of us can. I don't understand what that gets us. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster From sszabo@megazone23.bigpanda.com Fri Sep 20 11:29:09 2002 Return-path: Received: from west.navpoint.com (west.navpoint.com [207.106.42.13]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KFT6E24617 for ; Fri, 20 Sep 2002 11:29:08 -0400 (EDT) Received: from megazone.bigpanda.com (megazone.bigpanda.com [63.150.15.178]) by west.navpoint.com (8.11.6/8.10.1) with ESMTP id g8KFBZ217956 for ; Fri, 20 Sep 2002 11:11:35 -0400 (EDT) Received: by megazone.bigpanda.com (Postfix, from userid 1001) id 5F343D61C; Fri, 20 Sep 2002 08:10:19 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by megazone.bigpanda.com (Postfix) with ESMTP id 54E705C02; Fri, 20 Sep 2002 08:10:19 -0700 (PDT) Date: Fri, 20 Sep 2002 08:10:19 -0700 (PDT) From: Stephan Szabo To: Mike Mascari cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8B373C.7060102@mascari.com> Message-ID: <20020920080949.H40440-100000@megazone23.bigpanda.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO On Fri, 20 Sep 2002, Mike Mascari wrote: > Bruce Momjian wrote: > > I don't think we are not going to be supporting Win9X so there isn't an > > issue there. We will be supporting Win2000/NT/XP. > > > > I don't understand FILE_SHARE_DELETE. I read the description at: > > > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/createfile.asp > > > > but I don't understand it: > > > > FILE_SHARE_DELETE - Windows NT/2000/XP: Subsequent open operations on > > the object will succeed only if delete access is requested. > > I think that's a rather poor description. I think it just means > that if the file is opened once via CreateFile() with > FILE_SHARE_DELETE, then any subsequent CreateFile() calls will > fail unless they too have FILE_SHARE_DELETE. In other words, if > one of us can delete this file while its open, any of us can. The question is, what happens if two people have the file open and one goes and tries to delete it? Can the other still read from it? From pgsql-hackers-owner+M29216@postgresql.org Fri Sep 20 11:45:47 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KFjkE26656 for ; Fri, 20 Sep 2002 11:45:46 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 75E01476DE2; Fri, 20 Sep 2002 11:44:30 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id E1897476E29; Fri, 20 Sep 2002 11:44:16 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 96D5C47625F for ; Fri, 20 Sep 2002 11:38:15 -0400 (EDT) Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by postgresql.org (Postfix) with SMTP id B00BA476225 for ; Fri, 20 Sep 2002 11:38:14 -0400 (EDT) Received: from psc.progress.com (HELO Yahoo.com) (janwieck@192.233.92.200 with plain) by smtp.mail.vip.sc5.yahoo.com with SMTP; 20 Sep 2002 15:36:41 -0000 Message-ID: <3D8B4087.AF0CD803@Yahoo.com> Date: Fri, 20 Sep 2002 11:36:39 -0400 From: Jan Wieck Organization: Home X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en,ru MIME-Version: 1.0 To: Mike Mascari cc: Stephan Szabo , Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <20020919230827.A36505-100000@megazone23.bigpanda.com> <3D8ACA96.80504@mascari.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Mike Mascari wrote: > instead of fopen(). I'm not sure about ME, but I suspect it > behaves similarly to 95/98. I just checked with Katie and the good news (tm) is that the Win32 port we did here at PeerDirect doesn't support 95/98 and ME anyway. It does support NT4, 2000 and XP. So don't bother. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com # ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From pgsql-hackers-owner+M29222@postgresql.org Fri Sep 20 11:59:16 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KFxEE28389 for ; Fri, 20 Sep 2002 11:59:15 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 156F6476EFA; Fri, 20 Sep 2002 11:58:59 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 00F87476EE5; Fri, 20 Sep 2002 11:58:56 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 48990475E5E for ; Fri, 20 Sep 2002 11:57:18 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id F3EF1475D6E for ; Fri, 20 Sep 2002 11:57:16 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id LAA20244; Fri, 20 Sep 2002 11:54:31 -0400 Message-ID: <3D8B44CC.6070802@mascari.com> Date: Fri, 20 Sep 2002 11:54:52 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stephan Szabo cc: Bruce Momjian , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <20020920080949.H40440-100000@megazone23.bigpanda.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Stephan Szabo wrote: > On Fri, 20 Sep 2002, Mike Mascari wrote: > > >>I think that's a rather poor description. I think it just means >>that if the file is opened once via CreateFile() with >>FILE_SHARE_DELETE, then any subsequent CreateFile() calls will >>fail unless they too have FILE_SHARE_DELETE. In other words, if >>one of us can delete this file while its open, any of us can. > > > The question is, what happens if two people have the file open > and one goes and tries to delete it? Can the other still read > from it? Yes. I just tested it and it worked. I'll test Bruce's scenario as well: foo contains: "FOO" bar contains: "BAR" 1. Process 1 opens "foo" 2. Process 2 opens "foo" 3. Process 1 calls MoveFile("foo", "foo2"); 4. Process 3 opens "foo" <- Successful? 5. Process 1 calls MoveFile("bar", "foo"); 6. Process 4 opens "foo" <- Successful? 7. Process 1 calls DeleteFile("foo2"); 8. Process 1, 2, 3, 4 all read from their respective handles. I think the thing to worry about is a race condition between the two MoveFile() attempts. A very ugly hack would be to loop in a CreateFile() in an attempt to open "foo", giving up if the error is not a NOT EXISTS error code. Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org From pgsql-hackers-owner+M29230@postgresql.org Fri Sep 20 13:12:45 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KHChE07387 for ; Fri, 20 Sep 2002 13:12:44 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 54F85476F77; Fri, 20 Sep 2002 13:01:57 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 1CC51476F67; Fri, 20 Sep 2002 13:01:39 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id B253B476148 for ; Fri, 20 Sep 2002 12:29:02 -0400 (EDT) Received: from corvette.mascari.com (dhcp065-024-158-068.columbus.rr.com [65.24.158.68]) by postgresql.org (Postfix) with ESMTP id 7F8FC476135 for ; Fri, 20 Sep 2002 12:29:01 -0400 (EDT) Received: from mascari.com (ferrari.mascari.com [192.168.2.1]) by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id MAA20365; Fri, 20 Sep 2002 12:27:10 -0400 Message-ID: <3D8B4C74.2050708@mascari.com> Date: Fri, 20 Sep 2002 12:27:32 -0400 From: Mike Mascari User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: PostgreSQL-development cc: Stephan Szabo , Bruce Momjian Subject: Re: [HACKERS] Win32 rename()/unlink() questions References: <20020920080949.H40440-100000@megazone23.bigpanda.com> <3D8B44CC.6070802@mascari.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner: Found to be clean X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: ROr I wrote: > Stephan Szabo wrote: >> >> The question is, what happens if two people have the file open >> and one goes and tries to delete it? Can the other still read >> from it? > > Yes. I just tested it and it worked. I'll test Bruce's scenario as well: > > foo contains: "FOO" > bar contains: "BAR" > > 1. Process 1 opens "foo" > 2. Process 2 opens "foo" > 3. Process 1 calls MoveFile("foo", "foo2"); > 4. Process 3 opens "foo" <- Successful? > 5. Process 1 calls MoveFile("bar", "foo"); > 6. Process 4 opens "foo" <- Successful? > 7. Process 1 calls DeleteFile("foo2"); > 8. Process 1, 2, 3, 4 all read from their respective handles. Process 1: "FOO" Process 2: "FOO" Process 3: Error - File does not exist Process 4: "BAR" Its interesting in that it allows for Unix-style rename() and unlink() behavior, but with a race condition. Without Stephan's two MoveFile() trick and the FILE_SHARE_DELETE flag, however, the result would be Access Denied. Are the places in the backend that use rename() and unlink() renaming and unlinking files that are only opened for a brief moment by other backends? Mike Mascari mascarm@mascari.com ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M29235@postgresql.org Fri Sep 20 13:38:05 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KHc3E18565 for ; Fri, 20 Sep 2002 13:38:04 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id D4B2C4764A8; Fri, 20 Sep 2002 13:38:01 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id D071B476173; Fri, 20 Sep 2002 13:37:59 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 6DD60476791 for ; Fri, 20 Sep 2002 13:33:56 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id C679F476272 for ; Fri, 20 Sep 2002 13:33:54 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8KHVRu17060; Fri, 20 Sep 2002 13:31:27 -0400 (EDT) From: Bruce Momjian Message-ID: <200209201731.g8KHVRu17060@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8B4C74.2050708@mascari.com> To: Mike Mascari Date: Fri, 20 Sep 2002 13:31:27 -0400 (EDT) cc: PostgreSQL-development , Stephan Szabo X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Mike Mascari wrote: > Its interesting in that it allows for Unix-style rename() and > unlink() behavior, but with a race condition. Without Stephan's > two MoveFile() trick and the FILE_SHARE_DELETE flag, however, > the result would be Access Denied. Are the places in the backend > that use rename() and unlink() renaming and unlinking files that > are only opened for a brief moment by other backends? Yes, those files are only opened for a brief moment. They are not held open. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) From pgsql-hackers-owner+M29237@postgresql.org Fri Sep 20 13:57:39 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KHvcE21846 for ; Fri, 20 Sep 2002 13:57:38 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 1EC6A4762C8; Fri, 20 Sep 2002 13:57:36 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 321B14760E8; Fri, 20 Sep 2002 13:57:34 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id CD7E8476D24 for ; Fri, 20 Sep 2002 13:54:48 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id 0C6034762C8 for ; Fri, 20 Sep 2002 13:54:47 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8KHrnp21564; Fri, 20 Sep 2002 13:53:49 -0400 (EDT) From: Bruce Momjian Message-ID: <200209201753.g8KHrnp21564@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questions In-Reply-To: <3D8B4C74.2050708@mascari.com> To: Mike Mascari Date: Fri, 20 Sep 2002 13:53:49 -0400 (EDT) cc: PostgreSQL-development , Stephan Szabo X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Mike Mascari wrote: > > foo contains: "FOO" > > bar contains: "BAR" > > > > 1. Process 1 opens "foo" > > 2. Process 2 opens "foo" > > 3. Process 1 calls MoveFile("foo", "foo2"); > > 4. Process 3 opens "foo" <- Successful? > > 5. Process 1 calls MoveFile("bar", "foo"); > > 6. Process 4 opens "foo" <- Successful? > > 7. Process 1 calls DeleteFile("foo2"); > > 8. Process 1, 2, 3, 4 all read from their respective handles. > > Process 1: "FOO" > Process 2: "FOO" > Process 3: Error - File does not exist > Process 4: "BAR" > > Its interesting in that it allows for Unix-style rename() and > unlink() behavior, but with a race condition. Without Stephan's > two MoveFile() trick and the FILE_SHARE_DELETE flag, however, > the result would be Access Denied. Are the places in the backend > that use rename() and unlink() renaming and unlinking files that > are only opened for a brief moment by other backends? I think we are better off looping over MoveFileEx(MOVEFILE_REPLACE_EXISTING) until the file isn't opened by anyone. That localizes the changes to rename only and not out to all the opens. The open failure loops when the file isn't there seem much worse. I am a little concerned about starving the rename when there is a lot of activity but I don't see a better solution. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M29214@postgresql.org Fri Sep 20 11:31:33 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KFVWE25017 for ; Fri, 20 Sep 2002 11:31:32 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id D2062476F4F; Fri, 20 Sep 2002 11:29:20 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 1A194476EED; Fri, 20 Sep 2002 11:29:18 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 6C448476F71 for ; Fri, 20 Sep 2002 11:26:26 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id 2AA7E476F65 for ; Fri, 20 Sep 2002 11:26:24 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8KF4rj17150; Fri, 20 Sep 2002 11:04:53 -0400 (EDT) From: Bruce Momjian Message-ID: <200209201504.g8KF4rj17150@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questionst In-Reply-To: <3D8ABA3F.6030002@mascari.com> To: Mike Mascari Date: Fri, 20 Sep 2002 11:04:53 -0400 (EDT) cc: Stephan Szabo , PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO It is good that moving the file out of the way works, but it doesn't completely solve the problem. What we have now with Unix rename is ideal: 1) old opens continue seeing the old contents 2) new opens see the new contents 3) the file always exists under the fixed name We have that with MoveFileEx(), but we have to loop over the routine until is succeeds. If we move the old file out of the way, we loose the ability to know the file always exists and then we have to loop over open() until is succeeds. I think we may be best just looping on MoveFileEx() until is succeeds. We do the pg_pwd writes while holding an exclusive lock on pg_shadow so that will guarantee that no one else will slip an old version of the file in after we have written it. However, it also prevents pg_shadow access while we are doing the looping. Yuck. --------------------------------------------------------------------------- Mike Mascari wrote: > Stephan Szabo wrote: > > On Fri, 20 Sep 2002, Mike Mascari wrote: > >>Bruce Momjian wrote: > >>>Mike Mascari wrote: > >>>>Actually, looking at the pg_pwd code, you want to determine a > >>>>way for: > >>>> > >>>>1. Process 1 opens "foo" > >>>>2. Process 2 opens "foo" > >>>>3. Process 1 creates "bar" > >>>>4. Process 1 renames "bar" to "foo" > >>>>5. Process 2 can continue to read data from the open file handle > >>>>and get the original "foo" data. > >>> > >>> > >>>Yep, that's it. > >>> > >> > >>So far, MoveFileEx("foo", "bar", MOVEFILE_REPLACE_EXISTING) > >>returns "Access Denied" when Process 1 attempts the rename. But > >>I'm continuing to investigate the possibilities... > > > > > > Does a sequence like > > Process 1 opens "foo" > > Process 2 opens "foo" > > Process 1 creates "bar" > > Process 1 renames "foo" to > > - where something is generated to not overlap an existing file > > Process 1 renames "bar" to "foo" > > Process 2 continues reading > > let you do the replace and keep reading (at the penalty that > > you've now got to have a way to know when to remove the > > various s) > > Yes! Indeed that does work. > > Mike Mascari > mascarm@mascari.com > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html From tgl@sss.pgh.pa.us Fri Sep 20 11:50:26 2002 Return-path: Received: from sss.pgh.pa.us (root@[192.204.191.242]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KFoOE27327 for ; Fri, 20 Sep 2002 11:50:25 -0400 (EDT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.12.5/8.12.5) with ESMTP id g8KFoH5D016740; Fri, 20 Sep 2002 11:50:17 -0400 (EDT) To: Bruce Momjian cc: Mike Mascari , Stephan Szabo , PostgreSQL-development Subject: Re: [HACKERS] Win32 rename()/unlink() questionst In-Reply-To: <200209201504.g8KF4rj17150@candle.pha.pa.us> References: <200209201504.g8KF4rj17150@candle.pha.pa.us> Comments: In-reply-to Bruce Momjian message dated "Fri, 20 Sep 2002 11:04:53 -0400" Date: Fri, 20 Sep 2002 11:50:17 -0400 Message-ID: <16739.1032537017@sss.pgh.pa.us> From: Tom Lane Status: ROr Bruce Momjian writes: > I think we may be best just looping on MoveFileEx() until is succeeds. > We do the pg_pwd writes while holding an exclusive lock on pg_shadow so > that will guarantee that no one else will slip an old version of the > file in after we have written it. However, it also prevents pg_shadow > access while we are doing the looping. Yuck. Surely you're not evaluating this on the assumption that the pg_shadow triggers are the only places that use rename() ? I see other places in pgstat and relcache that expect rename() to work per Unix spec. regards, tom lane From pgsql-hackers-owner+M29223@postgresql.org Fri Sep 20 12:04:21 2002 Return-path: Received: from postgresql.org (postgresql.org [64.49.215.8]) by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g8KG06E28547 for ; Fri, 20 Sep 2002 12:04:20 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 0545E476F46; Fri, 20 Sep 2002 11:59:27 -0400 (EDT) Received: from postgresql.org (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with SMTP id 17069476EA0; Fri, 20 Sep 2002 11:59:24 -0400 (EDT) Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id DDE66476791 for ; Fri, 20 Sep 2002 11:57:45 -0400 (EDT) Received: from candle.pha.pa.us (momjian.navpoint.com [207.106.42.251]) by postgresql.org (Postfix) with ESMTP id C7B5F4762DE for ; Fri, 20 Sep 2002 11:57:43 -0400 (EDT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.10.1) id g8KFu2k28023; Fri, 20 Sep 2002 11:56:02 -0400 (EDT) From: Bruce Momjian Message-ID: <200209201556.g8KFu2k28023@candle.pha.pa.us> Subject: Re: [HACKERS] Win32 rename()/unlink() questionst In-Reply-To: <16739.1032537017@sss.pgh.pa.us> To: Tom Lane Date: Fri, 20 Sep 2002 11:56:02 -0400 (EDT) cc: Mike Mascari , Stephan Szabo , PostgreSQL-development X-Mailer: ELM [version 2.4ME+ PL99 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by AMaViS new-20020517 Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org X-Virus-Scanned: by AMaViS new-20020517 Status: RO Tom Lane wrote: > Bruce Momjian writes: > > I think we may be best just looping on MoveFileEx() until is succeeds. > > We do the pg_pwd writes while holding an exclusive lock on pg_shadow so > > that will guarantee that no one else will slip an old version of the > > file in after we have written it. However, it also prevents pg_shadow > > access while we are doing the looping. Yuck. > > Surely you're not evaluating this on the assumption that the pg_shadow > triggers are the only places that use rename() ? > > I see other places in pgstat and relcache that expect rename() to work > per Unix spec. Yes, I know there are others but I think we will need _a_ rename that works 100% and then replace that in all Win32 rename cases. Given what I have seen, I think a single rename with a loop that uses MoveFileEx() may be our best bet. It is localized, doesn't affect the open() code, and should work well. The only downside is that under heavy read activity the loop will loop around a few times but I just don't see another solution. I was initially concerned that the loop in rename could let old renames update the file overwriting newer contents but I realize now that rename() itself has the same issue (an old rename could hit in the code after a newer rename) so in all cases we must already have the proper locking in place. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly