This basically takes some build system code that was previously labeled
"Solaris" and ties it to the compiler rather than the operating system.
Author: Julius Stroffek <Julius.Stroffek@Sun.COM>
corrupted. (Neither is very important if SIGTERM is used to shut down the
whole database cluster together, but there's a problem if someone tries to
SIGTERM individual backends.) To do this, introduce new infrastructure
macros PG_ENSURE_ERROR_CLEANUP/PG_END_ENSURE_ERROR_CLEANUP that take care
of transiently pushing an on_shmem_exit cleanup hook. Also use this method
for createdb cleanup --- that wasn't a shared-memory-corruption problem,
but SIGTERM abort of createdb could leave orphaned files lying around.
Backpatch as far as 8.2. The shmem corruption cases don't exist in 8.1,
and the createdb usage doesn't seem important enough to risk backpatching
further.
errdetail except the string goes only to the server log, replacing the normal
errdetail there. This provides a reasonably clean way of dealing with error
details that are too security-sensitive or too bulky to send to the client.
This commit just adds the infrastructure --- actual uses to follow.
whether to execute an immediate interrupt, rather than testing whether
LockWaitCancel() cancelled a lock wait. The old way misclassified the case
where we were blocked in ProcWaitForSignal(), and arguably would misclassify
any other future additions of new ImmediateInterruptOK states too. This
allows reverting the old kluge that gave LockWaitCancel() a return value,
since no callers care anymore. Improve comments in the various
implementations of PGSemaphoreLock() to explain that on some platforms, the
assumption that semop() exits after a signal is wrong, and so we must ensure
that the signal handler itself throws elog if we want cancel or die interrupts
to be effective. Per testing related to bug #3883, though this patch doesn't
solve those problems fully.
Perhaps this change should be back-patched, but since pre-8.3 branches aren't
really relying on autovacuum to respond to SIGINT, it doesn't seem critical
for them.
SHGetFolderPath.
This removes the direct dependency on shell32.dll and user32.dll, which
eats a lot of "desktop heap" for each backend that's started. The
desktop heap is a very limited resource, causing backends to no
longer start once it's been exhausted.
We still have indirect depdendencies on user32.dll through third party
libraries, but those can't easily be removed.
Dave Page
This is a Linux kernel bug that apparently exists in every extant kernel
version: sometimes shmctl() will fail with EIDRM when EINVAL is correct.
We were assuming that EIDRM indicates a possible conflict with pre-existing
backends, and refusing to start the postmaster when this happens. Fortunately,
there does not seem to be any case where Linux can legitimately return EIDRM
(it doesn't track shmem segments in a way that would allow that), so we can
get away with just assuming that EIDRM means EINVAL on this platform.
Per reports from Michael Fuhr and Jon Lapham --- it's a bit surprising
we have not seen more reports, actually.
to be cases when at least Windows 2000 can do this even though select
just indicated that the socket is readable.
Per report and analysis from Cyril VELTER.
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
input in the stats collector. Our select() emulation is apparently buggy
for UDP sockets :-(. This should resolve problems with stats collection
(and hence autovacuum) failing under more than minimal load. Diagnosis
and patch by Magnus Hagander.
Patch probably needs to be back-ported to 8.1 and 8.0, but first let's
see if it makes the buildfarm happy...
FormatMessage() (This should have been in 8.2.0, patched to 8.2.X and
HEAD):
I think this problem to be complex....
http://archives.postgresql.org/pgsql-hackers/2006-11/msg00042.php
FormatMessage of windows cannot consider the encoding of the database.
However, I should try the solution now. It is necessary to clear the
problem.
Multi character-code exists together in message and log. It doesn't
consider
the data base encoding that the user intended....
The user in multi-byte country can try this.
http://inet.winpg.jp/~saito/pg_bug/MessageCheck.c
That is, it is likely to become it in this manner.(Japanese)
http://inet.winpg.jp/~saito/pg_bug/FormatMessage998.png
Hiroshi Saito
1) pgwin32_waitforsinglesocket(): WaitForMultipleObjectsEx now called with
finite timeout (100ms) in case of FP_WRITE and UDP socket. If timeout occurs
then pgwin32_waitforsinglesocket() tries to write empty packet goes to
WaitForMultipleObjectsEx again.
2) pgwin32_send(): add loop around WSASend and pgwin32_waitforsinglesocket().
The reason is: for overlapped socket, 'ok' result from
pgwin32_waitforsinglesocket() isn't guarantee that socket is still free,
it can become busy again and following WSASend call will fail with
WSAEWOULDBLOCK error.
See http://archives.postgresql.org/pgsql-hackers/2006-10/msg00561.php
as micro-seconds, rather than as 100 microseconds, as it does now. This
actually fixes all setitimer calls on Win32, but statement_timeout is
the most visible fix.
Backpatch to 8.1.X. 8.0 works as documented.
pg_usleep at all. Instead call the replacement function in
port/win32/signal.c by that name. Avoids tricky macro-redefinition
logic and suppresses a compiler warning; furthermore it ensures that
no one can accidentally use the non-signal-aware version of pg_usleep
in a Windows backend.
o remove many WIN32_CLIENT_ONLY defines
o add WIN32_ONLY_COMPILER define
o add 3rd argument to open() for portability
o add include/port/win32_msvc directory for
system includes
Magnus Hagander
more compliant with the error message style guide. In particular,
errdetail should begin with a capital letter and end with a period,
whereas errmsg should not. I also fixed a few related issues in
passing, such as fixing the repeated misspelling of "lexeme" in
contrib/tsearch2 (per Tom's suggestion).
comment line where output as too long, and update typedefs for /lib
directory. Also fix case where identifiers were used as variable names
in the backend, but as typedefs in ecpg (favor the backend for
indenting).
Backpatch to 8.1.X.
to the main thread. This allows removal of WaitForSingleObjectEx() calls
from the main thread, thereby allowing us to re-enable Qingqing Zhou's
CHECK_FOR_INTERRUPTS performance improvement. Qingqing, Magnus, et al.
a kernel call unless there's some evidence of a pending signal. This should
bring its performance on Windows into line with the Unix version. Problem
diagnosis and patch by Qingqing Zhou. Minor stylistic tweaks by moi ...
if it's broken, it's my fault.
level for unrecognized win32 error codes to LOG, and make messages
conform to style guide. Per old suggestion from Qingqing Zhou, which
seems to have gotten lost in the shuffle.
to 'Size' (that is, size_t), and install overflow detection checks in it.
This allows us to remove the former arbitrary restrictions on NBuffers
etc. It won't make any difference in a 32-bit machine, but in a 64-bit
machine you could theoretically have terabytes of shared buffers.
(How efficiently we could manage 'em remains to be seen.) Similarly,
num_temp_buffers, work_mem, and maintenance_work_mem can be set above
2Gb on a 64-bit machine. Original patch from Koichi Suzuki, additional
work by moi.
> >> 3) I restarted the postmaster both times. I got this error
> both times.
> >> :25: ERROR: could not load library "C:/Program
> >> Files/PostgreSQL/8.0/lib/testtrigfuncs.dll": dynamic load error
>
> > Yes. We really need to look at fixing that error message. I had
> > forgotten it completely :-(
>
> > Bruce, you think we can sneak that in after feature freeze? I would
> > call it a bugfix :-)
>
> Me too. That's been on the radar for awhile --- please do
> send in a patch.
Here we go, that wasn't too hard :-)
Apart from adding the error handling, it does one more thing: it changes
the errormode when loading the DLLs. Previously if a DLL was broken, or
referenced other DLLs that couldn't be found, a popup dialog box would
appear on the screen. Which had to be clicked before the backend could
continue. This patch also disables the popup error message for DLL
loads.
I think this is something we should consider doing for the entire
backend - disable those popups, and say we deal with it ourselves. What
do you other win32 hackers thinnk about this?
In the meantime, this patch fixes the error msgs. Please apply for 8.1
and please consider a backpatch to 8.0.
Magnus Hagander
before we can invoke fork() -- flush stdio buffers, save and restore the
profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This
patch moves that code into a single function, fork_process(), instead of
duplicating it at the various callsites of fork().
This patch doesn't address the EXEC_BACKEND case; there is room for
further cleanup there.
Also performed an initial run through of upgrading our Copyright date to
extend to 2005 ... first run here was very simple ... change everything
where: grep 1996-2004 && the word 'Copyright' ... scanned through the
generated list with 'less' first, and after, to make sure that I only
picked up the right entries ...
to shared memory as soon as possible, ie, right after read_backend_variables.
The effective difference from the original code is that this happens
before instead of after read_nondefault_variables(), which loads GUC
information and is apparently capable of expanding the backend's memory
allocation more than you'd think it should. This should fix the
failure-to-attach-to-shared-memory reports we've been seeing on Windows.
Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
even uglier than it was already :-(. Also, on Windows only, use temporary
shared memory segments instead of ordinary files to pass over critical
variable values from postmaster to child processes. Magnus Hagander
> pg specific, like "PostgreSQL.1". I have not done this since a new compile
> would not detect a running old beta. But now would be the time (or never).
Zeugswetter Andreas
shared memory segment ID. If we can't access the existing shmem segment,
it must not be relevant to our data directory. If we can access it,
then attach to it and check for an actual match to the data directory.
This should avoid some cases of failure-to-restart-after-boot without
introducing any significant risk of failing to detect a still-running
old backend.
- remove another senseless "extern" keyword that was applied to a
function definition
- change a foo more function signatures from "some_type foo()" to
"some_type foo(void)"
- rewrite another K&R style function definition
- make the type of the "action" function pointer in the KeyWord struct
in src/backend/utils/adt/formatting.c more precise
* Links with -leay32 and -lssleay32 instead of crypto and ssl. On win32,
"crypto and ssl" is only used for static linking.
* Initializes SSL in the backend and not just in the postmaster. We
cannot pass the SSL context from the postmaster through the parameter
file, because it contains function pointers.
* Split one error check in be-secure.c. Previously we could not tell
which of three calls actually failed. The previous code also returned
incorrect error messages if SSL_accept() failed - that function needs to
use SSL_get_error() on the return value, can't just use the error queue.
* Since the win32 implementation uses non-blocking sockets "behind the
scenes" in order to deliver signals correctly, implements a version of
SSL_accept() that can handle this. Also, add a wait function in case
SSL_read or SSL_write() needs more data.
Magnus Hagander
>>GetLastError will
>>> give much more details than errno.
>>
>>How much more, really? That mapping table gave me the impression that
>>the win32 error codes aren't all that much more detailed than errno...
>
>The mapping table is not complete. My winerror.h from the SDK
>lists 2209
>error codes, whereas errno.h lists 42...
>
>I still don't think we'll get that much more stuff. Right now,
>the Win32
>code paths that actually use the more advanced functions already write
>out the error number in case something happens. We can keep doing that
>for the other paths (ereport the error *number* when the mapping does
>not have a match). The map to errno will catch almost all cases, I
>think. And in the corner cases we can do with just the number, and use
>"net helpmsg" to get the actual message when checking...
Here's an attempt on this. new file goes in backend/port/win32.
Magnus Hagander
about a third, make it work on non-Windows platforms again. (But perhaps
I broke the WIN32 code, since I have no way to test that.) Fold all the
paths that fork postmaster child processes to go through the single
routine SubPostmasterMain, which takes care of resurrecting the state that
would normally be inherited from the postmaster (including GUC variables).
Clean up some places where there's no particularly good reason for the
EXEC and non-EXEC cases to work differently. Take care of one or two
FIXMEs that remained in the code.
* removed a few redundant defines
* get_user_name safe under win32
* rationalized pipe read EOF for win32 (UPDATED PATCH USED)
* changed all backend instances of sleep() to pg_usleep
- except for the SLEEP_ON_ASSERT in assert.c, as it would exceed a
32-bit long [Note to patcher: If a SLEEP_ON_ASSERT of 2000 seconds is
acceptable, please replace with pg_usleep(2000000000L)]
I added a comment to that part of the code:
/*
* It would be nice to use pg_usleep() here, but only does 2000 sec
* or 33 minutes, which seems too short.
*/
sleep(1000000);
Claudio Natoli
It works on the principle of turning sockets into non-blocking, and then
emulate blocking behaviour on top of that, while allowing signals to
run. Signals are now implemented using an event instead of APCs, thus
getting rid of the issue of APCs not being compatible with "old style"
sockets functions.
It also moves the win32 specific code away from pqsignal.h/c into
port/win32, and also removes the "thread style workaround" of the APC
issue previously in place.
In order to make things work, a few things are also changed in pgstat.c:
1) There is now a separate pipe to the collector and the bufferer. This
is required because the pipe will otherwise only be signalled in one of
the processes when the postmaster goes down. The MS winsock code for
select() must have some kind of workaround for this behaviour, but I
have found no stable way of doing that. You really are not supposed to
use the same socket from more than one process (unless you use
WSADuplicateSocket(), in which case the docs specifically say that only
one will be flagged).
2) The check for "postmaster death" is moved into a separate select()
call after the main loop. The previous behaviour select():ed on the
postmaster pipe, while later explicitly saying "we do NOT check for
postmaster exit inside the loop".
The issue was that the code relies on the same select() call seeing both
the postmaster pipe *and* the pgstat pipe go away. This does not always
happen, and it appears that useing WSAEventSelect() makes it even more
common that it does not.
Since it's only called when the process exits, I don't think using a
separate select() call will have any significant impact on how the stats
collector works.
Magnus Hagander
* Mostly, casting etc to remove compilation warnings in win32 only code.
* main.c: set _IONBF to stdout/stderr under win32 (under win32, _IOLBF
defaults to full buffering)
* pg_resetxlog/Makefile: ensures dirmod.o gets cleaned (got bitten by
this when, after "make clean"ing, switching compilation between Ming +
Cygwin)
Claudio Natoli
* Changes incorrect CYGWIN defines to __CYGWIN__
* Some localtime returns NULL checks (when unchecked cause SEGVs under
Win32
regression tests)
* Rationalized CreateSharedMemoryAndSemaphores and
AttachSharedMemoryAndSemaphores (Bruce, I finally remembered to do it);
requires attention.
Claudio Natoli
applied, deadlock detection and statement_timeout now works.
The file timer.c goes into src/backend/port/win32/.
The patch also removes two lines of "printf debugging" accidentally left
in pqsignal.h, in the console control handler.
Magnus Hagander
Natoli and Bruce Momjian (and some cosmetic fixes from Neil Conway).
Changes:
- remove duplicate signal definitions from pqsignal.h
- replace pqkill() with kill() and redefine kill() in Win32
- use ereport() in place of fprintf() in some error handling in
pqsignal.c
- export pg_queue_signal() and make use of it where necessary
- add a console control handler for Ctrl-C and similar handling
on Win32
- do WaitForSingleObjectEx() in CHECK_FOR_INTERRUPTS() on Win32;
query cancelling should now work on Win32
- various other fixes and cleanups
* configure + Makefile changes
* shared memory attaching in EXEC_BACKEND case (+ minor fix for apparent
cygwin bug under cygwin/EXEC_BACKEND case only)
* PATH env var separator differences
* missing win32 rand functions added
* placeholder replacements for sync etc under port.h
To those who are really interested, and there are a few of you: the attached
patch + file will allow the source base to be compiled (and, for some
definition, "run") under MingW, with the following caveats (I wanted to
first properly fix all but the last of these, but y'all won't quit asking
for a patch :-):
* child death: SIGCHLD not yet sent, so as a minimum, you'll need to
put in some sort of delay after StartupDatabase, and handle setting
StartupPID to 0 etc (ie. the stuff the reaper() signal function is supposed
to do)
* dirmod.c: comment out the elog calls
* dfmgr.c: some hackage required to substitute_libpath_macro
* slru/xact.c: comment out the errno checking after the readdir
(fixed by next version of MingW)
Again, this is only if you *really* want to see postgres compile and start,
and is a nice leg-up for working on the other Win32 TODO list items. Just
don't expect too much else from it at this point...
Claudio Natoli
against the latest shapshot. It also includes the replacement of kill()
with pqkill() and sigsetmask() with pqsigsetmask().
Passes all tests fine on my linux machine once applied. Still doesn't
link completely on Win32 - there are a few things still required. But
much closer than before.
At Bruce's request, I'm goint to write up a README file about the method
of signals delivery chosen and why the others were rejected (basically a
summary of the mailinglist discussions). I'll finish that up once/if the
patch is accepted.
Magnus Hagander
pointer type when it is not necessary to do so.
For future reference, casting NULL to a pointer type is only necessary
when (a) invoking a function AND either (b) the function has no prototype
OR (c) the function is a varargs function.
on 64-bit Solaris. Use a non-system-dependent datatype for UsedShmemSegID,
namely unsigned long (which we were already assuming could hold a shmem
key anyway, cf RecordSharedMemoryInLockFile).
max_connections at initdb time. Get rid of DEF_NBUFFERS and DEF_MAXBACKENDS
macros, which aren't doing anything useful anymore, and put more likely
defaults into postgresql.conf.sample.
that OS=hpux is the same as CPU=hppa. First steps at doing this.
With these patches, we still work on hppa with either gcc or HP's cc.
We might work on hpux/itanium with gcc, but I can't test it. Definitely
will not work on hpux/itanium with non-gcc compiler, for lack of spinlock
code.
and 100 respectively, if the platform will allow it. initdb selects
values that are not too large to allow the postmaster to start, and
places these values in the installed postgresql.conf file. This allows
us to continue to start up out-of-the-box on platforms with small SHMMAX,
while having somewhat-realistic default settings on platforms with
reasonable SHMMAX. Per recent pghackers discussion.
Win32 port is now called 'win32' rather than 'win'
add -lwsock32 on Win32
make gethostname() be only used when kerberos4 is enabled
use /port/getopt.c
new /port/opendir.c routines
disable GUC unix_socket_group on Win32
convert some keywords.c symbols to KEYWORD_P to prevent conflict
create new FCNTL_NONBLOCK macro to turn off socket blocking
create new /include/port.h file that has /port prototypes, move
out of c.h
new /include/port/win32_include dir to hold missing include files
work around ERROR being defined in Win32 includes
postgres.h or c.h includes a system header (such as stdio.h or
stdlib.h), there's no need to specifically include it in any of the .c
files in the backend.
Neil Conway
> > > > found in the postmaster and not included from elsewhere)
> >
> > shared libs on AIX need to be able to resolve all symbols at linkage time.
> > Those two symbols are in backend/utils/SUBSYS.o but not in the postgres
> > executable.
>
> They are defined in backend/utils/mb/conv.c and declared in
> include/mb/pg_wchar.h. They're also linked into the
> postmaster. I don't see anything unusual.
Attached is a patch to fix the mb linking problems on AIX. As a nice side effect
it reduces the duplicate symbol warnings to linking libpq.so and libecpg.so
(all shlibs that are not postmaster loadable modules).
Please apply to current (only affects AIX).
The _LARGE_FILES problem is unfortunately still open, unless Peter
has fixed it per his recent idea.
Zeugswetter Andreas SB SD
error handling, and simplifies the code that remains. Apparently,
the code that left Berkeley had a whole "error handling subsystem",
which exceptions and whatnot. Since we don't use that anymore,
there's no reason to keep it around.
The regression tests pass with the patch applied. Unless anyone
sees a problem, please apply.
Neil Conway
As proof of concept, provide an alternate implementation based on POSIX
semaphores. Also push the SysV shared-memory implementation into a
separate file so that it can be replaced conveniently.
src/GNUmakefile.in to src/Makefile
and
src/backend/port/Makefile.in to src/backend/port/Makefile
All configure substitutions are now done in Makefile.global.
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
This restores the Linux behavior to what it was in PG 7.0 and 7.1, and
causes other platforms to agree. (Other well-tested platforms like HPUX
were doing it this way already.) Per pghackers discussion over the past
month or so.
