width types and varlena types, since with the introduction of CSTRING as
a more-or-less-real type, these concepts aren't identical. I've tried to
use varlena consistently to denote datatypes with typlen = -1, ie, they
have a length word and are potentially TOASTable; while the term variable
width covers both varlena and cstring (and, perhaps, someday other types
with other rules for computing the actual width). No code changes in this
commit except for renaming a couple macros.
on the server, if DebugLvl >= 2.
The patch also includes a late addition to the last patch
(X509_check_private_key()). I'm not sure why it the currect
revision wasn't tagged.
Bear Giles
If the user has certificates in $HOME/.postgresql/postgresql.crt
and $HOME/.postgresql/postgresql.key exist, they are provided
to the server. The certificate used to sign this cert must be
known to the server, in $DataDir/root.crt. If successful, the
cert's "common name" is logged.
Client certs are not used for authentication, but they could be
via the port->peer (X509 *), port->peer_dn (char *) or
port->peer_cn (char *) fields. Or any other function could be
used, e.g., many sites like the issuer + serial number hash.
Bear Giles
In order to reduce the risk of cryptanalysis during extended
sessions (or brief ones involving a substantial amount of data),
this patch renegotiates the session key after 64kib has been
transferred.
Bear Giles
As the comment headers in be-secure.c discusses, EPH preserves
confidentiality even if the static private key (which is usually
kept unencrypted) is compromised.
Because of the value of this, common default values are hard-coded
to protect the confidentiality of the data even if an attacker
successfully deletes or modifies the external file.
Bear Giles
Attached are a revised set of SSL patches. Many of these patches
are motivated by security concerns, it's not just bug fixes. The key
differences (from stock 7.2.1) are:
*) almost all code that directly uses the OpenSSL library is in two
new files,
src/interfaces/libpq/fe-ssl.c
src/backend/postmaster/be-ssl.c
in the long run, it would be nice to merge these two files.
*) the legacy code to read and write network data have been
encapsulated into read_SSL() and write_SSL(). These functions
should probably be renamed - they handle both SSL and non-SSL
cases.
the remaining code should eliminate the problems identified
earlier, albeit not very cleanly.
*) both front- and back-ends will send a SSL shutdown via the
new close_SSL() function. This is necessary for sessions to
work properly.
(Sessions are not yet fully supported, but by cleanly closing
the SSL connection instead of just sending a TCP FIN packet
other SSL tools will be much happier.)
*) The client certificate and key are now expected in a subdirectory
of the user's home directory. Specifically,
- the directory .postgresql must be owned by the user, and
allow no access by 'group' or 'other.'
- the file .postgresql/postgresql.crt must be a regular file
owned by the user.
- the file .postgresql/postgresql.key must be a regular file
owned by the user, and allow no access by 'group' or 'other'.
At the current time encrypted private keys are not supported.
There should also be a way to support multiple client certs/keys.
*) the front-end performs minimal validation of the back-end cert.
Self-signed certs are permitted, but the common name *must*
match the hostname used by the front-end. (The cert itself
should always use a fully qualified domain name (FDQN) in its
common name field.)
This means that
psql -h eris db
will fail, but
psql -h eris.example.com db
will succeed. At the current time this must be an exact match;
future patches may support any FQDN that resolves to the address
returned by getpeername(2).
Another common "problem" is expiring certs. For now, it may be
a good idea to use a very-long-lived self-signed cert.
As a compile-time option, the front-end can specify a file
containing valid root certificates, but it is not yet required.
*) the back-end performs minimal validation of the client cert.
It allows self-signed certs. It checks for expiration. It
supports a compile-time option specifying a file containing
valid root certificates.
*) both front- and back-ends default to TLSv1, not SSLv3/SSLv2.
*) both front- and back-ends support DSA keys. DSA keys are
moderately more expensive on startup, but many people consider
them preferable than RSA keys. (E.g., SSH2 prefers DSA keys.)
*) if /dev/urandom exists, both client and server will read 16k
of randomization data from it.
*) the server can read empheral DH parameters from the files
$DataDir/dh512.pem
$DataDir/dh1024.pem
$DataDir/dh2048.pem
$DataDir/dh4096.pem
if none are provided, the server will default to hardcoded
parameter files provided by the OpenSSL project.
Remaining tasks:
*) the select() clauses need to be revisited - the SSL abstraction
layer may need to absorb more of the current code to avoid rare
deadlock conditions. This also touches on a true solution to
the pg_eof() problem.
*) the SIGPIPE signal handler may need to be revisited.
*) support encrypted private keys.
*) sessions are not yet fully supported. (SSL sessions can span
multiple "connections," and allow the client and server to avoid
costly renegotiations.)
*) makecert - a script that creates back-end certs.
*) pgkeygen - a tool that creates front-end certs.
*) the whole protocol issue, SASL, etc.
*) certs are fully validated - valid root certs must be available.
This is a hassle, but it means that you *can* trust the identity
of the server.
*) the client library can handle hardcoded root certificates, to
avoid the need to copy these files.
*) host name of server cert must resolve to IP address, or be a
recognized alias. This is more liberal than the previous
iteration.
*) the number of bytes transferred is tracked, and the session
key is periodically renegotiated.
*) basic cert generation scripts (mkcert.sh, pgkeygen.sh). The
configuration files have reasonable defaults for each type
of use.
Bear Giles
are motivated by security concerns, it's not just bug fixes. The key
differences (from stock 7.2.1) are:
*) almost all code that directly uses the OpenSSL library is in two
new files,
src/interfaces/libpq/fe-ssl.c
src/backend/postmaster/be-ssl.c
in the long run, it would be nice to merge these two files.
*) the legacy code to read and write network data have been
encapsulated into read_SSL() and write_SSL(). These functions
should probably be renamed - they handle both SSL and non-SSL
cases.
the remaining code should eliminate the problems identified
earlier, albeit not very cleanly.
*) both front- and back-ends will send a SSL shutdown via the
new close_SSL() function. This is necessary for sessions to
work properly.
(Sessions are not yet fully supported, but by cleanly closing
the SSL connection instead of just sending a TCP FIN packet
other SSL tools will be much happier.)
*) The client certificate and key are now expected in a subdirectory
of the user's home directory. Specifically,
- the directory .postgresql must be owned by the user, and
allow no access by 'group' or 'other.'
- the file .postgresql/postgresql.crt must be a regular file
owned by the user.
- the file .postgresql/postgresql.key must be a regular file
owned by the user, and allow no access by 'group' or 'other'.
At the current time encrypted private keys are not supported.
There should also be a way to support multiple client certs/keys.
*) the front-end performs minimal validation of the back-end cert.
Self-signed certs are permitted, but the common name *must*
match the hostname used by the front-end. (The cert itself
should always use a fully qualified domain name (FDQN) in its
common name field.)
This means that
psql -h eris db
will fail, but
psql -h eris.example.com db
will succeed. At the current time this must be an exact match;
future patches may support any FQDN that resolves to the address
returned by getpeername(2).
Another common "problem" is expiring certs. For now, it may be
a good idea to use a very-long-lived self-signed cert.
As a compile-time option, the front-end can specify a file
containing valid root certificates, but it is not yet required.
*) the back-end performs minimal validation of the client cert.
It allows self-signed certs. It checks for expiration. It
supports a compile-time option specifying a file containing
valid root certificates.
*) both front- and back-ends default to TLSv1, not SSLv3/SSLv2.
*) both front- and back-ends support DSA keys. DSA keys are
moderately more expensive on startup, but many people consider
them preferable than RSA keys. (E.g., SSH2 prefers DSA keys.)
*) if /dev/urandom exists, both client and server will read 16k
of randomization data from it.
*) the server can read empheral DH parameters from the files
$DataDir/dh512.pem
$DataDir/dh1024.pem
$DataDir/dh2048.pem
$DataDir/dh4096.pem
if none are provided, the server will default to hardcoded
parameter files provided by the OpenSSL project.
Remaining tasks:
*) the select() clauses need to be revisited - the SSL abstraction
layer may need to absorb more of the current code to avoid rare
deadlock conditions. This also touches on a true solution to
the pg_eof() problem.
*) the SIGPIPE signal handler may need to be revisited.
*) support encrypted private keys.
*) sessions are not yet fully supported. (SSL sessions can span
multiple "connections," and allow the client and server to avoid
costly renegotiations.)
*) makecert - a script that creates back-end certs.
*) pgkeygen - a tool that creates front-end certs.
*) the whole protocol issue, SASL, etc.
*) certs are fully validated - valid root certs must be available.
This is a hassle, but it means that you *can* trust the identity
of the server.
*) the client library can handle hardcoded root certificates, to
avoid the need to copy these files.
*) host name of server cert must resolve to IP address, or be a
recognized alias. This is more liberal than the previous
iteration.
*) the number of bytes transferred is tracked, and the session
key is periodically renegotiated.
*) basic cert generation scripts (mkcert.sh, pgkeygen.sh). The
configuration files have reasonable defaults for each type
of use.
Bear Giles
As proof of concept, provide an alternate implementation based on POSIX
semaphores. Also push the SysV shared-memory implementation into a
separate file so that it can be replaced conveniently.
A new pg_hba.conf column, USER
Allow specifiction of lists of users separated by commas
Allow group names specified by +
Allow include files containing lists of users specified by @
Allow lists of databases, and database files
Allow samegroup in database column to match group name matching dbname
Removal of secondary password files
Remove pg_passwd utility
Lots of code cleanup in user.c and hba.c
New data/global/pg_pwd format
New data/global/pg_group file
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
when to send what to which, prevent recursion by introducing new COMMERROR
elog level for client-communication problems, get rid of direct writes
to stderr in backend/libpq files, prevent non-error elogs from going to
client during the authentication cycle.
now just below FATAL in server_min_messages. Added more text to
highlight ordering difference between it and client_min_messages.
---------------------------------------------------------------------------
REALLYFATAL => PANIC
STOP => PANIC
New INFO level the prints to client by default
New LOG level the prints to server log by default
Cause VACUUM information to print only to the client
NOTICE => INFO where purely information messages are sent
DEBUG => LOG for purely server status messages
DEBUG removed, kept as backward compatible
DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1 added
DebugLvl removed in favor of new DEBUG[1-5] symbols
New server_min_messages GUC parameter with values:
DEBUG[5-1], INFO, NOTICE, ERROR, LOG, FATAL, PANIC
New client_min_messages GUC parameter with values:
DEBUG[5-1], LOG, INFO, NOTICE, ERROR, FATAL, PANIC
Server startup now logged with LOG instead of DEBUG
Remove debug_level GUC parameter
elog() numbers now start at 10
Add test to print error message if older elog() values are passed to elog()
Bootstrap mode now has a -d that requires an argument, like postmaster
> and that the right fix is to make each of the subsequent calls be in
> this same pattern, not to try to emulate their nonsensical style.
Dominic J. Eidson
to use and significantly faster. This tweak saves 25% (!) of the runtime
of COPY IN in a test with 8000-character lines. I wouldn't normally
commit a performance improvement this late in the cycle, but 25% got
my attention...
postmaster children before client auth step. Postmaster now rereads
pg_pwd on receipt of SIGHUP, the same way that pg_hba.conf is handled.
No cycles need be expended to validate password cache validity during
connection startup.
authentication failed' and a 'send() failed: Broken pipe' message
on every connection from psql in password auth mode. Problem is
that psql doesn't ask user for a password until it sees a password
challenge failure, and libpq just closes the connection unceremoniously
if it's challenged for a password when it hasn't got one to send.
Accordingly, EOF from the client after asking for a password is
normal behavior and should not result in postmaster log entries.
a hung client or lost connection can't indefinitely block a postmaster
child (not to mention the possibility of deliberate DoS attacks).
Timeout is controlled by new authentication_timeout GUC variable,
which I set to 60 seconds by default ... does that seem reasonable?
We will no longer try to send elog messages to the client before we have
initialized backend libpq (oops); however, reporting bogus commandline
switches via elog does work now (not irrelevant, because of PGOPTIONS).
Fix problem with inappropriate sending of checkpoint-process messages
to stderr.
> pam_strerror() should be used a few more times, rather than just saying
> "Error!". Also, the configure.in snippet seems wrong. You add
> -I$pam_prefix/include/security to $INCLUDES and then you #include
> <security/pam_appl.h>. This whole thing is probably unnecessary, since
> PAM is a system library on the systems where it exists, so the headers
> and libraries are found automatically, unlike OpenSSL and
> Kerberos.
See attached revised patch. (I'm sure the configure.in stuff can be done
right/better, I'm just not enough of a autoconf guru to know what to
change it to.)
Dominic J. Eidson
Allow pg_shadow to be MD5 encrypted.
Add ENCRYPTED/UNENCRYPTED option to CREATE/ALTER user.
Add password_encryption postgresql.conf option.
Update wire protocol version to 2.1.
that call is not needed to prepare for SO_PEERCRED. Also, simplify code
so that #ifdef SO_PEERCRED appears in only one place, to make it easier
to support other platforms with variants of this capability.
points out how silly it is to use Autoconf to test for a preprocessor
symbol, when one can equally easily #ifdef on the symbol itself.
Accordingly, revert configure to prior state and do it that way.
system supports SO_PEERCRED requests for Unix sockets. This is an
amalgamation of patches submitted by Helge Bahmann and Oliver Elphick,
with some editorializing by yours truly.
number in the data structure so that we can give at least a minimally
useful idea of where the mistake is when we issue syntax error messages.
Move the ClientAuthentication() call to where it should have been in
the first place, so that postmaster memory releasing can happen in a
reasonable place also. Update obsolete comments, correct one real bug
(auth_argument was not picked up correctly).
a lie on many Unixen), invoke listen() with MIN(MaxBackends*2, 10000).
The clamp value 10000 is configurable in config.h.in, if that proves
to be necessary --- hopefully it won't.
a new postmaster child process. This should eliminate problems with
authentication blocking (e.g., ident, SSL init) and also reduce problems
with the accept queue filling up under heavy load.
The option to send elog output to a different file per backend (postgres -o)
has been disabled for now because the initialization would have to happen
in a different order and it's not clear we want to keep this anyway.
database, including system catalogs (but not the shared catalogs,
since they don't really belong to his database). This is per recent
mailing list discussion. Clean up some other code that also checks
for database ownerness by introducing a test function is_dbadmin().
only possible failure is in pq_flush, which will log a (better!) report
anyway --- so pq_endmessage is just cluttering the log with a redundant
entry. This matters when a client crashes partway through a large query,
since we will emit many broken-pipe reports before finishing the query
and exiting.
are now separate files "postgres.h" and "postgres_fe.h", which are meant
to be the primary include files for backend .c files and frontend .c files
respectively. By default, only include files meant for frontend use are
installed into the installation include directory. There is a new make
target 'make install-all-headers' that adds the whole content of the
src/include tree to the installed fileset, for use by people who want to
develop server-side code without keeping the complete source tree on hand.
Cleaned up a whole lot of crufty and inconsistent header inclusions.
to ensure that we have released buffer refcounts and so forth, rather than
putting ad-hoc operations before (some of the calls to) proc_exit. Add
commentary to discourage future hackers from repeating that mistake.
socket file, in favor of having an ordinary lockfile beside the socket file.
Clean up a few robustness problems in the lockfile code. If postmaster is
going to reject a connection request based on database state, it will now
tell you so before authentication exchange not after. (Of course, a failure
after is still possible if conditions change meanwhile, but this makes life
easier for a yet-to-be-written pg_ping utility.)
hosting product, on both shared and dedicated machines. We currently
offer Oracle and MySQL, and it would be a nice middle-ground.
However, as shipped, PostgreSQL lacks the following features we need
that MySQL has:
1. The ability to listen only on a particular IP address. Each
hosting customer has their own IP address, on which all of their
servers (http, ftp, real media, etc.) run.
2. The ability to place the Unix-domain socket in a mode 700 directory.
This allows us to automatically create an empty database, with an
empty DBA password, for new or upgrading customers without having
to interactively set a DBA password and communicate it to (or from)
the customer. This in turn cuts down our install and upgrade times.
3. The ability to connect to the Unix-domain socket from within a
change-rooted environment. We run CGI programs chrooted to the
user's home directory, which is another reason why we need to be
able to specify where the Unix-domain socket is, instead of /tmp.
4. The ability to, if run as root, open a pid file in /var/run as
root, and then setuid to the desired user. (mysqld -u can almost
do this; I had to patch it, too).
The patch below fixes problem 1-3. I plan to address #4, also, but
haven't done so yet. These diffs are big enough that they should give
the PG development team something to think about in the meantime :-)
Also, I'm about to leave for 2 weeks' vacation, so I thought I'd get
out what I have, which works (for the problems it tackles), now.
With these changes, we can set up and run PostgreSQL with scripts the
same way we can with apache or proftpd or mysql.
In summary, this patch makes the following enhancements:
1. Adds an environment variable PGUNIXSOCKET, analogous to MYSQL_UNIX_PORT,
and command line options -k --unix-socket to the relevant programs.
2. Adds a -h option to postmaster to set the hostname or IP address to
listen on instead of the default INADDR_ANY.
3. Extends some library interfaces to support the above.
4. Fixes a few memory leaks in PQconnectdb().
The default behavior is unchanged from stock 7.0.2; if you don't use
any of these new features, they don't change the operation.
David J. MacKenzie
kibitzing from Tom Lane. Large objects are now all stored in a single
system relation "pg_largeobject" --- no more xinv or xinx files, no more
relkind 'l'. This should offer substantial performance improvement for
large numbers of LOs, since there won't be directory bloat anymore.
It'll also fix problems like running out of locktable space when you
access thousands of LOs in one transaction.
Also clean up cruft in read/write routines. LOs with "holes" in them
(never-written byte ranges) now work just like Unix files with holes do:
a hole reads as zeroes but doesn't occupy storage space.
INITDB forced!
* Makefile: Add more standard targets. Improve shell redirection in GNU
make detection.
* src/backend/access/transam/rmgr.c: Fix incorrect(?) C.
* src/backend/libpq/pqcomm.c (StreamConnection): Work around accept() bug.
* src/include/port/unixware.h: ...with help from here.
* src/backend/nodes/print.c (plannode_type): Remove some "break"s after
"return"s.
* src/backend/tcop/dest.c (DestToFunction): ditto.
* src/backend/nodes/readfuncs.c: Add proper prototypes.
* src/backend/utils/adt/numutils.c (pg_atoi): Cope specially with strtol()
setting EINVAL. This saves us from creating an extra set of regression test
output for the affected systems.
* src/include/storage/s_lock.h (tas): Correct prototype.
* src/interfaces/libpq/fe-connect.c (parseServiceInfo): Don't use variable
as dimension in array definition.
* src/makefiles/Makefile.unixware: Add support for GCC.
* src/template/unixware: same here
* src/test/regress/expected/abstime-solaris-1947.out: Adjust whitespace.
* src/test/regress/expected/horology-solaris-1947.out: Part of this file
was evidently missing.
* src/test/regress/pg_regress.sh: Fix shell. mkdir -p returns non-zero if
the directory exists.
* src/test/regress/resultmap: Add entries for Unixware.
> Regression tests opr_sanity and sanity_check are now failing.
Um, Bruce, I've said several times that I didn't think Perchine's large
object changes should be applied until someone had actually reviewed
them.
I tested it restoring my database with > 100000 BLOBS, and dumping it out.
But unfortunatly I can not restore it back due to problems in pg_dump.
--
Sincerely Yours,
Denis Perchine
As a result, backend/libpq/pqcomm.c and interfaces/libpq/fe-connect.c
fail to compile.
The <netinet/tcp.h> header needs to be preceded by <netinet/in.h>, at
least on IRIX, Solaris and AIX. The simple configure test fails.
(That header on Linux is idempotent.)
The basic problem is that <netinet/tcp.h> is a BSD header. The
correct header for TCP internals such as TCP_NODELAY on a UNIX system
is <xti.h>. By UNIX I mean UNIX95 (aka XPG4v2 or SUSv1) or later.
The current UNIX standard (UNIX98 aka SUSv2) is available online at
<http://www.opengroup.org/onlinepubs/7908799/>.
The fix is to add header support for <xti.h> into configure.in and
config.h.in.
The 2 files which conditionally include <netinet/tcp.h> need also to
conditionally include <xti.h>.
Pete Forman
> this is patch v 0.4 to support transactions with BLOBs.
> All BLOBs are in one table. You need to make initdb.
>
> --
> Sincerely Yours,
> Denis Perchine
working on the VERY latest version of BeOS. I'm sure there will be
alot of comments, but then if there weren't I'd be disappointed!
Thanks for your continuing efforts to get this into your tree.
Haven't bothered with the new files as they haven't changed.
BTW Peter, the compiler is "broken" about the bool define and so on.
I'm filing a bug report to try and get it addressed. Hopefully then we
can tidy up the code a bit.
I await the replies with interest :)
David Reid
Update the installation instructions (formerly misnamed "FAQ"), add configure
checks for some headers rather than having users copy stubs manually (ugh!).
Use Autoconf check for exe extension. This also avoids inheriting the value
of $(X) from the environment.
(rather than compile time). For libpq, even when Kerberos support is
compiled in, the default user name should still fall back to geteuid()
if it can't be determined via the Kerberos system.
A couple of fixes for string type configuration parameters, now that there
is one.
There's now only one transition value and transition function.
NULL handling in aggregates is a lot cleaner. Also, use Numeric
accumulators instead of integer accumulators for sum/avg on integer
datatypes --- this avoids overflow at the cost of being a little slower.
Implement VARIANCE() and STDDEV() aggregates in the standard backend.
Also, enable new LIKE selectivity estimators by default. Unrelated
change, but as long as I had to force initdb anyway...
* the result is not recorded anywhere
* the result is not used anywhere
* the result is only used in some places, whereas others have been getting away with it
* the result is used improperly
Also make command line options handling a little better (e.g., --disable-locale,
while redundant, should really still *dis*able).
* Add option to build with OpenSSL out of the box. Fix thusly exposed
bit rot. Although it compiles now, getting this to do something
useful is left as an exercise.
* Fix Kerberos options to defer checking for required libraries until
all the other libraries are checked for.
* Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc.
* Install work around for Autoconf's install-sh relative path anomaly.
Get rid of old INSTL_*_OPTS variables, now that we don't need them
anymore.
* Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX.
* Look for only one of readline.h or readline/readline.h, not both.
* Make check for PS_STRINGS cacheable. Don't test for the header files
separately.
* Disable fcntl(F_SETLK) test on Linux.
* Substitute the standard GCC warnings set into CFLAGS in configure,
don't add it on in Makefile.global.
* Sweep through contrib tree to teach makefiles standard semantics.
... and in completely unrelated news:
* Make postmaster.opts arbitrary options-aware. I still think we need to
save the environment as well.
backend functions via backend PQexec(). The SPI interface has long
been our only documented way to do this, and the backend pqexec/portal
code is unused and suffering bit-rot. I'm putting it out of its misery.
Does not work since it fetches one byte beyond the source data, and when
the phase of the moon is wrong, the source data is smack up against the
end of backend memory and you get SIGSEGV. Don't laugh, this is a fix
for an actual user bug report.
files to restrict the set of users that can connect to a database
but can still use the pg_shadow password. (You just leave off the
password field in the secondary file.)
Don't go through pg_exec_query_dest(), but directly to the execution
routines. Also, extend parameter lists so that there's no need to
change the global setting of allowSystemTableMods, a hack that was
certain to cause trouble in the event of any error.
for details). It doesn't really do that much yet, since there are no
short-term memory contexts in the executor, but the infrastructure is
in place and long-term contexts are handled reasonably. A few long-
standing bugs have been fixed, such as 'VACUUM; anything' in a single
query string crashing. Also, out-of-memory is now considered a
recoverable ERROR, not FATAL.
Eliminate a large amount of crufty, now-dead code in and around
memory management.
Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and
backend startup.
we'll get there one day.
Use `cat' to create aclocal.m4, not `aclocal'. Some people don't
have automake installed.
Only run the autoconf rule in the top-level GNUmakefile if the
invoker specified `make configure', don't run it automatically
because of CVS timestamp skew.
Interfaced a lot of the custom tests to the config.cache, in the process
made them separate macros and grouped them out into files. Made naming
adjustments.
Removed a couple of useless/unused configure tests.
Disabled C++ by default. C++ is no more special than Perl, Python, and Tcl.
And it breaks equally often. :(
That means you can now set your options in either or all of $PGDATA/configuration,
some postmaster option (--enable-fsync=off), or set a SET command. The list of
options is in backend/utils/misc/guc.c, documentation will be written post haste.
pg_options is gone, so is that pq_geqo config file. Also removed were backend -K,
-Q, and -T options (no longer applicable, although -d0 does the same as -Q).
Added to configure an --enable-syslog option.
changed all callers from TPRINTF to elog(DEBUG)
key call sites are changed, but most called functions are still oldstyle.
An exception is that the PL managers are updated (so, for example, NULL
handling now behaves as expected in plperl and plpgsql functions).
NOTE initdb is forced due to added column in pg_proc.
Most (nearly all) of the work was done by David Wragg <dpw@doc.ic.ac.uk>
He patched 6.5.3. I've updated it for 7.0RC5.
It works for MIT kerberos 1.1.1 (and previously for 1.0.6 as well).
I've got the patch against 6.5.3, plus kerberized RPMS.
Mike Wyer <mw@doc.ic.ac.uk> || "Woof?"
subsequent I/O attempts fail cleanly. I'm speculating about failure
scenarios in which we do pq_close, then something in a proc_exit routine
opens a file (re-using that kernel FD number), then something else
fails and tries to write an elog message to the frontend ... message
ends up in opened file, oops. No known examples of this but it seems
like a potential hole.
some platforms --- and I also see that it is documented as not thread-
safe on HPUX and possibly other platforms. No good reason not to just
use IPPROTO_TCP constant from <netinet/in.h> instead.
all platforms, not just SCO. The operation is undefined for Unix-domain
sockets anyway. It seems SCO is not the only platform that complains
instead of treating the call as a no-op.
running gcc and HP's cc with warnings cranked way up. Signed vs unsigned
comparisons, routines declared static and then defined not-static,
that kind of thing. Tedious, but perhaps useful...
eliminating some wildly inconsistent coding in various parts of the
system. I set MAXPGPATH = 1024 in config.h.in. If anyone is really
convinced that there ought to be a configure-time test to set the
value, go right ahead ... but I think it's a waste of time.
When drawing up a very simple "text-drawing" of how the negotiation is done,
I realised I had done this last part (fallback) in a very stupid way. Patch
#4 fixes this, and does it in a much better way.
Included is also the simple text-drawing of how the negotiation is done.
//Magnus
been applied. The patches are in the .tar.gz attachment at the end:
varchar-array.patch this patch adds support for arrays of bpchar() and
varchar(), which where always missing from postgres.
These datatypes can be used to replace the _char4,
_char8, etc., which were dropped some time ago.
block-size.patch this patch fixes many errors in the parser and other
program which happen with very large query statements
(> 8K) when using a page size larger than 8192.
This patch is needed if you want to submit queries
larger than 8K. Postgres supports tuples up to 32K
but you can't insert them because you can't submit
queries larger than 8K. My patch fixes this problem.
The patch also replaces all the occurrences of `8192'
and `1<<13' in the sources with the proper constants
defined in include files. You should now never find
8192 hardwired in C code, just to make code clearer.
--
Massimo Dal Zotto
can be generated in a buffer and then sent to the frontend in a single
libpq call. This solves problems with NOTICE and ERROR messages generated
in the middle of a data message or COPY OUT operation.
authentifica
tion
working with postgresql-6.4.2 and KTH-KRB Ebones
(http://www.pdc.kth.se/kth-kr
b) on a dec alpha running DU 4.0D using the native compiler. The
following
patch does the trick.
The rationale behind this is as follows. The KTH-KRB code header files
defines
lots of lengths like INST_SZ,REALM_SZ and KRB_SENDAUTH_VLEN. It also has
a
habit of doing things like
chararray[LENGTH] = '\0'
to ensure null terminated strings. In my instance this just happens to
blat
the kerberos principal instance string leading to error like
pg_krb4_recvauth: kerberos error: Can't decode authenticator
(krb_rd_req
)
The application code that comes with KTH-KRB uses "KRB_SENDAUTH_VLEN +
1" and
sometimes uses "INST_SZ + 1" so it seems safest to put that 1 char
buffer in
the appropriate place.
Rodney McDuff
so that fetching an attribute value needs only one SearchSysCacheTuple call
instead of two redundant searches. This speeds up a large SELECT by about
ten percent, and probably will help GROUP BY and SELECT DISTINCT too.
seem to be portable (HPUX doesn't like it, anyway). Also, clean up
StreamConnection(), which was mis-coded to assume that the address
family field is already set when it's called.
Here's another patch for the libpq backend areas. This patch removes all
usage of "FILE *" on the communications channel. It also cleans up the
comments and headers in the pqcomm.c file - a lot of things were either
missing or incorrect. Finally, it removes a couple of unused functions
(leftovers from the time of shared code between the libpq backend and
frontend).
Here is a first patch to cleanup the backend side of libpq.
This patch removes all external dependencies on the "Pfin" and "Pfout" that
are declared in pqcomm.h. These variables are also changed to "static" to
make sure.
Almost all the change is in the handler of the "copy" command - most other
areas of the backend already used the correct functions.
This change will make the way for cleanup of the internal stuff there - now
that all the functions accessing the file descriptors are confined to a
single directory.
where you state a format and arguments. the old behavior required
each appendStringInfo to have to have a sprintf() before it if any
formatting was required.
Also shortened several instances where there were multiple appendStringInfo()
calls in a row, doing nothing more then adding one more word to the String,
instead of doing them all in one call.
fail to consume the rest of the input string, and worse it would write
one more byte than it should into the buffer, probably resulting in coredump.
Fortunately there's a correct implementation next door in pqcomprim.c.
Here are patches needed to complie under AIX 4.2.
I changed configure.in, pqcomm.c, config.h.in, and fe-connect.c.
Also I had to install flex because lex did not want to translate pgc.l.
and make backend/libpq/pqcomm.c only try to lock the socket file when
the call exists. Also, change open-RDONLY to open-WRONLY; at least
on my platform, you can't get a write lock on a file you didn't open
for writing.
> socket-flock.patch
>
> use advisory locks to check if the unix socket can be deleted.
> A running postmaster keeps a lock on that file. A starting
> postmaster exits if the file exists and is locked, otherwise
> it deletes the sockets and proceeds.
> This avoid the need to remove manually the file after a postmaster
> or system crash.
> I don't know if flock is available on any system. If not we could
> define a HAVE_FLOCK set by configure.
patch is applied:
Rewrite rules on relation level work fine now.
Event qualifications on insert/update/delete rules work
fine now.
I added the new keyword OLD to reference the CURRENT
tuple. CURRENT will be removed in 6.5.
Update rules can reference NEW and OLD in the rule
qualification and the actions.
Insert/update/delete rules on views can be established to
let them behave like real tables.
For insert/update/delete rules multiple actions are
supported now. The actions can also be surrounded by
parantheses to make psql happy. Multiple actions are
required if update to a view requires updates to multiple
tables.
Regular users are permitted to create/drop rules on
tables they have RULE permissions for
(DefineQueryRewrite() is now able to get around the
access restrictions on pg_rewrite). This enables view
creation for regular users too. This required an extra
boolean parameter to pg_parse_and_plan() that tells to
set skipAcl on all rangetable entries of the resulting
queries. There is a new function
pg_exec_query_acl_override() that could be used by
backend utilities to use this facility.
All rule actions (not only views) inherit the permissions
of the event relations owner. Sample: User A creates
tables T1 and T2, creates rules that log
INSERT/UPDATE/DELETE on T1 in T2 (like in the regression
tests for rules I created) and grants ALL but RULE on T1
to user B. User B can now fully access T1 and the
logging happens in T2. But user B cannot access T2 at
all, only the rule actions can. And due to missing RULE
permissions on T1, user B cannot disable logging.
Rules on the attribute level are disabled (they don't
work properly and since regular users are now permitted
to create rules I decided to disable them).
Rules on select must have exactly one action that is a
select (so select rules must be a view definition).
UPDATE NEW/OLD rules are disabled (still broken, but
triggers can do it).
There are two new system views (pg_rule and pg_view) that
show the definition of the rules or views so the db admin
can see what the users do. They use two new functions
pg_get_ruledef() and pg_get_viewdef() that are builtins.
The functions pg_get_ruledef() and pg_get_viewdef() could
be used to implement rule and view support in pg_dump.
PostgreSQL is now the only database system I know, that
has rewrite rules on the query level. All others (where I
found a rule statement at all) use stored database
procedures or the like (triggers as we call them) for
active rules (as some call them).
Future of the rule system:
The now disabled parts of the rule system (attribute
level, multiple actions on select and update new stuff)
require a complete new rewrite handler from scratch. The
old one is too badly wired up.
After 6.4 I'll start to work on a new rewrite handler,
that fully supports the attribute level rules, multiple
actions on select and update new. This will be available
for 6.5 so we get full rewrite rule capabilities.
Jan
no longer returns buffer pointer, can be gotten from scan;
descriptor; bootstrap can create multi-key indexes;
pg_procname index now is multi-key index; oidint2, oidint4, oidname
are gone (must be removed from regression tests); use System Cache
rather than sequential scan in many places; heap_modifytuple no
longer takes buffer parameter; remove unused buffer parameter in
a few other functions; oid8 is not index-able; remove some use of
single-character variable names; cleanup Buffer variables usage
and scan descriptor looping; cleaned up allocation and freeing of
tuples; 18k lines of diff;
As Bruce mentioned, this is due to the conflict among changes we made.
Included patches should fix the problem(I changed all MB to
MULTIBYTE). Please let me know if you have further problem.
P.S. I did not include pathces to configure and gram.c to save the
file size(configure.in and gram.y modified).
calls. Outside a transaction, the backend detects them as buffer
leaks; it sends a NOTICE, and frees them. This sometimes cause a
segmentation fault (at least on Linux). These indexes are initialized
on the first lo_read/lo_write/lo_tell call, and (normally) closed
on a lo_close call. Thus the buffer leaks appear when lo direct
access functions are used, and not with lo_import/lo_export functions
(libpq version calls lo_close before ending the command, and the
backend version uses another path).
The included patches (against recent snapshot, and against 6.3.2)
cause indexes to be closed on transaction end (that is on explicit
'END' statment, or on command termination outside trasaction blocks),
thus preventing the buffer leaks while increasing performance inside
transactions. Some (all?) 'classic' memory leaks are also removed.
I hope it will be ok.
--- Pascal ANDRE, graduated from Ecole Centrale Paris andre@via.ecp.fr
Making PQrequestCancel safe to call in a signal handler turned out to be
much easier than I feared. So here are the diffs.
Some notes:
* I modified the postmaster's packet "iodone" callback interface to allow
the callback routine to return a continue-or-drop-connection return
code; this was necessary to allow the connection to be closed after
receiving a Cancel, rather than proceeding to launch a new backend...
Being a neatnik, I also made the iodone proc have a typechecked
parameter list.
* I deleted all code I could find that had to do with OOB.
* I made some edits to ensure that all signals mentioned in the code
are referred to symbolically not by numbers ("SIGUSR2" not "2").
I think Bruce may have already done at least some of the same edits;
I hope that merging these patches is not too painful.
I have implemented a framework of encoding translation between the
backend and the frontend. Also I have added a new variable setting
command:
SET CLIENT_ENCODING TO 'encoding';
Other features include:
Latin1 support more 8 bit cleaness
See doc/README.mb for more details. Note that the pacthes are
against May 30 snapshot.
Tatsuo Ishii
have > 20000 users and each (potentially) needs a separate database
which is > only accessible to them. Rather than having 20000 lines
in pg_hba.conf, > I've patched Postgres so that the special token
"sameuser" in the > database field of pg_hba.conf allows access
only to the username which > is connecting.
Attached you'll find a (big) patch that fixes make dep and make
depend in all Makefiles where I found it to be appropriate.
It also removes the dependency in Makefile.global for NAMEDATALEN
and OIDNAMELEN by making backend/catalog/genbki.sh and bin/initdb/initdb.sh
a little smarter.
This no longer requires initdb.sh that is turned into initdb with
a sed script when installing Postgres, hence initdb.sh should be
renamed to initdb (after the patch has been applied :-) )
This patch is against the 6.3 sources, as it took a while to
complete.
Please review and apply,
Cheers,
Jeroen van Vianen
yyerror ones from bison. It also includes a few 'enhancements' to
the C programming style (which are, of course, personal).
The other patch removes the compilation of backend/lib/qsort.c, as
qsort() is a standard function in stdlib.h and can be used any
where else (and it is). It was only used in
backend/optimizer/geqo/geqo_pool.c, backend/optimizer/path/predmig.c,
and backend/storage/page/bufpage.c
> > Some or all of these changes might not be appropriate for v6.3,
since we > > are in beta testing and since they do not affect the
current functionality. > > For those cases, how about submitting
patches based on the final v6.3 > > release?
There's more to come. Please review these patches. I ran the
regression tests and they only failed where this was expected
(random, geo, etc).
Cheers,
Jeroen
a while back I posted a patch for pg_ident, the patch worked but I didn't
diagnose the problem properly.
on my compiler(gcc2.7.2) this compiles with no errors...
char buf[1000]; if(buf != '\0') {
...but it doesn't compare '\0' with the first char of buf.
6.3 postmaster is supposed to work with pre 6.3 protocol. This is true
for little endian architecture servers. But for big endian machines
such as Sparc the backward compatibility function do not work.
Attached are patches to fix the problem.
seems that my last post didn't make it through. That's good
since the diff itself didn't covered the renaming of
pg_user.h to pg_shadow.h and it's new content.
Here it's again. The complete regression test passwd with
only some float diffs. createuser and destroyuser work.
pg_shadow cannot be read by ordinary user.
What it does:
It solves stupid problem with cyrillic charsets IP-based on-fly recoding.
take a look at /data/charset.conf for details.
You can use any tables for any charset.
Tables are from Russian Apache project.
Tables in this patch contains also Ukrainian characters.
Then run ./configure --enable-recode
I haven't had final confirmation from Peter yet, but the attached patch
needs to be applied for the Beta otherwise password and crypt
authentication just won't work.
It puts back the loop in libpq and also fixes a couple of problems with
maintaining compatability with pre-6.3 drivers.
Attached is the patch to fix the warning messages from my code. I also
fixed one which wasn't my code. Apart from the usual warnings about the
bison/yacc generated code I only have one other warning message. This
is in gramm.y around line 2234. I wasn't sure of the fix.
I've also replaced all the calls to free() in gramm.y to calls to
pfree(). Without these I was getting backend crashes with GRANT. This
might already have been fixed.
I've completed the patch to fix the protocol and authentication issues I
was discussing a couple of weeks ago. The particular changes are:
- the protocol has a version number
- network byte order is used throughout
- the pg_hba.conf file is used to specify what method is used to
authenticate a frontend (either password, ident, trust, reject, krb4
or krb5)
- support for multiplexed backends is removed
- appropriate changes to man pages
- the -a switch to many programs to specify an authentication service
no longer has any effect
- the libpq.so version number has changed to 1.1
The new backend still supports the old protocol so old interfaces won't
break.
o A new patch that contains the following changes:
-- The pg_pwd file is now cached in the postmaster's memory.
-- pg_pwd is reloaded when the postmaster detects a flag file creat()'ed
by a backend.
-- qsort() is used to sort loaded password entries, and bsearch() is
is used to find entries in the pg_pwd cache.
-- backends now copy the pg_user relation to pg_pwd.pid, and then
rename the temp file to be pg_pwd.
-- The delimiter for pg_pwd has been changed to a tab character.
tree "non-PORTNAME" dependent. Technically, anything that is PORTNAME
dependent should be able to be derived at compile time, through configure
or through gcc
Bruce Momjian
Peter Eisentraut
Tom Lane
Tatsuo Ishii
Jan Wieck
Marc G. Fournier
Vadim B. Mikheev
Thomas G. Lockhart