files rather than a header file where they belong. Pay some modicum
of attention to picking global routine names that aren't likely to
conflict with surrounding applications.
I am no longer pursuing a total non-blocking implementation. I haven't
found a good way to test it with the type of work that I do with
PostgreSQL. I do use blocking SSL sockets with this mod and have had no
problem whatsoever. The bug that I fixed in this patch is exceptionally
hard to reproduce reliably.
Jack Bates
file.
The program seems to compile ok, but when linking a program that uses
the call,
g++ chokes with an undefined reference error.
If you know how this problem might be fixed, list the solution below:
---------------------------------------------------------------------
I include the code:
Oid PgLargeObject::LOid(){
return pgObject;
}
in the .cc file.
Chris Traylor
PQExec(" ") in the wrapper around PQnotifies(), fix the Makefile for
the examples so that they will actually compile properly (with the
exception of #5, which depends on internal headers), make a minor change
to libpq++.h so that "make examples" now works on my machine, update
some documentation, fix some grammatical problems, and remove some of
the more hideous comments.
Neil Conway
Remove ODBC-compatible empty parentheses from calls to SQL99 functions
for which these parentheses do not match the standard.
Update the ODBC driver to ensure compatibility with the ODBC standard
for these functions (e.g. CURRENT_TIMESTAMP, CURRENT_USER, etc).
Include a new appendix in the User's Guide which lists the labeled features
for SQL99 (the labeled features replaced the "basic", "intermediate",
and "advanced" categories from SQL92). features.sgml does not yet split
this list into "supported" and "unsupported" lists.
on the server, if DebugLvl >= 2.
The patch also includes a late addition to the last patch
(X509_check_private_key()). I'm not sure why it the currect
revision wasn't tagged.
Bear Giles
If the user has certificates in $HOME/.postgresql/postgresql.crt
and $HOME/.postgresql/postgresql.key exist, they are provided
to the server. The certificate used to sign this cert must be
known to the server, in $DataDir/root.crt. If successful, the
cert's "common name" is logged.
Client certs are not used for authentication, but they could be
via the port->peer (X509 *), port->peer_dn (char *) or
port->peer_cn (char *) fields. Or any other function could be
used, e.g., many sites like the issuer + serial number hash.
Bear Giles
As the comment headers in be-secure.c discusses, EPH preserves
confidentiality even if the static private key (which is usually
kept unencrypted) is compromised.
Because of the value of this, common default values are hard-coded
to protect the confidentiality of the data even if an attacker
successfully deletes or modifies the external file.
Bear Giles
Attached are a revised set of SSL patches. Many of these patches
are motivated by security concerns, it's not just bug fixes. The key
differences (from stock 7.2.1) are:
*) almost all code that directly uses the OpenSSL library is in two
new files,
src/interfaces/libpq/fe-ssl.c
src/backend/postmaster/be-ssl.c
in the long run, it would be nice to merge these two files.
*) the legacy code to read and write network data have been
encapsulated into read_SSL() and write_SSL(). These functions
should probably be renamed - they handle both SSL and non-SSL
cases.
the remaining code should eliminate the problems identified
earlier, albeit not very cleanly.
*) both front- and back-ends will send a SSL shutdown via the
new close_SSL() function. This is necessary for sessions to
work properly.
(Sessions are not yet fully supported, but by cleanly closing
the SSL connection instead of just sending a TCP FIN packet
other SSL tools will be much happier.)
*) The client certificate and key are now expected in a subdirectory
of the user's home directory. Specifically,
- the directory .postgresql must be owned by the user, and
allow no access by 'group' or 'other.'
- the file .postgresql/postgresql.crt must be a regular file
owned by the user.
- the file .postgresql/postgresql.key must be a regular file
owned by the user, and allow no access by 'group' or 'other'.
At the current time encrypted private keys are not supported.
There should also be a way to support multiple client certs/keys.
*) the front-end performs minimal validation of the back-end cert.
Self-signed certs are permitted, but the common name *must*
match the hostname used by the front-end. (The cert itself
should always use a fully qualified domain name (FDQN) in its
common name field.)
This means that
psql -h eris db
will fail, but
psql -h eris.example.com db
will succeed. At the current time this must be an exact match;
future patches may support any FQDN that resolves to the address
returned by getpeername(2).
Another common "problem" is expiring certs. For now, it may be
a good idea to use a very-long-lived self-signed cert.
As a compile-time option, the front-end can specify a file
containing valid root certificates, but it is not yet required.
*) the back-end performs minimal validation of the client cert.
It allows self-signed certs. It checks for expiration. It
supports a compile-time option specifying a file containing
valid root certificates.
*) both front- and back-ends default to TLSv1, not SSLv3/SSLv2.
*) both front- and back-ends support DSA keys. DSA keys are
moderately more expensive on startup, but many people consider
them preferable than RSA keys. (E.g., SSH2 prefers DSA keys.)
*) if /dev/urandom exists, both client and server will read 16k
of randomization data from it.
*) the server can read empheral DH parameters from the files
$DataDir/dh512.pem
$DataDir/dh1024.pem
$DataDir/dh2048.pem
$DataDir/dh4096.pem
if none are provided, the server will default to hardcoded
parameter files provided by the OpenSSL project.
Remaining tasks:
*) the select() clauses need to be revisited - the SSL abstraction
layer may need to absorb more of the current code to avoid rare
deadlock conditions. This also touches on a true solution to
the pg_eof() problem.
*) the SIGPIPE signal handler may need to be revisited.
*) support encrypted private keys.
*) sessions are not yet fully supported. (SSL sessions can span
multiple "connections," and allow the client and server to avoid
costly renegotiations.)
*) makecert - a script that creates back-end certs.
*) pgkeygen - a tool that creates front-end certs.
*) the whole protocol issue, SASL, etc.
*) certs are fully validated - valid root certs must be available.
This is a hassle, but it means that you *can* trust the identity
of the server.
*) the client library can handle hardcoded root certificates, to
avoid the need to copy these files.
*) host name of server cert must resolve to IP address, or be a
recognized alias. This is more liberal than the previous
iteration.
*) the number of bytes transferred is tracked, and the session
key is periodically renegotiated.
*) basic cert generation scripts (mkcert.sh, pgkeygen.sh). The
configuration files have reasonable defaults for each type
of use.
Bear Giles
are motivated by security concerns, it's not just bug fixes. The key
differences (from stock 7.2.1) are:
*) almost all code that directly uses the OpenSSL library is in two
new files,
src/interfaces/libpq/fe-ssl.c
src/backend/postmaster/be-ssl.c
in the long run, it would be nice to merge these two files.
*) the legacy code to read and write network data have been
encapsulated into read_SSL() and write_SSL(). These functions
should probably be renamed - they handle both SSL and non-SSL
cases.
the remaining code should eliminate the problems identified
earlier, albeit not very cleanly.
*) both front- and back-ends will send a SSL shutdown via the
new close_SSL() function. This is necessary for sessions to
work properly.
(Sessions are not yet fully supported, but by cleanly closing
the SSL connection instead of just sending a TCP FIN packet
other SSL tools will be much happier.)
*) The client certificate and key are now expected in a subdirectory
of the user's home directory. Specifically,
- the directory .postgresql must be owned by the user, and
allow no access by 'group' or 'other.'
- the file .postgresql/postgresql.crt must be a regular file
owned by the user.
- the file .postgresql/postgresql.key must be a regular file
owned by the user, and allow no access by 'group' or 'other'.
At the current time encrypted private keys are not supported.
There should also be a way to support multiple client certs/keys.
*) the front-end performs minimal validation of the back-end cert.
Self-signed certs are permitted, but the common name *must*
match the hostname used by the front-end. (The cert itself
should always use a fully qualified domain name (FDQN) in its
common name field.)
This means that
psql -h eris db
will fail, but
psql -h eris.example.com db
will succeed. At the current time this must be an exact match;
future patches may support any FQDN that resolves to the address
returned by getpeername(2).
Another common "problem" is expiring certs. For now, it may be
a good idea to use a very-long-lived self-signed cert.
As a compile-time option, the front-end can specify a file
containing valid root certificates, but it is not yet required.
*) the back-end performs minimal validation of the client cert.
It allows self-signed certs. It checks for expiration. It
supports a compile-time option specifying a file containing
valid root certificates.
*) both front- and back-ends default to TLSv1, not SSLv3/SSLv2.
*) both front- and back-ends support DSA keys. DSA keys are
moderately more expensive on startup, but many people consider
them preferable than RSA keys. (E.g., SSH2 prefers DSA keys.)
*) if /dev/urandom exists, both client and server will read 16k
of randomization data from it.
*) the server can read empheral DH parameters from the files
$DataDir/dh512.pem
$DataDir/dh1024.pem
$DataDir/dh2048.pem
$DataDir/dh4096.pem
if none are provided, the server will default to hardcoded
parameter files provided by the OpenSSL project.
Remaining tasks:
*) the select() clauses need to be revisited - the SSL abstraction
layer may need to absorb more of the current code to avoid rare
deadlock conditions. This also touches on a true solution to
the pg_eof() problem.
*) the SIGPIPE signal handler may need to be revisited.
*) support encrypted private keys.
*) sessions are not yet fully supported. (SSL sessions can span
multiple "connections," and allow the client and server to avoid
costly renegotiations.)
*) makecert - a script that creates back-end certs.
*) pgkeygen - a tool that creates front-end certs.
*) the whole protocol issue, SASL, etc.
*) certs are fully validated - valid root certs must be available.
This is a hassle, but it means that you *can* trust the identity
of the server.
*) the client library can handle hardcoded root certificates, to
avoid the need to copy these files.
*) host name of server cert must resolve to IP address, or be a
recognized alias. This is more liberal than the previous
iteration.
*) the number of bytes transferred is tracked, and the session
key is periodically renegotiated.
*) basic cert generation scripts (mkcert.sh, pgkeygen.sh). The
configuration files have reasonable defaults for each type
of use.
Bear Giles
> Changes to avoid collisions with WIN32 & MFC names...
> 1. Renamed:
> a. PROC => PGPROC
> b. GetUserName() => GetUserNameFromId()
> c. GetCurrentTime() => GetCurrentDateTime()
> d. IGNORE => IGNORE_DTF in include/utils/datetime.h & utils/adt/datetim
>
> 2. Added _P to some lex/yacc tokens:
> CONST, CHAR, DELETE, FLOAT, GROUP, IN, OUT
Jan
Allows you to set the loglevel at runtime by adding ?loglevel=X to the connection URL, where 1 = INFO and 2 = DEBUG.
Automatically turns on logging by calling DriverManager.setPrintWriter(new PrintWriter(System.out)) if one is not already set.
Adds a Driver.info() message that prints out the version number
Adds member variables logDebug and logInfo that can be checked before making logging methods calls
Adds a build number to the version number string. This build number will need to be manually incremented when we see fit.
----------------------------------------------------------------------
Modified Files:
org/postgresql/Connection.java org/postgresql/Driver.java.in
org/postgresql/fastpath/Fastpath.java
org/postgresql/jdbc1/DatabaseMetaData.java
org/postgresql/jdbc2/Connection.java
org/postgresql/jdbc2/DatabaseMetaData.java
org/postgresql/largeobject/LargeObjectManager.java
org/postgresql/util/PSQLException.java
org/postgresql/util/Serialize.java
----------------------------------------------------------------------
with the Cursor object's fetchmany() method. The API and
inline documentation state that the default is 1. It
currently defaults to 5.
Patrick Macdonald
2) Supprt ARD precision/scale and SQL_C_NUEMRIC.
3) Minimal implementation of SQLGetDiagField().
4) SQLRowCount() reports the result of SQLSetPos and SQLBulkOperation.
5) int8 -> SQL_NUMERIC for Microsoft Jet.
6) Support isolation level change.
7) ODBC3.0 SQLSTATE code.
8) Append mode log files.
GUC support. It's now possible to set datestyle, timezone, and
client_encoding from postgresql.conf and per-database or per-user
settings. Also, implement rollback of SET commands that occur in a
transaction that later fails. Create a SET LOCAL var = value syntax
that sets the variable only for the duration of the current transaction.
All per previous discussions in pghackers.
by Marcelo Aceto <aceto@newinf.com.br> .
1) Wrong translations of embedded escape sequences inside outer join escape
sequences.
2) Wrong translation of parameter markers inside outer joins and function
escape sequences.
3) Bad concatenation of date, time, timestamp constants with next word in
statement:
work on all win9x machines, so i made it go thru a l ookup table
instead, using the DLL as last resort. I also moved this out of the
fe-misc.c file because of the size of the lookup ta ble. Who knows, we
might add more other win32 specific code there in the future.
I also fixed a small typo in the pg_config.h.win32 that made the
compiler compla in about the gnu snprintf declaration.
I tried to make this patch with psql coding style. I've successfully
tested this on win2k and win98 and it works fine (i.e. the mes sage
shows on win98 too, it didn't with the old implementation).
Magnus Naeslund
Slackware 8), and perhaps on other Pythons, haven't checked. Something in
the _pg.connect() call isn't working. I think the problem stems from the
fact that 'host' is a named parameter of both _pg.connect and pgdb.connect,
and so Python treats it as a variable assignment, not a named parameter.
Uses non-named parameters.
Andrew Johnson
In summary, if a software writer implements timer events or other events
which generate a signal with a timing fast enough to occur while libpq
is inside connect(), then connect returns -EINTR. The code following
the connect call does not handle this and generates an error message.
The sum result is that the pg_connect() fails. If the timer or other
event is right on the window of the connect() completion time, the
pg_connect() may appear to work sporadically. If the event is too slow,
pg_connect() will appear to always work and if the event is too fast,
pg_connect() will always fail.
David Ford
entries, per pghackers discussion. This fixes aggregates to live in
namespaces, and also simplifies/speeds up lookup in parse_func.c.
Also, add a 'proimplicit' flag to pg_proc that controls whether a type
coercion function may be invoked implicitly, or only explicitly. The
current settings of these flags are more permissive than I would like,
but we will need to debate and refine the behavior; for now, I avoided
breaking regression tests as much as I could.
This is necessary for mulibyte character sequences.
See "[HACKERS] PQescapeBytea is not multibyte aware" thread posted around
2002/04/05 for more details.
file, which is not the actual end of the file. One side effect of that
is that if you are i n a ifdef block, you get a wrong error telling you
that a endif is missing.
This patch corrects pgc.l and also adds a test of this problem to
test1.pgc. To convince you apply the patch to test1.pgc first then try
to compile the test the n apply the patch to pgc.l.
The patch moves the test of the scope of an ifdef block to the end of
the file b eeing parsed, including all includes files, ... .
Nicolas Bazin
From: Bradley McLean <brad@bradm.net>
Patch against 7,2 submitted for comment.
This seems to work just fine; Now, when our users submit a 2 hour
query with four million row sorts by accident, then cancel it 30 seconds
later, it doesn't bog down the server ...
2) Implement some options for SQLGetDescField().
3) Handle *Inifinity* timestamp for SQL_C_CHAR type output.
4) Separate Unicode conversions from common implementations.
5) Improve internal parse_statement() function.
1) Prepare to separate 4 kinds of Descriptor handles.
2) Detect the transaction status more naturally.
3) Improve Parse Statement functionality for the use
of updatable cursors.
4) Improve updatable cursors.
5) Implement SQLGetDescField() and improve SQLColAttribute().
6) etc.
these versions adhere to the backend protocol better than previous version
fixes problem when an error occurs on the backend, and the connection is still used
previous versions were throwing an exception half way through the protocol, leaving it
indeterminate.
also removes empty query code, should speed things up a bit
* Introduces a new class, StartupPacket.
* Moves a lot of constants from Connection to StartupPacket.
* Makes two instance variables in Connection into locals.
>
> I am running Python 1.5.
Therein lies the problem... :)
Since it appears you have the requirement of supporting old python
versions, attached is just the pgdb.py part of the patch (with a fix for
DateTime handling). It has the same functionality but certainly won't be
quite as fast. Given the absence of _PyString_Join in python1.5, it's a
pain to get the C variants working for all versions. The pgdb.py patch
does leaves the hooks in, should someone wish to do the optimization at a
later point.
Elliot Lee
If one is trying to compile a JDBC 1 driver and junit.jar is in the
CLASSPATH, then the build fails as ant tries to build the JDBC 2 test
classes. This patch fixes this problem by excluding the jdbc 2 files
unless the jdk1.2+ property is set.
previously it was throwing a SQLException as soon as the error message was
received from the backend. This did not allow the protocol to finish properly
now, simply collects error messages from the backend until the query is done
and throws exception at the end
Also added setLogLevel to Driver.java, and made the log levels public
1) Put back the error message for SQLError().
2) Change Disallow premature to handle the SELECTed
result.
3) Put back the behavior of AUTUCOMMIT mode change.
4) Fix SQLColumns for ODBC3.0.
5) Improve the handling of variable bookmark in ODBC3.0.
6) Enable Recognize Unique Index Button.
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
parts o f postgresql. The jdbc drivers are never compiled with debugging
support. This p atch make sure that debugging information is added to
the jdbc jar when the --en able-debug is added. This was usefull for me
for debugging some java jdbc poolin g objects but this might perhaps be
usefull for other people too?
Dries Verachtert
> > > > It was made to cope with encoding such as an Asian bloc in 7.2Beta2.
> > > >
> > > > Added ServerEncoding
> > > > Korean (JOHAB), Thai (WIN874),
> > > > Vietnamese (TCVN), Arabic (WIN1256)
> > > >
> > > > Added ClientEncoding
> > > > Simplified Chinese (GBK), Korean (UHC)
> > > >
> > > >
> > > >
> http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2b2.newencoding.diff.tar.gz
> > > > (608K)
> > >
> > > Looks good. I need some people to review this for me.
> >
> > For me they look good too. The only missing part is a
> > documentation. I will ask him to write it up. If he couldn't, I will
> > do it for him.
> > > The diff is 3mb
> > > but appears to address only additions to multibyte. I have attached a
> > > list of files it modifies. Also, look at the sizes of the mb/
> > > directory. It is getting large:
> > >
> > > 4 ./CVS
> > > 6 ./Unicode/CVS
> > > 3433 ./Unicode
> > > 6197 .
> >
> > Yes. We definitely need the on-the-fly encoding addition capability:
> > i.e. CREATE CHRACTER SET in the future...
> > --
> > Tatsuo Ishii
> >
> >
Address chainge.
http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2.newencoding.diff.gz
Add PsqlODBC and document ...etc patch.
Eiji Tokuya
queries over non-blocking connections with libpq. "Larger" here
basically means that it doesn't fit into the output buffer.
The basic strategy is to fix pqFlush and pqPutBytes.
The problem with pqFlush as it stands now is that it returns EOF when an
error occurs or when not all data could be sent. The latter case is
clearly not an error for a non-blocking connection but the caller can't
distringuish it from an error very well.
The first part of the fix is therefore to fix pqFlush. This is done by
to renaming it to pqSendSome which only differs from pqFlush in its
return values to allow the caller to make the above distinction and a
new pqFlush which is implemented in terms of pqSendSome and behaves
exactly like the old pqFlush.
The second part of the fix modifies pqPutBytes to use pqSendSome instead
of pqFlush and to either send all the data or if not all data can be
sent on a non-blocking connection to at least put all data into the
output buffer, enlarging it if necessary. The callers of pqPutBytes
don't have to be changed because from their point of view pqPutBytes
behaves like before. It either succeeds in queueing all output data or
fails with an error.
I've also added a new API function PQsendSome which analogously to
PQflush just calls pqSendSome. Programs using non-blocking queries
should use this new function. The main difference is that this function
will have to be called repeatedly (calling select() properly in between)
until all data has been written.
AFAICT, the code in CVS HEAD hasn't changed with respect to non-blocking
queries and this fix should work there, too, but I haven't tested that
yet.
Bernhard Herzog
An attached patch corrects problem of this bug and fractional second.
The handling of time zone was as follows:
(a) with time zone
using SimpleDateFormat("yyyy-MM-dd HH:mm:ss z")
(b) without time zone
using SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
About problem of fractional second,
Fractional second was changed from milli-second to nano-second
Elliot Lee wrote:
> This patch to the python bindings adds C versions of the often-used
query
> args quoting routines, as well as support for quoting lists e.g.
> dbc.execute("SELECT * FROM foo WHERE blah IN %s", ([1,2,3],))
getColumnClassName(int) is not implemented. This will futher fixes method
ResultSet.getObject(int) since it requires the getColumnClassName(int) method to return the proper java class used to map the database column.
auther Ed Yu
1) Handle parameter array.
2) Allow re-use of the connection handle after SQLDisconnect.
3) Reject NULL if no indicator specified.
4) Improve the handling of '_' in table name.
5) Unify internal begin/commit/abort operations.
6) Change SQLTables() to return null not "" for the
table_owner.
7) Fix a bug about parameter handling reported by Benoit Menendez.
8) Add cast in handling ODBC date/time escape sequences.
9) Fix a bug about cache_size handing in declare/fetch mode.
[ODBC3.0 related]
10) Improve the handling of descriptor handles(ODBC3.0).
11) Improve the type handling of some types for ODBC3.0.
[Thanks to Marcelo Aceto for his useful patches]
12) Allow nested ODBC escape.
13) Allow changing autocommit on/off inside the transaction
block.
14) Improve the handling of ODBC scalar functions.
compile in client apps that use the standard installed header set.
To allow removing that include, move DLLIMPORT definitions out of c.h
and into the appropriate port-specific header files.
a get on a bytea value the code was running the raw value from the server
through character set conversion, which if the character set was SQL_ASCII
would cause all 8bit characters to become ?'s.
wasn't updated to handle more than two decimal digits for fractional seconds
that now are possible in 7.2. This patch fixes the timestamp parsing logic.
I have built and tested on both jdbc1 and jdbc2.
to be used, causing extra sql statements to be executed. This was a
significant performance problem with the database meta data classes.
The fix is a simple one liner.
- Fixed bug that caused segfault when given incorrect DB name.
- Fixed bug in ecpglib causing indicator to list the size of the
variable instead of the size of the data.
the latest version and I wanted to make sure that there was a clean release.
I also change the build files as I discussed in my letter of Nov 6, 2001. At
the time I was asked to hold off until after the release.
Magnus Hagander that DLL only contains error strings for the Net***
functions, *not* WinSock. We need to look for a workable solution for
older Windows flavors ... but it won't happen for PG 7.2.
The bug was that any insert or update would fail if the returned oid was
larger than a signed int. Since OIDs are unsigned int's it was
a bug that the code used a java signed int to deal with the values. The bug
would result in the error message: "Unable to fathom update count".
While fixing the bug, it became apparent that other code made a similar
assumption about OIDs being signed ints. Therefore some methods that returned
or took OIDs are arguements also needed to be changed.
Since we are so close to the 7.2 release I have added new methods that
return longs and deprecated the old methods returning ints. Therefore all
old code should still work without requiring a code change to cast from long to int. Also note that the methods below are PostgreSQL specific extensions to
the JDBC api are are not part of the spec from Sun, thus it is unlikely that
they are used much or at all.
The deprecated methods are:
ResultSet.getInsertedOID()
Statement.getInsertedOID()
Serialize.store()
Connection.putObject()
and are replaced by:
ResultSet.getLastOID()
Statement.getLastOID()
Serialize.storeObject()
Connection.storeObject()
All the deprecated methods returned int, while their replacements return long
This patch also fixes two comments in MD5Digest that the author Jeremy Wohl
submitted.
--Barry
There's also a little fix for the getRow() method. While fixing
absolute(), I noticed that getRow() wasn't quite following the spec: it
wasn't returning 0 when the ResultSet wasn't positioned on a row. I've
started a ResultSet test case and included it as well.
Liam Stewart
Bruce Momjian
Michael Meskes
Dave Cramer
Tom Lane
Thomas G. Lockhart
Hiroshi Inoue
Jan Wieck
Barry Lind
Peter Eisentraut
Tatsuo Ishii
D'Arcy J.M. Cain