rewritten and the protocol is changed, but most elog calls are still
elog calls. Also, we need to contemplate mechanisms for controlling
all this functionality --- eg, how much stuff should appear in the
postmaster log? And what API should libpq expose for it?
have length words. COPY OUT reimplemented per new protocol: it doesn't
need \. anymore, thank goodness. COPY BINARY to/from frontend works,
at least as far as the backend is concerned --- libpq's PQgetline API
is not up to snuff, and will have to be replaced with something that is
null-safe. libpq uses message length words for performance improvement
(no cycles wasted rescanning long messages), but not yet for error
recovery.
value '-2' is used to indicate a variable-width type whose width is
computed as strlen(datum)+1. Everything that looks at typlen is updated
except for array support, which Joe Conway is working on; at the moment
it wouldn't work to try to create an array of cstring.
This is necessary for mulibyte character sequences.
See "[HACKERS] PQescapeBytea is not multibyte aware" thread posted around
2002/04/05 for more details.
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
queries over non-blocking connections with libpq. "Larger" here
basically means that it doesn't fit into the output buffer.
The basic strategy is to fix pqFlush and pqPutBytes.
The problem with pqFlush as it stands now is that it returns EOF when an
error occurs or when not all data could be sent. The latter case is
clearly not an error for a non-blocking connection but the caller can't
distringuish it from an error very well.
The first part of the fix is therefore to fix pqFlush. This is done by
to renaming it to pqSendSome which only differs from pqFlush in its
return values to allow the caller to make the above distinction and a
new pqFlush which is implemented in terms of pqSendSome and behaves
exactly like the old pqFlush.
The second part of the fix modifies pqPutBytes to use pqSendSome instead
of pqFlush and to either send all the data or if not all data can be
sent on a non-blocking connection to at least put all data into the
output buffer, enlarging it if necessary. The callers of pqPutBytes
don't have to be changed because from their point of view pqPutBytes
behaves like before. It either succeeds in queueing all output data or
fails with an error.
I've also added a new API function PQsendSome which analogously to
PQflush just calls pqSendSome. Programs using non-blocking queries
should use this new function. The main difference is that this function
will have to be called repeatedly (calling select() properly in between)
until all data has been written.
AFAICT, the code in CVS HEAD hasn't changed with respect to non-blocking
queries and this fix should work there, too, but I haven't tested that
yet.
Bernhard Herzog
>
> 1. Now outputs '\\' instead of '\134' when using encode(bytea, 'escape')
> Note that I ended up leaving \0 as \000 so that there are no ambiguities
> when decoding something like, for example, \0123.
>
> 2. Fixed bug in byteain which allowed input values which were not valid
> octals (e.g. \789), to be parsed as if they were octals.
>
> Joe
>
Here's rev 2 of the bytea string support patch. Changes:
1. Added missing declaration for MatchBytea function
2. Added PQescapeBytea to fe-exec.c
3. Applies cleanly on cvs tip from this afternoon
I'm hoping that someone can review/approve/apply this before beta starts, so
I guess I'd vote (not that it counts for much) to delay beta a few days :-)
Joe Conway
> null bytes to be literally '\0', the following can happen:
> 1. User inputs string value as "<null byte>##" where ## are digits in the
> range of 0 to 7.
> 2. PQescapeString converts this to "\0##"
> 3. Escaped string is used in a context that causes "\0##" to be evaluated as
> an octal escape sequence.
I agree that this is a problem, though it is not possible to do
anything harmful with it. In addition, it only occurs if there are
any NUL characters in its input, which is very unlikely if you are
using C strings.
The patch below addresses the issue by removing escaping of \0
characters entirely.
> If the goal is to "safely" encode null bytes, and preserve the rest of the
> string as it was entered, I think the null bytes should be escaped as \\000
> (note that if you simply use \000 the same string truncation problem
> occurs).
We can't do that, this would require 4n + 1 bytes of storage for the
result, breaking the interface.
Florian Weimer
discussion on pgsql-hackers (especially the frightening memory dump in
<12273.999562219@sss.pgh.pa.us>), we decided that it is best not to
use identifiers from an untrusted source at all. Therefore, all
claims of the suitability of PQescapeString() for identifiers have
been removed.
Florian Weimer
two additional files win32.mak and libpgtcl.def.
This patch allows to compile libpgtcl.dll on Windows
with tcl > 8.0. I've tested it on WinNT (VC6.0), SUSE Linux (7.0)
and Solaris 2.6 with tcl 8.3.3.
Mikhail Terekhov
> It seems that win9x doesn't have the "netmsg.dll" so it defaults to "normal"
> FormatMessage.
> I wonder if one could load wsock32.dll or winsock.dll on those systems
> instead of netmsg.dll.
>
> Mikhail, could you please test this code on your nt4 system?
> Could someone else test this code on a win98/95 system?
>
> It works on win2k over here.
It works on win2k here too but not on win98/95 or winNT.
Anyway, attached is the patch which uses Magnus's my_sock_strerror
function (renamed to winsock_strerror). The only difference is that
I put the code to load and unload netmsg.dll in the libpqdll.c
(is this OK Magnus?).
Mikhail Terekhov
Allow pg_shadow to be MD5 encrypted.
Add ENCRYPTED/UNENCRYPTED option to CREATE/ALTER user.
Add password_encryption postgresql.conf option.
Update wire protocol version to 2.1.
functions do not set errno, so some normal conditions are treated as
fatal errors. e.g. fetching large tuples fails, as at some point recv()
returns EWOULDBLOCK. here's a patch, which replaces errno with
WSAGetLastError(). i've tried to to affect non-win32 code.
Dmitry Yurtaev
are now separate files "postgres.h" and "postgres_fe.h", which are meant
to be the primary include files for backend .c files and frontend .c files
respectively. By default, only include files meant for frontend use are
installed into the installation include directory. There is a new make
target 'make install-all-headers' that adds the whole content of the
src/include tree to the installed fileset, for use by people who want to
develop server-side code without keeping the complete source tree on hand.
Cleaned up a whole lot of crufty and inconsistent header inclusions.
we'll get there one day.
Use `cat' to create aclocal.m4, not `aclocal'. Some people don't
have automake installed.
Only run the autoconf rule in the top-level GNUmakefile if the
invoker specified `make configure', don't run it automatically
because of CVS timestamp skew.
array. This allows processing of conninfo strings to be made thread-safe,
at the cost of a small memory leak in applications that use
PQconndefaults() and are not updated to free the returned array via
the new PQconninfoFree() function. But PQconndefaults() is probably not
used very much, so this seems like a good compromise.
thinks the connection is idle, the error message is displayed as if
it were a NOTICE. This seems better than dropping the message on
the floor ... particularly if the message is the backend telling us
why it's about to close the connection. The previous behavior was
Backend message type 0x45 arrived while idle
pqReadData() -- backend closed the channel unexpectedly.
which is not real helpful.
* Add use of 'const' for varibles in source tree
(which is misspelled, btw.)
I went through the front-end libpq code and did so. This affects in
particular the various accessor functions (such as PQdb() and
PQgetvalue()) as well as, by necessity, the internal helpers they use.
I have been really thorough in that regard, perhaps some people will find
it annoying that things like
char * foo = PQgetvalue(res, 0, 0)
will generate a warning. On the other hand it _should_ generate one. This
is no real compatibility break, although a few clients will have to be
fixed to suppress warnings. (Which again would be in the spirit of the
above TODO.)
In addition I replaced some int's by size_t's and removed some warnings
(and generated some new ones -- grmpf!). Also I rewrote PQoidStatus (so it
actually honors the const!) and supplied a new function PQoidValue that
returns a proper Oid type. This is only front-end stuff, none of the
communicaton stuff was touched.
The psql patch also adds some new consts to honor the new libpq situation,
as well as fixes a fatal condition that resulted when using the -V
(--version) option and there is no database listening.
So, to summarize, the psql you should definitely put in (with or without
the libpq). If you think I went too far with the const-mania in libpq, let
me know and I'll make adjustments. If you approve it, I will also update
the docs.
-Peter
--
Peter Eisentraut Sernanders vaeg 10:115
error/notice message lengths, and number of fields per tuple. Add
pqexpbuffer.c/.h, a frontend version of backend's stringinfo module.
This is first step in applying Mike Ansley's long-query patches,
even though he didn't do any of these particular changes...
do the right thing: look for a NOTICE message from the backend before we
close our side of the socket. 6.4 libpq did not reliably print the backend's
hara-kiri message, 'The Postmaster has informed me ...', because it only
did the right thing if connection closure was detected during a read
attempt instead of a write attempt.
would be a Bad Thing.
For what it's worth, I found another case in libpq where you can get a T
message without a D that my utility patch needs to handle. I have
attached
the updated patch against the 6.4.2 version of
src/interfaces/libpq/fe-exec.c
Jerry Gay
Just in case you'd like to see what I was talking about, I am
attaching
my patch to src/interfaces/libpq/fe-exec.c to prevent utility functions
called from SPI from locking up the client.
Jerry Gay
comparisons correctly. The psql monitor converts all table and field
names to lower case. If the PQfnumber function is called with a mixed
case name, it will always return -1.
Bahman Rafatjoo
structs from libpq-fe.h, as we previously discussed.
There turned out to be sloppy coding practices in more places than
I had realized :-(, but all in all I think it was a well-worth-while
exercise.
I ended up adding several routines to libpq's API in order to respond
to application requirements that were exposed by this work. I owe the
docs crew updates for libpq.sgml to describe these changes. I'm way too
tired to work on the docs tonight, however.
This is the last major change I intend to submit for 6.4. I do want
to see if I can make libpgtcl work with Tcl 8.0 before we go final,
but hopefully that will be a minor bug fix.
From: Tom Lane <tgl@sss.pgh.pa.us>
Attached is a patch for this weekend's work on libpq. I've dealt
with several issues:
<for details: see message, in pgsql-patches archive for above data>
couple weeks ago on the hackers and interfaces lists:
1. When the backend sends a NOTICE message and closes the connection
(typically, because it was told to by the postmaster after
another backend coredumped), libpq will now print the notice
and close the connection cleanly. Formerly, the frontend app
would usually terminate ungracefully due to a SIGPIPE. (I am
not sure if 6.3.2 behaved that way, but the current cvs sources
do...)
2. libpq's various printouts to stderr are now fed through a single
"notice processor" routine, which can be overridden by the
application to direct notices someplace else. This should ease
porting libpq to Windows.
I also noticed and fixed a problem in PQprint: when sending output
to a pager subprocess, it would disable SIGPIPE in case the pager
terminates early (this is good) --- but afterwards it reset SIGPIPE
to SIG_DFL, rather than restoring the application's prior setting
(bad).
regards, tom lane
Making PQrequestCancel safe to call in a signal handler turned out to be
much easier than I feared. So here are the diffs.
Some notes:
* I modified the postmaster's packet "iodone" callback interface to allow
the callback routine to return a continue-or-drop-connection return
code; this was necessary to allow the connection to be closed after
receiving a Cancel, rather than proceeding to launch a new backend...
Being a neatnik, I also made the iodone proc have a typechecked
parameter list.
* I deleted all code I could find that had to do with OOB.
* I made some edits to ensure that all signals mentioned in the code
are referred to symbolically not by numbers ("SIGUSR2" not "2").
I think Bruce may have already done at least some of the same edits;
I hope that merging these patches is not too painful.
Through some minor changes, I have been able to compile the libpq
client libraries on the Win32 platform. Since the libpq communications
part has been rewritten, this has become much easier. Enclosed is
a patch that will allow at least Microsoft Visual C++ to compile
libpq into both a static and a dynamic library. I will take a look
at porting the psql frontend as well, but I figured it was a good
idea to send in these patches first - so no major changes are done
to the files before it gets applied (if it does).
Regards,
Magnus Hagander
I have implemented a framework of encoding translation between the
backend and the frontend. Also I have added a new variable setting
command:
SET CLIENT_ENCODING TO 'encoding';
Other features include:
Latin1 support more 8 bit cleaness
See doc/README.mb for more details. Note that the pacthes are
against May 30 snapshot.
Tatsuo Ishii
1. Rewritten libpq to allow asynchronous clients.
2. Implemented client side of cancel protocol in library,
and patched psql.c to send a cancel request upon SIGINT. The
backend doesn't notice it yet :-(
3. Implemented 'Z' protocol message addition and renaming of
copy in/out start messages. These are implemented conditionally,
ie, the client protocol version is checked; so the code should
still work with 1.0 clients.
4. Revised protocol and libpq sgml documents (don't have an SGML
compiler, though, so there may be some markup glitches here).
What remains to be done:
1. Implement addition of atttypmod field to RowDescriptor messages.
The client-side code is there but ifdef'd out. I have no idea
what to change on the backend side. The field should be sent
only if protocol >= 2.0, of course.
2. Implement backend response to cancel requests received as OOB
messages. (This prolly need not be conditional on protocol
version; just do it if you get SIGURG.)
3. Update libpq.3. (I'm hoping this can be generated mechanically
from libpq.sgml... if not, will do it by hand.) Is there any
other doco to fix?
4. Update non-libpq interfaces as necessary. I patched libpgtcl
so that it would compile, but haven't tested it. Dunno what
needs to be done with the other interfaces.
Have at it!
Tom Lane
If PQfn() receives NOTICEs from the backend, it fails because there is no
provision to deal with them.
This patch (supplied by Anders Hammarquist <iko@netg.se> to me as Debian
maintainer of postgresql) cures the problem:
The following patch is to src/interfaces/libpq of postgresql-6.3.
The purpose of the patch is to make the initialization of
const char *pgresStatus[] match the ExecStatusType enum.
I've completed the patch to fix the protocol and authentication issues I
was discussing a couple of weeks ago. The particular changes are:
- the protocol has a version number
- network byte order is used throughout
- the pg_hba.conf file is used to specify what method is used to
authenticate a frontend (either password, ident, trust, reject, krb4
or krb5)
- support for multiplexed backends is removed
- appropriate changes to man pages
- the -a switch to many programs to specify an authentication service
no longer has any effect
- the libpq.so version number has changed to 1.1
The new backend still supports the old protocol so old interfaces won't
break.
Subject: [PATCHES] memory leak patches in libpq and psql
A couple of small memory leak patches (detected with Purify) primarily
in libpq.
* Fixed (NULL) border problem in psql (run psql, do \m, then select
something from a table...row separators will be nulls)
* Fixed memory leak with the abovementioned border not being freed
properly.
* Fixed memory leak in freePGconn() not freeing conn->port
* Fixed up PQclear() to free parts of PGresult only if these
parts are not null.
* Fixed a decent memory leak that occured after executing every command
in psql. PGresult *results was not freed most of the time.
There is still a leak being detected (2 bytes) in readline functions, but
I think this is old readline library. I will install new one and test it.