Here is a patch to bring both libpq and psql to a state where it compiles on
win32 (native) again. A lot of things have changed, and I have not been able
to keep up with them all, so it has been broken for quite a while.
After this patch, at least it compiles. It also talks "basic talk" to the
server, but I have not yet tested all things. Sending queries, and using
e.g. \d or \dt works fine. The rest will have to be tested further.
It also bumps the version on libpq.dll to 7.0.
Everything should be enclosed in #ifdef WIN32, unless I have missed
something. Except for one or maybe two places where I have moved a #include
that should not be used on win32 from the "global area" into a "#ifndef
WIN32 area".
//Magnus
Attached is a patch which patches cleanly against the Sunday afternoon
snapshot. It modifies pg_dump to dump COMMENT ON statements for
user-definable descriptions. In addition, it also modifies comment.c so
that the operator behavior is as Peter E. would like: a comment on an
operator is applied to the underlying function.
Thanks,
Mike Mascari
is considerably more robust and accurate than it used to be.
Also, get rid of numeric's private allocation freelist, which is no longer
a win since Jan rewrote palloc.
SQL cast constructs can be performed during expression transformation
instead of during parsing. This allows constructs like x::numeric(9,2)
and x::int2::float8 to behave as one would expect.
read is reused for successive attributes, instead of being deleted and
recreated from scratch for each value read in. This reduces palloc/pfree
overhead a lot. COPY IN still seems to be noticeably slower than it was
in 6.5 --- we need to figure out why. This change takes care of the only
major performance loss I can see in copy.c itself, so the performance
problem is at a lower level somewhere.
by creating a race condition. It wasn't waiting for select() to say
write-ready immediately after connect, which meant that you might get
an unhelpful 'broken pipe' error message if connect failed, rather than
the intended error message.
CommandCounterIncrement to make new relation visible before trying to
parse/deparse the expressions. Also, eliminate unnecessary
setheapoverride calls in AddNewAttributeTuples.
oidvector/int2vector. pg_dump code was assuming that it would see
exactly FUNC_MAX_ARGS integers in the string returned by the backend.
That's no longer true. (Perhaps that change wasn't such a good idea
after all --- will it break any other applications??)
functions, which would lead to trouble with datatypes that paid attention
to the typelem or typmod parameters to these functions. In particular,
incorrect code in pg_aggregate.c explains the platform-specific failures
that have been reported in NUMERIC avg().
- Prevent permissions on indexes
- Instituted --enable-multibyte option and tweaked the MB build process where necessary
- initdb prompts for superuser password
* Let unprivileged users change their own passwords.
* The password is now an Sconst in the parser, which better reflects its text datatype and also
forces users to quote them.
* If your password is NULL you won't be written to the password file, meaning you can't connect
until you have a password set up (if you use password authentication).
* When you drop a user that owns a database you get an error. The database is not gone.
in libpq --- mostly, poor response to error conditions. You now actually
get to see the postmaster's 'The Data Base System is starting up' message,
which you didn't before. I suspect the SSL code is still broken though.
choke on relation or attribute names containing spaces, quotes, or other
special characters. This fixes a TODO item. It also forces initdb,
since stored rule strings change.
errors. VACUUM normally compacts the table back-to-front, and stops
as soon as it gets to a page that it has moved some tuples onto.
(This logic doesn't make for a complete packing of the table, but it
should be pretty close.) But the way it was checking whether it had
got to a page with some moved-in tuples was to look at whether the
current page was the same as the last page of the list of pages that
have enough free space to be move-in targets. And there was other
code that would remove pages from that list once they got full.
There was a kluge that prevented the last list entry from being
removed, but it didn't get the job done. Fixed by keeping a separate
variable that contains the largest block number into which a tuple
has been moved. There's no longer any need to protect the last element
of the fraged_pages list.
Also, fix NOTICE messages to describe elapsed user/system CPU time
correctly.
Instead of hard-wiring one result file per platform, there is a map file
'resultmap' that says which one to use --- a lot like template/.similar.
I have only created entries in resultmap for my own platform (HPUX) so
far; feel free to add lines for other platforms.
which is broken in some weird way that I don't understand. I think it
may be exposing a bug in the new psql --- for one thing, I get different
results when I run psql by hand than the regress script gets. What
the heck???
quote_postgres(...) in ecpglib.c.
The code in CVS reads:
quote_postgres(char *arg, int lineno)
{
char *res = (char *) ecpg_alloc(2 * strlen(arg) + 3, lineno);
int i,
ri = 0;
if (!res)
return (res);
res[ri++] = '\'';
for (i = 0, ri=0; arg[i]; i++, ri++)
{
switch (arg[i])
{
case '\'':
res[ri++] = '\'';
break;
case '\\':
res[ri++] = '\\';
break;
default:
;
}
The problem here is that ri is reset to 0, thus overwriting the initial
quote.
Stephen Birch
if presented an uninitialized (all zeroes) page. The system no longer
crashes hard if an all-zeroes page is present in a relation. There seem
to be some boundary conditions where a page will be appended to a relation
and zeroed, but its page header is never initialized; until we can track
down and fix all of those, robustness seems like a good idea.
Also, clean up some obsolete and downright wrong comments.
1) datetime_pl_span() added the seconds field before adding the months
field. This lead to erroneous results for e.g.
select datetime '1999-11-30' + timespan '1 mon - 1 sec';
Reverse the order of operations to add months first.
2) tm2timespan() did all intermediate math as integer, converting to double
at the very end. This resulted in hidden overflows when given very large
integer days, hours, etc. For example,
select '74565 days'::timespan;
produced the wrong result. Change code to ensure that doubles are used
for intermediate calculations.
Thanks to Olivier PRENANT <ohp@pyrenet.fr> and
Tulassay Zsolt <zsolt@tek.bke.hu> for problem reports and to Tom Lane for
accurate analyses.
during InitProcessingMode and the CurrentTransactionState was neither
TRANS_DEFAULT nor TRANS_DISABLED. Unfortunately, after someone's recent
change to start the transaction manager earlier in startup than it used
to be started, that caused an abort() and consequent database system
reset on quite harmless errors (such as rejecting an invalid user name!).
As far as I can see, the test on CurrentTransactionState was completely
useless anyway, so I've removed it.
relcache entry no longer leaks a small amount of memory. index_endscan
now releases all the memory acquired by index_beginscan, so callers of it
should NOT pfree the scan descriptor anymore.
I finally got around to schlepping through pg_dump, to finish what I started
about three months (or more) ago. Attached is a gzipped diff file to apply
in the bin/pg_dump directory. This should remove all string length
dependencies, except one, which I'm working on. It has been through some
rudimentary unit testing, but that's about it, so if any of you would give
it a more strenuous run-through, I'd be grateful for the feedback.
Cheers...
Ansley, Michael
SELECT null::text;
SELECT int4fac(null);
work as expected now. In some cases a NULL must be surrounded by
parentheses:
SELECT 2 + null; fails
SELECT 2 + (null); OK
This is a grammatical ambiguity that seems difficult to avoid. Other
than that, NULLs seem to behave about like you'd expect. The internal
implementation is that NULL constants are typed as UNKNOWN (like
untyped string constants) until the parser can deduce the right type.
Locate path of postmaster in a portable way (stolen from initdb)
Add postmaster.opts.default.sample which should be copied into
$PGLIB in the installtion process. Also, it will be installed into
$PGDATA while initdb is running.
a ".pgc " extension. The second patch fixes a coredump when there is
more than one input file (in that case, cur and types were not set to
NULL before processing the second f ile)
The patch below modifies the accepted grammar of ecpg to accept
FETCH [direction] [amount] cursor name
i.e. the IN|FROM clause becomes optional (as in Oracle and Informix).
This removes the incompatibility mentioned in section "Porting From
Other RDBMS Packages" p169, PostgreSQL Programmer's Guide. The grammar
is modified in such a way as to avoid shift/reduce conflicts. It does
not accept the statement "EXEC SQL FETCH;" anymore, as the old grammar
did (this seems to be a bug of the old grammar anyway).
This patch cleans up the handling of space characters in the scanner;
some patte rns require \n to be in {space}, some do not. A second fix is
the handling of cpp continuati on lines; the old pattern did not match
these. The parser is patched to fix an off-by-one error in the #line
directives. The pa rser is also enhanced to report the correct location
of errors in declarations in the "E XEC SQL DECLARE SECTION". Finally,
some right recursions in the parser were replaced by left-recursions.
This patch adds preprocessor directives to ecpg; in particular
EXEC SQL IFDEF, EXEC SQL IFNDEF, EXEC SQL ELSE, EXEC SQL ELIF and EXEC SQL ENDIF
"EXEC SQL IFDEF" is used with defines made with "EXEC SQL DEFINE" and
defines, specified on the command line with -D. Defines, specified on
the command line are persistent across multiple input files. Defines can
be nested up to a maximum level of 128 (see patch). There is a fair
amount of error checking to make sure directives are matched properly. I
need preprocessor directives for porting code, that is written for an
Informix database, to a PostgreSQL database, while maintaining
compatibility with the original code. I decided not to extend the
already large ecpg grammar. Everything is done in the scanner by adding
some states, e.g. to skip all input except newlines and directives. The
preprocessor commands are compatible with Informix. Oracle uses a cpp
replacement.
Rene Hogendoorn
with DEC C.
DEC C doesn't handle double values greater than DBL_MAX, but some
PostgreSQL geo functions assign greater than DBL_MAX values to some vars
in some special cases - that couses SIGFPE. I dunno if that is the only place
to fix to work well with DEC C.
Kirill Nosov.
rather than returning a NaN for bogus input to pow(). Namely, HPUX 10.20.
I think this is sufficient evidence for what I thought all along, which
is that the float.c code *must* look at errno whether finite() exists or
not.
> > for them to actually set out and do it. Many new users are
> > of the not-so-knowledgable variety, and shell scripting isn't
> > something they want to undertake.
>
> Can someone modify the vacuumdb shell script to do that?
i tried it... it seems to work
neko@kredit.sth.sz
initdb. No more obscure dependencies on environment variables or paths.
It
now finds the templates and the right postgres itself (with cmd line
options as fallback). It also no longer depends on $USER (su safe), and
doesn't advertise that --username allows you to install the db as a
different user, since that doesn't work anyway. Also, recovery and
cleanup
on all errors. Consistent options, clearer documentation.
Please take a look at this and adopt it if you feel it's safe enough. I
have simulated all the stupid circumstances I could think of, but you
never know with shell scripts.
Oh yeah, you can give the postgres user a default password now.
--
Peter Eisentraut Sernanders väg 10:115
I sending promised patch with:
* getopt_long() - for pg_dump (portable)
* and "Usage: " changes in scripts in src/bin/
- this changes are cosmetic only, not change any
feature ...etc.
All PostgreSQL routines (scripts) support now long options and
help's output is alike for all scripts and all support -? or --help.
Karel
Karel Zak <zakkr@zf.jcu.cz> http://home.zf.jcu.cz/~zakkr/
>go about this. That will risk breaking existing applications that use
>those names as column names.
>
>It should actually almost work to write sq.nextval as things stand,
>because Postgres has for a long time considered table.function and
>function(table) to be interchangeable notations for certain kinds of
>functions. nextval doesn't seem to be one of that kind of function,
>at the moment. I'd suggest leaving the grammar as it was, and taking a
>look at ParseFuncOrColumn in parse_func.c to see if you can't persuade
>it to accept the sequence functions in that style.
OK, good point. I tried to implement it somewhere else and ended up
extending transformAttr. Attached you'll find the patch.
Jeroen van Vianen
didn't have time for documentation yet, but I'll write some. There are
still some things to work out what happens when you alter or drop users,
but the group stuff in and by itself is done.
--
Peter Eisentraut Sernanders väg 10:115
equality. The lobits macro is wrong and extracts the wrong set of
bits out of the structure.
To exhibit the problem:
select '000000:000000'::macaddr = '000000:110000'::macaddr ;
?column?
--------
t
(1 row)
Daniel Boyd
triggered
> function now returns the right datatype.
Oops, I got crossed up with Jan's improvements. Ignore this.
--
Peter Eisentraut Sernanders väg 10:115
peter_e@gmx.net 75262 Uppsala
anywhere from zero to two TODO items.
* Allow flag to control COPY input/output of NULLs
I got this:
COPY table .... [ WITH NULL AS 'string' ]
which does what you'd expect. The default is \N, otherwise you can use
empty strings, etc. On Copy In this acts like a filter: every data item
that looks like 'string' becomes a NULL. Pretty straightforward.
This also seems to be related to
* Make postgres user have a password by default
If I recall this discussion correctly, the problem was actually that the
default password for the postgres (or any) user is in fact "\N", because
of the way copy is used. With this change, the file pg_pwd is copied out
with nulls as empty strings, so if someone doesn't have a password, the
password is just '', which one would expect from a new account. I don't
think anyone really wants a hard-coded default password.
Peter Eisentraut Sernanders väg 10:115
* Document/trigger/rule so changes to pg_shadow recreate pg_pwd
I did it with a trigger and it seems to work like a charm. The function
that already updates the file for create and alter user has been made a
built-in "SQL" function and a trigger is created at initdb time.
Comments around the pg_pwd updating function seem to be worried about
this
routine being called concurrently, but I really don't see a reason to
worry about this. Verify for yourself. I guess we never had a system
trigger before, so treat this with care, and feel free to adjust the
nomenclature as well.
--
Peter Eisentraut Sernanders väg 10:115
at all, and because of shell quoting rules this can't be fixed, so I put
in error messages to that end.
Also, calling create or drop database in a transaction block is not so
good either, because the file system mysteriously refuses to roll back rm
calls on transaction aborts. :) So I put in checks to see if a transaction
is in progress and signal an error.
Also I put the whole call in a transaction of its own to be able to roll
back changes to pg_database in case the file system operations fail.
The alternative location issues I posted recently were untouched, awaiting
the outcome of that discussion. Other than that, this should be much more
fool-proof now.
The docs I cleaned up as well.
Peter Eisentraut Sernanders väg 10:115
yet, but at least we can give a better error message:
regression=> select count(distinct f1) from int4_tbl;
ERROR: aggregate(DISTINCT ...) is not implemented yet
instead of 'parser: parse error at or near distinct'.
I was able to crash postgres 6.5.3 when I did an 'alter user' command.
After I started a debugger I found the problem in the timezone handling
of
datetime (my Linux box lost its timezone information, that's how the
problem occurred).
Only 7 bytes are reserved for the timezone, without checking for
boundaries.
Attached is a patch that fixes this problem and emits a NOTICE if a
timezone is encountered that is longer than MAXTZLEN bytes, like this:
Jeroen van Vianen
against the sources from one hour ago and contain all the portable and
up
to date stuff.
A few other CVS "householding" things you might want to take care of:
* Remove the src/bin/cleardbdir directory
* Remove the file src/bin/psql/sql_help.h from the repository, as it is
a derived file and is build by the release_prep.
Peter Eisentraut
(which are palloc'd) instead of DLLists (which are malloc'd). Not very
significant, since this routine seldom has anything useful to do, but
a leak is a leak...
them into the scripts dir. I also added a --list option to show already
installed languages.
This whole moving and renaming totally confused CVS and my checked out
copy got completely fried last night. When you apply the source patch,
please make sure that all the directories src/bin/{create|destroy}* as
well as vacuumdb, cleardbdir are gone and that all the scripts (7) are
in
scripts/.
Meanwhile I am still puzzled about what happened with the docs patch.
Because I don't know what you got now, the second attachment contains
the
files
ref/allfiles.sgml
ref/commands.sgml
ref/createlang.sgml
ref/droplang.sgml
doc/src/sgml/Makefile
Peter Eisentraut Sernanders väg 10:115
parameter in some flavor of Unix, but Linux, HPUX, and SunOS all say
it's int. For now I'm just going to make it int so that I can compile.
If the other way is actually necessary on some Unix somewhere, I guess
we will need a configure test...
This one should work much better than the one I sent in previously. The
functionality is the same, but the patch was missing one file resulting
in
the compilation failing. The docs also received a minor fix.
Peter Eisentraut Sernanders väg 10:115
PQconnectStart
PQconnectPoll
PQresetStart
PQresetPoll
PQsetenvStart
PQsetenvPoll
PQsetenvAbort
and brings into the published interface
PQsetenv.
The first four are asynchronous analogues of PQconnectdb and PQreset -
they allow an application to connect to the DB without blocking on
remote I/O.
The PQsetenv functions perform an environment negotiation with the
server.
Internal to libpq, pqReadReady and pqWriteReady have been made available
across the library (they were previously static functions inside
fe-misc.c). A lot of internal rearrangement has been necessary to
support these changes.
The API documentation has been updated also.
Caveats:
o The Windows code does not default to using non-blocking sockets,
since I have no documentation: Define WIN32_NON_BLOCKING_CONNECTIONS to
do that.
o The SSL code still blocks.
Ewan Mellor.
You have CommLog and Debug enabled
You encounter in error in any operation (SQLConnect/SQLExec).
Previously, the extra logging didn't check for NULL pointers
when trying to print some of the strings- the socket error
message could frequently be NULL by design (if there was no socket
error)
and Solaris does not handle NULLS passed to things like printf
("%s\n",string);
gracefully.
This basically duplicates the functionality found in Linux where passing
a null pointer
to printf prints "(NULL)". No very elegant, but the logging is for debug
only anyway.
Dirk Niggemann
table owner in order to vacuum a table. This is mainly to prevent
denial-of-service attacks via repeated vacuums. Allow VACUUM to gather
statistics about system relations, except for pg_statistic itself ---
not clear that it's worth the trouble to make that case work cleanly.
Cope with possible tuple size overflow in pg_statistic tuples; I'm
surprised we never realized that could happen. Hold a couple of locks
a little longer to try to prevent deadlocks between concurrent VACUUMs.
There still seem to be some problems in that last area though :-(
expressions were written without spaces between operators and operands.
Problem was that something like "if new.f1=new.f2 then" would be translated
to "if $1=$2 then", and the Postgres lexer would tokenize that the wrong
way. Fix is to emit spaces around $paramno constructs to ensure they are
seen as separate tokens.
parallel --- and, not incidentally, removing a common reason for needing
manual cleanup by the DB admin after a crash. Remove initial global
delete of pg_statistics rows in VACUUM ANALYZE; this was not only bad
for performance of other backends that had to run without stats for a
while, but it was fundamentally broken because it was done outside any
transaction. Surprising we didn't see more consequences of that.
Detect attempt to run VACUUM inside a transaction block. Check for
query cancel request before starting vacuum of each table. Clean up
vacuum's private portal storage if vacuum is aborted.
By dropping stats rows here, we eliminate the need for VACUUM to do a
wholesale remove of stats rows. Before, pg_statistics was wiped clean
at the start of VACUUM, ensuring poor planning results for any backends
running in parallel until VACUUM got around to rebuilding the stats for
the relations they are accessing.
rate
it's better than what used to be there.
* Does proper SQL "host variable" substitution as pointed out by Andreas
Zeugwetter (thanks): select * from :foo; Also some changes in how ':'
and ';' are treated (escape with \ to send to backend). This does
_not_
affect the '::' cast operator, but perhaps others that contain : or ;
(but there are none right now).
* To show description with a <something> listing, append '?' to command
name, e.g., \df?. This seemed to be the convenient and logical
solution.
Or append a '+' to see more useless information, e.g., \df+.
* Fixed fflush()'ing bug pointed out by Jan during the regression test
discussion.
* Added LastOid variable. This ought to take care of TODO item "Add a
function to return the last inserted oid, for use in psql scripts"
(under CLIENTS)
E.g.,
insert into foo values(...);
insert into bar values(..., :LastOid);
\echo $LastOid
* \d command shows constraints, rules, and triggers defined on the table
(in addition to indices)
* Various fixes, optimizations, corrections
* Documentation update as well
Note: This now requires snprintf(), which, if necessary, is taken from
src/backend/port. This is certainly a little weird, but it should
suffice
until a source tree cleanup is done.
Enjoy.
--
Peter Eisentraut Sernanders väg 10:115
Make all system indexes unique.
Make all cache loads use system indexes.
Rename *rel to *relid in inheritance tables.
Rename cache names to be clearer.
for the case of errors in backend startup, and proc_exit's method for
coping with errors during proc_exit was *completely* busted. Fixed per
discussions on pghackers around 11/6/99.
it wants to release. This leads to a race condition: does the backend
that's trying to flush the buffer do so before the one that's deleting the
relation does so? Usually no problem, I expect, but on occasion this could
lead to hard-to-reproduce complaints from md.c, especially mdblindwrt.
returns a list of RelOptInfos, eliminating the need for static state
in index_info. That static state was a direct cause of coredumps; if
anything decided to elog(ERROR) partway through an index_info search of
pg_index, the next query would try to close a scan pointer that was
pointing at no-longer-valid memory. Another example of the reasons to
avoid static state variables...
of the index it wants to destroy. This ensures that no other backend is
actively scanning or updating that index. Getting exclusive access on
the index alone is NOT sufficient, because the executor is rather
cavalier about getting locks on indexes --- see ExecOpenIndices().
It might be better to grab index locks in the executor, but I'm not
sure the extra lockmanager traffic is really worth it just to make
index_destroy cleaner.
(whoever thought world-writable files were a good default????). Modify
the pg_pwd code so that pg_pwd is created with 600 permissions. Modify
initdb so that permissions on a pre-existing PGDATA directory are not
blindly accepted: if the dir is already there, it does chmod go-rwx
to be sure that the permissions are OK and the dir actually is owned
by postgres.
inval.c thought it could safely use the catcache to look up the OIDs of
system relations. Not good, considering that inval.c could be called
during catcache loading, if a shared-inval message arrives. Rip out the
lookup logic and instead use the known OIDs from pg_class.h.
table defaults or rules: translate them to a function call so that
parse_coerce doesn't reduce them to a date or time constant immediately.
Also, eliminate a lot of redundancy in the expression grammar by
defining a new nonterminal com_expr, which contains all the productions
that can be shared by a_expr and b_expr.
Warn_restart has been set by the backend main loop. This means that
elog(ERROR) or elog(FATAL) in the postmaster or during backend startup
now have well-defined behavior: proc_exit() rather than coredump.
In the case of elog() inside the postmaster, I think that proc_exit()
is probably not enough --- don't we want our child backends to be
forced to quit too? But I don't understand Vadim's recent changes in
this area, so I'll leave it to him to look over and tweak if needed.
subselects can only appear on the righthand side of a binary operator.
That's still true for quantified predicates like x = ANY (SELECT ...),
but a subselect that delivers a single result can now appear anywhere
in an expression. This is implemented by changing EXPR_SUBLINK sublinks
to represent just the (SELECT ...) expression, without any 'left hand
side' or combining operator --- so they're now more like EXISTS_SUBLINK.
To handle the case of '(x, y, z) = (SELECT ...)', I added a new sublink
type MULTIEXPR_SUBLINK, which acts just like EXPR_SUBLINK used to.
But the grammar will only generate one for a multiple-left-hand-side
row expression.
circumstances:
=> select * from foo\x\t\pset border 0 \p\g\\select * from bar;
Also the release prep update so the sql_help.h is generated before
packaging.
Peter.
nulls with non-nulls using proper three-valued boolean logic. Also clean
up ExecQual to make it clearer that ExecQual *does* follow the SQL spec
for boolean nulls. See '[BUGS] (null) != (null)' thread around 10/26/99
for more detail.
* Add use of 'const' for varibles in source tree
(which is misspelled, btw.)
I went through the front-end libpq code and did so. This affects in
particular the various accessor functions (such as PQdb() and
PQgetvalue()) as well as, by necessity, the internal helpers they use.
I have been really thorough in that regard, perhaps some people will find
it annoying that things like
char * foo = PQgetvalue(res, 0, 0)
will generate a warning. On the other hand it _should_ generate one. This
is no real compatibility break, although a few clients will have to be
fixed to suppress warnings. (Which again would be in the spirit of the
above TODO.)
In addition I replaced some int's by size_t's and removed some warnings
(and generated some new ones -- grmpf!). Also I rewrote PQoidStatus (so it
actually honors the const!) and supplied a new function PQoidValue that
returns a proper Oid type. This is only front-end stuff, none of the
communicaton stuff was touched.
The psql patch also adds some new consts to honor the new libpq situation,
as well as fixes a fatal condition that resulted when using the -V
(--version) option and there is no database listening.
So, to summarize, the psql you should definitely put in (with or without
the libpq). If you think I went too far with the const-mania in libpq, let
me know and I'll make adjustments. If you approve it, I will also update
the docs.
-Peter
--
Peter Eisentraut Sernanders vaeg 10:115