I have changed a bit the makefiles for the win32 port - the *.def files
(created when building shared libraries) are now clean from
Makefile.shlib.
I have also removed "-g" from CFLAGS in the "cygwin32" template - it can
be
enabled when running configure.
Dan
database, but they get truncated at the first NUL by lo_read
when they are read back. The reason for this is that lo_read in
Pg.xs is using the default:
OUTPUT:
RETVAL
buf
which uses C's strlen() to work out the length of the scalar.
The code ought to read something more like:
OUTPUT:
RETVAL
buf sv_setpvn((SV*)ST(2), buf, RETVAL);
I am not sure if this needs to be done on both lo_read methods
in this file, but I changed both and have not since had any
problems with truncated BLOBs.
Douglas Thomson <dougt@mugc.cc.monash.edu.au>
Cygwin snapshots (tested on 990115 which is recommended to use - it fixes
some errors in B20.1)
And I have another patch for including <sys/ipc.h> before <sys/sem.h> in
backend/storage/lmgr/proc.c - it is required due the design of cygipc
headers
Dan
mentioned in FROM but not elsewhere in the query: such tables should be
joined over anyway. Aside from being more standards-compliant, this allows
removal of some very ugly hacks for COUNT(*) processing. Also, allow
HAVING clause without aggregate functions, since SQL does. Clean up
CREATE RULE statement-list syntax the same way Bruce just fixed the
main stmtmulti production.
CAUTION: addition of a field to RangeTblEntry nodes breaks stored rules;
you will have to initdb if you have any rules.
quite the same way that transformInsertStatement does, so that an expression
could be accepted by CREATE TABLE and then fail when used. Also, put back
check that CONSTRAINT expressions must yield boolean...
expressions in CREATE TABLE. There is no longer an emasculated expression
syntax for these things; it's full a_expr for constraints, and b_expr
for defaults (unfortunately the fact that NOT NULL is a part of the
column constraint syntax causes a shift/reduce conflict if you try a_expr.
Oh well --- at least parenthesized boolean expressions work now). Also,
stored expression for a column default is not pre-coerced to the column
type; we rely on transformInsertStatement to do that when the default is
actually used. This means "f1 datetime default 'now'" behaves the way
people usually expect it to.
BTW, all the support code is now there to implement ALTER TABLE ADD
CONSTRAINT and ALTER TABLE ADD COLUMN with a default value. I didn't
actually teach ALTER TABLE to call it, but it wouldn't be much work.
--enable-debug adds -g (unconditionally)
--disable-debug removes -g (if it was already in there somehow)
(giving neither does nothing)
Since none of the templates default CFLAGS with a -g you're not likely
to
end up with two -g flags. Not that they'd hurt though.
It doesn't do anything about C++.
Peter Eisentraut
not just C, so that ISCACHABLE attribute can be specified for user-defined
functions. Get rid of ParamString node type, which wasn't actually being
generated by gram.y anymore, even though define.c thought that was what
it was getting. Clean up minor bug in dfmgr.c (premature heap_close).
modifyAggrefQual. This routine really, really needs to be retired, but
until we have subselects in FROM there's no chance of doing the job right.
In the meantime try to respond to unhandlable cases with elog rather than
coredump.
in the Expr nodes they produce. This fixes a few cases of errors like
'typeidTypeRelid: Invalid type - oid = 0' caused by calling parser-related
routines on expression trees that have already been processed by planner-
related routines.
they have no hardwired limit on the length of a rule's text. Fix a couple
of minor bugs in passing --- deparsed UPDATE queries didn't have quotes
around relation name, and quotes and backslashes in constant values weren't
backslash-quoted.
expression_tree_mutator rather than ad-hoc tree walking code. This shortens
the code materially and fixes a fair number of sins of omission. Also,
change modifyAggrefQual to *not* recurse into subselects, since its mission
is satisfied if it removes aggregate functions from the top level of a
WHERE clause. This cures problems with queries of the form SELECT ...
WHERE x IN (SELECT ... HAVING something-using-an-aggregate), which would
formerly get mucked up by modifyAggrefQual. The routine is still
fundamentally broken, of course, but I don't think there's any way to get
rid of it before we implement subselects in FROM ...
Implements the CREATE CONSTRAINT TRIGGER and SET CONSTRAINTS commands.
TODO:
Generic builtin trigger procedures
Automatic execution of appropriate CREATE CONSTRAINT... at CREATE TABLE
Support of new trigger type in pg_dump
Swapping of huge # of events to disk
Jan
functions. One problem that I have encountered with the function
manager is that it does not allow the user to define type conversion
functions that convert between user types. For instance if mytype1,
mytype2, and mytype3 are three Postgresql user types, and if I wish to
define Postgresql conversion functions like
I run into problems, because the Postgresql dynamic loader would look
for a single link symbol, mytype3, for both pieces of object code. If
I just change the name of one of the Postgresql functions (to make the
symbols distinct), the automatic type conversion that Postgresql uses,
for example, when matching operators to arguments no longer finds the
type conversion function.
The solution that I propose, and have implemented in the attatched
patch extends the CREATE FUNCTION syntax as follows. In the first case
above I use the link symbol mytype2_to_mytype3 for the link object
that implements the first conversion function, and define the
Postgresql operator with the following syntax
The patch includes changes to the parser to include the altered
syntax, changes to the ProcedureStmt node in nodes/parsenodes.h,
changes to commands/define.c to handle the extra information in the AS
clause, and changes to utils/fmgr/dfmgr.c that alter the way that the
dynamic loader figures out what link symbol to use. I store the
string for the link symbol in the prosrc text attribute of the pg_proc
table which is currently unused in rows that reference dynamically
loaded
functions.
Bernie Frankpitt
an empty targetlist *and* fails to return any tuples, as will happen
for example with 'SELECT COUNT(1) FROM table WHERE ...' if the where-
clause selects no tuples. It's so nice to make a fix by diking out code,
instead of adding more...
behavior as it was, apart from forbidding minus-terminated
operators. Seems that I have to break the habit of doing before
thinking properly :-/ The point is that my second patch breaks
constructs like a & b or a ! b. This patch is to be applied
instead of any of two other today's patches.
Leon
Two patches included:
- the first one enables the use of bool variables in fields which might
become NULL.
Up to now the lib told you that NULL is not a bool variable, even if
you provide a indicator.
- the second patch checks whether a value is null and issues an error if
no indicator is provided.
Sidenote: IIRC, the variable should be left alone if the value is NULL.
ECPGlib sets it's value to 0 on NULL. Is this a violation of the
standard?
Regards
Christof
is used to find start scan position of Indexscan-s.
To speed up finding scan start position,I have changed
_bt_first() to use as many keys as possible.
I'll attach the patch here.
Regards.
Hiroshi Inoue
When drawing up a very simple "text-drawing" of how the negotiation is done,
I realised I had done this last part (fallback) in a very stupid way. Patch
#4 fixes this, and does it in a much better way.
Included is also the simple text-drawing of how the negotiation is done.
//Magnus
with no input rows, per pghackers discussions around 7/22/99. Clean up
a bunch of ugly coding while at it; remove redundant re-lookup of
aggregate info at start of each new GROUP. Arrange to pfree intermediate
values when they are pass-by-ref types, so that aggregates on pass-by-ref
types no longer eat memory. This takes care of a couple of TODO items...
Frankpitt, plus some improvements from yours truly. The simplifier depends
on the proiscachable field of pg_proc to tell it whether a function is
safe to pre-evaluate --- things like nextval() are not, for example.
Update pg_proc.h to contain reasonable cacheability information; as of
6.5.* hardly any functions were marked cacheable. I may have erred too
far in the other direction; see recent mail to pghackers for more info.
This update does not force an initdb, exactly, but you won't see much
benefit from the simplifier until you do one.
* Buffer refcount cleanup (per my "progress report" to pghackers, 9/22).
* Add links to backend PROC structs to sinval's array of per-backend info,
and use these links for routines that need to check the state of all
backends (rather than the slow, complicated search of the ShmemIndex
hashtable that was used before). Add databaseOID to PROC structs.
* Use this to implement an interlock that prevents DESTROY DATABASE of
a database containing running backends. (It's a little tricky to prevent
a concurrently-starting backend from getting in there, since the new
backend is not able to lock anything at the time it tries to look up
its database in pg_database. My solution is to recheck that the DB is
OK at the end of InitPostgres. It may not be a 100% solution, but it's
a lot better than no interlock at all...)
* In ALTER TABLE RENAME, flush buffers for the relation before doing the
rename of the physical files, to ensure we don't get failures later from
mdblindwrt().
* Update TRUNCATE patch so that it actually compiles against current
sources :-(.
You should do "make clean all" after pulling these changes.
now that sequence names are properly quoted for field defaults, mixed
case sequence names are generated. These are properly quoted in the
CREATE SEQUENCE lines, but not in the SELECT nextval lines, as per
below:
CREATE SEQUENCE "Teams_TeamID_seq" start 10 increment 1 maxvalue
2147483647 minvalue 1 cache 1 ;
SELECT nextval ('Teams_TeamID_seq');
This needs to be:
SELECT nextval ('"Teams_TeamID_seq"');
Patch included below.
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
additional argument specifying the kind of lock to acquire/release (or
'NoLock' to do no lock processing). Ensure that all relations are locked
with some appropriate lock level before being examined --- this ensures
that relevant shared-inval messages have been processed and should prevent
problems caused by concurrent VACUUM. Fix several bugs having to do with
mismatched increment/decrement of relation ref count and mismatched
heap_open/close (which amounts to the same thing). A bogus ref count on
a relation doesn't matter much *unless* a SI Inval message happens to
arrive at the wrong time, which is probably why we got away with this
sloppiness for so long. Repair missing grab of AccessExclusiveLock in
DROP TABLE, ALTER/RENAME TABLE, etc, as noted by Hiroshi.
Recommend 'make clean all' after pulling this update; I modified the
Relation struct layout slightly.
Will post further discussion to pghackers list shortly.
See attached mail for more details.
-------------------------------------------------------------------
From: "Vadim Mikheev" <vadim@krs.ru>
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>
References: <000201befa94$42fe04c0$2801007e@cadzone.tpf.co.jp>
Subject: Re: elog(ERROR) in vacuum
Date: Fri, 10 Sep 1999 10:27:10 +0900
Organization: OJSC Rostelecom (Krasnoyarsk)
Message-ID: <37D85E6E.5AFA126D@krs.ru>
Hiroshi Inoue wrote:
>
> Hello Vadim,
>
> I have a question about vacuum.
>
> VACUUM has a phase like commit which calls TransactionIdCommit().
> But if elog(ERROR) occured after that,the status of transaction is
> changed from XID_COMMIT to XID_ABORT.
>
> Seems to me this causes inconsistency.
> Shoudn't AbortTransaction() be changed not to call TransacionIdAbort()
> in case of vacuum.
You're right!
As usual -:)
Vadim
Almost worked before, but forgot one place to check.
Reported by Tatsuo Ishii.
Still does not do the right thing if inserting into a non-string target
column. Should look for a type coersion later, but doesn't.
message under a kernel that only returns one packet per recv() call. This
didn't use to matter much, but it starts to get annoying with multi-megabyte
EXPLAIN VERBOSE responses...
conditions. There are some pretty bogus heuristics in prepqual.c that
try to decide whether to output CNF or DNF format; they need to be replaced,
likely. Right now the code is probably too willing to choose DNF form,
which might hurt performance in some cases that used to work OK.
But at least we have a foundation to build on.
in or_normalize, remove detection of duplicate subexpressions (since it's
highly unlikely to be worth the amount of time it takes), and introduce
a dnfify() entry point so that unintelligible backwards logic in UNION
processing can be eliminated. This is just an intermediate step ---
next thing is to look at not forcing the qual into CNF form when it would
be better off in DNF form.
This change seems necessary in conjunction with long queries, and it
cleans up some bogosity in connection with long EXPLAIN texts anyway.
Note that current libpq will accept any length error message (at least
until it runs out of memory); prior versions have a limit of 8K, but
will cleanly discard excess error text, so there shouldn't be any
big compatibility problems with old clients.
transaction abort --- before it only worked if there was exactly one level
of allocation context stacked in the blank portal. Now it does the right
thing for any depth, including zero...
was rejecting negative attnums as bogus, which of course they are not.
Add code to get_attdisbursion to produce a useful value for OID attribute,
since VACUUM does not store stats for system attributes.
Also, repair bug that's been in eqjoinsel for a long time: it was taking
the max of the two columns' disbursions, whereas it should use the min.
space consumption in pull_args, and avoid doing the full CNF transform on
operands of operator clauses, where it's really not particularly helpful.
This answers the TODO item about large numbers of OR clauses, at least
partially. I was able to do a ten-thousand-OR-clause query with about
20Mb memory consumption ... it took an obscenely long time, but it worked...
corrects flex myinput() routine so that it doesn't assume there is only
one bufferload of data. We still have the issue of getting rid of
YY_USES_REJECT so that the scanner can cope with tokens larger than its
initial buffer size.
before comparison; if fields being joined are different widths then hashing
will yield wrong answer. Also, remove hashjoinable mark from all uses of
array_eq, because array structures may have padding bytes between elements
and the pad bytes are of uncertain content. This could be revisited if
array code is cleaned up.
Modify opr_sanity regress test to complain if array_eq operator is marked
hashjoinable.
offended my aesthestic sensibility that there was so much unreadable code
doing so little. Rewritten code is about half the size, faster, and
(I hope) much more intelligible.
current transaction) are not flushed by shared-cache-inval reset message.
SI reset actually works now, for probably the first time in a long time.
I was able to run initdb and regression tests with a 16-element SI message
array, with a lot of NOTICE: cache state reset messages but no crashes.
We can't support these properly, since once the relation's physical files
are unlinked, there's no way to roll back the transaction. I suppose
we could postpone the unlink till transaction commit, but then what of
BEGIN; DROP TABLE foo; CREATE TABLE foo; ?
The code does allow dropping a table/index created in the current
transaction block, however, since the post-abort state would be that
the table doesn't exist anyway.
real name before doing lookup. We only want to index temp tables by their
real names in the relcache, to ensure there's not more than one relcache
entry for them.
has positive refcount, it is rebuilt from pg_class data. This ensures
that relcache entries will track changes made by other backends. Formerly,
a shared inval report would just be ignored if it happened to arrive while
the relcache entry was in use. Also, fix relcache to reset ref counts
to zero during transaction abort. Finally, change LockRelation() so that
it checks for shared inval reports after obtaining the lock. In this way,
once any kind of lock has been obtained on a rel, we can trust the relcache
entry to be up-to-date.
the SInval spinlock while it is calling the passed invalFunction or
resetFunction. This is necessary to avoid deadlock with lmgr change;
InvalidateSharedInvalid can be called recursively now. It should be
a good performance improvement anyway --- holding a spinlock for more
than a very short interval is a no-no.
insight that RelationFlushRelation ought to invoke smgrclose, and that the
way to make that work is to ensure that mdclose doesn't fail if the relation
is already closed (or unlinked, if we are looking at a DROP TABLE). While
I was testing that, I was able to identify several problems that we had
with multiple-segment relations. The system is now able to do initdb and
pass the regression tests with a very small segment size (I had it set to
64Kb per segment for testing). I don't believe that ever worked before.
File descriptor leaks seem to be gone too.
I have partially addressed the concerns we had about mdtruncate(), too.
On a Win32 or NFS filesystem it is not possible to unlink a file that
another backend is holding open, so what md.c now does is to truncate
unwanted files to zero length before trying to unlink them. The other
backends will be forced to close their open files by relation cache
invalidation --- but I think it would take considerable work to make
that happen before vacuum truncates the relation rather than after.
Leaving zero-length files lying around seems a usable compromise.
error/notice message lengths, and number of fields per tuple. Add
pqexpbuffer.c/.h, a frontend version of backend's stringinfo module.
This is first step in applying Mike Ansley's long-query patches,
even though he didn't do any of these particular changes...
and 1370 (timestamp(datetime)). This does not force an initdb, exactly,
but you won't see the effects of the bug fix until you do one.
BTW, OID 1358 for timespan(time) is still broken:
select timespan('21:11:26'::time);
ERROR: No such function 'time_timespan' with the specified attributes
But I couldn't figure out what it ought to be defined as, so I left it be.
references or CASE expressions, didn't parenthesize complex expressions
properly. Also, always output variable references as fully qualified
names to eliminate ambiguity bug recently reported. (This could be
smarter, but reliability comes first.)
Most parts of the planner should ignore, or indeed never even see, uplevel
Vars because they will be or have been replaced by Params. There were a
couple of places that got it wrong though, probably my fault from recent
changes...
last loop which would return the *first* surviving-to-that-point candidate
regardless of which one actually passed the test. This was producing
such curious results as 'oid % 2' getting translated to 'int2(oid) % 2'.
documented intepretation of the lefthand and oper fields. Fix a number of
obscure problems while at it --- for example, the old code failed if the parser
decided to insert a type-coercion function just below the operator of a
SubLink.
CAUTION: this will break stored rules that contain subplans. You may
need to initdb.
It will keep track the number of pages allocated so that
vacuum could allocate twice of the previous allocation.
This will greatly reduce the total memory consumption of
vacuum.
ALLOC_BIGCHUNK_LIMIT are always allocated as separate malloc() blocks,
and are free()d immediately upon pfree(). Also, if such a chunk is enlarged
with repalloc(), translate the operation into a realloc() so as to
minimize memory usage. Of course, these large chunks still get freed
automatically if the alloc set is reset.
I have set ALLOC_BIGCHUNK_LIMIT at 64K for now, but perhaps another
size would be better?
match then it tried for a self-commutative operator with the reversed input
data types. This is pretty silly; there could never be such an operator,
except maybe in binary-compatible-type scenarios, and we have oper_inexact
for that. Besides which, the oprsanity regress test would complain about
such an operator. Remove nonfunctional code and simplify routine calling
convention accordingly.
and fix_opids processing to a single recursive pass over the plan tree
executed at the very tail end of planning, rather than haphazardly here
and there at different places. Now that tlist Vars do not get modified
until the very end, it's possible to get rid of the klugy var_equal and
match_varid partial-matching routines, and just use plain equal()
throughout the optimizer. This is a step towards allowing merge and
hash joins to be done on expressions instead of only Vars ...
sort order down into planner, instead of handling it only at the very top
level of the planner. This fixes many things. An explicit sort is now
avoided if there is a cheaper alternative (typically an indexscan) not
only for ORDER BY, but also for the internal sort of GROUP BY. It works
even when there is no other reason (such as a WHERE condition) to consider
the indexscan. It works for indexes on functions. It works for indexes
on functions, backwards. It's just so cool...
CAUTION: I have changed the representation of SortClause nodes, therefore
THIS UPDATE BREAKS STORED RULES. You will need to initdb.
above a Sort or Materialize node. As far as I can tell, the only place
that actually needed that was set_tlist_references, which was being lazy
about checking to see if it had a noname node to fix or not...
store all ordering information in pathkeys lists (which are now lists of
lists of PathKeyItem nodes, not just lists of lists of vars). This was
a big win --- the code is smaller and IMHO more understandable than it
was, even though it handles more cases. I believe the node changes will
not force an initdb for anyone; planner nodes don't show up in stored
rules.
commuted (ie, the index var appears on the right). These are now handled
the same way as merge and hash join quals that need to be commuted: the
actual reversing of the clause only happens if we actually choose the path
and generate a plan from it. Furthermore, the clause is only reversed in
the 'indexqual' field of the plan, not in the 'indxqualorig' field. This
allows the clause to still be recognized and removed from qpquals of upper
level join plans. Also, simplify and generalize match_clause_to_indexkey;
now it recognizes binary-compatible indexes for join as well as restriction
clauses.
contains much code that looks like it will handle indexquals with the index
key on either side of the operator, in fact indexquals must have the index
key on the left because of limitations of the ScanKey machinery. Perhaps
someone will be motivated to fix that someday...
work under a wider range of scenarios than it did --- it formerly did not
handle a multi-pass inner scan, nor cases in which the inner scan's
indxqualorig or non-index qual contained outer var references. I am not
sure that these limitations could be hit in the existing optimizer, but
they need to be fixed for future expansion.
> >
> > was implemented by Jan Wieck.
> > His work is for ascending order cases.
> >
> > Here is a patch to prevent sorting also in descending
> > order cases.
> > Because I had already changed _bt_first() to position
> > backward correctly before v6.5,this patch would work.
> >
Hiroshi Inoue
Inoue@tpf.co.jp
multi-scan indexscan plans; it tried to use the same table-to-index
attribute mapping for all the scans, even if they used different indexes.
It would klugily work as long as OR indexquals never used multikey indexes,
but that's not likely to hold up much longer...
to go along with expression_tree_walker. (_walker is not suitable for
routines that need to alter the tree structure significantly.) Other minor
cleanups in clauses.c.
Also, move responsibility for calling vc_abort into main xact.c list of
things-to-call-at-abort. What in the world was it doing down inside of
TransactionIdAbort()?
hashjoinable clause, not one path for a randomly-chosen element of each
set of clauses with the same join operator. That is, if you wrote
SELECT ... WHERE t1.f1 = t2.f2 and t1.f3 = t2.f4,
and both '=' ops were the same opcode (say, all four fields are int4),
then the system would either consider hashing on f1=f2 or on f3=f4,
but it would *not* consider both possibilities. Boo hiss.
Also, revise estimation of hashjoin costs to include a penalty when the
inner join var has a high disbursion --- ie, the most common value is
pretty common. This tends to lead to badly skewed hash bucket occupancy
and way more comparisons than you'd expect on average.
I imagine that the cost calculation still needs tweaking, but at least
it generates a more reasonable plan than before on George Young's example.
(it should just call the given operator, not look up an = operator).
Fix intltsel() so that all numeric data types are converted to double
before trying to estimate where the given comparison value is in the
known range of column values. intltsel() still needs work, or replacement,
for non-numeric data types ... but for nonintegral numeric types it
should now be delivering reasonable estimates.
configure.in to determine if a system is ELF or not. Note that some
of the tests earlier may be redundant but I took the safest route.
D'Arcy J.M. Cain
neqsel now behave as per my suggestions in pghackers a few days ago.
selectivity for < > <= >= should work OK for integral types as well, but
still need work for nonintegral types. Since these routines have never
actually executed before :-(, this may result in some significant changes
in the optimizer's choices of execution plans. Let me know if you see
any serious misbehavior.
CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
so that Case works in WHERE join clauses. Temporary patch --- this routine
is one of many that ought to be changed to use centralized expression-tree-
walking logic.
Bruce Momjian
Tatsuo Ishii
Hiroshi Inoue
Jan Wieck
Michael Meskes
Tom Lane
Vadim B. Mikheev
Marc G. Fournier
Peter Mount
Thomas G. Lockhart
Byron Nikolaidis