before commit, not after :-( --- the original coding is not only unsafe
if an error occurs while it's processing, but it generates an invalid
sequence of WAL entries. Resurrect 7.2 logic for deleting items when
no longer needed. Use an enum instead of random macros. Editorialize
on names used for routines and constants. Teach backend/nodes routines
about new field in CreateTable struct. Add a regression test.
postgres.h or c.h includes a system header (such as stdio.h or
stdlib.h), there's no need to specifically include it in any of the .c
files in the backend.
Neil Conway
principled order; in particular ensure that all shared resources
are released before we release transaction locks. The code used
to release locks before buffer pins, which might explain an ancient
note I have about a bufmgr assertion failure I'd seen once several
years ago, and been unable to reproduce since. (Theory: someone
trying to drop a relation might be able to reach FlushRelationBuffers
before the last user of the relation had gotten around to dropping
his buffer pins.)
item, if the page containing the current item is split while the indexscan
is stopped and holds no read-lock on the page. The current item might
move right onto a page that the indexscan holds no pin on. In the prior
code this would allow btbulkdelete to reach and possibly delete the item,
causing 'my bits moved right off the end of the world!' when the indexscan
finally resumes. Fix by chaining read-locks to the right during
_bt_restscan and requiring btbulkdelete to LockBufferForCleanup on every
page it scans, not only those with deletable items. Per my pghackers
message of 25-May-02. (Too bad no one could think of a better way.)
in our write/flush operation any WAL entries that got queued while we
were waiting to get the WALWriteLock. This improves throughput when
transactions are small enough that several can be committed per WAL
write (ie, per disk revolution).
no reason to worry about the tuple commit status bits until the tuple
is inserted in a relation by heapam.c. Also, improve comments for
heap_addheader().
VACUUM FULL tuple moves. Store full-width t_infomask in WAL, rather
than storing low 8 bits and expecting to be able to reconstruct upper
bits. While at it, remove redundant t_oid field from WAL headers
(the OID, if present, is now recorded in the data portion of the tuple).
WAL version number bumped --- this does not force an initdb, you can
instead run pg_resetxlog after a clean shutdown of the old postmaster.
(overlaying low byte of page size) and add HEAP_HASOID bit to t_infomask,
per earlier discussion. Simplify scheme for overlaying fields in tuple
header (no need for cmax to live in more than one place). Don't try to
clear infomask status bits in tqual.c --- not safe to do it there. Don't
try to force output table of a SELECT INTO to have OIDs, either. Get rid
of unnecessarily complex three-state scheme for TupleDesc.tdhasoids, which
has already caused one recent failure. Improve documentation.
to false provides more SQL-spec-compliant behavior than we had before.
I am not sure that setting it false is actually a good idea yet; there
is a lot of client-side code that will probably be broken by turning
autocommit off. But it's a start.
Loosely based on a patch by David Van Wie.
column additions, deletions, and renames that would let a child table
get out of sync with its parent. Patch by Alvaro Herrera, with some
kibitzing by Tom Lane.
width types and varlena types, since with the introduction of CSTRING as
a more-or-less-real type, these concepts aren't identical. I've tried to
use varlena consistently to denote datatypes with typlen = -1, ie, they
have a length word and are potentially TOASTable; while the term variable
width covers both varlena and cstring (and, perhaps, someday other types
with other rules for computing the actual width). No code changes in this
commit except for renaming a couple macros.
value '-2' is used to indicate a variable-width type whose width is
computed as strlen(datum)+1. Everything that looks at typlen is updated
except for array support, which Joe Conway is working on; at the moment
it wouldn't work to try to create an array of cstring.
with OPAQUE, as per recent pghackers discussion. I still want to do some
more work on the 'cstring' pseudo-type, but I'm going to commit the bulk
of the changes now before the tree starts shifting under me ...
heap_mark4update; this avoids situations where a deleted tuple might
look like it is chained to something else. Also, cause all the WAL
redo routines to set t_ctid to equal t_self, rather than leaving it
undefined as before. Make heap_xlog_clean set the page's LSN and SUI
correctly. All per past discussions in pghackers, ranging back to
last December.
offset past the last-used-item-plus-one, since that would result in
leaving uninitialized holes in the item pointer array. AFAICT the only
place that was depending on this was btree index build, which was being
cavalier about when to fill in the P_HIKEY pointer; easily fixed.
Also a small performance improvement: shuffle itemid's by means of
memmove, not a one-at-a-time loop.
The local buffer manager is no longer used for newly-created relations
(unless they are TEMP); a new non-TEMP relation goes through the shared
bufmgr and thus will participate normally in checkpoints. But TEMP relations
use the local buffer manager throughout their lifespan. Also, operations
in TEMP relations are not logged in WAL, thus improving performance.
Since it's no longer necessary to fsync relations as they move out of the
local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code
is no longer needed and has been removed: there's no concept of a dirty
relation anymore in md.c/fd.c, and we never fsync anything but WAL.
Still TODO: improve local buffer management algorithms so that it would
be reasonable to increase NLocBuffer.
of functions returning domain types, update documentation for typtype,
move get_typtype to lsyscache.c (actually, resurrect the old version),
add defense against creating pseudo-typed table columns, fix some
bogus list-parsing in grammar. Issues remain with respect to alias
handling and type checking; Joe is on those.
types for Table Functions, as previously proposed on HACKERS. Here is a
brief explanation:
1. Creates a new pg_type typtype: 'p' for pseudo type (currently either
'b' for base or 'c' for catalog, i.e. a class).
2. Creates new builtin type of typtype='p' named RECORD. This is the
first of potentially several pseudo types.
3. Modify FROM clause grammer to accept:
SELECT * FROM my_func() AS m(colname1 type1, colname2 type1, ...)
where m is the table alias, colname1, etc are the column names, and
type1, etc are the column types.
4. When typtype == 'p' and the function return type is RECORD, a list
of column defs is required, and when typtype != 'p', it is
disallowed.
5. A check was added to ensure that the tupdesc provide via the parser
and the actual return tupdesc match in number and type of
attributes.
When creating a function you can do:
CREATE FUNCTION foo(text) RETURNS setof RECORD ...
When using it you can do:
SELECT * from foo(sqlstmt) AS (f1 int, f2 text, f3 timestamp)
or
SELECT * from foo(sqlstmt) AS f(f1 int, f2 text, f3 timestamp)
or
SELECT * from foo(sqlstmt) f(f1 int, f2 text, f3 timestamp)
Included in the patches are adjustments to the regression test sql and
expected files, and documentation.
p.s.
This potentially solves (or at least improves) the issue of builtin
Table Functions. They can be bootstrapped as returning RECORD, and
we can wrap system views around them with properly specified column
defs. For example:
CREATE VIEW pg_settings AS
SELECT s.name, s.setting
FROM show_all_settings()AS s(name text, setting text);
Then we can also add the UPDATE RULE that I previously posted to
pg_settings, and have pg_settings act like a virtual table, allowing
settings to be queried and set.
Joe Conway
in the relcache. It's rather silly that we have reference count leak
checks in bufmgr and in catcache, but not in relcache which will normally
have many fewer entries. Chris K-L would have caught at least one bug
in his recent DROP patch if he'd had this.
code review by Tom Lane. Remaining issues: functions that take or
return tuple types are likely to break if one drops (or adds!)
a column in the table defining the type. Need to think about what
to do here.
Along the way: some code review for recent COPY changes; mark system
columns attnotnull = true where appropriate, per discussion a month ago.
attstattarget to indicate 'use the default'. The default is now a GUC
variable default_statistics_target, and so may be changed on the fly. Along
the way we gain the ability to have pg_dump dump the per-column statistics
target when it's not the default. Patch by Neil Conway, with some kibitzing
from Tom Lane.
bitmap, if present).
Per Tom Lane's suggestion the information whether a tuple has an oid
or not is carried in the tuple descriptor. For debugging reasons
tdhasoid is of type char, not bool. There are predefined values for
WITHOID, WITHOUTOID and UNDEFOID.
This patch has been generated against a cvs snapshot from last week
and I don't expect it to apply cleanly to current sources. While I
post it here for public review, I'm working on a new version against a
current snapshot. (There's been heavy activity recently; hope to
catch up some day ...)
This is a long patch; if it is too hard to swallow, I can provide it
in smaller pieces:
Part 1: Accessor macros
Part 2: tdhasoid in TupDesc
Part 3: Regression test
Part 4: Parameter withoid to heap_addheader
Part 5: Eliminate t_oid from HeapTupleHeader
Part 2 is the most hairy part because of changes in the executor and
even in the parser; the other parts are straightforward.
Up to part 4 the patched postmaster stays binary compatible to
databases created with an unpatched version. Part 5 is small (100
lines) and finally breaks compatibility.
Manfred Koizar
>usefulness.
>
>> [...] should I post a patch that puts pagesize directly into
>> PageHeaderData?
>
>If you're so inclined. Given that pd_opaque is hidden in those macros,
>there wouldn't be much of any gain in readability either, so I haven't
>worried about changing the declaration.
Thanks for the clarification. Here is the patch. Not much gain, but at
least it saves the next junior hacker from scratching his head ...
Manfred Koizar
all places, where pd_linp is accessed. Also introduce new macros
SizeOfPageHeaderData and BTMaxItemSize. This is just source code
cosmetic, no behaviour changed.
Manfred Koizar
patch from 2002-06-10, is supposed to reduce the heap tuple header size
by four bytes on most architectures. Of course it changes the on-disk
tuple format and therefore requires initdb.
This overlays cmin/cmax/xmax fields into only two fields.
Manfred Koizar
two small changes to the API since last patch, which hopefully completes
the decoupling of composite function support from SRF specific support.
Joe Conway
HeapTupleHeaderData in setter and getter macros called
HeapTupleHeaderGetXmin, HeapTupleHeaderSetXmin etc.
It also introduces a "virtual" field xvac by defining
HeapTupleHeaderGetXvac and HeapTupleHeaderSetXvac. Xvac is used by
VACUUM, in fact it is stored in t_cmin.
Manfred Koizar
> Changes to avoid collisions with WIN32 & MFC names...
> 1. Renamed:
> a. PROC => PGPROC
> b. GetUserName() => GetUserNameFromId()
> c. GetCurrentTime() => GetCurrentDateTime()
> d. IGNORE => IGNORE_DTF in include/utils/datetime.h & utils/adt/datetim
>
> 2. Added _P to some lex/yacc tokens:
> CONST, CHAR, DELETE, FLOAT, GROUP, IN, OUT
Jan
where the latter is made slightly larger to allow for in-memory tuples
containing resjunk attributes. Responds to today's complaint that one
cannot UPDATE a table containing the allegedly-legal maximum number of
columns.
Also, apply Manfred Koizar's recent patch to avoid extra alignment padding
when there is a null bitmap. This saves bytes in some cases while not
creating any backward-compatibility problem AFAICS.
transaction, so as to avoid returning them out of the index AM. Saves
repeated heap_fetch operations on frequently-updated rows. Also detect
queries on unique keys (equality to all columns of a unique index), and
don't bother continuing scan once we have found first match.
Killing is implemented in the btree and hash AMs, but not yet in rtree
or gist, because there isn't an equally convenient place to do it in
those AMs (the outer amgetnext routine can't do it without re-pinning
the index page).
Did some small cleanup on APIs of HeapTupleSatisfies, heap_fetch, and
index_insert to make this a little easier.
exemplified by bug #671. Moving the storage to relcache turned out to
be a bad idea because relcache might decide to discard the info. Instead,
open and close the relcache entry on each sequence operation, and use
a record of the current XID to discover whether we already hold
AccessShareLock on the sequence.
in snapshots, per my proposal of a few days ago. Also, tweak heapam.c
routines (heap_insert, heap_update, heap_delete, heap_mark4update) to
be passed the command ID to use, instead of doing GetCurrentCommandID.
For catalog updates they'll still get passed current command ID, but
for updates generated from the main executor they'll get passed the
command ID saved in the snapshot the query is using. This should fix
some corner cases associated with functions and triggers that advance
current command ID while an outer query is still in progress.
yesterday's proposal to pghackers. Also remove unnecessary parameters
to heap_beginscan, heap_rescan. I modified pg_proc.h to reflect the
new numbers of parameters for the AM interface routines, but did not
force an initdb because nothing actually looks at those fields.
allows the example in the CREATE SCHEMA ref page to actually work now.
Also, clean up when the transaction that initially creates a temp-table
namespace is later aborted. Simplify internal representation of search
path by folding special cases into the main list.
GUC support. It's now possible to set datestyle, timezone, and
client_encoding from postgresql.conf and per-database or per-user
settings. Also, implement rollback of SET commands that occur in a
transaction that later fails. Create a SET LOCAL var = value syntax
that sets the variable only for the duration of the current transaction.
All per previous discussions in pghackers.
compiled for integer date/time storage and to check the length of
storage for the locale fields in the same data structure.
Slightly reword some of the error messages to be more accurate on
possible recovery options (e.g. recompile *or* re-initdb).
Bump version number on this file.
an 'opclass owner' column in pg_opclass. Nothing is done with it at
present, but since there are plans to invent a CREATE OPERATOR CLASS
command soon, we'll probably want DROP OPERATOR CLASS too, which
suggests that a notion of ownership would be a good idea.
path. The default behavior if no per-user schemas are created is that
all users share a 'public' namespace, thus providing behavior backwards
compatible with 7.2 and earlier releases. Probably the semantics and
default setting will need to be fine-tuned, but this is a start.
in schemas other than the system namespace; however, there's no search
path yet, and not all operations work yet on tables outside the system
namespace.
per recent pghackers discussion: force a new WAL record at first nextval
after a checkpoint, and ensure that xlog is flushed to disk if a nextval
record is the only thing emitted by a transaction.
PostgreSQL. This hash function replaces the one used by hash indexes and
the catalog cache. Hash joins use a different, relatively poor-quality
hash function, but I'll fix that later.
As suggested by Tom Lane, this patch also changes the size of the fixed
hash table used by the catalog cache to be a power-of-2 (instead of a
prime: I chose 256 instead of 257). This allows the catcache to lookup
hash buckets using a simple bitmask. This should improve the performance
of the catalog cache slightly, since the previous method (modulo a
prime) was slow.
In my tests, this improves the performance of hash indexes by between 4%
and 8%; the performance when using btree indexes or seqscans is
basically unchanged.
Neil Conway <neilconway@rogers.com>
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
(current as of a few hours ago.)
This patch:
1. Adds PG_GETARG_xxx_P_SLICE() macros and associated support routines.
2. Adds routines in src/backend/access/tuptoaster.c for fetching only
necessary chunks of a toasted value. (Modelled on latest changes to
assume chunks are returned in order).
3. Amends text_substr and bytea_substr to use new methods. It now
handles multibyte cases -and should still lead to a performance
improvement in the multibyte case where the substring is near the
beginning of the string.
4. Added new command: ALTER TABLE tabname ALTER COLUMN colname SET
STORAGE {PLAIN | EXTERNAL | EXTENDED | MAIN} to parser and documented in
alter-table.sgml. (NB I used ColId as the item type for the storage
mode string, rather than a new production - I hope this makes sense!).
All this does is sets attstorage for the specified column.
4. AlterTableAlterColumnStatistics is now AlterTableAlterColumnFlags and
handles both statistics and storage (it uses the subtype code to
distinguish). The previous version of my patch also re-arranged other
code in backend/commands/command.c but I have dropped that from this
patch.(I plan to return to it separately).
5. Documented new macros (and also the PG_GETARG_xxx_P_COPY macros) in
xfunc.sgml. ref/alter_table.sgml also contains documentation for ALTER
COLUMN SET STORAGE.
John Gray
"bad smell" in that code. Stuff like function parameters that aren't
used, typos in the comments, comparison between signed and unsigned
ints, etc.
Attached is a pretty trivial patch; it compiles, but beyond that
completely untested. Unless anyone sees any problems, please apply for
7.3.
Neil Conway
speed up repetitive failed searches; per pghackers discussion in late
January. inval.c logic substantially simplified, since we can now treat
inserts and deletes alike as far as inval events are concerned. Some
repair work needed in heap_create_with_catalog, which turns out to have
been doing CommandCounterIncrement at a point where the new relation has
non-self-consistent catalog entries. With the new inval code, that
resulted in assert failures during a relcache entry rebuild.
now just below FATAL in server_min_messages. Added more text to
highlight ordering difference between it and client_min_messages.
---------------------------------------------------------------------------
REALLYFATAL => PANIC
STOP => PANIC
New INFO level the prints to client by default
New LOG level the prints to server log by default
Cause VACUUM information to print only to the client
NOTICE => INFO where purely information messages are sent
DEBUG => LOG for purely server status messages
DEBUG removed, kept as backward compatible
DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1 added
DebugLvl removed in favor of new DEBUG[1-5] symbols
New server_min_messages GUC parameter with values:
DEBUG[5-1], INFO, NOTICE, ERROR, LOG, FATAL, PANIC
New client_min_messages GUC parameter with values:
DEBUG[5-1], LOG, INFO, NOTICE, ERROR, FATAL, PANIC
Server startup now logged with LOG instead of DEBUG
Remove debug_level GUC parameter
elog() numbers now start at 10
Add test to print error message if older elog() values are passed to elog()
Bootstrap mode now has a -d that requires an argument, like postmaster
are now both invoked once per received SQL command (raw parsetree) from
pg_exec_query_string. BeginCommand is actually just an empty routine
at the moment --- all its former operations have been pushed into tuple
receiver setup routines in printtup.c. This makes for a clean distinction
between BeginCommand/EndCommand (once per command) and the tuple receiver
setup/teardown routines (once per ExecutorRun call), whereas the old code
was quite ad hoc. Along the way, clean up the calling conventions for
ExecutorRun a little bit.
hashname() and reduce the penalty incured when NAMEDATALEN is increased.
I posted this to -hackers a couple days ago, and there haven't been any
major complaints. It passes the regression tests. See -hackers for more
discussion, as well as the suggestion from Tom Lane on which this patch
is based.
Unless anyone sees any problems, please apply for 7.3.
Cheers,
Neil Conway
Improve 'pg_internal.init' relcache entry preload mechanism so that it is
safe to use for all system catalogs, and arrange to preload a realistic
set of system-catalog entries instead of only the three nailed-in-cache
indexes that were formerly loaded this way. Fix mechanism for deleting
out-of-date pg_internal.init files: this must be synchronized with transaction
commit, not just done at random times within transactions. Drive it off
relcache invalidation mechanism so that no special-case tests are needed.
Cache additional information in relcache entries for indexes (their pg_index
tuples and index-operator OIDs) to eliminate repeated lookups. Also cache
index opclass info at the per-opclass level to avoid repeated lookups during
relcache load.
Generalize 'systable scan' utilities originally developed by Hiroshi,
move them into genam.c, use in a number of places where there was formerly
ugly code for choosing either heap or index scan. In particular this allows
simplification of the logic that prevents infinite recursion between syscache
and relcache during startup: we can easily switch to heapscans in relcache.c
when and where needed to avoid recursion, so IndexScanOK becomes simpler and
does not need any expensive initialization.
Eliminate useless opening of a heapscan data structure while doing an indexscan
(this saves an mdnblocks call and thus at least one kernel call).
of pointers is required. Patch from Teodor Sigaev per pghackers
discussion. It's an ugly kluge but avoids forcing initdb; we'll put
a better fix into 7.3 or later.
mess up after an aborted VACUUM FULL, per today's pghackers discussion.
Add a suitable HeapTupleSatisfiesToast routine. Remove useless special-
case test in HeapTupleSatisfiesVisibility macro for xmax =
BootstrapTransactionId; perhaps that was needed at one time, but it's
a waste of cycles now, not to mention actively wrong for SnapshotAny.
Along the way, add some much-needed comments to tqual.c, and simplify
toast_fetch_datum, which no longer needs to assume it may see chunks
out-of-order.
to prevent spreading of corruption when page header pointers are bad.
Merge PageZero into PageInit, since it was never used separately, and
remove separate memset calls used at most other PageInit call points.
Remove IndexPageCleanup, which wasn't used at all.
per my proposal of a couple days ago. This will eliminate the unable-
to-restart-database class of problem that we have seen reported half a
dozen times with 7.1.*.
to insert the same key into a supposedly unique index. The bug is of
low probability, and may not explain any of the recent reports of
duplicated rows; but a bug is a bug.
'volatile' pointers to access those structures, so that optimizing
compilers will not decide to move the structure accesses outside of the
spinlock-acquire-to-spinlock-release sequence. There are no known bugs
in these uses at present, but based on bad experience with lwlock.c,
it seems prudent to ensure that we protect these other uses too.
Per pghackers discussion around 12-Dec. (Note: it should not be
necessary to worry about structures protected by LWLocks, since the
LWLock acquire and release operations are not inline macros.)
do not use the undo pointer anyway. This is a quick-hack solution for
the three-way deadlock condition discussed in pghackers 17-Dec-01.
Need to find a better way of doing it.
where rightmost index page splits while we are waiting to obtain exclusive
lock on it. Not clear this would actually hurt (probably the callback
would always fail), but better safe than sorry.
Also, improve comments describing concurrency considerations in this code.
so that only one signal number is used not three. Flags in shared
memory tell the reason(s) for the current signal. This method is
extensible to handle more signal reasons without chewing up even more
signal numbers, but the immediate reason is to keep pg_pwd reloads
separate from SIGHUP processing in the postmaster.
Also clean up some problems in the postmaster with delayed response to
checkpoint status changes --- basically, it wouldn't schedule a checkpoint
if it wasn't getting connection requests on a regular basis.
recreated since the start of our transaction, our first reference to it
errored out because we'd try to reuse our old relcache entry for it.
Do this by accepting SI inval messages just before relcache search in
heap_openr, so that dead relcache entries will be flushed before we
search. Also, break heap_open/openr into two pairs of routines,
relation_open(r) and heap_open(r). The relation_open routines make
no tests on relkind and so can be used to open anything that has a
pg_class entry. The heap_open routines are wrappers that add a relkind
test to preserve their established behavior. Use the relation_open
routines in several places that had various kluge solutions for opening
rels that might be either heap or index rels.
Also, remove the old 'heap stats' code that's been superseded by Jan's
stats collector, and clean up some inconsistencies in error reporting
between the different types of ALTER TABLE.
Modified the parser and the SET handlers to use full Node structures
rather than simply a character string argument.
Implement INTERVAL() YEAR TO MONTH (etc) syntax per SQL99.
Does not yet accept the goofy string format that goes along with, but
this should be fairly straight forward to fix now as a bug or later
as a feature.
Implement precision for the INTERVAL() type.
Use the typmod mechanism for both of INTERVAL features.
Fix the INTERVAL syntax in the parser:
opt_interval was in the wrong place.
INTERVAL is now a reserved word, otherwise we get reduce/reduce errors.
Implement an explicit date_part() function for TIMETZ.
Should fix coersion problem with INTERVAL reported by Peter E.
Fix up some error messages for date/time types.
Use all caps for type names within message.
Fix recently introduced side-effect bug disabling 'epoch' as a recognized
field for date_part() etc. Reported by Peter E. (??)
Bump catalog version number.
Rename "microseconds" current transaction time field
from ...Msec to ...Usec. Duh!
date/time regression tests updated for reference platform, but a few
changes will be necessary for others.
lookup info in the relcache for index access method support functions.
This makes a huge difference for dynamically loaded support functions,
and should save a few cycles even for built-in ones. Also tweak dfmgr.c
so that load_external_function is called only once, not twice, when
doing fmgr_info for a dynamically loaded function. All per performance
gripe from Teodor Sigaev, 5-Oct-01.
readability. Bizarre '(long *) TRUE' return convention is gone,
in favor of just raising an error internally in dynahash.c when
we detect hashtable corruption. HashTableWalk is gone, in favor
of using hash_seq_search directly, since it had no hope of working
with non-LONGALIGNable datatypes. Simplify some other code that was
made undesirably grotty by promixity to HashTableWalk.
portability issues). Caller-visible data structures are now allocated
on MAXALIGN boundaries, allowing safe use of datatypes wider than 'long'.
Rejigger hash_create API so that caller specifies size of key and
total size of entry, not size of key and size of rest of entry.
This simplifies life considerably since each number is just a sizeof(),
and padding issues etc. are taken care of automatically.
rightmost on its tree level, we split 2/3 to the left and 1/3 to the
new right page, rather than the even split we use elsewhere. The idea
is that when faced with a steadily increasing series of inserted keys
(such as sequence or timestamp values), we'll end up with a btree that's
about 2/3ds full not 1/2 full, which is much closer to the desired
steady-state load for a btree. Per suggestion from Ann Harrison of
IBPhoenix.
existing lock manager and spinlocks: it understands exclusive vs shared
lock but has few other fancy features. Replace most uses of spinlocks
with lightweight locks. All remaining uses of spinlocks have very short
lock hold times (a few dozen instructions), so tweak spinlock backoff
code to work efficiently given this assumption. All per my proposal on
pghackers 26-Sep-01.
two performance improvements. I put an explanation of the
changes at
http://cs1.cs.nyu.edu/been/postgres-rtree.html
The performance improvements are quite significant.
All the changes are in the file src/backend/access/rtree/rtree.c
I was working with the 7.1.3 code.
I'm including the diff output as an attachment.
Kenneth Been
Define a new function, GetCurrentTransactionStartTimeUsec() to get the time
to this precision.
Allow now() and timestamp 'now' to use this higher precision result so
we now have fractional seconds in this "constant".
Add timestamp without time zone type.
Move previous timestamp type to timestamp with time zone.
Accept another ISO variant for date/time values: yyyy-mm-ddThh:mm:ss
(note the "T" separating the day from hours information).
Remove 'current' from date/time types; convert to 'now' in input.
Separate time and timetz regression tests.
Separate timestamp and timestamptz regression test.
Standardize on %X/%X as the formatting for XLOG position display --- we
had a couple of different formats before, and none of 'em were as useful
as hex offsets IMHO.
buffer manager with 'pg_clog', a specialized access method modeled
on pg_xlog. This simplifies startup (don't need to play games to
open pg_log; among other things, OverrideTransactionSystem goes away),
should improve performance a little, and opens the door to recycling
commit log space by removing no-longer-needed segments of the commit
log. Actual recycling is not there yet, but I felt I should commit
this part separately since it'd still be useful if we chose not to
do transaction ID wraparound.
pgsql-hackers. pg_opclass now has a row for each opclass supported by each
index AM, not a row for each opclass name. This allows pg_opclass to show
directly whether an AM supports an opclass, and furthermore makes it possible
to store additional information about an opclass that might be AM-dependent.
pg_opclass and pg_amop now store "lossy" and "haskeytype" information that we
previously expected the user to remember to provide in CREATE INDEX commands.
Lossiness is no longer an index-level property, but is associated with the
use of a particular operator in a particular index opclass.
Along the way, IndexSupportInitialize now uses the syscaches to retrieve
pg_amop and pg_amproc entries. I find this reduces backend launch time by
about ten percent, at the cost of a couple more special cases in catcache.c's
IndexScanOK.
Initial work by Oleg Bartunov and Teodor Sigaev, further hacking by Tom Lane.
initdb forced.
default, but OIDS are removed from many system catalogs that don't need them.
Some interesting side effects: TOAST pointers are 20 bytes not 32 now;
pg_description has a three-column key instead of one.
Bugs fixed in passing: BINARY cursors work again; pg_class.relhaspkey
has some usefulness; pg_dump dumps comments on indexes, rules, and
triggers in a valid order.
initdb forced.
(as proposed in http://fts.postgresql.org/db/mw/msg.html?mid=1028327)
2. support for 'pass-by-value' arguments - to test this
we used special opclass for int4 with values in range [0-2^15]
More testing will be done after resolving problem with
index_formtuple and implementation of B-tree using GiST
3. small patch to contrib modules (seg,cube,rtree_gist,intarray) -
mark functions as 'isstrict' where needed.
Oleg Bartunov
rather than deleting them only to have to create more. Steady state
is 2*CHECKPOINT_SEGMENTS + WAL_FILES + 1 segment files, which will
simply be renamed rather than constantly deleted and recreated.
To make this safe, added current XLOG file/offset number to page
header of XLOG pages, so that an un-overwritten page from an old
incarnation of a logfile can be reliably told from a valid page.
This change means that if you try to restart postmaster in a CVS-tip
database after installing the change, you'll get a complaint about
bad XLOG page magic number. If you don't want to initdb, run
contrib/pg_resetxlog (and be sure you shut down the old postmaster
cleanly).
in GetSnapshotData, GetNewTransactionId, CommitTransaction, AbortTransaction,
etc. Correct race condition in transaction status testing in
HeapTupleSatisfiesVacuum --- this wasn't important for old VACUUM with
exclusive lock on its table, but it sure is important now. All per
pghackers discussion 7/11/01 and 7/12/01.
per previous discussion on pghackers. Most of the duplicate code in
different AMs' ambuild routines has been moved out to a common routine
in index.c; this means that all index types now do the right things about
inserting recently-dead tuples, etc. (I also removed support for EXTEND
INDEX in the ambuild routines, since that's about to go away anyway, and
it cluttered the code a lot.) The retail indextuple deletion routines have
been replaced by a "bulk delete" routine in which the indexscan is inside
the access method. I haven't pushed this change as far as it should go yet,
but it should allow considerable simplification of the internal bookkeeping
for deletions. Also, add flag columns to pg_am to eliminate various
hardcoded tests on AM OIDs, and remove unused pg_am columns.
Fix rtree and gist index types to not attempt to store NULLs; before this,
gist usually crashed, while rtree managed not to crash but computed wacko
bounding boxes for NULL entries (which might have had something to do with
the performance problems we've heard about occasionally).
Add AtEOXact routines to hash, rtree, and gist, all of which have static
state that needs to be reset after an error. We discovered this need long
ago for btree, but missed the other guys.
Oh, one more thing: concurrent VACUUM is now the default.
validity checking rules for VACUUM. Make some other rearrangements of the
VACUUM code to allow more code to be shared between full and lazy VACUUM.
Minor code cleanups and added comments for TransactionId manipulations.
stub) into the rest of the system. Adopt a cleaner approach to preventing
deadlock in concurrent heap_updates: allow RelationGetBufferForTuple to
select any page of the rel, and put the onus on it to lock both buffers
in a consistent order. Remove no-longer-needed isExtend hack from
API of ReleaseAndReadBuffer.
do anything yet, but it has the necessary connections to initialization
and so forth. Make some gestures towards allowing number of blocks in
a relation to be BlockNumber, ie, unsigned int, rather than signed int.
(I doubt I got all the places that are sloppy about it, yet.) On the
way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC
variable.
IS TRUE, etc, with some degree of verisimilitude. Split out
selectivity support functions from builtins.h into a new header
file selfuncs.h, so as to reduce the number of header files builtins.h
must depend on. Fix a few missing inclusions exposed thereby.
From Joe Conway, with some kibitzing from Tom Lane.
SI messages now include the relevant database OID, so that operations
in one database do not cause useless cache flushes in backends attached
to other databases. Declare SI messages properly using a union, to
eliminate the former assumption that Oid is the same size as int or Index.
Rewrite the nearly-unreadable code in inval.c, and document it better.
Arrange for catcache flushes at end of command/transaction to happen before
relcache flushes do --- this avoids loading a new tuple into the catcache
while setting up new relcache entry, only to have it be flushed again
immediately.
pg_database now has unique indexes on oid and on datname.
pg_shadow now has unique indexes on usename and on usesysid.
pg_am now has unique index on oid.
pg_opclass now has unique index on oid.
pg_amproc now has unique index on amid+amopclaid+amprocnum.
Remove pg_rewrite's unnecessary index on oid, delete unused RULEOID syscache.
Remove index on pg_listener and associated syscache for performance reasons
(caching rows that are certain to change before you need 'em again is
rather pointless).
Change pg_attrdef's nonunique index on adrelid into a unique index on
adrelid+adnum.
Fix various incorrect settings of pg_class.relisshared, make that the
primary reference point for whether a relation is shared or not.
IsSharedSystemRelationName() is now only consulted to initialize relisshared
during initial creation of tables and indexes. In theory we might now
support shared user relations, though it's not clear how one would get
entries for them into pg_class &etc of multiple databases.
Fix recently reported bug that pg_attribute rows created for an index all have
the same OID. (Proof that non-unique OID doesn't matter unless it's
actually used to do lookups ;-))
There's no need to treat pg_trigger, pg_attrdef, pg_relcheck as bootstrap
relations. Convert them into plain system catalogs without hardwired
entries in pg_class and friends.
Unify global.bki and template1.bki into a single init script postgres.bki,
since the alleged distinction between them was misleading and pointless.
Not to mention that it didn't work for setting up indexes on shared
system relations.
Rationalize locking of pg_shadow, pg_group, pg_attrdef (no need to use
AccessExclusiveLock where ExclusiveLock or even RowExclusiveLock will do).
Also, hold locks until transaction commit where necessary.
appropriate pin-count manipulation, and instead use ReleaseAndReadBuffer.
Make use of the fact that the passed-in buffer (if there is one) must
be pinned to avoid grabbing the bufmgr spinlock when we are able to
return this same buffer. Eliminate unnecessary 'previous tuple' and
'next tuple' fields of HeapScanDesc and IndexScanDesc, thereby removing
a whole lot of bookkeeping from heap_getnext() and related routines.
checkpoint's redo pointer, not its undo pointer, per discussion in
pghackers a few days ago. No point in hanging onto undo information
until we have the ability to do something with it --- and this solves
a rather large problem with log space for long-running transactions.
Also, change all calls of write() to detect the case where write
returned a count less than requested, but failed to set errno.
Presume that this situation indicates ENOSPC, and give the appropriate
error message, rather than a random message associated with the previous
value of errno.
report on old-style functions invoked by RI triggers. We had a number of
other places that were being sloppy about which memory context FmgrInfo
subsidiary data will be allocated in. Turns out none of them actually
cause a problem in 7.1, but this is for arcane reasons such as the fact
that old-style triggers aren't supported anyway. To avoid getting burnt
later, I've restructured the trigger support so that we don't keep trigger
FmgrInfo structs in relcache memory. Some other related cleanups too:
it's not really necessary to call fmgr_info at all while setting up
the index support info in relcache entries, because those ScanKeyEntry
structs are never used to invoke the functions. This should speed up
relcache initialization a tiny bit.
Python) to support shared extension modules, I have learned that Guido
prefers the style of the attached patch to solve the above problem.
I feel that this solution is particularly appropriate in this case
because the following:
PglargeType
PgType
PgQueryType
are already being handled in the way that I am proposing for PgSourceType.
Jason Tishler
Bruce Momjian
Tom Lane
Peter Eisentraut
Thomas G. Lockhart
Jan Wieck
Hiroshi Inoue