The local buffer manager is no longer used for newly-created relations
(unless they are TEMP); a new non-TEMP relation goes through the shared
bufmgr and thus will participate normally in checkpoints. But TEMP relations
use the local buffer manager throughout their lifespan. Also, operations
in TEMP relations are not logged in WAL, thus improving performance.
Since it's no longer necessary to fsync relations as they move out of the
local buffers into shared buffers, quite a lot of smgr.c/md.c/fd.c code
is no longer needed and has been removed: there's no concept of a dirty
relation anymore in md.c/fd.c, and we never fsync anything but WAL.
Still TODO: improve local buffer management algorithms so that it would
be reasonable to increase NLocBuffer.
hardwired lists of index names for each catalog, use the relcache's
mechanism for caching lists of OIDs of indexes of any table. This
reduces the common case of updating system catalog indexes to a single
line, makes it much easier to add a new system index (in fact, you
can now do so on-the-fly if you want to), and as a nice side benefit
improves performance a little. Per recent pghackers discussion.
of functions returning domain types, update documentation for typtype,
move get_typtype to lsyscache.c (actually, resurrect the old version),
add defense against creating pseudo-typed table columns, fix some
bogus list-parsing in grammar. Issues remain with respect to alias
handling and type checking; Joe is on those.
in the relcache. It's rather silly that we have reference count leak
checks in bufmgr and in catcache, but not in relcache which will normally
have many fewer entries. Chris K-L would have caught at least one bug
in his recent DROP patch if he'd had this.
code review by Tom Lane. Remaining issues: functions that take or
return tuple types are likely to break if one drops (or adds!)
a column in the table defining the type. Need to think about what
to do here.
Along the way: some code review for recent COPY changes; mark system
columns attnotnull = true where appropriate, per discussion a month ago.
bitmap, if present).
Per Tom Lane's suggestion the information whether a tuple has an oid
or not is carried in the tuple descriptor. For debugging reasons
tdhasoid is of type char, not bool. There are predefined values for
WITHOID, WITHOUTOID and UNDEFOID.
This patch has been generated against a cvs snapshot from last week
and I don't expect it to apply cleanly to current sources. While I
post it here for public review, I'm working on a new version against a
current snapshot. (There's been heavy activity recently; hope to
catch up some day ...)
This is a long patch; if it is too hard to swallow, I can provide it
in smaller pieces:
Part 1: Accessor macros
Part 2: tdhasoid in TupDesc
Part 3: Regression test
Part 4: Parameter withoid to heap_addheader
Part 5: Eliminate t_oid from HeapTupleHeader
Part 2 is the most hairy part because of changes in the executor and
even in the parser; the other parts are straightforward.
Up to part 4 the patched postmaster stays binary compatible to
databases created with an unpatched version. Part 5 is small (100
lines) and finally breaks compatibility.
Manfred Koizar
extension to create binary compatible casts. Includes dependency tracking
as well.
pg_proc.proimplicit is now defunct, but will be removed in a separate
commit.
pg_dump provides a migration path from the previous scheme to declare
casts. Dumping binary compatible casts is currently impossible, though.
pg_relcheck is gone; CHECK, UNIQUE, PRIMARY KEY, and FOREIGN KEY
constraints all have real live entries in pg_constraint. pg_depend
exists, and RESTRICT/CASCADE options work on most kinds of DROP;
however, pg_depend is not yet very well populated with dependencies.
(Most of the ones that are present at this point just replace formerly
hardwired associations, such as the implicit drop of a relation's pg_type
entry when the relation is dropped.) Need to add more logic to create
dependency entries, improve pg_dump to dump constraints in place of
indexes and triggers, and add some regression tests.
This is the first cut toward CREATE CONVERSION/DROP CONVERSION implementaion.
The commands can now add/remove tuples to the new pg_conversion system
catalog, but that's all. Still need work to make them actually working.
Documentations, regression tests also need work.
are managed as per request.
Moved from merging with table attributes to applying themselves during
coerce_type() and coerce_type_typmod.
Regression tests altered to test the cast() scenarios.
Rod Taylor
returns-set boolean field in Func and Oper nodes. This allows cleaner,
more reliable tests for expressions returning sets in the planner and
parser. For example, a WHERE clause returning a set is now detected
and complained of in the parser, not only at runtime.
pg_database, pg_shadow, pg_group, all of which now have potentially-long
fields. Along the way, get rid of SharedSystemRelationNames list: shared
rels are now identified in their include/pg_catalog/*.h files by a
BKI_SHARED_RELATION macro, while indexes and toast rels inherit sharedness
automatically from their parent table. Fix some bugs with failure to detoast
pg_group.grolist during ALTER GROUP.
messages more uniform and internationalizable: the global array
aclcheck_error_strings[] is gone in favor of a subroutine
aclcheck_error(). Partial implementation of namespace-related
permission checks --- not all done yet.
per pghackers discussion. Add some more typsanity tests, and clean
up some problems exposed thereby (broken or missing array types for
some built-in types). Also, clean up loose ends from unknownin/out
patch.
different privilege bits (might as well make use of the space we were
wasting on padding). EXECUTE and USAGE bits for procedures, languages
now are separate privileges instead of being overlaid on SELECT. Add
privileges for namespaces and databases. The GRANT and REVOKE commands
work for these object types, but we don't actually enforce the privileges
yet...
(tgrelid, tgname). This provides an additional check on trigger name
uniqueness per-table (which was already enforced by the code anyway).
With this change, RelationBuildTriggers will read the triggers in
order by tgname, since it's scanning using this index. Since a
predictable trigger ordering has been requested for some time, document
this behavior as a feature. Also document that rules fire in name
order, since yesterday's changes to pg_rewrite indexing cause that too.
DROP RULE and COMMENT ON RULE syntax adds an 'ON tablename' clause,
similar to TRIGGER syntaxes. To allow loading of existing pg_dump
files containing COMMENT ON RULE, the COMMENT code will still accept
the old syntax --- but only if the target rulename is unique across
the whole database.
an 'opclass owner' column in pg_opclass. Nothing is done with it at
present, but since there are plans to invent a CREATE OPERATOR CLASS
command soon, we'll probably want DROP OPERATOR CLASS too, which
suggests that a notion of ownership would be a good idea.
qualified operator names directly, for example CREATE OPERATOR myschema.+
( ... ). To qualify an operator name in an expression you need to write
OPERATOR(myschema.+) (thanks to Peter for suggesting an escape hatch).
I also took advantage of having to reformat pg_operator to fix something
that'd been bugging me for a while: mergejoinable operators should have
explicit links to the associated cross-data-type comparison operators,
rather than hardwiring an assumption that they are named < and >.
entries, per pghackers discussion. This fixes aggregates to live in
namespaces, and also simplifies/speeds up lookup in parse_func.c.
Also, add a 'proimplicit' flag to pg_proc that controls whether a type
coercion function may be invoked implicitly, or only explicitly. The
current settings of these flags are more permissive than I would like,
but we will need to debate and refine the behavior; for now, I avoided
breaking regression tests as much as I could.
volatile), rather than the old cachable/noncachable distinction. This
allows indexscan optimizations in many places where we formerly didn't.
Also, add a pronamespace column to pg_proc (it doesn't do anything yet,
however).
in schemas other than the system namespace; however, there's no search
path yet, and not all operations work yet on tables outside the system
namespace.
addRangeTableEntry calls. Remove relname field from RTEs, since
it will no longer be a useful unique identifier of relations;
we want to encourage people to rely on the relation OID instead.
Further work on dumping qual expressions in EXPLAIN, too.
objects to be privilege-checked. Some change in their APIs would be
necessary no matter what in the schema environment, and simply getting
rid of the name-based interface entirely seems like the best way.
PostgreSQL. This hash function replaces the one used by hash indexes and
the catalog cache. Hash joins use a different, relatively poor-quality
hash function, but I'll fix that later.
As suggested by Tom Lane, this patch also changes the size of the fixed
hash table used by the catalog cache to be a power-of-2 (instead of a
prime: I chose 256 instead of 257). This allows the catcache to lookup
hash buckets using a simple bitmask. This should improve the performance
of the catalog cache slightly, since the previous method (modulo a
prime) was slow.
In my tests, this improves the performance of hash indexes by between 4%
and 8%; the performance when using btree indexes or seqscans is
basically unchanged.
Neil Conway <neilconway@rogers.com>
- domain.patch -> source patch against pgsql in cvs
- drop_domain.sgml and create_domain.sgml -> New doc/src/sgml/ref docs
- dominfo.txt -> basic domain related queries I used for testing
[ ADDED TO /doc]
Enables domains of array elements -> CREATE DOMAIN dom int4[3][2];
Uses a typbasetype column to describe the origin of the domain.
Copies data to attnotnull rather than processing in execMain().
Some documentation differences from earlier.
If this is approved, I'll start working on pg_dump, and a \dD <domain>
option in psql, and regression tests. I don't really feel like doing
those until the system table structure settles for pg_type.
CHECKS when added, will also be copied to to the table attributes. FK
Constraints (if I ever figure out how) will be done similarly. Both
will lbe handled by MergeDomainAttributes() which is called shortly
before MergeAttributes().
Rod Taylor
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
speed up repetitive failed searches; per pghackers discussion in late
January. inval.c logic substantially simplified, since we can now treat
inserts and deletes alike as far as inval events are concerned. Some
repair work needed in heap_create_with_catalog, which turns out to have
been doing CommandCounterIncrement at a point where the new relation has
non-self-consistent catalog entries. With the new inval code, that
resulted in assert failures during a relcache entry rebuild.
now just below FATAL in server_min_messages. Added more text to
highlight ordering difference between it and client_min_messages.
---------------------------------------------------------------------------
REALLYFATAL => PANIC
STOP => PANIC
New INFO level the prints to client by default
New LOG level the prints to server log by default
Cause VACUUM information to print only to the client
NOTICE => INFO where purely information messages are sent
DEBUG => LOG for purely server status messages
DEBUG removed, kept as backward compatible
DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1 added
DebugLvl removed in favor of new DEBUG[1-5] symbols
New server_min_messages GUC parameter with values:
DEBUG[5-1], INFO, NOTICE, ERROR, LOG, FATAL, PANIC
New client_min_messages GUC parameter with values:
DEBUG[5-1], LOG, INFO, NOTICE, ERROR, FATAL, PANIC
Server startup now logged with LOG instead of DEBUG
Remove debug_level GUC parameter
elog() numbers now start at 10
Add test to print error message if older elog() values are passed to elog()
Bootstrap mode now has a -d that requires an argument, like postmaster
both input streams to the end. If one variable's range is much less
than the other, an indexscan-based merge can win by not scanning all
of the other table. Per example from Reinhard Max.
hashname() and reduce the penalty incured when NAMEDATALEN is increased.
I posted this to -hackers a couple days ago, and there haven't been any
major complaints. It passes the regression tests. See -hackers for more
discussion, as well as the suggestion from Tom Lane on which this patch
is based.
Unless anyone sees any problems, please apply for 7.3.
Cheers,
Neil Conway
Improve 'pg_internal.init' relcache entry preload mechanism so that it is
safe to use for all system catalogs, and arrange to preload a realistic
set of system-catalog entries instead of only the three nailed-in-cache
indexes that were formerly loaded this way. Fix mechanism for deleting
out-of-date pg_internal.init files: this must be synchronized with transaction
commit, not just done at random times within transactions. Drive it off
relcache invalidation mechanism so that no special-case tests are needed.
Cache additional information in relcache entries for indexes (their pg_index
tuples and index-operator OIDs) to eliminate repeated lookups. Also cache
index opclass info at the per-opclass level to avoid repeated lookups during
relcache load.
Generalize 'systable scan' utilities originally developed by Hiroshi,
move them into genam.c, use in a number of places where there was formerly
ugly code for choosing either heap or index scan. In particular this allows
simplification of the logic that prevents infinite recursion between syscache
and relcache during startup: we can easily switch to heapscans in relcache.c
when and where needed to avoid recursion, so IndexScanOK becomes simpler and
does not need any expensive initialization.
Eliminate useless opening of a heapscan data structure while doing an indexscan
(this saves an mdnblocks call and thus at least one kernel call).
This saves one open file descriptor per backend, and avoids an
annoying NOTICE on Cygwin (which has trouble deleting open files).
Bug appears to date back to original coding of init_irels, circa 1992.
This seems the right thing for most usages, but I notice two places
where it is the wrong thing. One is that the default permissions on
TOAST rels should be no-access, not world-readable; the other is that
PrepareForTupleInvalidation doesn't really need to spend time looking
at tuples of TOAST relations.
lookup info in the relcache for index access method support functions.
This makes a huge difference for dynamically loaded support functions,
and should save a few cycles even for built-in ones. Also tweak dfmgr.c
so that load_external_function is called only once, not twice, when
doing fmgr_info for a dynamically loaded function. All per performance
gripe from Teodor Sigaev, 5-Oct-01.
readability. Bizarre '(long *) TRUE' return convention is gone,
in favor of just raising an error internally in dynahash.c when
we detect hashtable corruption. HashTableWalk is gone, in favor
of using hash_seq_search directly, since it had no hope of working
with non-LONGALIGNable datatypes. Simplify some other code that was
made undesirably grotty by promixity to HashTableWalk.
portability issues). Caller-visible data structures are now allocated
on MAXALIGN boundaries, allowing safe use of datatypes wider than 'long'.
Rejigger hash_create API so that caller specifies size of key and
total size of entry, not size of key and size of rest of entry.
This simplifies life considerably since each number is just a sizeof(),
and padding issues etc. are taken care of automatically.
Frank Miles 7-Sep-01. This is really just sticking a finger in the dike.
Frank's case works now, but we still couldn't support a recursive function
returning a set. Really need to restructure querytrees and execution
state so that the querytree is *read only*. We've run into this over and
over and over again ... it has to happen sometime soon.
If there's anyone out there who's actually using datatype-defined
default values, this will be an incompatible change in behavior ...
but the old behavior was so broken that I doubt anyone was using it.
buffer manager with 'pg_clog', a specialized access method modeled
on pg_xlog. This simplifies startup (don't need to play games to
open pg_log; among other things, OverrideTransactionSystem goes away),
should improve performance a little, and opens the door to recycling
commit log space by removing no-longer-needed segments of the commit
log. Actual recycling is not there yet, but I felt I should commit
this part separately since it'd still be useful if we chose not to
do transaction ID wraparound.
pgsql-hackers. pg_opclass now has a row for each opclass supported by each
index AM, not a row for each opclass name. This allows pg_opclass to show
directly whether an AM supports an opclass, and furthermore makes it possible
to store additional information about an opclass that might be AM-dependent.
pg_opclass and pg_amop now store "lossy" and "haskeytype" information that we
previously expected the user to remember to provide in CREATE INDEX commands.
Lossiness is no longer an index-level property, but is associated with the
use of a particular operator in a particular index opclass.
Along the way, IndexSupportInitialize now uses the syscaches to retrieve
pg_amop and pg_amproc entries. I find this reduces backend launch time by
about ten percent, at the cost of a couple more special cases in catcache.c's
IndexScanOK.
Initial work by Oleg Bartunov and Teodor Sigaev, further hacking by Tom Lane.
initdb forced.
default, but OIDS are removed from many system catalogs that don't need them.
Some interesting side effects: TOAST pointers are 20 bytes not 32 now;
pg_description has a three-column key instead of one.
Bugs fixed in passing: BINARY cursors work again; pg_class.relhaspkey
has some usefulness; pg_dump dumps comments on indexes, rules, and
triggers in a valid order.
initdb forced.
stub) into the rest of the system. Adopt a cleaner approach to preventing
deadlock in concurrent heap_updates: allow RelationGetBufferForTuple to
select any page of the rel, and put the onus on it to lock both buffers
in a consistent order. Remove no-longer-needed isExtend hack from
API of ReleaseAndReadBuffer.
do anything yet, but it has the necessary connections to initialization
and so forth. Make some gestures towards allowing number of blocks in
a relation to be BlockNumber, ie, unsigned int, rather than signed int.
(I doubt I got all the places that are sloppy about it, yet.) On the
way, replace the hardwired NLOCKS_PER_XACT fudge factor with a GUC
variable.
SI messages now include the relevant database OID, so that operations
in one database do not cause useless cache flushes in backends attached
to other databases. Declare SI messages properly using a union, to
eliminate the former assumption that Oid is the same size as int or Index.
Rewrite the nearly-unreadable code in inval.c, and document it better.
Arrange for catcache flushes at end of command/transaction to happen before
relcache flushes do --- this avoids loading a new tuple into the catcache
while setting up new relcache entry, only to have it be flushed again
immediately.
CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed)
how to distinguish tuples it should flush from those it needn't; this
means a relcache flush event now only removes the catcache entries
it ought to, rather than zapping the caches completely as it used to.
Testing with the regression tests indicates that this considerably
improves the lifespan of catcache entries. Also, rearrange catcache
data structures so that the limit on number of cached tuples applies
globally across all the catcaches, rather than being per-catcache.
It was a little silly to have the same size limit on both, say,
pg_attribute caches and pg_am caches (there being only four possible
rows in the latter...). Doing LRU removal across all the caches
instead of locally in each one should reduce cache reload traffic
in the more heavily used caches and improve the efficiency of
cache memory use.
Tom Lane). For the moment, only the OID/name variants are provided.
I didn't force initdb, but the additions to the 'privileges' regress
test won't pass until you do one.
pg_database now has unique indexes on oid and on datname.
pg_shadow now has unique indexes on usename and on usesysid.
pg_am now has unique index on oid.
pg_opclass now has unique index on oid.
pg_amproc now has unique index on amid+amopclaid+amprocnum.
Remove pg_rewrite's unnecessary index on oid, delete unused RULEOID syscache.
Remove index on pg_listener and associated syscache for performance reasons
(caching rows that are certain to change before you need 'em again is
rather pointless).
Change pg_attrdef's nonunique index on adrelid into a unique index on
adrelid+adnum.
Fix various incorrect settings of pg_class.relisshared, make that the
primary reference point for whether a relation is shared or not.
IsSharedSystemRelationName() is now only consulted to initialize relisshared
during initial creation of tables and indexes. In theory we might now
support shared user relations, though it's not clear how one would get
entries for them into pg_class &etc of multiple databases.
Fix recently reported bug that pg_attribute rows created for an index all have
the same OID. (Proof that non-unique OID doesn't matter unless it's
actually used to do lookups ;-))
There's no need to treat pg_trigger, pg_attrdef, pg_relcheck as bootstrap
relations. Convert them into plain system catalogs without hardwired
entries in pg_class and friends.
Unify global.bki and template1.bki into a single init script postgres.bki,
since the alleged distinction between them was misleading and pointless.
Not to mention that it didn't work for setting up indexes on shared
system relations.
Rationalize locking of pg_shadow, pg_group, pg_attrdef (no need to use
AccessExclusiveLock where ExclusiveLock or even RowExclusiveLock will do).
Also, hold locks until transaction commit where necessary.
report on old-style functions invoked by RI triggers. We had a number of
other places that were being sloppy about which memory context FmgrInfo
subsidiary data will be allocated in. Turns out none of them actually
cause a problem in 7.1, but this is for arcane reasons such as the fact
that old-style triggers aren't supported anyway. To avoid getting burnt
later, I've restructured the trigger support so that we don't keep trigger
FmgrInfo structs in relcache memory. Some other related cleanups too:
it's not really necessary to call fmgr_info at all while setting up
the index support info in relcache entries, because those ScanKeyEntry
structs are never used to invoke the functions. This should speed up
relcache initialization a tiny bit.
collected by ANALYZE. Also, add some modest amount of intelligence to
guesses that are used for varlena columns in the absence of any ANALYZE
statistics. The 'width' reported by EXPLAIN is finally something less
than totally bogus for varlena columns ... and, in consequence, hashjoin
estimating should be a little better ...
a separate statement (though it can still be invoked as part of VACUUM, too).
pg_statistic redesigned to be more flexible about what statistics are
stored. ANALYZE now collects a list of several of the most common values,
not just one, plus a histogram (not just the min and max values). Random
sampling is used to make the process reasonably fast even on very large
tables. The number of values and histogram bins collected is now
user-settable via an ALTER TABLE command.
There is more still to do; the new stats are not being used everywhere
they could be in the planner. But the remaining changes for this project
should be localized, and the behavior is already better than before.
A not-very-related change is that sorting now makes use of btree comparison
routines if it can find one, rather than invoking '<' twice.
physical file has disappeared. There is no really good reason why
relcache should be opening the underlying file at all, AFAICS.
In any case we needn't raise a hard error here.
are now critical sections, so as to ensure die() won't interrupt us while
we are munging shared-memory data structures. Avoid insecure intermediate
states in some code that proc_exit will call, like palloc/pfree. Rename
START/END_CRIT_CODE to START/END_CRIT_SECTION, since that seems to be
what people tend to call them anyway, and make them be called with () like
a function call, in hopes of not confusing pg_indent.
I doubt that this is sufficient to make SIGTERM safe anywhere; there's
just too much code that could get invoked during proc_exit().
in per-entry sub-memory-context, where they were supposed to go, rather
than in CacheMemoryContext where the code was putting them. Must've
suffered a severe brain fade when I wrote this :-(
starting a new hashtable search no longer clobbers any other search
active anywhere in the system. Fix RelationCacheInvalidate() so that
it will not crash or go into an infinite loop if invoked recursively,
as for example by a second SI Reset message arriving while we are still
processing a prior one.
1. Distinguish cases where a Datum representing a tuple datatype is an OID
from cases where it is a pointer to TupleTableSlot, and make sure we use
the right typlen in each case.
2. Make fetchatt() and related code support 8-byte by-value datatypes on
machines where Datum is 8 bytes. Centralize knowledge of the available
by-value datatype sizes in two macros in tupmacs.h, so that this will be
easier if we ever have to do it again.
table that inherits from a temp table. Make sure the right things happen
if one creates a temp table, creates another temp that inherits from it,
then renames the first one. (Previously, system would end up trying to
delete the temp tables in the wrong order.)
both MULTIBYTE and TOAST prevent char(n) from being truly fixed-size.
Simplify and speed up fastgetattr() and index_getattr() macros by
eliminating special cases for attnum=1. It's just as fast to handle
the first attribute by presetting its attcacheoff to zero; so do that
instead when loading the tupledesc in relcache.c.
in pghackers list. Support for oldstyle internal functions is gone
(no longer needed, since conversion is complete) and pg_language entry
'internal' now implies newstyle call convention. pg_language entry
'newC' is gone; both old and newstyle dynamically loaded C functions
are now called language 'C'. A newstyle function must be identified
by an associated info routine. See src/backend/utils/fmgr/README.
maintained for each cache entry. A cache entry will not be freed until
the matching ReleaseSysCache call has been executed. This eliminates
worries about cache entries getting dropped while still in use. See
my posting to pg-hackers of even date for more info.
message about recursive use of a syscache. Also remove most of the
specialized indexscan routines in indexing.c --- it turns out that
catcache.c is perfectly able to perform the indexscan for itself,
in fact has already looked up all the information needed to do so!
This should be faster as well as needing far less boilerplate code.
(WAL logging for this is not done yet, however.) Clean up a number of really
crufty things that are no longer needed now that DROP behaves nicely. Make
temp table mapper do the right things when drop or rename affecting a temp
table is rolled back. Also, remove "relation modified while in use" error
check, in favor of locking tables at first reference and holding that lock
throughout the statement.
from bufmgr - it would be nice to have separate hash in smgr
for node <--> fd mappings, but for the moment it's easy to
add new hash to relcache.
Fixed small bug in xlog.c:ReadRecord.
for views. Views are now have a "relkind" of
RELKIND_VIEW instead of RELKIND_RELATION.
Also, views no longer have actual heap storage
files.
The following changes were made
1. CREATE VIEW sets the new relkind
2. The executor complains if a DELETE or
INSERT references a view.
3. DROP RULE complains if an attempt is made
to delete a view SELECT rule.
4. CREATE RULE "_RETmytable" AS ON SELECT TO mytable DO INSTEAD ...
1. checks to make sure mytable is empty.
2. sets the relkind to RELKIND_VIEW.
3. deletes the heap storage files.
5. LOCK myview is not allowed. :)
6. the regression test type_sanity was changed to
account for the new relkind value.
7. CREATE INDEX ON myview ... is not allowed.
8. VACUUM myview is not allowed.
VACUUM automatically skips views when do the entire
database.
9. TRUNCATE myview is not allowed.
THINGS LEFT TO THINK ABOUT
o pg_views
o pg_dump
o pgsql (\d \dv)
o Do we really want to be able to inherit from views?
o Is 'DROP TABLE myview' OK?
--
Mark Hollomon
=# create table t (id int4 unique);
NOTICE: CREATE TABLE/UNIQUE will create implicit index 't_id_key' for table 't'
=# begin;
query: drop table t;
NOTICE: Caution: DROP TABLE cannot be rolled back, so don't abort now
NOTICE: Caution: DROP INDEX cannot be rolled back, so don't abort now
=# rollback;
=# drop table t;
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: RelationIdBuildRelation: smgropen(t): No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: mdopen: couldn't open t_id_key: No such file or directory
NOTICE: RelationIdBuildRelation: smgropen(t_id_key): No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: RelationIdBuildRelation: smgropen(t): No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
ERROR: cannot open relation t
for example, an SQL function can be used in a functional index. (I make
no promises about speed, but it'll work ;-).) Clean up and simplify
handling of functions returning sets.
right thing with variable-free clauses that contain noncachable functions,
such as 'WHERE random() < 0.5' --- these are evaluated once per
potential output tuple. Expressions that contain only Params are
now candidates to be indexscan quals --- for example, 'var = ($1 + 1)'
can now be indexed. Cope with RelabelType nodes atop potential indexscan
variables --- this oversight prevents 7.0.* from recognizing some
potentially indexscanable situations.
from Param nodes, per discussion a few days ago on pghackers. Add new
expression node type FieldSelect that implements the functionality where
it's actually needed. Clean up some other unused fields in Func nodes
as well.
NOTE: initdb forced due to change in stored expression trees for rules.
that giving pg_proc a toast table required solving the same problems
we'd have to solve for pg_class --- pg_proc is one of the relations
that gets bootstrapped in relcache.c. Solution is to go back at the
end of initialization and read in the *real* pg_class row to replace
the phony entry created by formrdesc(). This should work as long as
there's no need to touch any toasted values during initialization,
which seems a reasonable assumption.
Although I did not add a toast-table for every single system table
with a varlena attribute, I believe that it would work to just do
ALTER TABLE pg_class CREATE TOAST TABLE. So anyone who's really
intent on having several thousand ACL entries for a rel could do it.
NOTE: I didn't force initdb, but you must do one to see the effects
of this patch.
(Sorry, couldn't help it...)
Removed type filename as well, since it's unused and probably useless.
INITDB FORCED, because pg_rewrite columns are now plain text again.
pass-by-ref data types --- eg, an index on lower(textfield) --- no longer
leak memory during index creation or update. Clean up a lot of redundant
code ... did you know that copy, vacuum, truncate, reindex, extend index,
and bootstrap each basically duplicated the main executor's logic for
extracting information about an index and preparing index entries?
Functional indexes should be a little faster now too, due to removal
of repeated function lookups.
CREATE INDEX 'opt_type' clause is deimplemented by these changes,
but I haven't removed it from the parser yet (need to merge with
Thomas' latest change set first).
memory contexts. Currently, only leaks in expressions executed as
quals or projections are handled. Clean up some old dead cruft in
executor while at it --- unused fields in state nodes, that sort of thing.
Don't go through pg_exec_query_dest(), but directly to the execution
routines. Also, extend parameter lists so that there's no need to
change the global setting of allowSystemTableMods, a hack that was
certain to cause trouble in the event of any error.
entry that has rules. This allows us to release the rule parsetrees
on relcache flush without needing a working freeObject() routine.
Formerly, the rule trees were leaked permanently at relcache flush.
Also, clean up handling of rule creation and deletion --- there was
not sufficient locking of the relation being modified, and there was
no reliable notification of other backends that a relcache reload
was needed. Also, clean up relcache.c code so that scans of system
tables needed to load a relcache entry are done in the caller's
memory context, not in CacheMemoryContext. This prevents any
un-pfreed memory from those scans from becoming a permanent memory
leak.
for details). It doesn't really do that much yet, since there are no
short-term memory contexts in the executor, but the infrastructure is
in place and long-term contexts are handled reasonably. A few long-
standing bugs have been fixed, such as 'VACUUM; anything' in a single
query string crashing. Also, out-of-memory is now considered a
recoverable ERROR, not FATAL.
Eliminate a large amount of crufty, now-dead code in and around
memory management.
Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and
backend startup.
to apply the tempname->realname mapping to type name lookup as well
as relation name lookup, else the type tuple will not be found when
wanted. This fixes bugs like this one:
create temp table foo (f1 int);
select foo.f2 from foo;
ERROR: Unable to locate type name 'foo' in catalog
are opened in a consistent order by different backends (I ordered them
by index OID because that's easy, but any other consistent order would
do as well). This avoids potential deadlock for index types that we
acquire exclusive locks on ... ie, rtree.
entries now for int8 and network hash indexes. int24_ops and int42_ops
are gone. pg_opclass no longer contains multiple entries claiming to be
the default opclass for the same datatype. opr_sanity regress test
extended to catch errors like these in the future.
materialized tupleset is small enough) instead of a temporary relation.
This was something I was thinking of doing anyway for performance, and Jan
says he needs it for TOAST because he doesn't want to cope with toasting
noname relations. With this change, the 'noname table' support in heap.c
is dead code, and I have accordingly removed it. Also clean up 'noname'
plan handling in planner --- nonames are either sort or materialize plans,
and it seems less confusing to handle them separately under those names.
discussion of 5/19/00). pg_index is now searched for indexes of a
relation using an indexscan. Moreover, this is done once and cached
in the relcache entry for the relation, in the form of a list of OIDs
for the indexes. This list is used by the parser and executor to drive
lookups in the pg_index syscache when they want to know the properties
of the indexes. Net result: index information will be fully cached
for repetitive operations such as inserts.
pointers, namely the catcache tuple fetch routines. Also get rid of
the unused and possibly confusing 'size' field in struct cachedesc.
Since it doesn't allow for variable-length fields, anyone who
actually trusted it would likely be making a mistake...
over multiple lookups --- it should use SearchSysCacheTupleCopy instead.
This accounts for rare failures like 'init_fcache: null probin for procedure 481'
when running concurrently with a VACUUM.
inputs have been converted to newstyle. This should go a long way towards
fixing our portability problems with platforms where char and short
parameters are passed differently from int-width parameters. Still
more to do for the Alpha port however.
key call sites are changed, but most called functions are still oldstyle.
An exception is that the PL managers are updated (so, for example, NULL
handling now behaves as expected in plperl and plpgsql functions).
NOTE initdb is forced due to added column in pg_proc.
really ought to fix relcache entry construction so that it does not
do so much with CurrentMemoryContext = CacheCxt. As is, relatively
harmless leaks in either sequential or index scanning translate to
permanent leaks if they occur when called from relcache build.
For the moment, however, the path of least resistance is to repair
all such leaks...
pg_internal.init file in-place, which meant that if another backend
started at about the same time, it might read the incomplete file.
init_irels tries to guard against that, but I have now seen a crash
due to reading bad data from a partly-written file. (This may indicate
a kernel bug on my platform? Not sure.) Anyway, clearly the safest
course is to write the new pg_internal.init file under a unique temporary
filename, and rename it into place only after it's all written.
running gcc and HP's cc with warnings cranked way up. Signed vs unsigned
comparisons, routines declared static and then defined not-static,
that kind of thing. Tedious, but perhaps useful...
the cache context, it didn't bother to free the tuple that
CatalogIndexFetchTuple had allocated in the transaction context.
Do enough cache lookups in the same xact, and you start to notice...
syscache and relcache flushes). Relcache entry rebuild now preserves
original tupledesc, rewrite rules, and triggers if possible, so that pointers
to these things remain valid --- if these things change while relcache entry
has positive refcount, we elog(ERROR) to avoid later crash. Arrange for
xact-local rels to be rebuilt when an SI inval message is seen for them,
so that they are updated by CommandCounterIncrement the same as regular rels.
(This is useful because of Hiroshi's recent changes to process our own SI
messages at CommandCounterIncrement time.) This allows simplification of
some routines that previously hacked around the lack of an automatic update.
catcache now keeps its own copy of tupledesc for its relation, rather than
depending on the relcache's copy; this avoids needing to reinitialize catcache
during a cache flush, which saves some cycles and eliminates nasty circularity
problems that occur if a cache flush happens while trying to initialize a
catcache.
Eliminate a number of permanent memory leaks that used to happen during
catcache or relcache flush; not least of which was that catcache never
freed any cached tuples! (Rule parsetree storage is still leaked, however;
will fix that separately.)
Nothing done yet about code that uses tuples retrieved by SearchSysCache
for longer than is safe.
we *always* rebuild, rather than deleting, an invalidated relcache entry
that has positive refcount. Otherwise an SI cache overrun leads to
dangling Relation pointers all over the place!
an attribute of a tuple previously fetched with SearchSysCacheTuple.
This avoids a lot of redundant cache lookups, particularly in selfuncs.c.
Also, remove SearchSysCacheStruct, which was unused and grotty.
relcache entry no longer leaks a small amount of memory. index_endscan
now releases all the memory acquired by index_beginscan, so callers of it
should NOT pfree the scan descriptor anymore.
Make all system indexes unique.
Make all cache loads use system indexes.
Rename *rel to *relid in inheritance tables.
Rename cache names to be clearer.
inval.c thought it could safely use the catcache to look up the OIDs of
system relations. Not good, considering that inval.c could be called
during catcache loading, if a shared-inval message arrives. Rip out the
lookup logic and instead use the known OIDs from pg_class.h.
Now indexes of pg_class and pg_type are unique indexes
and guarantee the uniqueness of correponding attributes.
heap_create() was changed to take another boolean parameter
which allows to postpone the creation of disk file.
The name of rd_nonameunlinked was changed to rd_unlinked.
It is used generally(not only for noname relations) now.
Requires initdb.
expressions in CREATE TABLE. There is no longer an emasculated expression
syntax for these things; it's full a_expr for constraints, and b_expr
for defaults (unfortunately the fact that NOT NULL is a part of the
column constraint syntax causes a shift/reduce conflict if you try a_expr.
Oh well --- at least parenthesized boolean expressions work now). Also,
stored expression for a column default is not pre-coerced to the column
type; we rely on transformInsertStatement to do that when the default is
actually used. This means "f1 datetime default 'now'" behaves the way
people usually expect it to.
BTW, all the support code is now there to implement ALTER TABLE ADD
CONSTRAINT and ALTER TABLE ADD COLUMN with a default value. I didn't
actually teach ALTER TABLE to call it, but it wouldn't be much work.
additional argument specifying the kind of lock to acquire/release (or
'NoLock' to do no lock processing). Ensure that all relations are locked
with some appropriate lock level before being examined --- this ensures
that relevant shared-inval messages have been processed and should prevent
problems caused by concurrent VACUUM. Fix several bugs having to do with
mismatched increment/decrement of relation ref count and mismatched
heap_open/close (which amounts to the same thing). A bogus ref count on
a relation doesn't matter much *unless* a SI Inval message happens to
arrive at the wrong time, which is probably why we got away with this
sloppiness for so long. Repair missing grab of AccessExclusiveLock in
DROP TABLE, ALTER/RENAME TABLE, etc, as noted by Hiroshi.
Recommend 'make clean all' after pulling this update; I modified the
Relation struct layout slightly.
Will post further discussion to pghackers list shortly.
was rejecting negative attnums as bogus, which of course they are not.
Add code to get_attdisbursion to produce a useful value for OID attribute,
since VACUUM does not store stats for system attributes.
Also, repair bug that's been in eqjoinsel for a long time: it was taking
the max of the two columns' disbursions, whereas it should use the min.
current transaction) are not flushed by shared-cache-inval reset message.
SI reset actually works now, for probably the first time in a long time.
I was able to run initdb and regression tests with a 16-element SI message
array, with a lot of NOTICE: cache state reset messages but no crashes.
real name before doing lookup. We only want to index temp tables by their
real names in the relcache, to ensure there's not more than one relcache
entry for them.
has positive refcount, it is rebuilt from pg_class data. This ensures
that relcache entries will track changes made by other backends. Formerly,
a shared inval report would just be ignored if it happened to arrive while
the relcache entry was in use. Also, fix relcache to reset ref counts
to zero during transaction abort. Finally, change LockRelation() so that
it checks for shared inval reports after obtaining the lock. In this way,
once any kind of lock has been obtained on a rel, we can trust the relcache
entry to be up-to-date.
insight that RelationFlushRelation ought to invoke smgrclose, and that the
way to make that work is to ensure that mdclose doesn't fail if the relation
is already closed (or unlinked, if we are looking at a DROP TABLE). While
I was testing that, I was able to identify several problems that we had
with multiple-segment relations. The system is now able to do initdb and
pass the regression tests with a very small segment size (I had it set to
64Kb per segment for testing). I don't believe that ever worked before.
File descriptor leaks seem to be gone too.
I have partially addressed the concerns we had about mdtruncate(), too.
On a Win32 or NFS filesystem it is not possible to unlink a file that
another backend is holding open, so what md.c now does is to truncate
unwanted files to zero length before trying to unlink them. The other
backends will be forced to close their open files by relation cache
invalidation --- but I think it would take considerable work to make
that happen before vacuum truncates the relation rather than after.
Leaving zero-length files lying around seems a usable compromise.
to save a little bit of backend startup time. This way, the first
backend started after a VACUUM will rebuild the init file with up-to-date
statistics for the critical system indexes.
Ok. I made patches replacing all of "#if FALSE" or "#if 0" to "#ifdef
NOT_USED" for current. I have tested these patches in that the
postgres binaries are identical.
I don't know if this is really related to the initdb problem
discussion (haven't followed it enough). But seems so because
it fixes a damn problem during index tuple insertion on
CREATE TABLE into pg_attribute_relid_attnum_index.
Anyway - this bug was really hard to find. During startup the
relcache reads in some prepared information about index
strategies from a file and then reinitializes the function
pointers inside the scanKey data. But for sake it assumed
single attribute index tuples (hasn't that changed recently).
Thus not all the strategies scanKey entries where initialized
properly, resulting in invalid addresses for the btree
comparision functions.
With the patch at the end the regression tests passed
excellent except for the sanity_check that crashed at vacuum
and the misc test where the select unique1 from onek2 outputs
the two rows in different order.
Jan
if MULTIBYTE is not enabled. So be sure to run initdb.
o these patches are made against the latest source tree (after
Bruce's massive patch, I think) BTW, I noticed that after running
regression, the oid field of pg_type seems disappeared.
regression=> select oid from pg_type; ERROR: attribute
'oid' not found
this happens after the constraints test. This occures with/without
my patches. strange...
o pg_database_mb.h, pg_class_mb.h, pg_attribute_mb.h are no longer
used, and shoud be removed.
o GetDatabaseInfo() in utils/misc/database.c removed (actually in
#ifdef 0). seems nobody uses.
t-ishii@sra.co.jp
no longer returns buffer pointer, can be gotten from scan;
descriptor; bootstrap can create multi-key indexes;
pg_procname index now is multi-key index; oidint2, oidint4, oidname
are gone (must be removed from regression tests); use System Cache
rather than sequential scan in many places; heap_modifytuple no
longer takes buffer parameter; remove unused buffer parameter in
a few other functions; oid8 is not index-able; remove some use of
single-character variable names; cleanup Buffer variables usage
and scan descriptor looping; cleaned up allocation and freeing of
tuples; 18k lines of diff;
As Bruce mentioned, this is due to the conflict among changes we made.
Included patches should fix the problem(I changed all MB to
MULTIBYTE). Please let me know if you have further problem.
P.S. I did not include pathces to configure and gram.c to save the
file size(configure.in and gram.y modified).
From: t-ishii@sra.co.jp
Attached are patches to enhance the multi-byte support. (patches are
against 7/18 snapshot)
* determine encoding at initdb/createdb rather than compile time
Now initdb/createdb has an option to specify the encoding. Also, I
modified the syntax of CREATE DATABASE to accept encoding option. See
README.mb for more details.
For this purpose I have added new column "encoding" to pg_database.
Also pg_attribute and pg_class are changed to catch up the
modification to pg_database. Actually I haved added pg_database_mb.h,
pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is
enabled. The reason having separate files is I couldn't find a way to
use ifdef or whatever in those files. I have to admit it looks
ugly. No way.
* support for PGCLIENTENCODING when issuing COPY command
commands/copy.c modified.
* support for SQL92 syntax "SET NAMES"
See gram.y.
* support for LATIN2-5
* add UNICODE regression test case
* new test suite for MB
New directory test/mb added.
* clean up source files
Basic idea is to have MB's own subdirectory for easier maintenance.
These are include/mb and backend/utils/mb.
1. Removes the unnecessary "#define AbcRegProcedure 123"'s from
pg_proc.h.
2. Changes those #defines to use the names already defined in
fmgr.h.
3. Forces the make of fmgr.h in backend/Makefile instead of having
it
made as a dependency in access/common/Makefile *hack*hack*hack*
4. Rearranged the #includes to a less helter-skelter arrangement,
also
changing <file.h> to "file.h" to signify a non-system header.
5. Removed "pg_proc.h" from files where its only purpose was for
the
#defines removed in item #1.
6. Added "fmgr.h" to each file changed for completeness sake.
Turns out that #6 was not necessary for some files because fmgr.h
was being included in a roundabout way SIX levels deep by the first
include.
"access/genam.h"
->"access/relscan.h"
->"utils/rel.h"
->"access/strat.h"
->"access/skey.h"
->"fmgr.h"
So adding fmgr.h really didn't add anything to the compile, hopefully
just made it clearer to the programmer.
S Darren.
Attached you'll find a (big) patch that fixes make dep and make
depend in all Makefiles where I found it to be appropriate.
It also removes the dependency in Makefile.global for NAMEDATALEN
and OIDNAMELEN by making backend/catalog/genbki.sh and bin/initdb/initdb.sh
a little smarter.
This no longer requires initdb.sh that is turned into initdb with
a sed script when installing Postgres, hence initdb.sh should be
renamed to initdb (after the patch has been applied :-) )
This patch is against the 6.3 sources, as it took a while to
complete.
Please review and apply,
Cheers,
Jeroen van Vianen
1. Remove the char2, char4, char8 and char16 types from postgresql
2. Change references of char16 to name in the regression tests.
3. Rename the char16.sql regression test to name.sql. 4. Modify
the regression test scripts and outputs to match up.
Might require new regression.{SYSTEM} files...
Darren King
seems that my last post didn't make it through. That's good
since the diff itself didn't covered the renaming of
pg_user.h to pg_shadow.h and it's new content.
Here it's again. The complete regression test passwd with
only some float diffs. createuser and destroyuser work.
pg_shadow cannot be read by ordinary user.
The diff looks so simple and easy. But to find it wasn't fun.
It must have been there for a long time. What happened:
When a tuple in one of some central catalogs was updated, the
referenced relation got flushed, so it would be reopened on
the next access (to reflect new triggers, rules and table
structure changes into the relation cache).
Some data (the tupleDescriptor e.g.) is used in the system
cache too. So when a relation is subject to the system cache,
this must know too that a cached system relation got flushed
because the tupleDesc data gets freed during the flush!
For the GRANT/REVOKE on pg_class it was slightly different.
There is some local data in inval.c that gets initialized on
the first invalidation of a tuple in some central catalogs.
This needs a SysCache lookup in pg_class. But when the first
of all commands is a GRANT on pg_class, exactly the needed
tuple is the one actually invalidated. So I added little code
snippets that the initialization of the local variables in
inval.c will already happen during InitPostgres().
Attached is the patch to fix the warning messages from my code. I also
fixed one which wasn't my code. Apart from the usual warnings about the
bison/yacc generated code I only have one other warning message. This
is in gramm.y around line 2234. I wasn't sure of the fix.
I've also replaced all the calls to free() in gramm.y to calls to
pfree(). Without these I was getting backend crashes with GRANT. This
might already have been fixed.
Patch by: wieck@sapserv.debis.de (Jan Wieck)
One of the design rules of PostgreSQL is extensibility. And
to follow this rule means (at least for me) that there should
not only be a builtin PL. Instead I would prefer a defined
interface for PL implemetations.
Makefile.global.
End result, if all goes well, should allow for much easier porting, since
there will no longer be a concept of a "port". Most, if not everything,
*should* be determined by configure, or by the compiler itself. Still
work to be done though :)
/*
* RelationFlushRelation () below will flush relation information
* from the cache. We must call smgrclose to flush relation
* information from SMGR & FMGR, too. We assume that for temp
* relations smgrunlink is already called by heap_destroyr
* and we skip smgrclose for them. - vadim 05/22/97
*/
smgrclose(reln->rd_rel->relsmgr, reln);
- it avoids memory leaks in SMGR & VFD.
RelationFlushRelation():
there is no more call FileInvalidate(RelationGetSystemPort(relation));
- invalid (FileInvalidate() expects File, not SMGR' fd)
- unuseful anyway.
Changes:
* Unique index capability works using the syntax 'create unique
index'.
* Duplicate OID's in the system tables are removed. I put
little scripts called 'duplicate_oids' and 'find_oid' in
include/catalog that help to find and remove duplicate OID's.
I also moved 'unused_oids' from backend/catalog to
include/catalog, since it has to be in the same directory
as the include files in order to work.
* The backend tries converting the name of a function or aggregate
to all lowercase if the original name given doesn't work (mostly
for compatibility with ODBC).
* You can 'SELECT NULL' to your heart's content.
* I put my _bt_updateitem fix in instead, which uses
_bt_insertonpg so that even if the new key is so big that
the page has to be split, everything still works.
* All literal references to system catalog OID's have been
replaced with references to define'd constants from the catalog
header files.
* I added a couple of node copy functions. I think this was a
preliminary attempt to get rules to work.
Tom Lane
Tatsuo Ishii
Bruce Momjian
Peter Eisentraut
Jan Wieck
Hiroshi Inoue
Vadim B. Mikheev
Marc G. Fournier
Thomas G. Lockhart
PostgreSQL Daemon
Bryan Henderson