maintained for each cache entry. A cache entry will not be freed until
the matching ReleaseSysCache call has been executed. This eliminates
worries about cache entries getting dropped while still in use. See
my posting to pg-hackers of even date for more info.
message about recursive use of a syscache. Also remove most of the
specialized indexscan routines in indexing.c --- it turns out that
catcache.c is perfectly able to perform the indexscan for itself,
in fact has already looked up all the information needed to do so!
This should be faster as well as needing far less boilerplate code.
(WAL logging for this is not done yet, however.) Clean up a number of really
crufty things that are no longer needed now that DROP behaves nicely. Make
temp table mapper do the right things when drop or rename affecting a temp
table is rolled back. Also, remove "relation modified while in use" error
check, in favor of locking tables at first reference and holding that lock
throughout the statement.
from bufmgr - it would be nice to have separate hash in smgr
for node <--> fd mappings, but for the moment it's easy to
add new hash to relcache.
Fixed small bug in xlog.c:ReadRecord.
for views. Views are now have a "relkind" of
RELKIND_VIEW instead of RELKIND_RELATION.
Also, views no longer have actual heap storage
files.
The following changes were made
1. CREATE VIEW sets the new relkind
2. The executor complains if a DELETE or
INSERT references a view.
3. DROP RULE complains if an attempt is made
to delete a view SELECT rule.
4. CREATE RULE "_RETmytable" AS ON SELECT TO mytable DO INSTEAD ...
1. checks to make sure mytable is empty.
2. sets the relkind to RELKIND_VIEW.
3. deletes the heap storage files.
5. LOCK myview is not allowed. :)
6. the regression test type_sanity was changed to
account for the new relkind value.
7. CREATE INDEX ON myview ... is not allowed.
8. VACUUM myview is not allowed.
VACUUM automatically skips views when do the entire
database.
9. TRUNCATE myview is not allowed.
THINGS LEFT TO THINK ABOUT
o pg_views
o pg_dump
o pgsql (\d \dv)
o Do we really want to be able to inherit from views?
o Is 'DROP TABLE myview' OK?
--
Mark Hollomon
=# create table t (id int4 unique);
NOTICE: CREATE TABLE/UNIQUE will create implicit index 't_id_key' for table 't'
=# begin;
query: drop table t;
NOTICE: Caution: DROP TABLE cannot be rolled back, so don't abort now
NOTICE: Caution: DROP INDEX cannot be rolled back, so don't abort now
=# rollback;
=# drop table t;
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: RelationIdBuildRelation: smgropen(t): No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: mdopen: couldn't open t_id_key: No such file or directory
NOTICE: RelationIdBuildRelation: smgropen(t_id_key): No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
NOTICE: RelationIdBuildRelation: smgropen(t): No such file or directory
NOTICE: mdopen: couldn't open t: No such file or directory
ERROR: cannot open relation t
for example, an SQL function can be used in a functional index. (I make
no promises about speed, but it'll work ;-).) Clean up and simplify
handling of functions returning sets.
right thing with variable-free clauses that contain noncachable functions,
such as 'WHERE random() < 0.5' --- these are evaluated once per
potential output tuple. Expressions that contain only Params are
now candidates to be indexscan quals --- for example, 'var = ($1 + 1)'
can now be indexed. Cope with RelabelType nodes atop potential indexscan
variables --- this oversight prevents 7.0.* from recognizing some
potentially indexscanable situations.
from Param nodes, per discussion a few days ago on pghackers. Add new
expression node type FieldSelect that implements the functionality where
it's actually needed. Clean up some other unused fields in Func nodes
as well.
NOTE: initdb forced due to change in stored expression trees for rules.
that giving pg_proc a toast table required solving the same problems
we'd have to solve for pg_class --- pg_proc is one of the relations
that gets bootstrapped in relcache.c. Solution is to go back at the
end of initialization and read in the *real* pg_class row to replace
the phony entry created by formrdesc(). This should work as long as
there's no need to touch any toasted values during initialization,
which seems a reasonable assumption.
Although I did not add a toast-table for every single system table
with a varlena attribute, I believe that it would work to just do
ALTER TABLE pg_class CREATE TOAST TABLE. So anyone who's really
intent on having several thousand ACL entries for a rel could do it.
NOTE: I didn't force initdb, but you must do one to see the effects
of this patch.
(Sorry, couldn't help it...)
Removed type filename as well, since it's unused and probably useless.
INITDB FORCED, because pg_rewrite columns are now plain text again.
pass-by-ref data types --- eg, an index on lower(textfield) --- no longer
leak memory during index creation or update. Clean up a lot of redundant
code ... did you know that copy, vacuum, truncate, reindex, extend index,
and bootstrap each basically duplicated the main executor's logic for
extracting information about an index and preparing index entries?
Functional indexes should be a little faster now too, due to removal
of repeated function lookups.
CREATE INDEX 'opt_type' clause is deimplemented by these changes,
but I haven't removed it from the parser yet (need to merge with
Thomas' latest change set first).
memory contexts. Currently, only leaks in expressions executed as
quals or projections are handled. Clean up some old dead cruft in
executor while at it --- unused fields in state nodes, that sort of thing.
Don't go through pg_exec_query_dest(), but directly to the execution
routines. Also, extend parameter lists so that there's no need to
change the global setting of allowSystemTableMods, a hack that was
certain to cause trouble in the event of any error.
entry that has rules. This allows us to release the rule parsetrees
on relcache flush without needing a working freeObject() routine.
Formerly, the rule trees were leaked permanently at relcache flush.
Also, clean up handling of rule creation and deletion --- there was
not sufficient locking of the relation being modified, and there was
no reliable notification of other backends that a relcache reload
was needed. Also, clean up relcache.c code so that scans of system
tables needed to load a relcache entry are done in the caller's
memory context, not in CacheMemoryContext. This prevents any
un-pfreed memory from those scans from becoming a permanent memory
leak.
for details). It doesn't really do that much yet, since there are no
short-term memory contexts in the executor, but the infrastructure is
in place and long-term contexts are handled reasonably. A few long-
standing bugs have been fixed, such as 'VACUUM; anything' in a single
query string crashing. Also, out-of-memory is now considered a
recoverable ERROR, not FATAL.
Eliminate a large amount of crufty, now-dead code in and around
memory management.
Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and
backend startup.
to apply the tempname->realname mapping to type name lookup as well
as relation name lookup, else the type tuple will not be found when
wanted. This fixes bugs like this one:
create temp table foo (f1 int);
select foo.f2 from foo;
ERROR: Unable to locate type name 'foo' in catalog
are opened in a consistent order by different backends (I ordered them
by index OID because that's easy, but any other consistent order would
do as well). This avoids potential deadlock for index types that we
acquire exclusive locks on ... ie, rtree.
entries now for int8 and network hash indexes. int24_ops and int42_ops
are gone. pg_opclass no longer contains multiple entries claiming to be
the default opclass for the same datatype. opr_sanity regress test
extended to catch errors like these in the future.
materialized tupleset is small enough) instead of a temporary relation.
This was something I was thinking of doing anyway for performance, and Jan
says he needs it for TOAST because he doesn't want to cope with toasting
noname relations. With this change, the 'noname table' support in heap.c
is dead code, and I have accordingly removed it. Also clean up 'noname'
plan handling in planner --- nonames are either sort or materialize plans,
and it seems less confusing to handle them separately under those names.
discussion of 5/19/00). pg_index is now searched for indexes of a
relation using an indexscan. Moreover, this is done once and cached
in the relcache entry for the relation, in the form of a list of OIDs
for the indexes. This list is used by the parser and executor to drive
lookups in the pg_index syscache when they want to know the properties
of the indexes. Net result: index information will be fully cached
for repetitive operations such as inserts.
pointers, namely the catcache tuple fetch routines. Also get rid of
the unused and possibly confusing 'size' field in struct cachedesc.
Since it doesn't allow for variable-length fields, anyone who
actually trusted it would likely be making a mistake...
over multiple lookups --- it should use SearchSysCacheTupleCopy instead.
This accounts for rare failures like 'init_fcache: null probin for procedure 481'
when running concurrently with a VACUUM.
inputs have been converted to newstyle. This should go a long way towards
fixing our portability problems with platforms where char and short
parameters are passed differently from int-width parameters. Still
more to do for the Alpha port however.
key call sites are changed, but most called functions are still oldstyle.
An exception is that the PL managers are updated (so, for example, NULL
handling now behaves as expected in plperl and plpgsql functions).
NOTE initdb is forced due to added column in pg_proc.
really ought to fix relcache entry construction so that it does not
do so much with CurrentMemoryContext = CacheCxt. As is, relatively
harmless leaks in either sequential or index scanning translate to
permanent leaks if they occur when called from relcache build.
For the moment, however, the path of least resistance is to repair
all such leaks...
pg_internal.init file in-place, which meant that if another backend
started at about the same time, it might read the incomplete file.
init_irels tries to guard against that, but I have now seen a crash
due to reading bad data from a partly-written file. (This may indicate
a kernel bug on my platform? Not sure.) Anyway, clearly the safest
course is to write the new pg_internal.init file under a unique temporary
filename, and rename it into place only after it's all written.
running gcc and HP's cc with warnings cranked way up. Signed vs unsigned
comparisons, routines declared static and then defined not-static,
that kind of thing. Tedious, but perhaps useful...
the cache context, it didn't bother to free the tuple that
CatalogIndexFetchTuple had allocated in the transaction context.
Do enough cache lookups in the same xact, and you start to notice...
syscache and relcache flushes). Relcache entry rebuild now preserves
original tupledesc, rewrite rules, and triggers if possible, so that pointers
to these things remain valid --- if these things change while relcache entry
has positive refcount, we elog(ERROR) to avoid later crash. Arrange for
xact-local rels to be rebuilt when an SI inval message is seen for them,
so that they are updated by CommandCounterIncrement the same as regular rels.
(This is useful because of Hiroshi's recent changes to process our own SI
messages at CommandCounterIncrement time.) This allows simplification of
some routines that previously hacked around the lack of an automatic update.
catcache now keeps its own copy of tupledesc for its relation, rather than
depending on the relcache's copy; this avoids needing to reinitialize catcache
during a cache flush, which saves some cycles and eliminates nasty circularity
problems that occur if a cache flush happens while trying to initialize a
catcache.
Eliminate a number of permanent memory leaks that used to happen during
catcache or relcache flush; not least of which was that catcache never
freed any cached tuples! (Rule parsetree storage is still leaked, however;
will fix that separately.)
Nothing done yet about code that uses tuples retrieved by SearchSysCache
for longer than is safe.
we *always* rebuild, rather than deleting, an invalidated relcache entry
that has positive refcount. Otherwise an SI cache overrun leads to
dangling Relation pointers all over the place!
an attribute of a tuple previously fetched with SearchSysCacheTuple.
This avoids a lot of redundant cache lookups, particularly in selfuncs.c.
Also, remove SearchSysCacheStruct, which was unused and grotty.
relcache entry no longer leaks a small amount of memory. index_endscan
now releases all the memory acquired by index_beginscan, so callers of it
should NOT pfree the scan descriptor anymore.
Make all system indexes unique.
Make all cache loads use system indexes.
Rename *rel to *relid in inheritance tables.
Rename cache names to be clearer.
inval.c thought it could safely use the catcache to look up the OIDs of
system relations. Not good, considering that inval.c could be called
during catcache loading, if a shared-inval message arrives. Rip out the
lookup logic and instead use the known OIDs from pg_class.h.
Now indexes of pg_class and pg_type are unique indexes
and guarantee the uniqueness of correponding attributes.
heap_create() was changed to take another boolean parameter
which allows to postpone the creation of disk file.
The name of rd_nonameunlinked was changed to rd_unlinked.
It is used generally(not only for noname relations) now.
Requires initdb.
expressions in CREATE TABLE. There is no longer an emasculated expression
syntax for these things; it's full a_expr for constraints, and b_expr
for defaults (unfortunately the fact that NOT NULL is a part of the
column constraint syntax causes a shift/reduce conflict if you try a_expr.
Oh well --- at least parenthesized boolean expressions work now). Also,
stored expression for a column default is not pre-coerced to the column
type; we rely on transformInsertStatement to do that when the default is
actually used. This means "f1 datetime default 'now'" behaves the way
people usually expect it to.
BTW, all the support code is now there to implement ALTER TABLE ADD
CONSTRAINT and ALTER TABLE ADD COLUMN with a default value. I didn't
actually teach ALTER TABLE to call it, but it wouldn't be much work.
additional argument specifying the kind of lock to acquire/release (or
'NoLock' to do no lock processing). Ensure that all relations are locked
with some appropriate lock level before being examined --- this ensures
that relevant shared-inval messages have been processed and should prevent
problems caused by concurrent VACUUM. Fix several bugs having to do with
mismatched increment/decrement of relation ref count and mismatched
heap_open/close (which amounts to the same thing). A bogus ref count on
a relation doesn't matter much *unless* a SI Inval message happens to
arrive at the wrong time, which is probably why we got away with this
sloppiness for so long. Repair missing grab of AccessExclusiveLock in
DROP TABLE, ALTER/RENAME TABLE, etc, as noted by Hiroshi.
Recommend 'make clean all' after pulling this update; I modified the
Relation struct layout slightly.
Will post further discussion to pghackers list shortly.
was rejecting negative attnums as bogus, which of course they are not.
Add code to get_attdisbursion to produce a useful value for OID attribute,
since VACUUM does not store stats for system attributes.
Also, repair bug that's been in eqjoinsel for a long time: it was taking
the max of the two columns' disbursions, whereas it should use the min.
current transaction) are not flushed by shared-cache-inval reset message.
SI reset actually works now, for probably the first time in a long time.
I was able to run initdb and regression tests with a 16-element SI message
array, with a lot of NOTICE: cache state reset messages but no crashes.
real name before doing lookup. We only want to index temp tables by their
real names in the relcache, to ensure there's not more than one relcache
entry for them.
has positive refcount, it is rebuilt from pg_class data. This ensures
that relcache entries will track changes made by other backends. Formerly,
a shared inval report would just be ignored if it happened to arrive while
the relcache entry was in use. Also, fix relcache to reset ref counts
to zero during transaction abort. Finally, change LockRelation() so that
it checks for shared inval reports after obtaining the lock. In this way,
once any kind of lock has been obtained on a rel, we can trust the relcache
entry to be up-to-date.
insight that RelationFlushRelation ought to invoke smgrclose, and that the
way to make that work is to ensure that mdclose doesn't fail if the relation
is already closed (or unlinked, if we are looking at a DROP TABLE). While
I was testing that, I was able to identify several problems that we had
with multiple-segment relations. The system is now able to do initdb and
pass the regression tests with a very small segment size (I had it set to
64Kb per segment for testing). I don't believe that ever worked before.
File descriptor leaks seem to be gone too.
I have partially addressed the concerns we had about mdtruncate(), too.
On a Win32 or NFS filesystem it is not possible to unlink a file that
another backend is holding open, so what md.c now does is to truncate
unwanted files to zero length before trying to unlink them. The other
backends will be forced to close their open files by relation cache
invalidation --- but I think it would take considerable work to make
that happen before vacuum truncates the relation rather than after.
Leaving zero-length files lying around seems a usable compromise.
to save a little bit of backend startup time. This way, the first
backend started after a VACUUM will rebuild the init file with up-to-date
statistics for the critical system indexes.
Ok. I made patches replacing all of "#if FALSE" or "#if 0" to "#ifdef
NOT_USED" for current. I have tested these patches in that the
postgres binaries are identical.
I don't know if this is really related to the initdb problem
discussion (haven't followed it enough). But seems so because
it fixes a damn problem during index tuple insertion on
CREATE TABLE into pg_attribute_relid_attnum_index.
Anyway - this bug was really hard to find. During startup the
relcache reads in some prepared information about index
strategies from a file and then reinitializes the function
pointers inside the scanKey data. But for sake it assumed
single attribute index tuples (hasn't that changed recently).
Thus not all the strategies scanKey entries where initialized
properly, resulting in invalid addresses for the btree
comparision functions.
With the patch at the end the regression tests passed
excellent except for the sanity_check that crashed at vacuum
and the misc test where the select unique1 from onek2 outputs
the two rows in different order.
Jan
if MULTIBYTE is not enabled. So be sure to run initdb.
o these patches are made against the latest source tree (after
Bruce's massive patch, I think) BTW, I noticed that after running
regression, the oid field of pg_type seems disappeared.
regression=> select oid from pg_type; ERROR: attribute
'oid' not found
this happens after the constraints test. This occures with/without
my patches. strange...
o pg_database_mb.h, pg_class_mb.h, pg_attribute_mb.h are no longer
used, and shoud be removed.
o GetDatabaseInfo() in utils/misc/database.c removed (actually in
#ifdef 0). seems nobody uses.
t-ishii@sra.co.jp
no longer returns buffer pointer, can be gotten from scan;
descriptor; bootstrap can create multi-key indexes;
pg_procname index now is multi-key index; oidint2, oidint4, oidname
are gone (must be removed from regression tests); use System Cache
rather than sequential scan in many places; heap_modifytuple no
longer takes buffer parameter; remove unused buffer parameter in
a few other functions; oid8 is not index-able; remove some use of
single-character variable names; cleanup Buffer variables usage
and scan descriptor looping; cleaned up allocation and freeing of
tuples; 18k lines of diff;
As Bruce mentioned, this is due to the conflict among changes we made.
Included patches should fix the problem(I changed all MB to
MULTIBYTE). Please let me know if you have further problem.
P.S. I did not include pathces to configure and gram.c to save the
file size(configure.in and gram.y modified).
From: t-ishii@sra.co.jp
Attached are patches to enhance the multi-byte support. (patches are
against 7/18 snapshot)
* determine encoding at initdb/createdb rather than compile time
Now initdb/createdb has an option to specify the encoding. Also, I
modified the syntax of CREATE DATABASE to accept encoding option. See
README.mb for more details.
For this purpose I have added new column "encoding" to pg_database.
Also pg_attribute and pg_class are changed to catch up the
modification to pg_database. Actually I haved added pg_database_mb.h,
pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is
enabled. The reason having separate files is I couldn't find a way to
use ifdef or whatever in those files. I have to admit it looks
ugly. No way.
* support for PGCLIENTENCODING when issuing COPY command
commands/copy.c modified.
* support for SQL92 syntax "SET NAMES"
See gram.y.
* support for LATIN2-5
* add UNICODE regression test case
* new test suite for MB
New directory test/mb added.
* clean up source files
Basic idea is to have MB's own subdirectory for easier maintenance.
These are include/mb and backend/utils/mb.
1. Removes the unnecessary "#define AbcRegProcedure 123"'s from
pg_proc.h.
2. Changes those #defines to use the names already defined in
fmgr.h.
3. Forces the make of fmgr.h in backend/Makefile instead of having
it
made as a dependency in access/common/Makefile *hack*hack*hack*
4. Rearranged the #includes to a less helter-skelter arrangement,
also
changing <file.h> to "file.h" to signify a non-system header.
5. Removed "pg_proc.h" from files where its only purpose was for
the
#defines removed in item #1.
6. Added "fmgr.h" to each file changed for completeness sake.
Turns out that #6 was not necessary for some files because fmgr.h
was being included in a roundabout way SIX levels deep by the first
include.
"access/genam.h"
->"access/relscan.h"
->"utils/rel.h"
->"access/strat.h"
->"access/skey.h"
->"fmgr.h"
So adding fmgr.h really didn't add anything to the compile, hopefully
just made it clearer to the programmer.
S Darren.
Attached you'll find a (big) patch that fixes make dep and make
depend in all Makefiles where I found it to be appropriate.
It also removes the dependency in Makefile.global for NAMEDATALEN
and OIDNAMELEN by making backend/catalog/genbki.sh and bin/initdb/initdb.sh
a little smarter.
This no longer requires initdb.sh that is turned into initdb with
a sed script when installing Postgres, hence initdb.sh should be
renamed to initdb (after the patch has been applied :-) )
This patch is against the 6.3 sources, as it took a while to
complete.
Please review and apply,
Cheers,
Jeroen van Vianen
1. Remove the char2, char4, char8 and char16 types from postgresql
2. Change references of char16 to name in the regression tests.
3. Rename the char16.sql regression test to name.sql. 4. Modify
the regression test scripts and outputs to match up.
Might require new regression.{SYSTEM} files...
Darren King
seems that my last post didn't make it through. That's good
since the diff itself didn't covered the renaming of
pg_user.h to pg_shadow.h and it's new content.
Here it's again. The complete regression test passwd with
only some float diffs. createuser and destroyuser work.
pg_shadow cannot be read by ordinary user.
The diff looks so simple and easy. But to find it wasn't fun.
It must have been there for a long time. What happened:
When a tuple in one of some central catalogs was updated, the
referenced relation got flushed, so it would be reopened on
the next access (to reflect new triggers, rules and table
structure changes into the relation cache).
Some data (the tupleDescriptor e.g.) is used in the system
cache too. So when a relation is subject to the system cache,
this must know too that a cached system relation got flushed
because the tupleDesc data gets freed during the flush!
For the GRANT/REVOKE on pg_class it was slightly different.
There is some local data in inval.c that gets initialized on
the first invalidation of a tuple in some central catalogs.
This needs a SysCache lookup in pg_class. But when the first
of all commands is a GRANT on pg_class, exactly the needed
tuple is the one actually invalidated. So I added little code
snippets that the initialization of the local variables in
inval.c will already happen during InitPostgres().
Attached is the patch to fix the warning messages from my code. I also
fixed one which wasn't my code. Apart from the usual warnings about the
bison/yacc generated code I only have one other warning message. This
is in gramm.y around line 2234. I wasn't sure of the fix.
I've also replaced all the calls to free() in gramm.y to calls to
pfree(). Without these I was getting backend crashes with GRANT. This
might already have been fixed.
Patch by: wieck@sapserv.debis.de (Jan Wieck)
One of the design rules of PostgreSQL is extensibility. And
to follow this rule means (at least for me) that there should
not only be a builtin PL. Instead I would prefer a defined
interface for PL implemetations.
Makefile.global.
End result, if all goes well, should allow for much easier porting, since
there will no longer be a concept of a "port". Most, if not everything,
*should* be determined by configure, or by the compiler itself. Still
work to be done though :)
/*
* RelationFlushRelation () below will flush relation information
* from the cache. We must call smgrclose to flush relation
* information from SMGR & FMGR, too. We assume that for temp
* relations smgrunlink is already called by heap_destroyr
* and we skip smgrclose for them. - vadim 05/22/97
*/
smgrclose(reln->rd_rel->relsmgr, reln);
- it avoids memory leaks in SMGR & VFD.
RelationFlushRelation():
there is no more call FileInvalidate(RelationGetSystemPort(relation));
- invalid (FileInvalidate() expects File, not SMGR' fd)
- unuseful anyway.
Changes:
* Unique index capability works using the syntax 'create unique
index'.
* Duplicate OID's in the system tables are removed. I put
little scripts called 'duplicate_oids' and 'find_oid' in
include/catalog that help to find and remove duplicate OID's.
I also moved 'unused_oids' from backend/catalog to
include/catalog, since it has to be in the same directory
as the include files in order to work.
* The backend tries converting the name of a function or aggregate
to all lowercase if the original name given doesn't work (mostly
for compatibility with ODBC).
* You can 'SELECT NULL' to your heart's content.
* I put my _bt_updateitem fix in instead, which uses
_bt_insertonpg so that even if the new key is so big that
the page has to be split, everything still works.
* All literal references to system catalog OID's have been
replaced with references to define'd constants from the catalog
header files.
* I added a couple of node copy functions. I think this was a
preliminary attempt to get rules to work.