< Currently indexes do not have enough tuple tuple visibility
< information to allow data to be pulled from the index without
< also accessing the heap. One way to allow this is to set a bit
< to index tuples to indicate if a tuple is currently visible to
< all transactions when the first valid heap lookup happens. This
< bit would have to be cleared when a heap tuple is expired.
> Currently indexes do not have enough tuple visibility information
> to allow data to be pulled from the index without also accessing
> the heap. One way to allow this is to set a bit to index tuples
> to indicate if a tuple is currently visible to all transactions
> when the first valid heap lookup happens. This bit would have to
> be cleared when a heap tuple is expired.
logic operations during planning. Seems cleaner to create two new Path
node types, instead --- this avoids duplication of cost-estimation code.
Also, create an enable_bitmapscan GUC parameter to control use of bitmap
plans.
< Bitmap indexes index single columns that can be combined with other bitmap
< indexes to dynamically create a composite index to match a specific query.
< Each index is a bitmap, and the bitmaps are bitwise AND'ed or OR'ed to be
< combined. They can index by tid or can be lossy requiring a scan of the
< heap page to find matching rows, or perhaps use a mixed solution where
< tids are recorded for pages with only a few matches and per-page bitmaps
< are used for more dense pages. Another idea is to use a 32-bit bitmap
< for every page and set a bit based on the item number mod(32).
> This feature allows separate indexes to be ANDed or ORed together. This
> is particularly useful for data warehousing applications that need to
> query the database in an many permutations. This feature scans an index
> and creates an in-memory bitmap, and allows that bitmap to be combined
> with other bitmap created in a similar way. The bitmap can either index
> all TIDs, or be lossy, meaning it records just page numbers and each
> page tuple has to be checked for validity in a separate pass.
>>>
>>>No, and I think it should be in the manual as an example.
>>>
>>>You will need to enter a loop that uses exception handling to detect
>>>unique_violation.
>>
>>Pursuant to an IRC discussion to which Dennis Bjorklund and
>>Christopher Kings-Lynne made most of the contributions, please find
>>enclosed an example patch demonstrating an UPSERT-like capability.
>>
David Fetter
>
> No, and I think it should be in the manual as an example.
>
> You will need to enter a loop that uses exception handling to detect
> unique_violation.
Pursuant to an IRC discussion to which Dennis Bjorklund and
Christopher Kings-Lynne made most of the contributions, please find
enclosed an example patch demonstrating an UPSERT-like capability.
David Fetter
< failure.
> failure. This could be triggered by a user command or a timer.
< * Force archiving of partially-full WAL files when pg_stop_backup() is
< called or the server is stopped
> * Automatically force archiving of partially-filled WAL files when
> pg_stop_backup() is called or the server is stopped
indexes. Extend the macros in include/catalog/*.h to carry the info
about hand-assigned OIDs, and adjust the genbki script and bootstrap
code to make the relations actually get those OIDs. Remove the small
number of RelOid_pg_foo macros that we had in favor of a complete
set named like the catname.h and indexing.h macros. Next phase will
get rid of internal use of names for looking up catalogs and indexes;
but this completes the changes forcing an initdb, so it looks like a
good place to commit.
Along the way, I made the shared relations (pg_database etc) not be
'bootstrap' relations any more, so as to reduce the number of hardwired
entries and simplify changing those relations in future. I'm not
sure whether they ever really needed to be handled as bootstrap
relations, but it seems to work fine to not do so now.
avoid encroaching on the 'user' range of OIDs by allowing automatic
OID assignment to use values below 16k until we reach normal operation.
initdb not forced since this doesn't make any incompatible change;
however a lot of stuff will have different OIDs after your next initdb.
be supported for all datatypes. Add CREATE AGGREGATE and pg_dump support
too. Add specialized min/max aggregates for bpchar, instead of depending
on text's min/max, because otherwise the possible use of bpchar indexes
cannot be recognized.
initdb forced because of catalog changes.
the long-term plan for this behavior for quite some time, but it is only
possible now that DELETE has a USING clause so that the user can join
other tables in a DELETE statement without relying on this behavior.
output parameters or VOID or a set. There seems no particular reason to
insist on a RETURN in these cases, since the function return value is
determined by other elements anyway. Per recent discussion.
in UPDATE. We also now issue a NOTICE if a query has _any_ implicit
range table entries -- in the past, we would only warn about implicit
RTEs in SELECTs with at least one explicit RTE.
As a result of the warning change, 25 of the regression tests had to
be updated. I also took the opportunity to remove some bogus whitespace
differences between some of the float4 and float8 variants. I believe
I have correctly updated all the platform-specific variants, but let
me know if that's not the case.
Original patch for DELETE ... USING from Euler Taveira de Oliveira,
reworked by Neil Conway.
OPENed on non-SELECT commands such as EXPLAIN or SHOW (anything that
returns tuples is allowed). This flexibility already existed for
bound cursors, but OPEN was artificially restricting what it would
take. Per a gripe some months back.
proposal for OUT parameter support. The columns don't actually *do*
anything yet, they are just left NULLs. But I thought I'd commit this
part separately as a fairly pure example of the tasks needed when adding
a column to pg_proc or one of the other core system tables.
change saves a great deal of space in pg_proc and its primary index,
and it eliminates the former requirement that INDEX_MAX_KEYS and
FUNC_MAX_ARGS have the same value. INDEX_MAX_KEYS is still embedded
in the on-disk representation (because it affects index tuple header
size), but FUNC_MAX_ARGS is not. I believe it would now be possible
to increase FUNC_MAX_ARGS at little cost, but haven't experimented yet.
There are still a lot of vestigial references to FUNC_MAX_ARGS, which
I will clean up in a separate pass. However, getting rid of it
altogether would require changing the FunctionCallInfoData struct,
and I'm not sure I want to buy into that.
access: define new index access method functions 'amgetmulti' that can
fetch multiple TIDs per call. (The functions exist but are totally
untested as yet.) Since I was modifying pg_am anyway, remove the
no-longer-needed 'rel' parameter from amcostestimate functions, and
also remove the vestigial amowner column that was creating useless
work for Alvaro's shared-object-dependencies project.
Initdb forced due to changes in pg_am.
executing a statement that fires triggers. Formerly this time was
included in "Total runtime" but not otherwise accounted for.
As a side benefit, we avoid re-opening relations when firing non-deferred
AFTER triggers, because the trigger code can re-use the main executor's
ResultRelInfo data structure.
currently does. This is now the default Win32 wal sync method because
we perfer o_datasync to fsync.
Also, change Win32 fsync to a new wal sync method called
fsync_writethrough because that is the behavior of _commit, which is
what is used for fsync on Win32.
Backpatch to 8.0.X.
< * Add ANSI INTERVAL handling
> * Add ISo INTERVAL handling
< o Interpret syntax that isn't uniquely ANSI or PG, like '1:30' or
< '1' as ANSI syntax, e.g. interpret '1:30' MINUTE TO SECOND as
> o Interpret syntax that isn't uniquely ISO or PG, like '1:30' or
> '1' as ISO syntax, e.g. interpret '1:30' MINUTE TO SECOND as
649c649
< * Add pre-parsing phase that converts non-ANSI syntax to supported
> * Add pre-parsing phase that converts non-ISO syntax to supported
< o Process mixed ANSI/PG syntax, and round value to requested
< precision or generate an error
< o Interpret INTERVAL '1 year' MONTH as CAST (INTERVAL '1 year' AS
< INTERVAL MONTH), and this should return '12 months'
194a191,194
> o Interpret INTERVAL '1 year' MONTH as CAST (INTERVAL '1 year' AS
> INTERVAL MONTH), and this should return '12 months'
> o Round or truncate values to the requested precision, e.g.
> INTERVAL '11 months' AS YEAR should return one or zero
< o Add support for day-time syntax, INTERVAL '1 2:03:04'
> o Add support for day-time syntax, INTERVAL '1 2:03:04'
192c192,194
< o Interpret INTERVAL '1:30' MINUTE TO SECOND as '1 minute 30 seconds'
> o Interpret syntax that isn't uniquely ANSI or PG, like '1:30' or
> '1' as ANSI syntax, e.g. interpret '1:30' MINUTE TO SECOND as
> '1 minute 30 seconds'
< * Add support for ANSI time INTERVAL syntax, INTERVAL '1 2:03:04' DAY TO SECOND
< * Add support for ANSI date INTERVAL syntax, INTERVAL '20-6' YEAR TO MONTH
< * Process mixed ANSI/PG INTERVAL syntax, and round value to requested precision
<
< Interpret INTERVAL '1 year' MONTH as CAST (INTERVAL '1 year' AS INTERVAL
< MONTH), and this should return '12 months'
<
< * Interpret INTERVAL '1:30' MINUTE TO SECOND as '1 minute 30 seconds'
> * Add ANSI INTERVAL handling
> o Add support for day-time syntax, INTERVAL '1 2:03:04'
> DAY TO SECOND
> o Add support for year-month syntax, INTERVAL '50-6' YEAR TO MONTH
> o Process mixed ANSI/PG syntax, and round value to requested
> precision or generate an error
> o Interpret INTERVAL '1 year' MONTH as CAST (INTERVAL '1 year' AS
> INTERVAL MONTH), and this should return '12 months'
> o Interpret INTERVAL '1:30' MINUTE TO SECOND as '1 minute 30 seconds'
> o Support precision, CREATE TABLE foo (a INTERVAL MONTH(3))
< * Add support for ANSI date INTERVAL syntax, INTERVAL '9-3' YEAR TO MONTH
> * Add support for ANSI date INTERVAL syntax, INTERVAL '20-6' YEAR TO MONTH
< * Add support for ANSI date INTERVAL syntax, INTERVAL '1-2' YEAR TO MONTH
> * Add support for ANSI date INTERVAL syntax, INTERVAL '9-3' YEAR TO MONTH
ExclusiveLock rather than AccessExclusiveLock. This will allow concurrent
SELECT queries to proceed on the table. Per discussion with Andrew at
SuperNews.
> * Add support for ANSI time INTERVAL syntax, INTERVAL '1 2:03:04' DAY TO SECOND
> * Add support for ANSI date INTERVAL syntax, INTERVAL '1-2' YEAR TO MONTH
> * Process mixed ANSI/PG INTERVAL syntax, and round value to requested precision
184a188,189
> Interpret INTERVAL '1 year' MONTH as CAST (INTERVAL '1 year' AS INTERVAL
> MONTH), and this should return '12 months'
>
> * Support table partitioning that allows a single table to be stored
> in subtables that are partitioned based on the primary key or a WHERE
> clause
convention for isnull flags. Also, remove the useless InsertIndexResult
return struct from index AM aminsert calls --- there is no reason for
the caller to know where in the index the tuple was inserted, and we
were wasting a palloc cycle per insert to deliver this uninteresting
value (plus nontrivial complexity in some AMs).
I forced initdb because of the change in the signature of the aminsert
routines, even though nothing really looks at those pg_proc entries...
< SQL-spec compliant, so allow such handling to be disabled.
> SQL-spec compliant, so allow such handling to be disabled. However,
> disabling backslashes could break many third-party applications and tools.
of tuples when passing data up through multiple plan nodes. A slot can now
hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting
of Datum/isnull arrays. Upper plan levels can usually just copy the Datum
arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr()
calls to extract the data again. This work extends Atsushi Ogawa's earlier
patch, which provided the key idea of adding Datum arrays to TupleTableSlots.
(I believe however that something like this was foreseen way back in Berkeley
days --- see the old comment on ExecProject.) A test case involving many
levels of join of fairly wide tables (about 80 columns altogether) showed
about 3x overall speedup, though simple queries will probably not be
helped very much.
I have also duplicated some code in heaptuple.c in order to provide versions
of heap_formtuple and friends that use "bool" arrays to indicate null
attributes, instead of the old convention of "char" arrays containing either
'n' or ' '. This provides a better match to the convention used by
ExecEvalExpr. While I have not made a concerted effort to get rid of uses
of the old routines, I think they should be deprecated and eventually removed.
< o Disallow encodings like UTF8 which PostgreSQL supports
< but the operating system does not (already disallowed by
< pginstaller)
> o Add support for Unicode
< To fix UTF8, the data needs to be converted to UTF16 and then
< the Win32 wcscoll() can be used, and perhaps other functions
> To fix this, the data needs to be converted to/from UTF16/UTF8
> so the Win32 wcscoll() can be used, and perhaps other functions
< locales but provides no ordering.
<
> locales but provides no ordering or character set classes.
whether or not it is a security definer. Changing a function's strictness
is required by SQL2003, and the other capabilities make sense. Also, allow
an optional RESTRICT noise word to be specified, for SQL conformance.
Some trivial regression tests added and the documentation has been
updated.
can tell whether it is being used as an aggregate or not. This allows
such a function to avoid re-pallocing a pass-by-reference transition
value; normally it would be unsafe for a function to scribble on an input,
but in the aggregate case it's safe to reuse the old transition value.
Make int8inc() do this. This gets a useful improvement in the speed of
COUNT(*), at least on narrow tables (it seems to be swamped by I/O when
the table rows are wide). Per a discussion in early December with
Neil Conway. I also fixed int_aggregate.c to check this, thereby
turning it into something approaching a supportable technique instead
of being a crude hack.
Formerly, if such a clause contained no aggregate functions we mistakenly
treated it as equivalent to WHERE. Per spec it must cause the query to
be treated as a grouped query of a single group, the same as appearance
of aggregate functions would do. Also, the HAVING filter must execute
after aggregate function computation even if it itself contains no
aggregate functions.
the freelist, plus per-buffer spinlocks that protect access to individual
shared buffer headers. This requires abandoning a global freelist (since
the freelist is a global contention point), which shoots down ARC and 2Q
as well as plain LRU management. Adopt a clock sweep algorithm instead.
Preliminary results show substantial improvement in multi-backend situations.
adjusting values:
> But to be on the safe side, it would make sense to do something similar
> to the BSD section, and comment about older distributions maybe needing
> to manipulate /proc/kernel/* directly.
Mark Kirkwood
! authentication. Use of this environment variable is not
! recommended for security reasons (some operating systems
! allow non-root users to see process environment variables via
! <application>ps</>); instead consider using the
! <filename>~/.pgpass</> file (see <xref linkend="libpq-pgpass">).
< * Allow server configuration parameters to be remotely modified
> * Allow pg_hba.conf settings to be controlled via SQL
>
> This would require a new global table that is dumped to flat file for
> use by the postmaster. We do a similar thing for pg_shadow currently.
>
< * Consider use of open/fcntl(O_DIRECT) to minimize OS caching
> * Consider use of open/fcntl(O_DIRECT) to minimize OS caching,
> especially for WAL writes
in GetNewTransactionId(). Since the limit value has to be computed
before we run any real transactions, this requires adding code to database
startup to scan pg_database and determine the oldest datfrozenxid.
This can conveniently be combined with the first stage of an attack on
the problem that the 'flat file' copies of pg_shadow and pg_group are
not properly updated during WAL recovery. The code I've added to
startup resides in a new file src/backend/utils/init/flatfiles.c, and
it is responsible for rewriting the flat files as well as initializing
the XID wraparound limit value. This will eventually allow us to get
rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add
a trigger to pg_database.
more variables can be found in the libpq manual section.
Mention .pgpass in the psql manual page section dealing with connection
parameters and point to the libpq section for more details.
Backpatch to 8.0.X.
the item:
< o Automatic failover
<
< The proper solution to this will probably the use of a master/slave
< replication solution like Sloney and a connection pooling tool like
< pgpool.
<
< all temporary tables, removal of any NOTIFYs, etc. This could be used
< for connection pooling. We could also change RESET ALL to have this
< functionality.
> all temporary tables, removal of any NOTIFYs, cursors, prepared
> queries(?), currval()s, etc. This could be used for connection pooling.
> We could also change RESET ALL to have this functionality.
tests. Contributed by Koju Iijima, review from Neil Conway, Gavin Sherry
and Tom Lane.
Also, fix error in description of WITH CHECK OPTION clause in the CREATE
VIEW reference page: it should be "CASCADED", not "CASCADE".
