Commit Graph

2440 Commits

Author SHA1 Message Date
Andrew Dunstan
bbe4deac1b Return valid json when converting an empty hstore.
Oskari Saarenmaa.
2013-10-17 11:12:48 -04:00
Peter Eisentraut
5b6d08cd29 Add use of asprintf()
Add asprintf(), pg_asprintf(), and psprintf() to simplify string
allocation and composition.  Replacement implementations taken from
NetBSD.

Reviewed-by: Álvaro Herrera <alvherre@2ndquadrant.com>
Reviewed-by: Asif Naeem <anaeem.it@gmail.com>
2013-10-13 00:09:18 -04:00
Peter Eisentraut
264aa14a2f pg_upgrade: Split off pg_fatal() from pg_log()
This allows decorating pg_fatal() with noreturn compiler hints, leading
to better diagnostics.

Reviewed-by: Marko Tiikkaja <marko@joh.to>
2013-10-09 22:48:41 -04:00
Kevin Grittner
f566515192 Add record_image_ops opclass for matview concurrent refresh.
REFRESH MATERIALIZED VIEW CONCURRENTLY was broken for any matview
containing a column of a type without a default btree operator
class.  It also did not produce results consistent with a non-
concurrent REFRESH or a normal view if any column was of a type
which allowed user-visible differences between values which
compared as equal according to the type's default btree opclass.
Concurrent matview refresh was modified to use the new operators
to solve these problems.

Documentation was added for record comparison, both for the
default btree operator class for record, and the newly added
operators.  Regression tests now check for proper behavior both
for a matview with a box column and a matview containing a citext
column.

Reviewed by Steve Singer, who suggested some of the doc language.
2013-10-09 14:26:09 -05:00
Bruce Momjian
9299f61798 unaccent: mark unaccent() functions as immutable
Suggestion from Pavel Stehule
2013-10-08 12:20:36 -04:00
Noah Misch
7db0dc0eac pgbench: Comment on thread timing hazards.
Reviewed by Fabien COELHO.
2013-10-06 09:57:26 -04:00
Noah Misch
2d6c0f10ef pgbench: Elaborate latency reporting.
Isolate transaction latency (elapsed time between submitting first
command and receiving response to last command) from client-side delays
pertaining to the --rate schedule.  Under --rate, report schedule lag as
defined in the documentation.  Report latency standard deviation
whenever we collect the measurements to do so.  All of these changes
affect --progress messages and the final report.

Fabien COELHO, reviewed by Pavel Stehule.
2013-10-05 17:33:38 -04:00
Noah Misch
8e00a3850d pgbench: Remove stray use of "float" math.
Oversight in commit 4a87f308b3.

Fabien COELHO
2013-10-05 17:19:37 -04:00
Alvaro Herrera
dddc91ddd3 Remove broken PGXS code for pg_xlogdump
With the PGXS boilerplate in place, pg_xlogdump currently fails with an
ominous error message that certain targets cannot be built because
certain files do not exist.  Remove that and instead throw a quick error
message alerting the user of the actual problem, which should be easier
to diagnose that the statu quo.

Andres Freund
2013-10-01 17:36:15 -03:00
Andrew Dunstan
7ce569babd Add missing condition for pg_depend in hstore migration script.
Error noted by Andres Freund.
2013-09-30 11:33:54 -04:00
Fujii Masao
514b3194e8 Correct comment of pgbench "filler" columns.
Pavan Deolasee
2013-09-30 12:35:54 +09:00
Andrew Dunstan
42bf7fc1de Fix makefile broken by hstore fix. 2013-09-29 22:46:55 -04:00
Andrew Dunstan
a18167510f Use a new hstore extension version for added json functions.
This should have been done when the json functionality was added to
hstore in 9.3.0. To handle this correctly, the upgrade script therefore
uses conditional logic by using plpgsql in a DO statement to add the two
new functions and the new cast. If hstore_to_json_loose is detected as
already present and dependent on the hstore extension nothing is done.
This will require that the database be loaded with plpgsql.

People who have installed the earlier and spurious 1.1 version of hstore
will need to do:

	ALTER EXTENSION hstore UPDATE;

to pick up the new functions properly.
2013-09-29 17:41:56 -04:00
Noah Misch
c2df45a37c pgbench: Correct for bias in --rate schedule generation.
Previous code gave a mean delay 0.44% below target.  This change also
has the effect of increasing the maximum possible delay.

Fabien COELHO
2013-09-25 23:11:34 -04:00
Noah Misch
825da2aba8 pgbench: Tweak documentation.
Fabien COELHO
2013-09-23 15:49:21 -04:00
Bruce Momjian
ff2a1f5e84 pg_upgrade: more C comment fixes 2013-09-23 11:12:09 -04:00
Bruce Momjian
f7cf5fa262 pg_upgrade: fix C comment typo 2013-09-23 11:06:19 -04:00
Kevin Grittner
a49d0b75ce Create index on srt table in citext regression tests.
Comments and the tests make clear that the intent is to test with
and without an index, but there was no index.
2013-09-11 16:53:23 -05:00
Bruce Momjian
f0e86930dd fuzzystrmatch: replace broken link in C comment
Albe Laurenz
2013-09-10 21:34:01 -04:00
Bruce Momjian
c155f654b4 intarray: return empty zero-dimensional array for an empty array
Previously a one-dimensional empty array was returned, but its text
representation matched a zero-dimensional array, and there is no way to
dump/reload a one-dimensional empty array.

BACKWARD INCOMPATIBILITY

Per report from elein
2013-09-07 11:44:33 -04:00
Bruce Momjian
66d85c5356 Remove dead URL mention in OSX startup script
Backpatch to 9.3.

Per suggestion from Gavan Schneider
2013-09-04 17:04:33 -04:00
Greg Stark
cfa1cd711c Fix thinko in worker_spi, count(*) returns a bigint. Thanks RhodiumToad 2013-09-03 13:27:34 +01:00
Robert Haas
090d0f2050 Allow discovery of whether a dynamic background worker is running.
Using the infrastructure provided by this patch, it's possible either
to wait for the startup of a dynamically-registered background worker,
or to poll the status of such a worker without waiting.  In either
case, the current PID of the worker process can also be obtained.
As usual, worker_spi is updated to demonstrate the new functionality.

Patch by me.  Review by Andres Freund.
2013-08-28 14:08:13 -04:00
Tom Lane
9e7e29c75a Fix planner problems with LATERAL references in PlaceHolderVars.
The planner largely failed to consider the possibility that a
PlaceHolderVar's expression might contain a lateral reference to a Var
coming from somewhere outside the PHV's syntactic scope.  We had a previous
report of a problem in this area, which I tried to fix in a quick-hack way
in commit 4da6439bd8, but Antonin Houska
pointed out that there were still some problems, and investigation turned
up other issues.  This patch largely reverts that commit in favor of a more
thoroughly thought-through solution.  The new theory is that a PHV's
ph_eval_at level cannot be higher than its original syntactic level.  If it
contains lateral references, those don't change the ph_eval_at level, but
rather they create a lateral-reference requirement for the ph_eval_at join
relation.  The code in joinpath.c needs to handle that.

Another issue is that createplan.c wasn't handling nested PlaceHolderVars
properly.

In passing, push knowledge of lateral-reference checks for join clauses
into join_clause_is_movable_to.  This is mainly so that FDWs don't need
to deal with it.

This patch doesn't fix the original join-qual-placement problem reported by
Jeremy Evans (and indeed, one of the new regression test cases shows the
wrong answer because of that).  But the PlaceHolderVar problems need to be
fixed before that issue can be addressed, so committing this separately
seems reasonable.
2013-08-17 20:22:37 -04:00
Bruce Momjian
ac76ec27b9 pg_upgrade: shut down server after auth failure
Register atexit() server shutdown if pg_ctl successfully started the
server, but we can't connect to it.

Backpatch to 9.3.

Pavel Raiskup
2013-08-16 11:09:09 -04:00
Bruce Momjian
43f6388931 pg_test_fsync: expand ops/sec display
Expand ops/sec by two digits to maintain alignment on servers with fast
I/O subsystems, e.g. can now display < 10M ops/sec with consistent
alignment.
2013-08-02 00:45:19 -04:00
Bruce Momjian
000b65fd2b pg_upgrade: clarify C comment about Windows thread struct pointers
Backpatch to 9.3 to keep source trees consistent.
2013-07-30 09:23:31 -04:00
Tom Lane
55cbfa5366 Fix contrib/cube and contrib/seg to build with bison 3.0.
These modules used the YYPARSE_PARAM macro, which has been deprecated
by the bison folk since 1.875, and which they finally removed in 3.0.
Adjust the code to use the replacement facility, %parse-param, which
is a much better solution anyway since it allows specification of the
type of the extra parser parameter.  We can thus get rid of a lot of
unsightly casting.

Back-patch to all active branches, since somebody might try to build
a back branch with up-to-date tools.
2013-07-29 10:42:37 -04:00
Bruce Momjian
815fcd050f pg_upgrade: fix -j race condition on Windows
Pg_Upgrade cannot write the command string to the log file and then call
system() to write to the same file without causing occasional file-share
errors on Windows.  So instead, write the command string to the log file
after system(), in those cases.
Backpatch to 9.3.
2013-07-27 15:00:58 -04:00
Robert Haas
80c79ab2a8 pgstattuple: Use SnapshotDirty, not SnapshotNow.
Tuples belonging to uncommitted transactions should not be
counted as dead.

This is arguably a bug fix that should be back-patched, but
as no one ever noticed until it came time to try to get rid
of SnapshotNow, I'm only doing this in master for now.
2013-07-25 16:21:13 -04:00
Bruce Momjian
76a7650c40 pg_upgrade: adjust umask() calls
Since pg_upgrade -j on Windows uses threads, calling umask()
before/after opening a file via fopen_priv() is no longer possible, so
set umask() as we enter the thread-creating loop, and reset it on exit.
Also adjust internal fopen_priv() calls to just use fopen().
Backpatch to 9.3beta.
2013-07-25 11:33:15 -04:00
Bruce Momjian
e4c6cccd8c pg_upgrade: fix initialization of thread argument
Reorder initialization of thread argument marker to it happens before
reap_child() is called.
Backpatch to 9.3.
2013-07-24 22:01:31 -04:00
Bruce Momjian
910d3a458c pg_upgrade: more Windows parallel/-j fixes
More fixes to handle Windows thread parameter passing.
Backpatch to 9.3 beta.
Patch originally from Andrew Dunstan
2013-07-24 13:15:47 -04:00
Bruce Momjian
d7de6a4790 pg_upgrade: fix parallel/-j crash on Windows
This fixes the problem of passing the wrong function pointer when doing
parallel copy/link operations on Windows.
Backpatched to 9.3beta.
Found and patch supplied by Andrew Dunstan
2013-07-24 10:00:46 -04:00
Tatsuo Ishii
fc9f4e9f8c Add --rate option.
This controls the target transaction rate to certain tps, rather than
maximum. Patch contributed by Fabien COELHO, reviewed by Greg Smith,
and slight editing by me.
2013-07-23 08:40:22 +09:00
Robert Haas
2e44770fa3 pgrowlocks: Use GetActiveSnapshot() rather than SnapshotNow.
Per discussion, it's desirable to eliminate all remaining uses of
SnapshotNow, because it has unpleasant semantics: race conditions
can result in seeing multiple versions of a concurrently updated
row, or none at all.  By using GetActiveSnapshot() here, callers
will see exactly those rows that would have been visible if the
invoking query had scanned the table using, for example, a SELECT
statement.

This is slightly different from the old behavior, because commits
that happen concurrently with the scan will not affect the results.
In REPEATABLE READ or SERIALIZABLE modes, where transaction
snapshots are used, commits that have happened since the start of
the transaction will also not affect the results.  It is hoped
that this minor incompatibility will be thought an improvement,
or at least no worse than what we did before.
2013-07-22 16:21:14 -04:00
Robert Haas
f40a318eea Remove bgw_sighup and bgw_sigterm.
Per discussion on pgsql-hackers, these aren't really needed.  Interim
versions of the background worker patch had the worker starting with
signals already unblocked, which would have made this necessary.
But the final version does not, so we don't really need it; and it
doesn't work well with the new facility for starting dynamic background
workers, so just rip it out.

Also per discussion on pgsql-hackers, back-patch this change to 9.3.
It's best to get the API break out of the way before we do an
official release of this facility, to avoid more pain for extension
authors later.
2013-07-22 14:13:00 -04:00
Robert Haas
0518eceec3 Adjust HeapTupleSatisfies* routines to take a HeapTuple.
Previously, these functions took a HeapTupleHeader, but upcoming
patches for logical replication will introduce new a new snapshot
type under which the tuple's TID will be used to lookup (CMIN, CMAX)
for visibility determination purposes.  This makes that information
available.  Code churn is minimal since HeapTupleSatisfiesVisibility
took the HeapTuple anyway, and deferenced it before calling the
satisfies function.

Independently of logical replication, this allows t_tableOid and
t_self to be cross-checked via assertions in tqual.c.  This seems
like a useful way to make sure that all callers are setting these
values properly, which has been previously put forward as
desirable.

Andres Freund, reviewed by Álvaro Herrera
2013-07-22 13:38:44 -04:00
Fujii Masao
6f9e39bc99 Fix typo in update scripts for some contrib modules. 2013-07-19 04:13:01 +09:00
Fujii Masao
1dc118660b Fix pgstattuple functions to use regclass-type as the argument.
This allows us to specify the target relation with several expressions,
'relname', 'schemaname.relname' and OID in all pgstattuple functions.
pgstatindex() and pg_relpages() could not accept OID as the argument
so far.

Per discussion on -hackers, we decided to keep two types of interfaces,
with regclass-type and TEXT-type argument, for each pgstattuple
function because of the backward-compatibility issue. The functions
which have TEXT-type argument will be deprecated in the future release.

Patch by Satoshi Nagayasu, reviewed by Rushabh Lathia and Fujii Masao.
2013-07-19 03:50:20 +09:00
Fujii Masao
750f43685b Fix typo in previous pgbench --progress patch. 2013-07-18 01:21:30 +09:00
Noah Misch
b560ec1b0d Implement the FILTER clause for aggregate function calls.
This is SQL-standard with a few extensions, namely support for
subqueries and outer references in clause expressions.

catversion bump due to change in Aggref and WindowFunc.

David Fetter, reviewed by Dean Rasheed.
2013-07-16 20:15:36 -04:00
Tatsuo Ishii
4a87f308b3 Add --progress option to show progress report
Patch contributed by Fabien COELHO, reviewed by KONDO Mitsumasa.
2013-07-17 08:05:37 +09:00
Robert Haas
7f7485a0cd Allow background workers to be started dynamically.
There is a new API, RegisterDynamicBackgroundWorker, which allows
an ordinary user backend to register a new background writer during
normal running.  This means that it's no longer necessary for all
background workers to be registered during processing of
shared_preload_libraries, although the option of registering workers
at that time remains available.

When a background worker exits and will not be restarted, the
slot previously used by that background worker is automatically
released and becomes available for reuse.  Slots used by background
workers that are configured for automatic restart can't (yet) be
released without shutting down the system.

This commit adds a new source file, bgworker.c, and moves some
of the existing control logic for background workers there.
Previously, there was little enough logic that it made sense to
keep everything in postmaster.c, but not any more.

This commit also makes the worker_spi contrib module into an
extension and adds a new function, worker_spi_launch, which can
be used to demonstrate the new facility.
2013-07-16 13:02:15 -04:00
Robert Haas
67ccbb080d vacuumlo: Use a cursor to limit client-side memory usage.
This prevents the client from gobbling up too much memory when the
number of large objects to be removed is very large.

Andrew Dunstan, reviewed by Josh Kupershmidt
2013-07-15 10:51:27 -04:00
Heikki Linnakangas
b5ed21998c Fix pg_test_fsync, broken by xloginsert scaling patch.
I didn't realize that ALIGNOF_XLOG_BUFFER was used in pg_test_fsync.
2013-07-08 13:04:07 +03:00
Peter Eisentraut
55f100efc6 pgbench: Silence compiler warning
Explicitly ignore return value from write(), to silence warning.  This
warning only appeared under --disable-thread-safety.
2013-07-07 07:27:22 -04:00
Fujii Masao
2ef085d0e6 Get rid of pg_class.reltoastidxid.
Treat TOAST index just the same as normal one and get the OID
of TOAST index from pg_index but not pg_class.reltoastidxid.
This change allows us to handle multiple TOAST indexes, and
which is required infrastructure for upcoming
REINDEX CONCURRENTLY feature.

Patch by Michael Paquier, reviewed by Andres Freund and me.
2013-07-04 03:24:09 +09:00
Bruce Momjian
cce5d851ed pg_upgrade: revert changing '' to ""
On the command line, GUC option strings are handled by the guc parser,
not by the shell parser, so '' is the proper way to represent a
zero-length string.  This reverts commit
3132a9b7ab.
2013-07-02 10:29:49 -04:00
Robert Haas
568d4138c6 Use an MVCC snapshot, rather than SnapshotNow, for catalog scans.
SnapshotNow scans have the undesirable property that, in the face of
concurrent updates, the scan can fail to see either the old or the new
versions of the row.  In many cases, we work around this by requiring
DDL operations to hold AccessExclusiveLock on the object being
modified; in some cases, the existing locking is inadequate and random
failures occur as a result.  This commit doesn't change anything
related to locking, but will hopefully pave the way to allowing lock
strength reductions in the future.

The major issue has held us back from making this change in the past
is that taking an MVCC snapshot is significantly more expensive than
using a static special snapshot such as SnapshotNow.  However, testing
of various worst-case scenarios reveals that this problem is not
severe except under fairly extreme workloads.  To mitigate those
problems, we avoid retaking the MVCC snapshot for each new scan;
instead, we take a new snapshot only when invalidation messages have
been processed.  The catcache machinery already requires that
invalidation messages be sent before releasing the related heavyweight
lock; else other backends might rely on locally-cached data rather
than scanning the catalog at all.  Thus, making snapshot reuse
dependent on the same guarantees shouldn't break anything that wasn't
already subtly broken.

Patch by me.  Review by Michael Paquier and Andres Freund.
2013-07-02 09:47:01 -04:00