Commit Graph

2162 Commits

Author SHA1 Message Date
Tom Lane
d9b31e4859 Fix some more bugs in contrib/xml2's xslt_process().
It failed to check for error return from xsltApplyStylesheet(), as reported
by Peter Gagarinov.  (So far as I can tell, libxslt provides no convenient
way to get a useful error message in failure cases.  There might be some
inconvenient way, but considering that this code is deprecated it's hard to
get enthusiastic about putting lots of work into it.  So I just made it say
"failed to apply stylesheet", in line with the existing error checks.)

While looking at the code I also noticed that the string returned by
xsltSaveResultToString was never freed, resulting in a session-lifespan
memory leak.

Back-patch to all supported versions.
2012-06-04 20:12:50 -04:00
Bruce Momjian
8fcbfea788 In pg_upgrade, report pre-PG 8.1 plpython helper functions left in the
public schema that no longer point to valid shared object libraries, and
suggest a solution.
2012-06-01 11:40:04 -04:00
Tom Lane
932ded2ed5 Fix incorrect password transformation in contrib/pgcrypto's DES crypt().
Overly tight coding caused the password transformation loop to stop
examining input once it had processed a byte equal to 0x80.  Thus, if the
given password string contained such a byte (which is possible though not
highly likely in UTF8, and perhaps also in other non-ASCII encodings), all
subsequent characters would not contribute to the hash, making the password
much weaker than it appears on the surface.

This would only affect cases where applications used DES crypt() to encode
passwords before storing them in the database.  If a weak password has been
created in this fashion, the hash will stop matching after this update has
been applied, so it will be easy to tell if any passwords were unexpectedly
weak.  Changing to a different password would be a good idea in such a case.
(Since DES has been considered inadequately secure for some time, changing
to a different encryption algorithm can also be recommended.)

This code, and the bug, are shared with at least PHP, FreeBSD, and OpenBSD.
Since the other projects have already published their fixes, there is no
point in trying to keep this commit private.

This bug has been assigned CVE-2012-2143, and credit for its discovery goes
to Rubin Xu and Joseph Bonneau.
2012-05-30 10:53:30 -04:00
Bruce Momjian
4498a3407a Improve pg_upgrade C comment. 2012-05-27 06:45:29 -04:00
Bruce Momjian
20a3830b81 Add C comment explaining why we can't exclude checking functions in the
pg_catalog schema, even though they are not explicitly dumped (they are
implicitly dumped, e.g. create language plperl).
2012-05-27 06:35:10 -04:00
Bruce Momjian
d4696fa68f Add pg_update C comment about problems with plpython_call_handler(). 2012-05-27 06:29:45 -04:00
Magnus Hagander
2b97db61dd Fix handling of pg_stat_statements.stat temporary file
Write the file to a temporary name and then rename() it into the
permanent name, to ensure it can't end up half-written and corrupt
in case of a crash during shutdown.

Unlink the file after it has been read so it's removed from the data
directory and not included in base backups going to replication slaves.
2012-05-27 10:54:31 +02:00
Bruce Momjian
45ca31d6a7 Have pg_upgrade only use one extra log file for Win32, not two. 2012-05-25 09:10:07 -04:00
Bruce Momjian
d226e236c6 On Windows, have pg_upgrade use different two files to log pg_ctl
start/stop output, to fix file share error reported by Edmund Horner
2012-05-24 20:30:39 -04:00
Bruce Momjian
476291bef0 Adjust pg_upgrade to output a separate log file for pg_ctl output on
Windows, to avoid opening a file by multiple processes.
2012-05-23 20:19:34 -04:00
Peter Eisentraut
72155b29a8 pg_standby: Remove tabs from string literals
And align a bit better with the rest of the debug output.
2012-05-23 19:58:17 +03:00
Robert Haas
8fbe5a317d Fix error message for COMMENT/SECURITY LABEL ON COLUMN xxx IS 'yyy'
When the column name is an unqualified name, rather than table.column,
the error message complains about too many dotted names, which is
wrong.  Report by Peter Eisentraut based on examination of the
sepgsql regression test output, but the problem also affects COMMENT.
New wording as suggested by Tom Lane.
2012-05-22 11:23:36 -04:00
Peter Eisentraut
510860a6a5 Small cleanups of contrib --help output 2012-05-20 01:14:25 +03:00
Peter Eisentraut
8402fab411 file_fdw: Improve error message
The hint looked a bit confusing when there were no valid options to
hint about.
2012-05-17 20:20:33 +03:00
Peter Eisentraut
c8e086795a Remove whitespace from end of lines
pgindent and perltidy should clean up the rest.
2012-05-15 22:19:41 +03:00
Tom Lane
63fecc9177 Fix contrib/citext's upgrade script to handle array and domain cases.
We previously recognized that citext wouldn't get marked as collatable
during pg_upgrade from a pre-9.1 installation, and hacked its
create-from-unpackaged script to manually perform the necessary catalog
adjustments.  However, we overlooked the fact that domains over citext,
as well as the citext[] array type, need the same adjustments.  Extend
the script to handle those cases.

Also, the documentation suggested that this was only an issue in pg_upgrade
scenarios, which is quite wrong; loading any dump containing citext from a
pre-9.1 server will also result in the type being wrongly marked.

I approached the documentation problem by changing the 9.1.2 release note
paragraphs about this issue, which is historically inaccurate.  But it
seems better than having the information scattered in multiple places, and
leaving incorrect info in the 9.1.2 notes would be bad anyway.  We'll still
need to mention the issue again in the 9.1.4 notes, but perhaps they can
just reference 9.1.2 for fix instructions.

Per report from Evan Carroll.  Back-patch into 9.1.
2012-05-11 15:22:30 -04:00
Peter Eisentraut
081ca7a0d1 Tweak contrib --help output to match common style
Placeholders such as OPTION are typically kept in singular.
2012-05-08 21:55:05 +03:00
Peter Eisentraut
dcb2c58381 Fix misleading comments
Josh Kupershmidt
2012-05-08 19:35:22 +03:00
Peter Eisentraut
e9605a039b Even more duplicate word removal, in the spirit of the season 2012-05-02 20:56:03 +03:00
Robert Haas
e01e66f808 More duplicate word removal. 2012-05-02 09:28:16 -04:00
Tom Lane
1dd89eadcd Rename I/O timing statistics columns to blk_read_time and blk_write_time.
This seems more consistent with the pre-existing choices for names of
other statistics columns.  Rename assorted internal identifiers to match.
2012-04-29 18:13:33 -04:00
Tom Lane
93f94e356d Adjust timing units in pg_stat_statements.
Display total time and I/O timings in milliseconds, for consistency with
the units used for timings in the core statistics views.  The columns
remain of float8 type, so that sub-msec precision is available.  (At some
point we will probably want to convert the core views to use float8 type
for the same reason, but this patch does not touch that issue.)

This is a release-note-requiring change in the meaning of the total_time
column.  The I/O timing columns are new as of 9.2, so there is no
compatibility impact from redefining them.

Do some minor copy-editing in the documentation, too.
2012-04-28 16:03:57 -04:00
Robert Haas
e4f06b70c9 Another typographical correction.
Noted by Guillaume Smet.
2012-04-24 08:15:45 -04:00
Robert Haas
5d4b60f2f2 Lots of doc corrections.
Josh Kupershmidt
2012-04-23 22:43:09 -04:00
Peter Eisentraut
48658a1b81 Fix some typos
Josh Kupershmidt
2012-04-22 19:23:47 +03:00
Tom Lane
5b7b5518d0 Revise parameterized-path mechanism to fix assorted issues.
This patch adjusts the treatment of parameterized paths so that all paths
with the same parameterization (same set of required outer rels) for the
same relation will have the same rowcount estimate.  We cache the rowcount
estimates to ensure that property, and hopefully save a few cycles too.
Doing this makes it practical for add_path_precheck to operate without
a rowcount estimate: it need only assume that paths with different
parameterizations never dominate each other, which is close enough to
true anyway for coarse filtering, because normally a more-parameterized
path should yield fewer rows thanks to having more join clauses to apply.

In add_path, we do the full nine yards of comparing rowcount estimates
along with everything else, so that we can discard parameterized paths that
don't actually have an advantage.  This fixes some issues I'd found with
add_path rejecting parameterized paths on the grounds that they were more
expensive than not-parameterized ones, even though they yielded many fewer
rows and hence would be cheaper once subsequent joining was considered.

To make the same-rowcounts assumption valid, we have to require that any
parameterized path enforce *all* join clauses that could be obtained from
the particular set of outer rels, even if not all of them are useful for
indexing.  This is required at both base scans and joins.  It's a good
thing anyway since the net impact is that join quals are checked at the
lowest practical level in the join tree.  Hence, discard the original
rather ad-hoc mechanism for choosing parameterization joinquals, and build
a better one that has a more principled rule for when clauses can be moved.
The original rule was actually buggy anyway for lack of knowledge about
which relations are part of an outer join's outer side; getting this right
requires adding an outer_relids field to RestrictInfo.
2012-04-19 15:53:47 -04:00
Peter Eisentraut
48ea558313 Update contrib/README
Remove lots of outdated information that is duplicated by the
better-maintained SGML documentation.  In particular, remove the
outdated listing of contrib modules.  Update the installation
instructions to mention CREATE EXTENSION, but don't go into too much
detail.
2012-04-14 09:29:54 +03:00
Bruce Momjian
38458e4573 Fix pg_upgrade to properly upgrade a table that is stored in the cluster
default tablespace, but part of a database that is in a user-defined
tablespace.  Caused "file not found" error during upgrade.

Per bug report from Ants Aasma.

Backpatch to 9.1 and 9.0.
2012-04-10 19:57:14 -04:00
Tom Lane
e969f9a780 Save a few cycles while creating "sticky" entries in pg_stat_statements.
There's no need to sit there and increment the stats when we know all the
increments would be zero anyway.  The actual additions might not be very
expensive, but skipping acquisition of the spinlock seems like a good
thing.  Pushing the logic about initialization of the usage count down into
entry_alloc() allows us to do that while making the code actually simpler,
not more complex.  Expansion on a suggestion by Peter Geoghegan.
2012-04-09 11:16:04 -04:00
Tom Lane
d5375491f8 Improve management of "sticky" entries in contrib/pg_stat_statements.
This patch addresses a deficiency in the previous pg_stat_statements patch.
We want to give sticky entries an initial "usage" factor high enough that
they probably will stick around until their query is completed.  However,
if the query never completes (eg it gets an error during execution), the
entry shouldn't persist indefinitely.  Manage this by starting out with
a usage setting equal to the (approximate) median usage value within the
whole hashtable, but decaying the value much more aggressively than we
do for normal entries.

Peter Geoghegan
2012-04-08 15:49:47 -04:00
Tom Lane
cea49fe82f Dept of second thoughts: improve the API for AnalyzeForeignTable.
If we make the initially-called function return the table physical-size
estimate, acquire_inherited_sample_rows will be able to use that to
allocate numbers of samples among child tables, when the day comes that
we want to support foreign tables in inheritance trees.
2012-04-06 16:04:10 -04:00
Tom Lane
263d9de66b Allow statistics to be collected for foreign tables.
ANALYZE now accepts foreign tables and allows the table's FDW to control
how the sample rows are collected.  (But only manual ANALYZEs will touch
foreign tables, for the moment, since among other things it's not very
clear how to handle remote permissions checks in an auto-analyze.)

contrib/file_fdw is extended to support this.

Etsuro Fujita, reviewed by Shigeru Hanada, some further tweaking by me.
2012-04-06 15:02:35 -04:00
Robert Haas
bbc02243fc Allow pg_archivecleanup to strip optional file extensions.
Greg Smith and Jaime Casanova, reviewed by Alex Shulgin and myself.
e
2012-04-05 14:18:42 -04:00
Tom Lane
6f922ef88e Improve efficiency of dblink by using libpq's new row processor API.
This patch provides a test case for libpq's row processor API.
contrib/dblink can deal with very large result sets by dumping them into
a tuplestore (which can spill to disk) --- but until now, the intermediate
storage of the query result in a PGresult meant memory bloat for any large
result.  Now we use a row processor to convert the data to tuple form and
dump it directly into the tuplestore.

A limitation is that this only works for plain dblink() queries, not
dblink_send_query() followed by dblink_get_result().  In the latter
case we don't know the desired tuple rowtype soon enough.  While hack
solutions to that are possible, a different user-level API would
probably be a better answer.

Kyotaro Horiguchi, reviewed by Marko Kreen and Tom Lane
2012-04-04 18:39:08 -04:00
Tom Lane
d843ed2116 Fix a couple of contrib/dblink bugs.
dblink_exec leaked temporary database connections if any error occurred
after connection setup, for example
	SELECT dblink_exec('...connect string...', 'select 1/0');
Add a PG_TRY block to ensure PQfinish gets done when it is needed.
(dblink_record_internal is on the hairy edge of needing similar treatment,
but seems not to be actively broken at the moment.)

Also, in 9.0 and up, only one of the three functions using tuplestore
return mode was properly checking that the query context would allow
a tuplestore result.

Noted while reviewing dblink patch.  Back-patch to all supported branches.
2012-04-03 20:43:15 -04:00
Robert Haas
3f427c1353 pg_test_timing: Lame hack to work around compiler warning.
Fujii Masao, plus a comment by me.  While I'm at it, correctly tabify
this chunk of code.
2012-03-30 08:17:54 -04:00
Tom Lane
b75fbe9191 Fix dblink's failure to report correct connection name in error messages.
The DBLINK_GET_CONN and DBLINK_GET_NAMED_CONN macros did not set the
surrounding function's conname variable, causing errors to be incorrectly
reported as having occurred on the "unnamed" connection in some cases.
This bug was actually visible in two cases in the regression tests,
but apparently whoever added those cases wasn't paying attention.

Noted by Kyotaro Horiguchi, though this is different from his proposed
patch.

Back-patch to 8.4; 8.3 does not have the same type of error reporting
so the patch is not relevant.
2012-03-29 17:52:28 -04:00
Tom Lane
566a1d43cf Improve contrib/pg_stat_statements' handling of PREPARE/EXECUTE statements.
It's actually more useful for the module to ignore these.  Ignoring
EXECUTE (and not incrementing the nesting level) allows the executor
hooks to charge the time to the underlying prepared query, which
shows up as a stats entry with the original PREPARE as query string
(possibly modified by suppression of constants, which might not be
terribly useful here but it's not worth avoiding).  This is much more
useful than cluttering the stats table with a distinct entry for each
textually distinct EXECUTE.

Experimentation with this idea shows that it's also preferable to ignore
PREPARE.  If we don't, we get two stats table entries, one with the query
string hash and one with the jumble-derived hash, but with the same visible
query string (modulo those constants).  This is confusing and not very
helpful, since the first entry will only receive costs associated with
initial planning of the query, which is not something counted at all
normally by pg_stat_statements.  (And if we do start tracking planning
costs, we'd want them blamed on the other hash table entry anyway.)
2012-03-29 16:42:09 -04:00
Tom Lane
e0e4ebe384 Improve handling of utility statements containing plannable statements.
When tracking nested statements, contrib/pg_stat_statements formerly
double-counted the execution costs of utility statements that directly
contain an executable statement, such as EXPLAIN and DECLARE CURSOR.
This was not obvious since the ProcessUtility and Executor hooks
would each add their measured costs to the same stats table entry.
However, with the new implementation that hashes utility and plannable
statements differently, this showed up as seemingly-duplicate stats
entries.  Fix that by disabling the Executor hooks when the query has a
queryId of zero, which was the case already for such statements but is now
more clearly specified in the code.  (The zero queryId was causing problems
anyway because all such statements would add to a single bogus entry.)

The PREPARE/EXECUTE case still results in counting the same execution
in two different stats table entries, but it should be much less surprising
to users that there are two entries in such cases.

In passing, include a CommonTableExpr's ctename in the query hash.
I had left it out originally on the grounds that we wanted to omit all
inessential aliases, but since RTE_CTE RTEs are hashing their referenced
names, we'd better hash the CTE names too to make sure we don't hash
semantically different queries the same.
2012-03-29 15:32:59 -04:00
Tom Lane
7313cc0163 Improve contrib/pg_stat_statements to lump "similar" queries together.
pg_stat_statements now hashes selected fields of the analyzed parse tree
to assign a "fingerprint" to each query, and groups all queries with the
same fingerprint into a single entry in the pg_stat_statements view.
In practice it is expected that queries with the same fingerprint will be
equivalent except for values of literal constants.  To make the display
more useful, such constants are replaced by "?" in the displayed query
strings.

This mechanism currently supports only optimizable queries (SELECT,
INSERT, UPDATE, DELETE).  Utility commands are still matched on the
basis of their literal query strings.

There remain some open questions about how to deal with utility statements
that contain optimizable queries (such as EXPLAIN and SELECT INTO) and how
to deal with expiring speculative hashtable entries that are made to save
the normalized form of a query string.  However, fixing these issues should
require only localized changes, and since there are other open patches
involving contrib/pg_stat_statements, it seems best to go ahead and commit
what we've got.

Peter Geoghegan, reviewed by Daniel Farina
2012-03-28 21:01:23 -04:00
Robert Haas
cee523867d pg_test_timing utility, to measure clock monotonicity and timing cost.
Ants Aasma, Greg Smith
2012-03-27 16:14:00 -04:00
Robert Haas
5b4f346611 Expose track_iotiming information via pg_stat_statements.
Ants Aasma, reviewed by Greg Smith, with very minor tweaks by me.
2012-03-27 15:18:49 -04:00
Tom Lane
98316e211b Silence compiler warning about uninitialized variable. 2012-03-27 14:47:23 -04:00
Tom Lane
64c604898e Improve the -l (limit) option recently added to contrib/vacuumlo.
Instead of just stopping after removing an arbitrary subset of orphaned
large objects, commit and start a new transaction after each -l objects.
This is just as effective as the original patch at limiting the number of
locks used, and it doesn't require doing the OID collection process
repeatedly to get everything.  Since the option no longer changes the
fundamental behavior of vacuumlo, and it avoids a known server-side
limitation, enable it by default (with a default limit of 1000 LOs per
transaction).

In passing, be more careful about properly quoting the names of tables
and fields, and do some other cosmetic cleanup.
2012-03-20 19:05:44 -04:00
Peter Eisentraut
1b12c3e836 pg_upgrade: Add new generated file to .gitignore 2012-03-20 20:40:31 +02:00
Bruce Momjian
02b183acb4 In pg_upgrade, remove dependency on pg_config, as that might not be in
the non-development install.  Instead, use the LOAD mechanism to check
for the pg_upgrade_support shared object, like we do for other shared
object checks.

Backpatch to 9.1.

Report from Àlvaro
2012-03-19 09:31:50 -04:00
Bruce Momjian
ced53c38d7 In pg_upgrade, move new echo quote define into include file. 2012-03-17 09:56:56 -04:00
Bruce Momjian
faec2815c8 In pg_upgrade, create a script to incrementally generate more accurate
optimizer statistics so the cluster can be made available sooner.
2012-03-16 18:54:23 -04:00
Robert Haas
523176cbf1 sepgsql_setcon().
This is intended as infrastructure to allow sepgsql to cooperate with
connection pooling software, by allowing the effective security label
to be set for each new connection.

KaiGai Kohei, reviewed by Yeb Havinga.
2012-03-15 16:08:40 -04:00
Bruce Momjian
7c8cfed78a Uppercase pg_upgrade status output title. 2012-03-14 19:22:11 -04:00