Commit Graph

90 Commits

Author SHA1 Message Date
Tom Lane
c59f94e81e Revert "Delete contrib/xml2's legacy implementation of xml_is_well_formed()."
This partly reverts commit 20540710e8.
Since we've given up on adding PGDLLEXPORT markers to PG_FUNCTION_INFO_V1,
there's no need to remove the legacy compatibility function.  I kept the
documentation changes, though, as they seem appropriate anyway.
2016-11-07 10:27:52 -05:00
Tom Lane
20540710e8 Delete contrib/xml2's legacy implementation of xml_is_well_formed().
This function is unreferenced in modern usage; it was superseded in 9.1
by a core function of the same name.  It has been left in place in the C
code only so that pre-9.1 SQL definitions of the contrib/xml2 functions
would continue to work.  Six years seems like enough time for people to
have updated to the extension-style version of the xml2 module, so let's
drop this.

The key reason for not keeping it any longer is that we want to stick
an explicit PGDLLEXPORT into PG_FUNCTION_INFO_V1(), and the similarity
of name to the core function creates a conflict that compilers will
complain about.

Extracted from a larger patch for that purpose.  I'm committing this
change separately to give it more visibility in the commit logs.

While at it, remove the documentation entry that claimed that
xml_is_well_formed() is a function provided by contrib/xml2, and
instead mention the even more ancient alias xml_valid().

Laurenz Albe, doc change by me

Patch: <A737B7A37273E048B164557ADEF4A58B53962ED8@ntex2010a.host.magwien.gv.at>
2016-11-04 18:29:53 -04:00
Robert Haas
13e7453135 Update xml2 extension for parallel query.
All functions provided by this extension are PARALLEL SAFE.

Andreas Karlsson
2016-06-14 15:49:32 -04:00
Tom Lane
23a27b039d Widen query numbers-of-tuples-processed counters to uint64.
This patch widens SPI_processed, EState's es_processed field, PortalData's
portalPos field, FuncCallContext's call_cntr and max_calls fields,
ExecutorRun's count argument, PortalRunFetch's result, and the max number
of rows in a SPITupleTable to uint64, and deals with (I hope) all the
ensuing fallout.  Some of these values were declared uint32 before, and
others "long".

I also removed PortalData's posOverflow field, since that logic seems
pretty useless given that portalPos is now always 64 bits.

The user-visible results are that command tags for SELECT etc will
correctly report tuple counts larger than 4G, as will plpgsql's GET
GET DIAGNOSTICS ... ROW_COUNT command.  Queries processing more tuples
than that are still not exactly the norm, but they're becoming more
common.

Most values associated with FETCH/MOVE distances, such as PortalRun's count
argument and the count argument of most SPI functions that have one, remain
declared as "long".  It's not clear whether it would be worth promoting
those to int64; but it would definitely be a large dollop of additional
API churn on top of this, and it would only help 32-bit platforms which
seem relatively less likely to see any benefit.

Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me
2016-03-12 16:05:29 -05:00
Tom Lane
586dd5d6a5 Replace a bunch more uses of strncpy() with safer coding.
strncpy() has a well-deserved reputation for being unsafe, so make an
effort to get rid of nearly all occurrences in HEAD.

A large fraction of the remaining uses were passing length less than or
equal to the known strlen() of the source, in which case no null-padding
can occur and the behavior is equivalent to memcpy(), though doubtless
slower and certainly harder to reason about.  So just use memcpy() in
these cases.

In other cases, use either StrNCpy() or strlcpy() as appropriate (depending
on whether padding to the full length of the destination buffer seems
useful).

I left a few strncpy() calls alone in the src/timezone/ code, to keep it
in sync with upstream (the IANA tzcode distribution).  There are also a
few such calls in ecpg that could possibly do with more analysis.

AFAICT, none of these changes are more than cosmetic, except for the four
occurrences in fe-secure-openssl.c, which are in fact buggy: an overlength
source leads to a non-null-terminated destination buffer and ensuing
misbehavior.  These don't seem like security issues, first because no stack
clobber is possible and second because if your values of sslcert etc are
coming from untrusted sources then you've got problems way worse than this.
Still, it's undesirable to have unpredictable behavior for overlength
inputs, so back-patch those four changes to all active branches.
2015-01-24 13:05:42 -05:00
Tom Lane
c168ba3112 Free libxml2/libxslt resources in a safer order.
Mark Simonetti reported that libxslt sometimes crashes for him, and that
swapping xslt_process's object-freeing calls around to do them in reverse
order of creation seemed to fix it.  I've not reproduced the crash, but
valgrind clearly shows a reference to already-freed memory, which is
consistent with the idea that shutdown of the xsltTransformContext is
trying to reference the already-freed stylesheet or input document.
With this patch, valgrind is no longer unhappy.

I have an inquiry in to see if this is a libxslt bug or if we're just
abusing the library; but even if it's a library bug, we'd want to adjust
our code so it doesn't fail with unpatched libraries.

Back-patch to all supported branches, because we've been doing this in
the wrong(?) order for a long time.
2014-11-27 11:13:29 -05:00
Andres Freund
d153b80161 Fix typos in some error messages thrown by extension scripts when fed to psql.
Some of the many error messages introduced in 458857cc missed 'FROM
unpackaged'. Also e016b724 and 45ffeb7e forgot to quote extension
version numbers.

Backpatch to 9.1, just like 458857cc which introduced the messages. Do
so because the error messages thrown when the wrong command is copy &
pasted aren't easy to understand.
2014-08-25 18:30:37 +02:00
Noah Misch
0ffc201a51 Add file version information to most installed Windows binaries.
Prominent binaries already had this metadata.  A handful of minor
binaries, such as pg_regress.exe, still lack it; efforts to eliminate
such exceptions are welcome.

Michael Paquier, reviewed by MauMau.
2014-07-14 14:07:52 -04:00
Bruce Momjian
6a605cd6bd Adjust blank lines around PG_MODULE_MAGIC defines, for consistency
Report by Robert Haas
2014-07-10 14:02:08 -04:00
Bruce Momjian
0a78320057 pgindent run for 9.4
This includes removing tabs after periods in C comments, which was
applied to back branches, so this change should not effect backpatching.
2014-05-06 12:12:18 -04:00
Peter Eisentraut
e7128e8dbb Create function prototype as part of PG_FUNCTION_INFO_V1 macro
Because of gcc -Wmissing-prototypes, all functions in dynamically
loadable modules must have a separate prototype declaration.  This is
meant to detect global functions that are not declared in header files,
but in cases where the function is called via dfmgr, this is redundant.
Besides filling up space with boilerplate, this is a frequent source of
compiler warnings in extension modules.

We can fix that by creating the function prototype as part of the
PG_FUNCTION_INFO_V1 macro, which such modules have to use anyway.  That
makes the code of modules cleaner, because there is one less place where
the entry points have to be listed, and creates an additional check that
functions have the right prototype.

Remove now redundant prototypes from contrib and other modules.
2014-04-18 00:03:19 -04:00
Alvaro Herrera
c219d9b0a5 Split tuple struct defs from htup.h to htup_details.h
This reduces unnecessary exposure of other headers through htup.h, which
is very widely included by many files.

I have chosen to move the function prototypes to the new file as well,
because that means htup.h no longer needs to include tupdesc.h.  In
itself this doesn't have much effect in indirect inclusion of tupdesc.h
throughout the tree, because it's also required by execnodes.h; but it's
something to explore in the future, and it seemed best to do the htup.h
change now while I'm busy with it.
2012-08-30 16:52:35 -04:00
Tom Lane
adc97d03b9 Prevent access to external files/URLs via contrib/xml2's xslt_process().
libxslt offers the ability to read and write both files and URLs through
stylesheet commands, thus allowing unprivileged database users to both read
and write data with the privileges of the database server.  Disable that
through proper use of libxslt's security options.

Also, remove xslt_process()'s ability to fetch documents and stylesheets
from external files/URLs.  While this was a documented "feature", it was
long regarded as a terrible idea.  The fix for CVE-2012-3489 broke that
capability, and rather than expend effort on trying to fix it, we're just
going to summarily remove it.

While the ability to write as well as read makes this security hole
considerably worse than CVE-2012-3489, the problem is mitigated by the fact
that xslt_process() is not available unless contrib/xml2 is installed,
and the longstanding warnings about security risks from that should have
discouraged prudent DBAs from installing it in security-exposed databases.

Reported and fixed by Peter Eisentraut.

Security: CVE-2012-3488
2012-08-14 18:31:18 -04:00
Bruce Momjian
927d61eeff Run pgindent on 9.2 source tree in preparation for first 9.3
commit-fest.
2012-06-10 15:20:04 -04:00
Tom Lane
d9b31e4859 Fix some more bugs in contrib/xml2's xslt_process().
It failed to check for error return from xsltApplyStylesheet(), as reported
by Peter Gagarinov.  (So far as I can tell, libxslt provides no convenient
way to get a useful error message in failure cases.  There might be some
inconvenient way, but considering that this code is deprecated it's hard to
get enthusiastic about putting lots of work into it.  So I just made it say
"failed to apply stylesheet", in line with the existing error checks.)

While looking at the code I also noticed that the string returned by
xsltSaveResultToString was never freed, resulting in a session-lifespan
memory leak.

Back-patch to all supported versions.
2012-06-04 20:12:50 -04:00
Tom Lane
458857cc9d Throw a useful error message if an extension script file is fed to psql.
We have seen one too many reports of people trying to use 9.1 extension
files in the old-fashioned way of sourcing them in psql.  Not only does
that usually not work (due to failure to substitute for MODULE_PATHNAME
and/or @extschema@), but if it did work they'd get a collection of loose
objects not an extension.  To prevent this, insert an \echo ... \quit
line that prints a suitable error message into each extension script file,
and teach commands/extension.c to ignore lines starting with \echo.
That should not only prevent any adverse consequences of loading a script
file the wrong way, but make it crystal clear to users that they need to
do it differently now.

Tom Lane, following an idea of Andrew Dunstan's.  Back-patch into 9.1
... there is not going to be much value in this if we wait till 9.2.
2011-10-12 15:45:03 -04:00
Robert Haas
10c378f235 Fix contrib/sepgsql and contrib/xml2 to always link required libraries.
contrib/xml2 can get by without libxslt; the relevant features just
won't work.  But if doesn't have libxml2, or if sepgsql doesn't have
libselinux, the link succeeds but the module then fails to work at load
time.  To avoid that, link the require libraries unconditionally, so
that it will be clear at link-time that there is a problem.

Per discussion with Tom Lane and KaiGai Kohei.
2011-08-19 12:00:45 -04:00
Tom Lane
cacd42d62c Rewrite libxml error handling to be more robust.
libxml reports some errors (like invalid xmlns attributes) via the error
handler hook, but still returns a success indicator to the library caller.
This causes us to miss some errors that are important to report.  Since the
"generic" error handler hook doesn't know whether the message it's getting
is for an error, warning, or notice, stop using that and instead start
using the "structured" error handler hook, which gets enough information
to be useful.

While at it, arrange to save and restore the error handler hook setting in
each libxml-using function, rather than assuming we can set and forget the
hook.  This should improve the odds of working nicely with third-party
libraries that also use libxml.

In passing, volatile-ize some local variables that get modified within
PG_TRY blocks.  I noticed this while testing with an older gcc version
than I'd previously tried to compile xml.c with.

Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 13:03:49 -04:00
Peter Eisentraut
f8ebe3bcc5 Support "make check" in contrib
Added a new option --extra-install to pg_regress to arrange installing
the respective contrib directory into the temporary installation.
This is currently not yet supported for Windows MSVC builds.

Updated the .gitignore files for contrib modules to ignore the
leftovers of a temp-install check run.

Changed the exit status of "make check" in a pgxs build (which still
does nothing) to 0 from 1.

Added "make check" in contrib to top-level "make check-world".
2011-04-25 22:27:11 +03:00
Bruce Momjian
bf50caf105 pgindent run before PG 9.1 beta 1. 2011-04-10 11:42:00 -04:00
Tom Lane
de623f3335 Fix upgrade of contrib/xml2 from 9.0.
Update script was being sloppy about two functions that have been changed
since 9.0.
2011-02-17 18:11:28 -05:00
Tom Lane
de06cfe834 More fixups for "unpackaged" conversion scripts. 2011-02-13 23:33:18 -05:00
Tom Lane
029fac2264 Avoid use of CREATE OR REPLACE FUNCTION in extension installation files.
It was never terribly consistent to use OR REPLACE (because of the lack of
comparable functionality for data types, operators, etc), and
experimentation shows that it's now positively pernicious in the extension
world.  We really want a failure to occur if there are any conflicts, else
it's unclear what the extension-ownership state of the conflicted object
ought to be.  Most of the time, CREATE EXTENSION will fail anyway because
of conflicts on other object types, but an extension defining only
functions can succeed, with bad results.
2011-02-13 22:54:52 -05:00
Tom Lane
629b3af27d Convert contrib modules to use the extension facility.
This isn't fully tested as yet, in particular I'm not sure that the
"foo--unpackaged--1.0.sql" scripts are OK.  But it's time to get some
buildfarm cycles on it.

sepgsql is not converted to an extension, mainly because it seems to
require a very nonstandard installation process.

Dimitri Fontaine and Tom Lane
2011-02-13 22:54:49 -05:00
Tom Lane
4a2516a7f9 Fix significant memory leak in contrib/xml2 functions.
Most of the functions that execute XPath queries leaked the data structures
created by libxml2.  This memory would not be recovered until end of
session, so it mounts up pretty quickly in any serious use of the feature.
Per report from Pavel Stehule, though this isn't his patch.

Back-patch to all supported branches.
2010-11-26 15:21:53 -05:00
Peter Eisentraut
fc946c39ae Remove useless whitespace at end of lines 2010-11-23 22:34:55 +02:00
Robert Haas
0a8ed2cdb4 Add contrib/xml2/pgxml.sql to .gitignore
Kevin Grittner
2010-09-23 22:08:22 -04:00
Tom Lane
cc2c8152e6 Some more gitignore cleanups: cover contrib and PL regression test outputs.
Also do some further work in the back branches, where quite a bit wasn't
covered by Magnus' original back-patch.
2010-09-22 17:22:40 -04:00
Magnus Hagander
9f2e211386 Remove cvs keywords from all files. 2010-09-20 22:08:53 +02:00
Tom Lane
a0b7b717a4 Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content
functions to the core XML code.  Per discussion, the former depends on
XMLOPTION while the others do not.  These supersede a version previously
offered by contrib/xml2.

Mike Fowler, reviewed by Pavel Stehule
2010-08-13 18:36:26 +00:00
Tom Lane
c04fd1b9db Remove the arbitrary (and undocumented) limit on the number of parameter=value
pairs that can be handled by xslt_process().

There is much else to do here, but this patch seems useful in its own right
for as long as this code survives.

Pavel Stehule, reviewed by Mike Fowler
2010-08-10 23:02:00 +00:00
Bruce Momjian
239d769e7e pgindent run for 9.0, second run 2010-07-06 19:19:02 +00:00
Tom Lane
5e47403be3 Make contrib/xml2 use core xml.c's error handler, when available (that is,
in versions >= 8.3).  The core code is more robust and efficient than what
was there before, and this also reduces risks involved in swapping different
libxml error handler settings.

Before 8.3, there is still some risk of problems if add-on modules such as
Perl invoke libxml without setting their own error handler.  Given the lack
of reports I'm not sure there's a risk in practice, so I didn't take the
step of actually duplicating the core code into older contrib/xml2 branches.
Instead I just tweaked the existing code to ensure it didn't leave a dangling
pointer to short-lived memory when throwing an error.
2010-03-03 19:10:22 +00:00
Tom Lane
d6a6f8c6be Fix contrib/xml2 so regression test still works when it's built without libxslt.
This involves modifying the module to have a stable ABI, that is, the
xslt_process() function still exists even without libxslt.  It throws a
runtime error if called, but doesn't prevent executing the CREATE FUNCTION
call.  This is a good thing anyway to simplify cross-version upgrades.
2010-03-01 18:07:59 +00:00
Tom Lane
a102090995 Remove xmlCleanupParser calls from contrib/xml2.
These are unnecessary and probably dangerous.  I don't see any immediate
risk situations in the core XML support or contrib/xml2 itself, but there
could be issues with external uses of libxml2, and in any case it's an
accident waiting to happen.
2010-03-01 05:16:35 +00:00
Tom Lane
936c4af1c6 Fix up memory management problems in contrib/xml2.
Get rid of the code that attempted to funnel libxml2's memory allocations
into palloc.   We already knew from experience with the core xml datatype
that trying to do this is simply not reliable.  Unlike the core code, I
did not bother adding a lot of PG_TRY/PG_CATCH logic to try to ensure that
everything is cleaned up on error exit.  Hence, we might leak some memory
if one of these functions fails partway through.  Given the deprecated
status of this contrib module and the fact that errors partway through
the functions shouldn't be too common, it doesn't seem worth worrying about.

Also fix a separate bug in xpath_table, that it did the wrong things
if given a result tuple descriptor with less than 2 columns.  While
such a case isn't very useful in practice, we shouldn't fail or stomp
memory when it occurs.

Add some simple regression tests based on all the reported crash cases
that I have on hand.

This should be back-patched, but let's see if the buildfarm likes it first.
2010-02-28 21:31:57 +00:00
Tom Lane
a12333eed2 Assorted code cleanup for contrib/xml2. No change in functionality,
just make it a bit less ugly in places.
2010-02-28 19:51:37 +00:00
Magnus Hagander
94fa8301f7 Silence compiler warning, noted by Stefan Kaltenbrunner 2010-01-17 12:11:25 +00:00
Heikki Linnakangas
84d723b6ce Previous fix for temporary file management broke returning a set from
PL/pgSQL function within an exception handler. Make sure we use the right
resource owner when we create the tuplestore to hold returned tuples.

Simplify tuplestore API so that the caller doesn't need to be in the right
memory context when calling tuplestore_put* functions. tuplestore.c
automatically switches to the memory context used when the tuplestore was
created. Tuplesort was already modified like this earlier. This patch also
removes the now useless MemoryContextSwitch calls from callers.

Report by Aleksei on pgsql-bugs on Dec 22 2009. Backpatch to 8.1, like
the previous patch that broke this.
2009-12-29 17:40:59 +00:00
Tom Lane
e7370babd1 Fix xslt_process() to ensure that it inserts a NULL terminator after the
last pair of parameter name/value strings, even when there are MAXPARAMS
of them.  Aboriginal bug in contrib/xml2, noted while studying bug #4912
(though I'm not sure whether there's something else involved in that
report).

This might be thought a security issue, since it's a potential backend
crash; but considering that untrustworthy users shouldn't be allowed
to get their hands on xslt_process() anyway, it's probably not worth
getting excited about.
2009-07-10 00:32:00 +00:00
Bruce Momjian
d747140279 8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list
provided by Andrew.
2009-06-11 14:49:15 +00:00
Tom Lane
1cfd9e8834 Fix executor/spi.h to follow our usual conventions for include files, ie,
not include postgres.h nor anything else it doesn't directly need.  Add
#includes to calling files as needed to compensate.  Per my proposal of
yesterday.

This should be noted as a source code change in the 8.4 release notes,
since it's likely to require changes in add-on modules.
2009-01-07 13:44:37 +00:00
Tom Lane
05bba3d176 Be more tense about not creating tuplestores with randomAccess = true unless
backwards scan could actually happen.  In particular, pass a flag to
materialize-mode SRFs that tells them whether they need to require random
access.  In passing, also suppress unneeded backward-scan overhead for a
Portal's holdStore tuplestore.  Per my proposal about reducing I/O costs for
tuplestores.
2008-10-29 00:00:39 +00:00
Andrew Dunstan
53972b460c Add $PostgreSQL$ markers to a lot of files that were missing them.
This particular batch was just for *.c and *.h file.

The changes were made with the following 2 commands:

find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o  \( -name '*.[ch]'  \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | while read file ; do head -n 1 < $file | grep -q '^/\*' && echo $file; done | xargs -l sed -i -e '1s/^\// /' -e '1i/*\n * $PostgreSQL:$ \n *'

find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o  \( -name '*.[ch]'  \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | xargs -l sed -i -e '1i/*\n * $PostgreSQL:$ \n */'
2008-05-17 01:28:26 +00:00
Tom Lane
eb915caf92 Fix contrib/xml2 makefile to not override CFLAGS, and in passing make it
auto-configure properly for libxslt present or not.
2008-05-08 16:49:37 +00:00
Tom Lane
45173ae24e Use new cstring/text conversion functions in some additional places.
These changes assume that the varchar and xml data types are represented
the same as text.  (I did not, however, accept the portions of the proposed
patch that wanted to assume bytea is the same as text --- tgl.)

Brendan Jurd
2008-05-04 16:42:41 +00:00
Tom Lane
ff8a1204b4 Fix core dump in contrib/xml2's xpath_table() when the input query returns
a NULL value.  Per bug #4058.
2008-03-26 01:19:04 +00:00
Tom Lane
220db7ccd8 Simplify and standardize conversions between TEXT datums and ordinary C
strings.  This patch introduces four support functions cstring_to_text,
cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and
two macros CStringGetTextDatum and TextDatumGetCString.  A number of
existing macros that provided variants on these themes were removed.

Most of the places that need to make such conversions now require just one
function or macro call, in place of the multiple notational layers that used
to be needed.  There are no longer any direct calls of textout or textin,
and we got most of the places that were using handmade conversions via
memcpy (there may be a few still lurking, though).

This commit doesn't make any serious effort to eliminate transient memory
leaks caused by detoasting toasted text objects before they reach
text_to_cstring.  We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few
places where it was easy, but much more could be done.

Brendan Jurd and Tom Lane
2008-03-25 22:42:46 +00:00
Bruce Momjian
33e2e02493 Add CVS version labels to all install/uninstall scripts. 2007-11-13 04:24:29 +00:00
Bruce Momjian
926bbab448 Make /contrib install/uninstall script consistent:
remove transactions
	use create or replace function
	make formatting consistent
	set search patch on first line

Add documentation on modifying *.sql to set the search patch, and
mention that major upgrades should still run the installation scripts.

Some of these issues were spotted by Tom today.
2007-11-11 03:25:35 +00:00