rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/utils/adt/xml.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

5045 lines
129 KiB
C
Raw Normal View History

Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
/*-------------------------------------------------------------------------
*
* xml.c
* XML data type support.
*
*
Update copyright for 2024 Reported-by: Michael Paquier Discussion: https://postgr.es/m/ZZKTDPxBBMt3C0J9@paquier.xyz Backpatch-through: 12
2024-01-04 02:49:05 +01:00
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
*
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/utils/adt/xml.c
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*
*-------------------------------------------------------------------------
*/
/*
* Generally, XML type support is only available when libxml use was
* configured during the build. But even if that is not done, the
* type and all the functions are available, but most of them will
* fail. For one thing, this avoids having to manage variant catalog
* installations. But it also has nice effects such as that you can
* dump a database containing XML type data even if the server is not
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
* linked with libxml. Thus, make sure xml_out() works even if nothing
* else does.
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
Reactivate libxml memory management via palloc, now that I think I've classified the conditions under which this is safe to do (see source code comment).
2007-01-27 15:50:51 +01:00
/*
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
* Notes on memory management:
*
* Sometimes libxml allocates global structures in the hope that it can reuse
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
* them later on. This makes it impractical to change the xmlMemSetup
* functions on-the-fly; that is likely to lead to trying to pfree() chunks
* allocated with malloc() or vice versa. Since libxml might be used by
* loadable modules, eg libperl, our only safe choices are to change the
* functions at postmaster/backend launch or not at all. Since we'd rather
* not activate libxml in sessions that might never use it, the latter choice
* is the preferred one. However, for debugging purposes it can be awfully
* handy to constrain libxml's allocations to be done in a specific palloc
* context, where they're easy to track. Therefore there is code here that
* can be enabled in debug builds to redirect libxml's allocations into a
* special context LibxmlContext. It's not recommended to turn this on in
* a production build because of the possibility of bad interactions with
* external modules.
Reactivate libxml memory management via palloc, now that I think I've classified the conditions under which this is safe to do (see source code comment).
2007-01-27 15:50:51 +01:00
*/
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
/* #define USE_LIBXMLCONTEXT */
Reactivate libxml memory management via palloc, now that I think I've classified the conditions under which this is safe to do (see source code comment).
2007-01-27 15:50:51 +01:00
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include "postgres.h"
#ifdef USE_LIBXML
#include <libxml/chvalid.h>
Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predefined entities (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. This also adds a note in features.sgml about not supporting XML(SEQUENCE). The SQL specification defines a RETURNING clause to a set of XML functions, where RETURNING CONTENT or RETURNING SEQUENCE can be defined. Since PostgreSQL doesn't support XML(SEQUENCE) all of these functions operate with an implicit RETURNING CONTENT. Author: Jim Jones <jim.jones@uni-muenster.de> Reviewed-by: Vik Fearing <vik@postgresfriends.org> Discussion: https://postgr.es/m/86617a66-ec95-581f-8d54-08059cca8885@uni-muenster.de
2023-11-06 09:38:29 +01:00
#include <libxml/entities.h>
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include <libxml/parser.h>
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
#include <libxml/parserInternals.h>
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include <libxml/tree.h>
#include <libxml/uri.h>
#include <libxml/xmlerror.h>
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
#include <libxml/xmlsave.h>
Check LIBXML_VERSION instead of testing in configure script. We had put a test for libxml2's xmlStructuredErrorContext variable in configure, but of course that doesn't work on Windows builds. The next best alternative seems to be to test the LIBXML_VERSION symbol provided by xmlversion.h. Per report from Talha Bin Rizwan, though this fixes it in a different way than his proposed patch.
2012-08-17 06:05:26 +02:00
#include <libxml/xmlversion.h>
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
#include <libxml/xmlwriter.h>
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
#include <libxml/xpath.h>
#include <libxml/xpathInternals.h>
Check LIBXML_VERSION instead of testing in configure script. We had put a test for libxml2's xmlStructuredErrorContext variable in configure, but of course that doesn't work on Windows builds. The next best alternative seems to be to test the LIBXML_VERSION symbol provided by xmlversion.h. Per report from Talha Bin Rizwan, though this fixes it in a different way than his proposed patch.
2012-08-17 06:05:26 +02:00
/*
* We used to check for xmlStructuredErrorContext via a configure test; but
* that doesn't work on Windows, so instead use this grottier method of
* testing the library version number.
*/
#if LIBXML_VERSION >= 20704
#define HAVE_XMLSTRUCTUREDERRORCONTEXT 1
#endif
Fix incompatibilities with libxml2 >= 2.12.0. libxml2 changed the required signature of error handler callbacks to make the passed xmlError struct "const". This is causing build failures on buildfarm member caiman, and no doubt will start showing up in the field quite soon. Add a version check to adjust the declaration of xml_errorHandler() according to LIBXML_VERSION. 2.12.x also produces deprecation warnings for contrib/xml2/xpath.c's assignment to xmlLoadExtDtdDefaultValue. I see no good reason for that to still be there, seeing that we disabled external DTDs (at a lower level) years ago for security reasons. Let's just remove it. Back-patch to all supported branches, since they might all get built with newer libxml2 once it gets a bit more popular. (The back branches produce another deprecation warning about xpath.c's use of xmlSubstituteEntitiesDefault(). We ought to consider whether to back-patch all or part of commit 65c5864d7 to silence that. It's less urgent though, since it won't break the buildfarm.) Discussion: https://postgr.es/m/1389505.1706382262@sss.pgh.pa.us
2024-01-29 18:06:07 +01:00
/*
* libxml2 2.12 decided to insert "const" into the error handler API.
*/
#if LIBXML_VERSION >= 21200
#define PgXmlErrorPtr const xmlError *
#else
#define PgXmlErrorPtr xmlErrorPtr
#endif
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#endif /* USE_LIBXML */
Split tuple struct defs from htup.h to htup_details.h This reduces unnecessary exposure of other headers through htup.h, which is very widely included by many files. I have chosen to move the function prototypes to the new file as well, because that means htup.h no longer needs to include tupdesc.h. In itself this doesn't have much effect in indirect inclusion of tupdesc.h throughout the tree, because it's also required by execnodes.h; but it's something to explore in the future, and it seemed best to do the htup.h change now while I'm busy with it.
2012-08-30 22:15:44 +02:00
#include "access/htup_details.h"
Replace heapam.h includes with {table, relation}.h where applicable. A lot of files only included heapam.h for relation_open, heap_open etc - replace the heapam.h include in those files with the narrower header. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:18:20 +01:00
#include "access/table.h"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
#include "catalog/namespace.h"
Fix hard-coded relkind constants in assorted other files. Although it's reasonable to expect that most of these constants will never change, that does not make it good programming style to hard-code the value rather than using the RELKIND_FOO macros. I think I've now gotten all the hard-coded references in C code. Unfortunately there's no equally convenient way to parameterize SQL files ... Discussion: https://postgr.es/m/11145.1488931324@sss.pgh.pa.us
2017-03-10 05:36:44 +01:00
#include "catalog/pg_class.h"
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
#include "catalog/pg_type.h"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
#include "commands/dbcommands.h"
#include "executor/spi.h"
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
#include "executor/tablefunc.h"
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include "fmgr.h"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
#include "lib/stringinfo.h"
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
#include "libpq/pqformat.h"
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include "mb/pg_wchar.h"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
#include "miscadmin.h"
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include "nodes/execnodes.h"
Clean up dubious error handling in wellformed_xml(). This ancient bit of code was summarily trapping any ereport longjmp whatsoever and assuming that it must represent an invalid-XML report. It's not really appropriate to handle OOM-like situations that way: maybe the input is valid or maybe not, but we couldn't find out. And it'd be a seriously bad idea to ignore, say, a query cancel error that way. (Perhaps that can't happen because there is no CHECK_FOR_INTERRUPTS anywhere within xml_parse, but even if that's true today it's obviously a very fragile assumption.) But in the wake of the previous commit, we can drop the PG_TRY here altogether, and use the soft error mechanism to catch only the kinds of errors that are legitimate to treat as invalid-XML. (This is our first use of the soft error mechanism for something not directly related to a datatype input function. It won't be the last.) xml_is_document can be converted in the same way. That one is not actively broken, because it was checking specifically for ERRCODE_INVALID_XML_DOCUMENT rather than trapping everything; but the code is still shorter and probably faster this way. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 17:10:36 +01:00
#include "nodes/miscnodes.h"
Move exprType(), exprTypmod(), expression_tree_walker(), and related routines into nodes/nodeFuncs, so as to reduce wanton cross-subsystem #includes inside the backend. There's probably more that should be done along this line, but this is a start anyway.
2008-08-26 00:42:34 +02:00
#include "nodes/nodeFuncs.h"
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
#include "utils/array.h"
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include "utils/builtins.h"
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
#include "utils/date.h"
#include "utils/datetime.h"
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
#include "utils/lsyscache.h"
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
#include "utils/memutils.h"
Move Trigger and TriggerDesc structs out of rel.h into a new reltrigger.h This lets us stop including rel.h into execnodes.h, which is a widely used header.
2011-02-23 18:18:09 +01:00
#include "utils/rel.h"
Fix executor/spi.h to follow our usual conventions for include files, ie, not include postgres.h nor anything else it doesn't directly need. Add #includes to calling files as needed to compensate. Per my proposal of yesterday. This should be noted as a source code change in the 8.4 release notes, since it's likely to require changes in add-on modules.
2009-01-07 14:44:37 +01:00
#include "utils/syscache.h"
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#include "utils/xml.h"
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
/* GUC variables */
Clean up some inconsistencies with GUC declarations This is similar to 7d25958, and this commit takes care of all the remaining inconsistencies between the initial value used in the C variable associated to a GUC and its default value stored in the GUC tables (as of pg_settings.boot_val). Some of the initial values of the GUCs updated rely on a compile-time default. These are refactored so as the GUC table and its C declaration use the same values. This makes everything consistent with other places, backend_flush_after, bgwriter_flush_after, port, checkpoint_flush_after doing so already, for example. Extracted from a larger patch by Peter Smith. The spots updated in the modules are from me. Author: Peter Smith, Michael Paquier Reviewed-by: Nathan Bossart, Tom Lane, Justin Pryzby Discussion: https://postgr.es/m/CAHut+PtHE0XSfjjRQ6D4v7+dqzCw=d+1a64ujra4EX8aoc_Z+w@mail.gmail.com
2022-10-31 04:44:48 +01:00
int xmlbinary = XMLBINARY_BASE64;
int xmloption = XMLOPTION_CONTENT;
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#ifdef USE_LIBXML
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* random number to identify PgXmlErrorContext */
#define ERRCXT_MAGIC 68275028
struct PgXmlErrorContext
{
int magic;
/* strictness argument passed to pg_xml_init */
PgXmlStrictness strictness;
/* current error status and accumulated message, if any */
bool err_occurred;
StringInfoData err_buf;
/* previous libxml error handling state (saved by pg_xml_init) */
xmlStructuredErrorFunc saved_errfunc;
void *saved_errcxt;
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
/* previous libxml entity handler (saved by pg_xml_init) */
xmlExternalEntityLoader saved_entityfunc;
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
};
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
static xmlParserInputPtr xmlPgEntityLoader(const char *URL, const char *ID,
xmlParserCtxtPtr ctxt);
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
static void xml_errsave(Node *escontext, PgXmlErrorContext *errcxt,
int sqlcode, const char *msg);
Fix incompatibilities with libxml2 >= 2.12.0. libxml2 changed the required signature of error handler callbacks to make the passed xmlError struct "const". This is causing build failures on buildfarm member caiman, and no doubt will start showing up in the field quite soon. Add a version check to adjust the declaration of xml_errorHandler() according to LIBXML_VERSION. 2.12.x also produces deprecation warnings for contrib/xml2/xpath.c's assignment to xmlLoadExtDtdDefaultValue. I see no good reason for that to still be there, seeing that we disabled external DTDs (at a lower level) years ago for security reasons. Let's just remove it. Back-patch to all supported branches, since they might all get built with newer libxml2 once it gets a bit more popular. (The back branches produce another deprecation warning about xpath.c's use of xmlSubstituteEntitiesDefault(). We ought to consider whether to back-patch all or part of commit 65c5864d7 to silence that. It's less urgent though, since it won't break the buildfarm.) Discussion: https://postgr.es/m/1389505.1706382262@sss.pgh.pa.us
2024-01-29 18:06:07 +01:00
static void xml_errorHandler(void *data, PgXmlErrorPtr error);
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
static int errdetail_for_xml_code(int code);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
static void chopStringInfoNewlines(StringInfo str);
static void appendStringInfoLineSeparator(StringInfo str);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
#ifdef USE_LIBXMLCONTEXT
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
static MemoryContext LibxmlContext = NULL;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
static void xml_memory_init(void);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
static void *xml_palloc(size_t size);
static void *xml_repalloc(void *ptr, size_t size);
static void xml_pfree(void *ptr);
static char *xml_pstrdup(const char *string);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
#endif /* USE_LIBXMLCONTEXT */
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
static xmlChar *xml_text2xmlChar(text *in);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
static int parse_xml_decl(const xmlChar *str, size_t *lenp,
xmlChar **version, xmlChar **encoding, int *standalone);
static bool print_xml_decl(StringInfo buf, const xmlChar *version,
pg_enc encoding, int standalone);
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
static bool xml_doctype_in_content(const xmlChar *str);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
static xmlDocPtr xml_parse(text *data, XmlOptionType xmloption_arg,
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
bool preserve_whitespace, int encoding,
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
XmlOptionType *parsed_xmloptiontype,
xmlNodePtr *parsed_nodes,
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
Node *escontext);
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
static text *xml_xmlnodetoxmltype(xmlNodePtr cur, PgXmlErrorContext *xmlerrcxt);
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
static int xml_xpathobjtoxmlarray(xmlXPathObjectPtr xpathobj,
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
ArrayBuildState *astate,
PgXmlErrorContext *xmlerrcxt);
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
static xmlChar *pg_xmlCharStrndup(const char *str, size_t len);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#endif /* USE_LIBXML */
Fix cursor_to_xml in tableforest false mode It only produced <row> elements but no wrapping <table> element. By contrast, cursor_to_xmlschema produced a schema that is now correct but did not previously match the XML data produced by cursor_to_xml. In passing, also fix a minor misunderstanding about moving cursors in the tests related to this. Reported-by: filip@jirsak.org Based-on-patch-by: Thomas Munro <thomas.munro@enterprisedb.com>
2017-05-04 03:25:01 +02:00
static void xmldata_root_element_start(StringInfo result, const char *eltname,
const char *xmlschema, const char *targetns,
bool top_level);
static void xmldata_root_element_end(StringInfo result, const char *eltname);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
static StringInfo query_to_xml_internal(const char *query, char *tablename,
const char *xmlschema, bool nulls, bool tableforest,
const char *targetns, bool top_level);
static const char *map_sql_table_to_xmlschema(TupleDesc tupdesc, Oid relid,
bool nulls, bool tableforest, const char *targetns);
static const char *map_sql_schema_to_xmlschema_types(Oid nspid,
List *relid_list, bool nulls,
bool tableforest, const char *targetns);
static const char *map_sql_catalog_to_xmlschema_types(List *nspid_list,
bool nulls, bool tableforest,
const char *targetns);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
static const char *map_sql_type_to_xml_name(Oid typeoid, int typmod);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
static const char *map_sql_typecoll_to_xmlschema_types(List *tupdesc_list);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
static const char *map_sql_type_to_xmlschema_type(Oid typeoid, int typmod);
Widen query numbers-of-tuples-processed counters to uint64. This patch widens SPI_processed, EState's es_processed field, PortalData's portalPos field, FuncCallContext's call_cntr and max_calls fields, ExecutorRun's count argument, PortalRunFetch's result, and the max number of rows in a SPITupleTable to uint64, and deals with (I hope) all the ensuing fallout. Some of these values were declared uint32 before, and others "long". I also removed PortalData's posOverflow field, since that logic seems pretty useless given that portalPos is now always 64 bits. The user-visible results are that command tags for SELECT etc will correctly report tuple counts larger than 4G, as will plpgsql's GET GET DIAGNOSTICS ... ROW_COUNT command. Queries processing more tuples than that are still not exactly the norm, but they're becoming more common. Most values associated with FETCH/MOVE distances, such as PortalRun's count argument and the count argument of most SPI functions that have one, remain declared as "long". It's not clear whether it would be worth promoting those to int64; but it would definitely be a large dollop of additional API churn on top of this, and it would only help 32-bit platforms which seem relatively less likely to see any benefit. Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me
2016-03-12 22:05:10 +01:00
static void SPI_sql_row_to_xmlelement(uint64 rownum, StringInfo result,
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
char *tablename, bool nulls, bool tableforest,
const char *targetns, bool top_level);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
/* XMLTABLE support */
#ifdef USE_LIBXML
/* random number to identify XmlTableContext */
#define XMLTABLE_CONTEXT_MAGIC 46922182
typedef struct XmlTableBuilderData
{
int magic;
int natts;
long int row_count;
PgXmlErrorContext *xmlerrcxt;
xmlParserCtxtPtr ctxt;
xmlDocPtr doc;
xmlXPathContextPtr xpathcxt;
xmlXPathCompExprPtr xpathcomp;
xmlXPathObjectPtr xpathobj;
xmlXPathCompExprPtr *xpathscomp;
} XmlTableBuilderData;
#endif
static void XmlTableInitOpaque(struct TableFuncScanState *state, int natts);
static void XmlTableSetDocument(struct TableFuncScanState *state, Datum value);
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
static void XmlTableSetNamespace(struct TableFuncScanState *state, const char *name,
const char *uri);
static void XmlTableSetRowFilter(struct TableFuncScanState *state, const char *path);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
static void XmlTableSetColumnFilter(struct TableFuncScanState *state,
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
const char *path, int colnum);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
static bool XmlTableFetchRow(struct TableFuncScanState *state);
static Datum XmlTableGetValue(struct TableFuncScanState *state, int colnum,
Oid typid, int32 typmod, bool *isnull);
static void XmlTableDestroyOpaque(struct TableFuncScanState *state);
const TableFuncRoutine XmlTableRoutine =
{
struct XmlTableRoutine: use C99 designated initializers As in c27f8621eed et al. Not as critical as other cases we've handled, but I figure if we're going to add JsonbTableRoutine using TableFuncRoutine, this makes it easier to jump around the code.
2024-01-16 12:48:30 +01:00
.InitOpaque = XmlTableInitOpaque,
.SetDocument = XmlTableSetDocument,
.SetNamespace = XmlTableSetNamespace,
.SetRowFilter = XmlTableSetRowFilter,
.SetColumnFilter = XmlTableSetColumnFilter,
.FetchRow = XmlTableFetchRow,
.GetValue = XmlTableGetValue,
.DestroyOpaque = XmlTableDestroyOpaque
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
};
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
#define NO_XML_SUPPORT() \
ereport(ERROR, \
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
Update XML error message text for missing libxml; update regression output to match.
2007-04-05 15:53:24 +02:00
errmsg("unsupported XML feature"), \
Remove error message hints mentioning configure options These are usually not useful since users will use packaged distributions and won't be interested in rebuilding their installation from source. Also, we have only used these kinds of hints for some features and in some places, not consistently throughout. Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/2552aed7-d0e9-280a-54aa-2dc7073f371d%40enterprisedb.com
2022-04-08 07:41:09 +02:00
errdetail("This functionality requires the server to be built with libxml support.")))
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
/* from SQL/XML:2008 section 4.9 */
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
#define NAMESPACE_XSD "http://www.w3.org/2001/XMLSchema"
#define NAMESPACE_XSI "http://www.w3.org/2001/XMLSchema-instance"
#define NAMESPACE_SQLXML "http://standards.iso.org/iso/9075/2003/sqlxml"
Fix the inadvertent libpq ABI breakage discovered by Martin Pitt: the renumbering of encoding IDs done between 8.2 and 8.3 turns out to break 8.2 initdb and psql if they are run with an 8.3beta1 libpq.so. For the moment we can rearrange the order of enum pg_enc to keep the same number for everything except PG_JOHAB, which isn't a problem since there are no direct references to it in the 8.2 programs anyway. (This does force initdb unfortunately.) Going forward, we want to fix things so that encoding IDs can be changed without an ABI break, and this commit includes the changes needed to allow libpq's encoding IDs to be treated as fully independent of the backend's. The main issue is that libpq clients should not include pg_wchar.h or otherwise assume they know the specific values of libpq's encoding IDs, since they might encounter version skew between pg_wchar.h and the libpq.so they are using. To fix, have libpq officially export functions needed for encoding name<=>ID conversion and validity checking; it was doing this anyway unofficially. It's still the case that we can't renumber backend encoding IDs until the next bump in libpq's major version number, since doing so will break the 8.2-era client programs. However the code is now prepared to avoid this type of problem in future. Note that initdb is no longer a libpq client: we just pull in the two source files we need directly. The patch also fixes a few places that were being sloppy about checking for an unrecognized encoding name.
2007-10-13 22:18:42 +02:00
#ifdef USE_LIBXML
static int
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
xmlChar_to_encoding(const xmlChar *encoding_name)
Fix the inadvertent libpq ABI breakage discovered by Martin Pitt: the renumbering of encoding IDs done between 8.2 and 8.3 turns out to break 8.2 initdb and psql if they are run with an 8.3beta1 libpq.so. For the moment we can rearrange the order of enum pg_enc to keep the same number for everything except PG_JOHAB, which isn't a problem since there are no direct references to it in the 8.2 programs anyway. (This does force initdb unfortunately.) Going forward, we want to fix things so that encoding IDs can be changed without an ABI break, and this commit includes the changes needed to allow libpq's encoding IDs to be treated as fully independent of the backend's. The main issue is that libpq clients should not include pg_wchar.h or otherwise assume they know the specific values of libpq's encoding IDs, since they might encounter version skew between pg_wchar.h and the libpq.so they are using. To fix, have libpq officially export functions needed for encoding name<=>ID conversion and validity checking; it was doing this anyway unofficially. It's still the case that we can't renumber backend encoding IDs until the next bump in libpq's major version number, since doing so will break the 8.2-era client programs. However the code is now prepared to avoid this type of problem in future. Note that initdb is no longer a libpq client: we just pull in the two source files we need directly. The patch also fixes a few places that were being sloppy about checking for an unrecognized encoding name.
2007-10-13 22:18:42 +02:00
{
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
int encoding = pg_char_to_encoding((const char *) encoding_name);
Fix the inadvertent libpq ABI breakage discovered by Martin Pitt: the renumbering of encoding IDs done between 8.2 and 8.3 turns out to break 8.2 initdb and psql if they are run with an 8.3beta1 libpq.so. For the moment we can rearrange the order of enum pg_enc to keep the same number for everything except PG_JOHAB, which isn't a problem since there are no direct references to it in the 8.2 programs anyway. (This does force initdb unfortunately.) Going forward, we want to fix things so that encoding IDs can be changed without an ABI break, and this commit includes the changes needed to allow libpq's encoding IDs to be treated as fully independent of the backend's. The main issue is that libpq clients should not include pg_wchar.h or otherwise assume they know the specific values of libpq's encoding IDs, since they might encounter version skew between pg_wchar.h and the libpq.so they are using. To fix, have libpq officially export functions needed for encoding name<=>ID conversion and validity checking; it was doing this anyway unofficially. It's still the case that we can't renumber backend encoding IDs until the next bump in libpq's major version number, since doing so will break the 8.2-era client programs. However the code is now prepared to avoid this type of problem in future. Note that initdb is no longer a libpq client: we just pull in the two source files we need directly. The patch also fixes a few places that were being sloppy about checking for an unrecognized encoding name.
2007-10-13 22:18:42 +02:00
if (encoding < 0)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("invalid encoding name \"%s\"",
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
(const char *) encoding_name)));
Fix the inadvertent libpq ABI breakage discovered by Martin Pitt: the renumbering of encoding IDs done between 8.2 and 8.3 turns out to break 8.2 initdb and psql if they are run with an 8.3beta1 libpq.so. For the moment we can rearrange the order of enum pg_enc to keep the same number for everything except PG_JOHAB, which isn't a problem since there are no direct references to it in the 8.2 programs anyway. (This does force initdb unfortunately.) Going forward, we want to fix things so that encoding IDs can be changed without an ABI break, and this commit includes the changes needed to allow libpq's encoding IDs to be treated as fully independent of the backend's. The main issue is that libpq clients should not include pg_wchar.h or otherwise assume they know the specific values of libpq's encoding IDs, since they might encounter version skew between pg_wchar.h and the libpq.so they are using. To fix, have libpq officially export functions needed for encoding name<=>ID conversion and validity checking; it was doing this anyway unofficially. It's still the case that we can't renumber backend encoding IDs until the next bump in libpq's major version number, since doing so will break the 8.2-era client programs. However the code is now prepared to avoid this type of problem in future. Note that initdb is no longer a libpq client: we just pull in the two source files we need directly. The patch also fixes a few places that were being sloppy about checking for an unrecognized encoding name.
2007-10-13 22:18:42 +02:00
return encoding;
}
#endif
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
/*
* xml_in uses a plain C string to VARDATA conversion, so for the time being
* we use the conversion function for the text datatype.
*
* This is only acceptable so long as xmltype and text use the same
* representation.
*/
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Datum
xml_in(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
char *s = PG_GETARG_CSTRING(0);
xmltype *vardata;
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
xmlDocPtr doc;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
/* Build the result object. */
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
vardata = (xmltype *) cstring_to_text(s);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
/*
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
* Parse the data to check if it is well-formed XML data.
*
* Note: we don't need to worry about whether a soft error is detected.
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
doc = xml_parse(vardata, xmloption, true, GetDatabaseEncoding(),
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
NULL, NULL, fcinfo->context);
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
if (doc != NULL)
xmlFreeDoc(doc);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
PG_RETURN_XML_P(vardata);
#else
NO_XML_SUPPORT();
return 0;
#endif
}
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
#define PG_XML_DEFAULT_VERSION "1.0"
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
/*
* xml_out_internal uses a plain VARDATA to C string conversion, so for the
* time being we use the conversion function for the text datatype.
*
* This is only acceptable so long as xmltype and text use the same
* representation.
*/
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
static char *
xml_out_internal(xmltype *x, pg_enc target_encoding)
{
Fix multiple memory leaks in xml_out(). Per report from Matt Magoffin.
2008-09-16 02:49:41 +02:00
char *str = text_to_cstring((text *) x);
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
#ifdef USE_LIBXML
Fix multiple memory leaks in xml_out(). Per report from Matt Magoffin.
2008-09-16 02:49:41 +02:00
size_t len = strlen(str);
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
xmlChar *version;
int standalone;
int res_code;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
if ((res_code = parse_xml_decl((xmlChar *) str,
Fix multiple memory leaks in xml_out(). Per report from Matt Magoffin.
2008-09-16 02:49:41 +02:00
&len, &version, NULL, &standalone)) == 0)
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
{
StringInfoData buf;
initStringInfo(&buf);
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
if (!print_xml_decl(&buf, version, target_encoding, standalone))
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
{
/*
* If we are not going to produce an XML declaration, eat a single
* newline in the original string to prevent empty first lines in
* the output.
*/
if (*(str + len) == '\n')
len += 1;
}
appendStringInfoString(&buf, str + len);
Fix multiple memory leaks in xml_out(). Per report from Matt Magoffin.
2008-09-16 02:49:41 +02:00
pfree(str);
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
return buf.data;
}
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
ereport(WARNING,
errcode(ERRCODE_INTERNAL_ERROR),
errmsg_internal("could not parse XML declaration in stored value"),
errdetail_for_xml_code(res_code));
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
#endif
return str;
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Datum
xml_out(PG_FUNCTION_ARGS)
{
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
xmltype *x = PG_GETARG_XML_P(0);
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
/*
* xml_out removes the encoding property in all cases. This is because we
* cannot control from here whether the datum will be converted to a
* different client encoding, so we'd do more harm than good by including
* it.
*/
PG_RETURN_CSTRING(xml_out_internal(x, 0));
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
Datum
xml_recv(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
StringInfo buf = (StringInfo) PG_GETARG_POINTER(0);
xmltype *result;
char *str;
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
char *newstr;
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
int nbytes;
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
xmlDocPtr doc;
Fix encoding handling in xml binary input function. If the XML header didn't specify an encoding explicitly, we used to treat it as being in database encoding when we parsed it, but then perform a UTF-8 -> database encoding conversion on it, which was completely bogus. It's now consistently treated as UTF-8.
2009-09-04 12:49:29 +02:00
xmlChar *encodingStr = NULL;
int encoding;
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
/*
* Read the data in raw format. We don't know yet what the encoding is, as
* that information is embedded in the xml declaration; so we have to
* parse that before converting to server encoding.
*/
nbytes = buf->len - buf->cursor;
str = (char *) pq_getmsgbytes(buf, nbytes);
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
/*
* We need a null-terminated string to pass to parse_xml_decl(). Rather
* than make a separate copy, make the temporary result one byte bigger
* than it needs to be.
*/
result = palloc(nbytes + 1 + VARHDRSZ);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(result, nbytes + VARHDRSZ);
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
memcpy(VARDATA(result), str, nbytes);
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
str = VARDATA(result);
str[nbytes] = '\0';
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
Add const qualifiers where they are accidentally cast away This only produces warnings under -Wcast-qual, but it's more correct and consistent in any case.
2012-02-28 11:42:08 +01:00
parse_xml_decl((const xmlChar *) str, NULL, NULL, &encodingStr, NULL);
Fix encoding handling in xml binary input function. If the XML header didn't specify an encoding explicitly, we used to treat it as being in database encoding when we parsed it, but then perform a UTF-8 -> database encoding conversion on it, which was completely bogus. It's now consistently treated as UTF-8.
2009-09-04 12:49:29 +02:00
/*
* If encoding wasn't explicitly specified in the XML header, treat it as
* UTF-8, as that's the default in XML. This is different from xml_in(),
* where the input has to go through the normal client to server encoding
* conversion.
*/
encoding = encodingStr ? xmlChar_to_encoding(encodingStr) : PG_UTF8;
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
/*
* Parse the data to check if it is well-formed XML data. Assume that
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
* xml_parse will throw ERROR if not.
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
*/
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
doc = xml_parse(result, xmloption, true, encoding, NULL, NULL, NULL);
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
xmlFreeDoc(doc);
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
/* Now that we know what we're dealing with, convert to server encoding */
Prefer pg_any_to_server/pg_server_to_any over pg_do_encoding_conversion. A large majority of the callers of pg_do_encoding_conversion were specifying the database encoding as either source or target of the conversion, meaning that we can use the less general functions pg_any_to_server/pg_server_to_any instead. The main advantage of using the latter functions is that they can make use of a cached conversion-function lookup in the common case that the other encoding is the current client_encoding. It's notationally cleaner too in most cases, not least because of the historical artifact that the latter functions use "char *" rather than "unsigned char *" in their APIs. Note that pg_any_to_server will apply an encoding verification step in some cases where pg_do_encoding_conversion would have just done nothing. This seems to me to be a good idea at most of these call sites, though it partially negates the performance benefit. Per discussion of bug #9210.
2014-02-23 22:59:05 +01:00
newstr = pg_any_to_server(str, nbytes, encoding);
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
if (newstr != str)
{
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
pfree(result);
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
result = (xmltype *) cstring_to_text(newstr);
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
pfree(newstr);
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
}
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
PG_RETURN_XML_P(result);
#else
NO_XML_SUPPORT();
return 0;
#endif
}
Datum
xml_send(PG_FUNCTION_ARGS)
{
xmltype *x = PG_GETARG_XML_P(0);
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
char *outval;
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
StringInfoData buf;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
/*
* xml_out_internal doesn't convert the encoding, it just prints the right
* declaration. pq_sendtext will do the conversion.
*/
outval = xml_out_internal(x, pg_get_client_encoding());
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
pq_begintypsend(&buf);
Fix bugs in XML binary I/O functions. Heikki and Tom
2007-09-23 23:36:42 +02:00
pq_sendtext(&buf, outval, strlen(outval));
pfree(outval);
Add send and recv functions for xml type.
2006-12-28 15:28:36 +01:00
PG_RETURN_BYTEA_P(pq_endtypsend(&buf));
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#ifdef USE_LIBXML
static void
appendStringInfoText(StringInfo str, const text *t)
{
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
appendBinaryStringInfo(str, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t));
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
#endif
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
static xmltype *
stringinfo_to_xmltype(StringInfo buf)
{
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
return (xmltype *) cstring_to_text_with_len(buf->data, buf->len);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
static xmltype *
cstring_to_xmltype(const char *string)
{
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
return (xmltype *) cstring_to_text(string);
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
#ifdef USE_LIBXML
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
static xmltype *
xmlBuffer_to_xmltype(xmlBufferPtr buf)
{
Remove many -Wcast-qual warnings This addresses only those cases that are easy to fix by adding or moving a const qualifier or removing an unnecessary cast. There are many more complicated cases remaining.
2011-09-11 20:54:32 +02:00
return (xmltype *) cstring_to_text_with_len((const char *) xmlBufferContent(buf),
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
xmlBufferLength(buf));
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#endif
Datum
xmlcomment(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *arg = PG_GETARG_TEXT_PP(0);
char *argdata = VARDATA_ANY(arg);
int len = VARSIZE_ANY_EXHDR(arg);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
StringInfoData buf;
int i;
/* check for "--" in string or "-" at the end */
for (i = 1; i < len; i++)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
{
if (argdata[i] == '-' && argdata[i - 1] == '-')
ereport(ERROR,
(errcode(ERRCODE_INVALID_XML_COMMENT),
errmsg("invalid XML comment")));
}
if (len > 0 && argdata[len - 1] == '-')
ereport(ERROR,
(errcode(ERRCODE_INVALID_XML_COMMENT),
errmsg("invalid XML comment")));
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
initStringInfo(&buf);
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&buf, "<!--");
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
appendStringInfoText(&buf, arg);
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&buf, "-->");
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(&buf));
#else
NO_XML_SUPPORT();
return 0;
#endif
}
Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predefined entities (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. This also adds a note in features.sgml about not supporting XML(SEQUENCE). The SQL specification defines a RETURNING clause to a set of XML functions, where RETURNING CONTENT or RETURNING SEQUENCE can be defined. Since PostgreSQL doesn't support XML(SEQUENCE) all of these functions operate with an implicit RETURNING CONTENT. Author: Jim Jones <jim.jones@uni-muenster.de> Reviewed-by: Vik Fearing <vik@postgresfriends.org> Discussion: https://postgr.es/m/86617a66-ec95-581f-8d54-08059cca8885@uni-muenster.de
2023-11-06 09:38:29 +01:00
Datum
xmltext(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
text *arg = PG_GETARG_TEXT_PP(0);
text *result;
xmlChar *xmlbuf = NULL;
xmlbuf = xmlEncodeSpecialChars(NULL, xml_text2xmlChar(arg));
Assert(xmlbuf);
result = cstring_to_text_with_len((const char *) xmlbuf, xmlStrlen(xmlbuf));
xmlFree(xmlbuf);
PG_RETURN_XML_P(result);
#else
NO_XML_SUPPORT();
return 0;
#endif /* not USE_LIBXML */
}
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
/*
* TODO: xmlconcat needs to merge the notations and unparsed entities
* of the argument values. Not very important in practice, though.
*/
xmltype *
xmlconcat(List *args)
{
#ifdef USE_LIBXML
int global_standalone = 1;
xmlChar *global_version = NULL;
bool global_version_no_value = false;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
StringInfoData buf;
ListCell *v;
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
initStringInfo(&buf);
foreach(v, args)
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmltype *x = DatumGetXmlP(PointerGetDatum(lfirst(v)));
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
size_t len;
xmlChar *version;
int standalone;
char *str;
len = VARSIZE(x) - VARHDRSZ;
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
str = text_to_cstring((text *) x);
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
parse_xml_decl((xmlChar *) str, &len, &version, NULL, &standalone);
if (standalone == 0 && global_standalone == 1)
global_standalone = 0;
if (standalone < 0)
global_standalone = -1;
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
if (!version)
global_version_no_value = true;
else if (!global_version)
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
global_version = version;
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
else if (xmlStrcmp(version, global_version) != 0)
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
global_version_no_value = true;
appendStringInfoString(&buf, str + len);
pfree(str);
}
if (!global_version_no_value || global_standalone >= 0)
{
StringInfoData buf2;
initStringInfo(&buf2);
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
print_xml_decl(&buf2,
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
(!global_version_no_value) ? global_version : NULL,
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
0,
global_standalone);
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
Use appendBinaryStringInfo in more places where the length is known When we already know the length that we're going to append, then it makes sense to use appendBinaryStringInfo instead of appendStringInfoString so that the append can be performed with a simple memcpy() using a known length rather than having to first perform a strlen() call to obtain the length. Discussion: https://postgr.es/m/CAKJS1f8+FRAM1s5+mAa3isajeEoAaicJ=4e0WzrH3tAusbbiMQ@mail.gmail.com
2019-07-22 14:14:11 +02:00
appendBinaryStringInfo(&buf2, buf.data, buf.len);
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
buf = buf2;
}
return stringinfo_to_xmltype(&buf);
#else
NO_XML_SUPPORT();
return NULL;
#endif
}
/*
* XMLAGG support
*/
Datum
xmlconcat2(PG_FUNCTION_ARGS)
{
if (PG_ARGISNULL(0))
{
if (PG_ARGISNULL(1))
PG_RETURN_NULL();
else
PG_RETURN_XML_P(PG_GETARG_XML_P(1));
}
else if (PG_ARGISNULL(1))
PG_RETURN_XML_P(PG_GETARG_XML_P(0));
else
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(xmlconcat(list_make2(PG_GETARG_XML_P(0),
PG_GETARG_XML_P(1))));
Fix xmlconcat by properly merging the XML declarations. Add aggregate function xmlagg.
2007-01-20 10:27:20 +01:00
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Datum
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
texttoxml(PG_FUNCTION_ARGS)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *data = PG_GETARG_TEXT_PP(0);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Implement XMLSERIALIZE for real. Analogously, make the xml to text cast observe the xmloption. Reorganize the representation of the XML option in the parse tree and the API to make it easier to manage and understand. Add regression tests for parsing back XML expressions.
2007-02-03 15:06:56 +01:00
PG_RETURN_XML_P(xmlparse(data, xmloption, true));
}
Datum
xmltotext(PG_FUNCTION_ARGS)
{
xmltype *data = PG_GETARG_XML_P(0);
Make casts from xml to text independent of the XML option setting, thus immutable and indexable. Also fix the volatility settings of some other XML-related functions.
2007-11-27 13:21:05 +01:00
/* It's actually binary compatible. */
Suppress compiler warning.
2007-11-27 19:13:01 +01:00
PG_RETURN_TEXT_P((text *) data);
Implement XMLSERIALIZE for real. Analogously, make the xml to text cast observe the xmloption. Reorganize the representation of the XML option in the parse tree and the API to make it easier to manage and understand. Add regression tests for parsing back XML expressions.
2007-02-03 15:06:56 +01:00
}
text *
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
xmltotext_with_options(xmltype *data, XmlOptionType xmloption_arg, bool indent)
Implement XMLSERIALIZE for real. Analogously, make the xml to text cast observe the xmloption. Reorganize the representation of the XML option in the parse tree and the API to make it easier to manage and understand. Add regression tests for parsing back XML expressions.
2007-02-03 15:06:56 +01:00
{
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
#ifdef USE_LIBXML
text *volatile result;
xmlDocPtr doc;
XmlOptionType parsed_xmloptiontype;
xmlNodePtr content_nodes;
volatile xmlBufferPtr buf = NULL;
volatile xmlSaveCtxtPtr ctxt = NULL;
ErrorSaveContext escontext = {T_ErrorSaveContext};
PgXmlErrorContext *xmlerrcxt;
#endif
if (xmloption_arg != XMLOPTION_DOCUMENT && !indent)
{
/*
* We don't actually need to do anything, so just return the
* binary-compatible input. For backwards-compatibility reasons,
* allow such cases to succeed even without USE_LIBXML.
*/
return (text *) data;
}
#ifdef USE_LIBXML
/* Parse the input according to the xmloption */
doc = xml_parse(data, xmloption_arg, true, GetDatabaseEncoding(),
&parsed_xmloptiontype, &content_nodes,
(Node *) &escontext);
if (doc == NULL || escontext.error_occurred)
{
if (doc)
xmlFreeDoc(doc);
/* A soft error must be failure to conform to XMLOPTION_DOCUMENT */
Implement XMLSERIALIZE for real. Analogously, make the xml to text cast observe the xmloption. Reorganize the representation of the XML option in the parse tree and the API to make it easier to manage and understand. Add regression tests for parsing back XML expressions.
2007-02-03 15:06:56 +01:00
ereport(ERROR,
(errcode(ERRCODE_NOT_AN_XML_DOCUMENT),
errmsg("not an XML document")));
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
}
/* If we weren't asked to indent, we're done. */
if (!indent)
{
xmlFreeDoc(doc);
return (text *) data;
}
/* Otherwise, we gotta spin up some error handling. */
xmlerrcxt = pg_xml_init(PG_XML_STRICTNESS_ALL);
PG_TRY();
{
size_t decl_len = 0;
/* The serialized data will go into this buffer. */
buf = xmlBufferCreate();
if (buf == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate xmlBuffer");
/* Detect whether there's an XML declaration */
parse_xml_decl(xml_text2xmlChar(data), &decl_len, NULL, NULL, NULL);
/*
* Emit declaration only if the input had one. Note: some versions of
* xmlSaveToBuffer leak memory if a non-null encoding argument is
* passed, so don't do that. We don't want any encoding conversion
* anyway.
*/
if (decl_len == 0)
ctxt = xmlSaveToBuffer(buf, NULL,
XML_SAVE_NO_DECL | XML_SAVE_FORMAT);
else
ctxt = xmlSaveToBuffer(buf, NULL,
XML_SAVE_FORMAT);
if (ctxt == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate xmlSaveCtxt");
if (parsed_xmloptiontype == XMLOPTION_DOCUMENT)
{
/* If it's a document, saving is easy. */
if (xmlSaveDoc(ctxt, doc) == -1 || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
"could not save document to xmlBuffer");
}
else if (content_nodes != NULL)
{
/*
* Deal with the case where we have non-singly-rooted XML.
* libxml's dump functions don't work well for that without help.
* We build a fake root node that serves as a container for the
* content nodes, and then iterate over the nodes.
*/
xmlNodePtr root;
xmlNodePtr newline;
root = xmlNewNode(NULL, (const xmlChar *) "content-root");
if (root == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate xml node");
/* This attaches root to doc, so we need not free it separately. */
xmlDocSetRootElement(doc, root);
xmlAddChild(root, content_nodes);
Implement XMLSERIALIZE for real. Analogously, make the xml to text cast observe the xmloption. Reorganize the representation of the XML option in the parse tree and the API to make it easier to manage and understand. Add regression tests for parsing back XML expressions.
2007-02-03 15:06:56 +01:00
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
/*
* We use this node to insert newlines in the dump. Note: in at
* least some libxml versions, xmlNewDocText would not attach the
* node to the document even if we passed it. Therefore, manage
* freeing of this node manually, and pass NULL here to make sure
* there's not a dangling link.
*/
newline = xmlNewDocText(NULL, (const xmlChar *) "\n");
if (newline == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate xml node");
for (xmlNodePtr node = root->children; node; node = node->next)
{
/* insert newlines between nodes */
if (node->type != XML_TEXT_NODE && node->prev != NULL)
{
if (xmlSaveTree(ctxt, newline) == -1 || xmlerrcxt->err_occurred)
{
xmlFreeNode(newline);
xml_ereport(xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
"could not save newline to xmlBuffer");
}
}
if (xmlSaveTree(ctxt, node) == -1 || xmlerrcxt->err_occurred)
{
xmlFreeNode(newline);
xml_ereport(xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
"could not save content to xmlBuffer");
}
}
xmlFreeNode(newline);
}
if (xmlSaveClose(ctxt) == -1 || xmlerrcxt->err_occurred)
{
ctxt = NULL; /* don't try to close it again */
xml_ereport(xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
"could not close xmlSaveCtxtPtr");
}
result = (text *) xmlBuffer_to_xmltype(buf);
}
PG_CATCH();
{
if (ctxt)
xmlSaveClose(ctxt);
if (buf)
xmlBufferFree(buf);
if (doc)
xmlFreeDoc(doc);
pg_xml_done(xmlerrcxt, true);
PG_RE_THROW();
}
PG_END_TRY();
xmlBufferFree(buf);
xmlFreeDoc(doc);
pg_xml_done(xmlerrcxt, false);
return result;
#else
NO_XML_SUPPORT();
return NULL;
#endif
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
xmltype *
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
xmlelement(XmlExpr *xexpr,
Datum *named_argvalue, bool *named_argnull,
Datum *argvalue, bool *argnull)
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
{
#ifdef USE_LIBXML
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
xmltype *result;
List *named_arg_strings;
List *arg_strings;
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
int i;
ListCell *arg;
ListCell *narg;
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
PgXmlErrorContext *xmlerrcxt;
volatile xmlBufferPtr buf = NULL;
volatile xmlTextWriterPtr writer = NULL;
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
/*
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
* All arguments are already evaluated, and their values are passed in the
* named_argvalue/named_argnull or argvalue/argnull arrays. This avoids
* issues if one of the arguments involves a call to some other function
* or subsystem that wants to use libxml on its own terms. We examine the
* original XmlExpr to identify the numbers and types of the arguments.
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
*/
named_arg_strings = NIL;
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
i = 0;
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
foreach(arg, xexpr->named_args)
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
{
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
Expr *e = (Expr *) lfirst(arg);
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
char *str;
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
if (named_argnull[i])
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
str = NULL;
else
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
str = map_sql_value_to_xml_value(named_argvalue[i],
exprType((Node *) e),
false);
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
named_arg_strings = lappend(named_arg_strings, str);
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
i++;
}
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
arg_strings = NIL;
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
i = 0;
foreach(arg, xexpr->args)
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
{
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
Expr *e = (Expr *) lfirst(arg);
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
char *str;
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
/* here we can just forget NULL elements immediately */
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
if (!argnull[i])
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
{
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
str = map_sql_value_to_xml_value(argvalue[i],
exprType((Node *) e),
true);
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
arg_strings = lappend(arg_strings, str);
}
Faster expression evaluation and targetlist projection. This replaces the old, recursive tree-walk based evaluation, with non-recursive, opcode dispatch based, expression evaluation. Projection is now implemented as part of expression evaluation. This both leads to significant performance improvements, and makes future just-in-time compilation of expressions easier. The speed gains primarily come from: - non-recursive implementation reduces stack usage / overhead - simple sub-expressions are implemented with a single jump, without function calls - sharing some state between different sub-expressions - reduced amount of indirect/hard to predict memory accesses by laying out operation metadata sequentially; including the avoidance of nearly all of the previously used linked lists - more code has been moved to expression initialization, avoiding constant re-checks at evaluation time Future just-in-time compilation (JIT) has become easier, as demonstrated by released patches intended to be merged in a later release, for primarily two reasons: Firstly, due to a stricter split between expression initialization and evaluation, less code has to be handled by the JIT. Secondly, due to the non-recursive nature of the generated "instructions", less performance-critical code-paths can easily be shared between interpreted and compiled evaluation. The new framework allows for significant future optimizations. E.g.: - basic infrastructure for to later reduce the per executor-startup overhead of expression evaluation, by caching state in prepared statements. That'd be helpful in OLTPish scenarios where initialization overhead is measurable. - optimizing the generated "code". A number of proposals for potential work has already been made. - optimizing the interpreter. Similarly a number of proposals have been made here too. The move of logic into the expression initialization step leads to some backward-incompatible changes: - Function permission checks are now done during expression initialization, whereas previously they were done during execution. In edge cases this can lead to errors being raised that previously wouldn't have been, e.g. a NULL array being coerced to a different array type previously didn't perform checks. - The set of domain constraints to be checked, is now evaluated once during expression initialization, previously it was re-built every time a domain check was evaluated. For normal queries this doesn't change much, but e.g. for plpgsql functions, which caches ExprStates, the old set could stick around longer. The behavior around might still change. Author: Andres Freund, with significant changes by Tom Lane, changes by Heikki Linnakangas Reviewed-By: Tom Lane, Heikki Linnakangas Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
2017-03-14 23:45:36 +01:00
i++;
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
}
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlerrcxt = pg_xml_init(PG_XML_STRICTNESS_ALL);
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_TRY();
{
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
buf = xmlBufferCreate();
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (buf == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
"could not allocate xmlBuffer");
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
writer = xmlNewTextWriterMemory(buf, 0);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (writer == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
"could not allocate xmlTextWriter");
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
xmlTextWriterStartElement(writer, (xmlChar *) xexpr->name);
forboth(arg, named_arg_strings, narg, xexpr->arg_names)
{
char *str = (char *) lfirst(arg);
char *argname = strVal(lfirst(narg));
if (str)
xmlTextWriterWriteAttribute(writer,
(xmlChar *) argname,
(xmlChar *) str);
}
foreach(arg, arg_strings)
{
char *str = (char *) lfirst(arg);
xmlTextWriterWriteRaw(writer, (xmlChar *) str);
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
}
xmlTextWriterEndElement(writer);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
/* we MUST do this now to flush data out to the buffer ... */
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
xmlFreeTextWriter(writer);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
writer = NULL;
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
result = xmlBuffer_to_xmltype(buf);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
}
PG_CATCH();
{
if (writer)
xmlFreeTextWriter(writer);
if (buf)
xmlBufferFree(buf);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_done(xmlerrcxt, true);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_RE_THROW();
}
PG_END_TRY();
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
xmlBufferFree(buf);
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_done(xmlerrcxt, false);
Use libxml's xmlwriter API for producing XML elements, instead of doing our own printing dance. This does a better job of quoting and escaping the values.
2007-01-10 21:33:54 +01:00
return result;
#else
NO_XML_SUPPORT();
return NULL;
#endif
}
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
xmltype *
Implement XMLSERIALIZE for real. Analogously, make the xml to text cast observe the xmloption. Reorganize the representation of the XML option in the parse tree and the API to make it easier to manage and understand. Add regression tests for parsing back XML expressions.
2007-02-03 15:06:56 +01:00
xmlparse(text *data, XmlOptionType xmloption_arg, bool preserve_whitespace)
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
{
#ifdef USE_LIBXML
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
xmlDocPtr doc;
Fix encoding handling in xml binary input function. If the XML header didn't specify an encoding explicitly, we used to treat it as being in database encoding when we parsed it, but then perform a UTF-8 -> database encoding conversion on it, which was completely bogus. It's now consistently treated as UTF-8.
2009-09-04 12:49:29 +02:00
doc = xml_parse(data, xmloption_arg, preserve_whitespace,
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
GetDatabaseEncoding(), NULL, NULL, NULL);
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
xmlFreeDoc(doc);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
return (xmltype *) data;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#else
NO_XML_SUPPORT();
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
return NULL;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#endif
}
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
xmltype *
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
xmlpi(const char *target, text *arg, bool arg_is_null, bool *result_is_null)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
#ifdef USE_LIBXML
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
xmltype *result;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
StringInfoData buf;
Allow XML processing instructions starting with "xml" while prohibiting those being exactly "xml". Bug #3735 from Ben Leslie
2007-11-09 16:52:51 +01:00
if (pg_strcasecmp(target, "xml") == 0)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
ereport(ERROR,
Some fine-tuning of xmlpi in corner cases: - correct error codes - do syntax checks in correct order - strip leading spaces of argument
2007-01-07 23:49:56 +01:00
(errcode(ERRCODE_SYNTAX_ERROR), /* really */
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
errmsg("invalid XML processing instruction"),
Allow XML processing instructions starting with "xml" while prohibiting those being exactly "xml". Bug #3735 from Ben Leslie
2007-11-09 16:52:51 +01:00
errdetail("XML processing instruction target name cannot be \"%s\".", target)));
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Some fine-tuning of xmlpi in corner cases: - correct error codes - do syntax checks in correct order - strip leading spaces of argument
2007-01-07 23:49:56 +01:00
/*
* Following the SQL standard, the null check comes after the syntax check
* above.
*/
*result_is_null = arg_is_null;
if (*result_is_null)
return NULL;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
initStringInfo(&buf);
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
appendStringInfo(&buf, "<?%s", target);
if (arg != NULL)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
char *string;
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
string = text_to_cstring(arg);
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
if (strstr(string, "?>") != NULL)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_XML_PROCESSING_INSTRUCTION),
errmsg("invalid XML processing instruction"),
errdetail("XML processing instruction cannot contain \"?>\".")));
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
appendStringInfoChar(&buf, ' ');
Some fine-tuning of xmlpi in corner cases: - correct error codes - do syntax checks in correct order - strip leading spaces of argument
2007-01-07 23:49:56 +01:00
appendStringInfoString(&buf, string + strspn(string, " "));
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
pfree(string);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
appendStringInfoString(&buf, "?>");
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
result = stringinfo_to_xmltype(&buf);
pfree(buf.data);
return result;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#else
NO_XML_SUPPORT();
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
return NULL;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#endif
}
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
xmltype *
xmlroot(xmltype *data, text *version, int standalone)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
#ifdef USE_LIBXML
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
char *str;
size_t len;
xmlChar *orig_version;
int orig_standalone;
StringInfoData buf;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
len = VARSIZE(data) - VARHDRSZ;
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
str = text_to_cstring((text *) data);
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
parse_xml_decl((xmlChar *) str, &len, &orig_version, NULL, &orig_standalone);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
if (version)
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
orig_version = xml_text2xmlChar(version);
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
else
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
orig_version = NULL;
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
switch (standalone)
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
{
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
case XML_STANDALONE_YES:
orig_standalone = 1;
break;
case XML_STANDALONE_NO:
orig_standalone = 0;
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
break;
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
case XML_STANDALONE_NO_VALUE:
orig_standalone = -1;
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
break;
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
case XML_STANDALONE_OMITTED:
/* leave original value */
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
break;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
initStringInfo(&buf);
print_xml_decl(&buf, orig_version, 0, orig_standalone);
appendStringInfoString(&buf, str + len);
Replace xmlroot with a properly functioning version that parses the value, sets the items, and serializes the value back (rather than adding an arbitrary number of XML preambles as before). The libxml memory management via palloc had to be disabled because it crashes when libxml tries to access memory that was helpfully freed earlier by PostgreSQL. This needs further thought.
2007-01-06 20:18:36 +01:00
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
return stringinfo_to_xmltype(&buf);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#else
NO_XML_SUPPORT();
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
return NULL;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#endif
}
/*
* Validate document (given as string) against DTD (given as external link)
Disable the undocumented xmlvalidate() function, which was unintentionally left in the code though it was not meant to be provided. It represents a security hole because unprivileged users could use it to look at (at least the first line of) any file readable by the backend. Fortunately, this is only possible if the backend was built with XML support, so the damage is at least mitigated; and 8.3 probably hasn't propagated into any security-critical uses yet anyway. Per report from Sergey Burladyan.
2008-03-01 03:46:49 +01:00
*
* This has been removed because it is a security hole: unprivileged users
* should not be able to use Postgres to fetch arbitrary external files,
* which unfortunately is exactly what libxml is willing to do with the DTD
* parameter.
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
Datum
xmlvalidate(PG_FUNCTION_ARGS)
{
Disable the undocumented xmlvalidate() function, which was unintentionally left in the code though it was not meant to be provided. It represents a security hole because unprivileged users could use it to look at (at least the first line of) any file readable by the backend. Fortunately, this is only possible if the backend was built with XML support, so the damage is at least mitigated; and 8.3 probably hasn't propagated into any security-critical uses yet anyway. Per report from Sergey Burladyan.
2008-03-01 03:46:49 +01:00
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("xmlvalidate is not implemented")));
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
return 0;
}
Add support for xmlval IS DOCUMENT expression.
2007-01-14 14:11:54 +01:00
bool
xml_is_document(xmltype *arg)
{
#ifdef USE_LIBXML
Clean up dubious error handling in wellformed_xml(). This ancient bit of code was summarily trapping any ereport longjmp whatsoever and assuming that it must represent an invalid-XML report. It's not really appropriate to handle OOM-like situations that way: maybe the input is valid or maybe not, but we couldn't find out. And it'd be a seriously bad idea to ignore, say, a query cancel error that way. (Perhaps that can't happen because there is no CHECK_FOR_INTERRUPTS anywhere within xml_parse, but even if that's true today it's obviously a very fragile assumption.) But in the wake of the previous commit, we can drop the PG_TRY here altogether, and use the soft error mechanism to catch only the kinds of errors that are legitimate to treat as invalid-XML. (This is our first use of the soft error mechanism for something not directly related to a datatype input function. It won't be the last.) xml_is_document can be converted in the same way. That one is not actively broken, because it was checking specifically for ERRCODE_INVALID_XML_DOCUMENT rather than trapping everything; but the code is still shorter and probably faster this way. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 17:10:36 +01:00
xmlDocPtr doc;
ErrorSaveContext escontext = {T_ErrorSaveContext};
Add support for xmlval IS DOCUMENT expression.
2007-01-14 14:11:54 +01:00
Clean up dubious error handling in wellformed_xml(). This ancient bit of code was summarily trapping any ereport longjmp whatsoever and assuming that it must represent an invalid-XML report. It's not really appropriate to handle OOM-like situations that way: maybe the input is valid or maybe not, but we couldn't find out. And it'd be a seriously bad idea to ignore, say, a query cancel error that way. (Perhaps that can't happen because there is no CHECK_FOR_INTERRUPTS anywhere within xml_parse, but even if that's true today it's obviously a very fragile assumption.) But in the wake of the previous commit, we can drop the PG_TRY here altogether, and use the soft error mechanism to catch only the kinds of errors that are legitimate to treat as invalid-XML. (This is our first use of the soft error mechanism for something not directly related to a datatype input function. It won't be the last.) xml_is_document can be converted in the same way. That one is not actively broken, because it was checking specifically for ERRCODE_INVALID_XML_DOCUMENT rather than trapping everything; but the code is still shorter and probably faster this way. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 17:10:36 +01:00
/*
* We'll report "true" if no soft error is reported by xml_parse().
*/
doc = xml_parse((text *) arg, XMLOPTION_DOCUMENT, true,
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
GetDatabaseEncoding(), NULL, NULL, (Node *) &escontext);
Add support for xmlval IS DOCUMENT expression.
2007-01-14 14:11:54 +01:00
if (doc)
xmlFreeDoc(doc);
Clean up dubious error handling in wellformed_xml(). This ancient bit of code was summarily trapping any ereport longjmp whatsoever and assuming that it must represent an invalid-XML report. It's not really appropriate to handle OOM-like situations that way: maybe the input is valid or maybe not, but we couldn't find out. And it'd be a seriously bad idea to ignore, say, a query cancel error that way. (Perhaps that can't happen because there is no CHECK_FOR_INTERRUPTS anywhere within xml_parse, but even if that's true today it's obviously a very fragile assumption.) But in the wake of the previous commit, we can drop the PG_TRY here altogether, and use the soft error mechanism to catch only the kinds of errors that are legitimate to treat as invalid-XML. (This is our first use of the soft error mechanism for something not directly related to a datatype input function. It won't be the last.) xml_is_document can be converted in the same way. That one is not actively broken, because it was checking specifically for ERRCODE_INVALID_XML_DOCUMENT rather than trapping everything; but the code is still shorter and probably faster this way. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 17:10:36 +01:00
return !escontext.error_occurred;
Add support for xmlval IS DOCUMENT expression.
2007-01-14 14:11:54 +01:00
#else /* not USE_LIBXML */
NO_XML_SUPPORT();
return false;
#endif /* not USE_LIBXML */
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
#ifdef USE_LIBXML
/*
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
* pg_xml_init_library --- set up for use of libxml
Export xml.c's libxml-error-handling support so that contrib/xml2 can use it too, instead of duplicating the functionality (badly). I renamed xml_init to pg_xml_init, because the former seemed just a bit too generic to be safe as a global symbol. I considered likewise renaming xml_ereport to pg_xml_ereport, but felt that the reference to ereport probably made it sufficiently PG-centric already.
2010-03-03 18:29:45 +01:00
*
* This should be called by each function that is about to use libxml
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
* facilities but doesn't require error handling. It initializes libxml
* and verifies compatibility with the loaded libxml version. These are
* once-per-session activities.
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
*
* TODO: xmlChar is utf8-char, make proper tuning (initdb with enc!=utf8 and
* check)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
Export xml.c's libxml-error-handling support so that contrib/xml2 can use it too, instead of duplicating the functionality (badly). I renamed xml_init to pg_xml_init, because the former seemed just a bit too generic to be safe as a global symbol. I considered likewise renaming xml_ereport to pg_xml_ereport, but felt that the reference to ereport probably made it sufficiently PG-centric already.
2010-03-03 18:29:45 +01:00
void
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_init_library(void)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
static bool first_time = true;
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
if (first_time)
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
{
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
/* Stuff we need do only once per session */
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
/*
* Currently, we have no pure UTF-8 support for internals -- check if
* we can work.
*/
if (sizeof(char) != sizeof(xmlChar))
ereport(ERROR,
(errmsg("could not initialize XML library"),
Remove some unnecessary casts in format arguments We can use %zd or %zu directly, no need to cast to int. Conversely, some code was casting away from int when it could be using %d directly.
2021-08-08 22:05:42 +02:00
errdetail("libxml2 has incompatible char type: sizeof(char)=%zu, sizeof(xmlChar)=%zu.",
sizeof(char), sizeof(xmlChar))));
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
#ifdef USE_LIBXMLCONTEXT
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Set up libxml's memory allocation our way */
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
xml_memory_init();
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
#endif
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
/* Check library compatibility */
LIBXML_TEST_VERSION;
first_time = false;
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
}
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
}
Reactivate libxml memory management via palloc, now that I think I've classified the conditions under which this is safe to do (see source code comment).
2007-01-27 15:50:51 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* pg_xml_init --- set up for use of libxml and register an error handler
*
* This should be called by each function that is about to use libxml
* facilities and requires error handling. It initializes libxml with
* pg_xml_init_library() and establishes our libxml error handler.
*
* strictness determines which errors are reported and which are ignored.
*
* Calls to this function MUST be followed by a PG_TRY block that guarantees
* that pg_xml_done() is called during either normal or error exit.
*
* This is exported for use by contrib/xml2, as well as other code that might
* wish to share use of this module's libxml error handler.
*/
PgXmlErrorContext *
pg_xml_init(PgXmlStrictness strictness)
{
PgXmlErrorContext *errcxt;
Check to see whether libxml2 handles error context the way we expect. It turns out to be possible to link against a libxml2.so that does this differently than the version we configured and built against, so we need a runtime check to avoid bizarre behavior. Per report from Bernd Helmle. Patch by Florian Pflug.
2011-07-26 22:29:53 +02:00
void *new_errcxt;
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Do one-time setup if needed */
pg_xml_init_library();
/* Create error handling context structure */
errcxt = (PgXmlErrorContext *) palloc(sizeof(PgXmlErrorContext));
errcxt->magic = ERRCXT_MAGIC;
errcxt->strictness = strictness;
errcxt->err_occurred = false;
initStringInfo(&errcxt->err_buf);
/*
* Save original error handler and install ours. libxml originally didn't
* distinguish between the contexts for generic and for structured error
* handlers. If we're using an old libxml version, we must thus save the
* generic error context, even though we're using a structured error
* handler.
*/
errcxt->saved_errfunc = xmlStructuredError;
#ifdef HAVE_XMLSTRUCTUREDERRORCONTEXT
errcxt->saved_errcxt = xmlStructuredErrorContext;
#else
errcxt->saved_errcxt = xmlGenericErrorContext;
#endif
xmlSetStructuredErrorFunc((void *) errcxt, xml_errorHandler);
Check to see whether libxml2 handles error context the way we expect. It turns out to be possible to link against a libxml2.so that does this differently than the version we configured and built against, so we need a runtime check to avoid bizarre behavior. Per report from Bernd Helmle. Patch by Florian Pflug.
2011-07-26 22:29:53 +02:00
/*
* Verify that xmlSetStructuredErrorFunc set the context variable we
* expected it to. If not, the error context pointer we just saved is not
* the correct thing to restore, and since that leaves us without a way to
* restore the context in pg_xml_done, we must fail.
*
* The only known situation in which this test fails is if we compile with
* headers from a libxml2 that doesn't track the structured error context
Check LIBXML_VERSION instead of testing in configure script. We had put a test for libxml2's xmlStructuredErrorContext variable in configure, but of course that doesn't work on Windows builds. The next best alternative seems to be to test the LIBXML_VERSION symbol provided by xmlversion.h. Per report from Talha Bin Rizwan, though this fixes it in a different way than his proposed patch.
2012-08-17 06:05:26 +02:00
* separately (< 2.7.4), but at runtime use a version that does, or vice
Check to see whether libxml2 handles error context the way we expect. It turns out to be possible to link against a libxml2.so that does this differently than the version we configured and built against, so we need a runtime check to avoid bizarre behavior. Per report from Bernd Helmle. Patch by Florian Pflug.
2011-07-26 22:29:53 +02:00
* versa. The libxml2 authors did not treat that change as constituting
* an ABI break, so the LIBXML_TEST_VERSION test in pg_xml_init_library
* fails to protect us from this.
*/
#ifdef HAVE_XMLSTRUCTUREDERRORCONTEXT
new_errcxt = xmlStructuredErrorContext;
#else
new_errcxt = xmlGenericErrorContext;
#endif
if (new_errcxt != (void *) errcxt)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("could not set up XML error handler"),
errhint("This probably indicates that the version of libxml2"
" being used is not compatible with the libxml2"
" header files that PostgreSQL was built with.")));
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
/*
* Also, install an entity loader to prevent unwanted fetches of external
* files and URLs.
*/
errcxt->saved_entityfunc = xmlGetExternalEntityLoader();
xmlSetExternalEntityLoader(xmlPgEntityLoader);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
return errcxt;
}
/*
* pg_xml_done --- restore previous libxml error handling
*
* Resets libxml's global error-handling state to what it was before
* pg_xml_init() was called.
*
* This routine verifies that all pending errors have been dealt with
* (in assert-enabled builds, anyway).
*/
void
pg_xml_done(PgXmlErrorContext *errcxt, bool isError)
{
void *cur_errcxt;
/* An assert seems like enough protection here */
Assert(errcxt->magic == ERRCXT_MAGIC);
/*
* In a normal exit, there should be no un-handled libxml errors. But we
* shouldn't try to enforce this during error recovery, since the longjmp
* could have been thrown before xml_ereport had a chance to run.
*/
Assert(!errcxt->err_occurred || isError);
/*
* Check that libxml's global state is correct, warn if not. This is a
* real test and not an Assert because it has a higher probability of
* happening.
*/
#ifdef HAVE_XMLSTRUCTUREDERRORCONTEXT
cur_errcxt = xmlStructuredErrorContext;
#else
cur_errcxt = xmlGenericErrorContext;
#endif
if (cur_errcxt != (void *) errcxt)
elog(WARNING, "libxml error handling state is out of sync with xml.c");
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
/* Restore the saved handlers */
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlSetStructuredErrorFunc(errcxt->saved_errcxt, errcxt->saved_errfunc);
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
xmlSetExternalEntityLoader(errcxt->saved_entityfunc);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* Mark the struct as invalid, just in case somebody somehow manages to
* call xml_errorHandler or xml_ereport with it.
*/
errcxt->magic = 0;
/* Release memory */
pfree(errcxt->err_buf.data);
pfree(errcxt);
}
/*
* pg_xml_error_occurred() --- test the error flag
*/
bool
pg_xml_error_occurred(PgXmlErrorContext *errcxt)
{
return errcxt->err_occurred;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
/*
* SQL/XML allows storing "XML documents" or "XML content". "XML
* documents" are specified by the XML specification and are parsed
* easily by libxml. "XML content" is specified by SQL/XML as the
* production "XMLDecl? content". But libxml can only parse the
* "content" part, so we have to parse the XML declaration ourselves
* to complete this.
*/
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
#define CHECK_XML_SPACE(p) \
do { \
if (!xmlIsBlank_ch(*(p))) \
return XML_ERR_SPACE_REQUIRED; \
} while (0)
#define SKIP_XML_SPACE(p) \
while (xmlIsBlank_ch(*(p))) (p)++
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
Allow XML processing instructions starting with "xml" while prohibiting those being exactly "xml". Bug #3735 from Ben Leslie
2007-11-09 16:52:51 +01:00
/* Letter | Digit | '.' | '-' | '_' | ':' | CombiningChar | Extender */
xmlGetUTF8Char()'s second argument is both input and output. Fix uninitialized value, and avoid invoking the function nine separate times in the pg_xmlIsNameChar macro. Should resolve buildfarm failures. Per report from Ben Leslie.
2007-11-10 19:51:20 +01:00
/* Beware of multiple evaluations of argument! */
#define PG_XMLISNAMECHAR(c) \
Allow XML processing instructions starting with "xml" while prohibiting those being exactly "xml". Bug #3735 from Ben Leslie
2007-11-09 16:52:51 +01:00
(xmlIsBaseChar_ch(c) || xmlIsIdeographicQ(c) \
|| xmlIsDigit_ch(c) \
|| c == '.' || c == '-' || c == '_' || c == ':' \
|| xmlIsCombiningQ(c) \
|| xmlIsExtender_ch(c))
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
/* pnstrdup, but deal with xmlChar not char; len is measured in xmlChars */
static xmlChar *
xml_pnstrdup(const xmlChar *str, size_t len)
{
xmlChar *result;
result = (xmlChar *) palloc((len + 1) * sizeof(xmlChar));
memcpy(result, str, len * sizeof(xmlChar));
result[len] = 0;
return result;
}
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
/* Ditto, except input is char* */
static xmlChar *
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
pg_xmlCharStrndup(const char *str, size_t len)
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
xmlChar *result;
result = (xmlChar *) palloc((len + 1) * sizeof(xmlChar));
memcpy(result, str, len);
result[len] = '\0';
return result;
}
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
/*
* Copy xmlChar string to PostgreSQL-owned memory, freeing the input.
*
* The input xmlChar is freed regardless of success of the copy.
*/
static char *
xml_pstrdup_and_free(xmlChar *str)
{
char *result;
if (str)
{
PG_TRY();
{
result = pstrdup((char *) str);
}
PG_FINALLY This gives an alternative way of catching exceptions, for the common case where the cleanup code is the same in the error and non-error cases. So instead of PG_TRY(); { ... code that might throw ereport(ERROR) ... } PG_CATCH(); { cleanup(); PG_RE_THROW(); } PG_END_TRY(); cleanup(); one can write PG_TRY(); { ... code that might throw ereport(ERROR) ... } PG_FINALLY(); { cleanup(); } PG_END_TRY(); Discussion: https://www.postgresql.org/message-id/flat/95a822c3-728b-af0e-d7e5-71890507ae0c%402ndquadrant.com
2019-11-01 11:09:52 +01:00
PG_FINALLY();
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
{
xmlFree(str);
}
PG_END_TRY();
}
else
result = NULL;
return result;
}
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
/*
* str is the null-terminated input string. Remaining arguments are
* output arguments; each can be NULL if value is not wanted.
* version and encoding are returned as locally-palloc'd strings.
* Result is 0 if OK, an error code if not.
*/
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
static int
xmlGetUTF8Char()'s second argument is both input and output. Fix uninitialized value, and avoid invoking the function nine separate times in the pg_xmlIsNameChar macro. Should resolve buildfarm failures. Per report from Ben Leslie.
2007-11-10 19:51:20 +01:00
parse_xml_decl(const xmlChar *str, size_t *lenp,
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlChar **version, xmlChar **encoding, int *standalone)
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
{
const xmlChar *p;
const xmlChar *save_p;
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
size_t len;
xmlGetUTF8Char()'s second argument is both input and output. Fix uninitialized value, and avoid invoking the function nine separate times in the pg_xmlIsNameChar macro. Should resolve buildfarm failures. Per report from Ben Leslie.
2007-11-10 19:51:20 +01:00
int utf8char;
Allow XML processing instructions starting with "xml" while prohibiting those being exactly "xml". Bug #3735 from Ben Leslie
2007-11-09 16:52:51 +01:00
int utf8len;
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* Only initialize libxml. We don't need error handling here, but we do
* need to make sure libxml is initialized before calling any of its
* functions. Note that this is safe (and a no-op) if caller has already
* done pg_xml_init().
*/
pg_xml_init_library();
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
/* Initialize output arguments to "not present" */
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
if (version)
*version = NULL;
if (encoding)
*encoding = NULL;
if (standalone)
*standalone = -1;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
p = str;
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
if (xmlStrncmp(p, (xmlChar *) "<?xml", 5) != 0)
goto finished;
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
/*
* If next char is a name char, it's a PI like <?xml-stylesheet ...?>
* rather than an XMLDecl, so we have done what we came to do and found no
* XMLDecl.
*
* We need an input length value for xmlGetUTF8Char, but there's no need
* to count the whole document size, so use strnlen not strlen.
*/
utf8len = strnlen((const char *) (p + 5), MAX_MULTIBYTE_CHAR_LEN);
xmlGetUTF8Char()'s second argument is both input and output. Fix uninitialized value, and avoid invoking the function nine separate times in the pg_xmlIsNameChar macro. Should resolve buildfarm failures. Per report from Ben Leslie.
2007-11-10 19:51:20 +01:00
utf8char = xmlGetUTF8Char(p + 5, &utf8len);
if (PG_XMLISNAMECHAR(utf8char))
Allow XML processing instructions starting with "xml" while prohibiting those being exactly "xml". Bug #3735 from Ben Leslie
2007-11-09 16:52:51 +01:00
goto finished;
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
p += 5;
/* version */
CHECK_XML_SPACE(p);
SKIP_XML_SPACE(p);
if (xmlStrncmp(p, (xmlChar *) "version", 7) != 0)
return XML_ERR_VERSION_MISSING;
p += 7;
SKIP_XML_SPACE(p);
if (*p != '=')
return XML_ERR_VERSION_MISSING;
p += 1;
SKIP_XML_SPACE(p);
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
if (*p == '\'' || *p == '"')
{
const xmlChar *q;
q = xmlStrchr(p + 1, *p);
if (!q)
return XML_ERR_VERSION_MISSING;
if (version)
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
*version = xml_pnstrdup(p + 1, q - p - 1);
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
p = q + 1;
}
else
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
return XML_ERR_VERSION_MISSING;
/* encoding */
save_p = p;
SKIP_XML_SPACE(p);
if (xmlStrncmp(p, (xmlChar *) "encoding", 8) == 0)
{
CHECK_XML_SPACE(save_p);
p += 8;
SKIP_XML_SPACE(p);
if (*p != '=')
return XML_ERR_MISSING_ENCODING;
p += 1;
SKIP_XML_SPACE(p);
if (*p == '\'' || *p == '"')
{
const xmlChar *q;
q = xmlStrchr(p + 1, *p);
if (!q)
return XML_ERR_MISSING_ENCODING;
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
if (encoding)
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
*encoding = xml_pnstrdup(p + 1, q - p - 1);
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
p = q + 1;
}
else
return XML_ERR_MISSING_ENCODING;
}
else
{
p = save_p;
}
/* standalone */
save_p = p;
SKIP_XML_SPACE(p);
if (xmlStrncmp(p, (xmlChar *) "standalone", 10) == 0)
{
CHECK_XML_SPACE(save_p);
p += 10;
SKIP_XML_SPACE(p);
if (*p != '=')
return XML_ERR_STANDALONE_VALUE;
p += 1;
SKIP_XML_SPACE(p);
Fix bogus length calculation that could lead to crash if the string happened to be right up against the end of memory, per report from Matt Magoffin. While at it, avoid useless multiple copying of string by not depending on xmlStrncatNew.
2007-11-21 00:14:41 +01:00
if (xmlStrncmp(p, (xmlChar *) "'yes'", 5) == 0 ||
xmlStrncmp(p, (xmlChar *) "\"yes\"", 5) == 0)
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
{
Fix null-dereference crash in parse_xml_decl(). parse_xml_decl's header comment says you can pass NULL for any unwanted output parameter, but it failed to honor this contract for the "standalone" flag. The only currently-affected caller is xml_recv, so the net effect is that sending a binary XML value containing a standalone parameter in its xml declaration would crash the backend. Per bug #6044 from Christopher Dillard. In passing, remove useless initializations of parse_xml_decl's output parameters in xml_parse. Back-patch to 8.3, where this code was introduced.
2011-05-28 18:36:04 +02:00
if (standalone)
*standalone = 1;
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
p += 5;
}
Fix bogus length calculation that could lead to crash if the string happened to be right up against the end of memory, per report from Matt Magoffin. While at it, avoid useless multiple copying of string by not depending on xmlStrncatNew.
2007-11-21 00:14:41 +01:00
else if (xmlStrncmp(p, (xmlChar *) "'no'", 4) == 0 ||
xmlStrncmp(p, (xmlChar *) "\"no\"", 4) == 0)
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
{
Fix null-dereference crash in parse_xml_decl(). parse_xml_decl's header comment says you can pass NULL for any unwanted output parameter, but it failed to honor this contract for the "standalone" flag. The only currently-affected caller is xml_recv, so the net effect is that sending a binary XML value containing a standalone parameter in its xml declaration would crash the backend. Per bug #6044 from Christopher Dillard. In passing, remove useless initializations of parse_xml_decl's output parameters in xml_parse. Back-patch to 8.3, where this code was introduced.
2011-05-28 18:36:04 +02:00
if (standalone)
*standalone = 0;
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
p += 4;
}
else
return XML_ERR_STANDALONE_VALUE;
}
else
{
p = save_p;
}
SKIP_XML_SPACE(p);
if (xmlStrncmp(p, (xmlChar *) "?>", 2) != 0)
return XML_ERR_XMLDECL_NOT_FINISHED;
p += 2;
finished:
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
len = p - str;
for (p = str; p < str + len; p++)
if (*p > 127)
return XML_ERR_INVALID_CHAR;
if (lenp)
*lenp = len;
Allow XML fragment to contain a XML declaration. For that, we need a small hand-crafted parser for the XML declaration, because libxml doesn't seem to allow this.
2007-01-07 01:13:55 +01:00
return XML_ERR_OK;
}
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
/*
* Write an XML declaration. On output, we adjust the XML declaration
* as follows. (These rules are the moral equivalent of the clause
* "Serialization of an XML value" in the SQL standard.)
*
* We try to avoid generating an XML declaration if possible. This is
* so that you don't get trivial things like xml '<foo/>' resulting in
* '<?xml version="1.0"?><foo/>', which would surely be annoying. We
* must provide a declaration if the standalone property is specified
* or if we include an encoding declaration. If we have a
* declaration, we must specify a version (XML requires this).
* Otherwise we only make a declaration if the version is not "1.0",
* which is the default version specified in SQL:2003.
*/
static bool
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
print_xml_decl(StringInfo buf, const xmlChar *version,
pg_enc encoding, int standalone)
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
{
Remove many -Wcast-qual warnings This addresses only those cases that are easy to fix by adding or moving a const qualifier or removing an unnecessary cast. There are many more complicated cases remaining.
2011-09-11 20:54:32 +02:00
if ((version && strcmp((const char *) version, PG_XML_DEFAULT_VERSION) != 0)
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
|| (encoding && encoding != PG_UTF8)
|| standalone != -1)
{
appendStringInfoString(buf, "<?xml");
if (version)
appendStringInfo(buf, " version=\"%s\"", version);
else
appendStringInfo(buf, " version=\"%s\"", PG_XML_DEFAULT_VERSION);
if (encoding && encoding != PG_UTF8)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
{
Fix xmlelement() to initialize libxml correctly before using it, and to avoid assuming that evaluation of its input expressions won't change the state of libxml. This requires refactoring xml_init() to not call xmlInitParser(), since now not all of its callers want that. I also tweaked things to avoid repeated execution of one-time-only tests inside xml_init(), though this is mostly for clarity rather than in hopes of saving any noticeable amount of runtime. Per report from Sheikh Amjad and subsequent discussion. In passing, fix a couple of inadequately schema-qualified queries.
2007-11-05 23:23:07 +01:00
/*
* XXX might be useful to convert this to IANA names (ISO-8859-1
* instead of LATIN1 etc.); needs field experience
*/
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
appendStringInfo(buf, " encoding=\"%s\"",
pg_encoding_to_char(encoding));
}
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
if (standalone == 1)
appendStringInfoString(buf, " standalone=\"yes\"");
else if (standalone == 0)
appendStringInfoString(buf, " standalone=\"no\"");
appendStringInfoString(buf, "?>");
return true;
}
else
return false;
}
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
/*
* Test whether an input that is to be parsed as CONTENT contains a DTD.
*
* The SQL/XML:2003 definition of CONTENT ("XMLDecl? content") is not
* satisfied by a document with a DTD, which is a bit of a wart, as it means
* the CONTENT type is not a proper superset of DOCUMENT. SQL/XML:2006 and
* later fix that, by redefining content with reference to the "more
* permissive" Document Node of the XQuery/XPath Data Model, such that any
* DOCUMENT value is indeed also a CONTENT value. That definition is more
* useful, as CONTENT becomes usable for parsing input of unknown form (think
* pg_restore).
*
* As used below in parse_xml when parsing for CONTENT, libxml does not give
* us the 2006+ behavior, but only the 2003; it will choke if the input has
* a DTD. But we can provide the 2006+ definition of CONTENT easily enough,
* by detecting this case first and simply doing the parse as DOCUMENT.
*
* A DTD can be found arbitrarily far in, but that would be a contrived case;
* it will ordinarily start within a few dozen characters. The only things
* that can precede it are an XMLDecl (here, the caller will have called
* parse_xml_decl already), whitespace, comments, and processing instructions.
* This function need only return true if it sees a valid sequence of such
* things leading to <!DOCTYPE. It can simply return false in any other
* cases, including malformed input; that will mean the input gets parsed as
* CONTENT as originally planned, with libxml reporting any errors.
*
* This is only to be called from xml_parse, when pg_xml_init has already
* been called. The input is already in UTF8 encoding.
*/
static bool
xml_doctype_in_content(const xmlChar *str)
{
const xmlChar *p = str;
for (;;)
{
const xmlChar *e;
SKIP_XML_SPACE(p);
if (*p != '<')
return false;
p++;
if (*p == '!')
{
p++;
/* if we see <!DOCTYPE, we can return true */
if (xmlStrncmp(p, (xmlChar *) "DOCTYPE", 7) == 0)
return true;
/* otherwise, if it's not a comment, fail */
if (xmlStrncmp(p, (xmlChar *) "--", 2) != 0)
return false;
/* find end of comment: find -- and a > must follow */
p = xmlStrstr(p + 2, (xmlChar *) "--");
if (!p || p[2] != '>')
return false;
/* advance over comment, and keep scanning */
p += 3;
continue;
}
/* otherwise, if it's not a PI <?target something?>, fail */
if (*p != '?')
return false;
p++;
/* find end of PI (the string ?> is forbidden within a PI) */
e = xmlStrstr(p, (xmlChar *) "?>");
if (!e)
return false;
/* advance over PI, keep scanning */
p = e + 2;
}
}
Various fixes in the logic of XML functions: - Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. - Subtle corrections in the handling of the standalone property in xmlroot(). - Allow xmlroot() to work on content fragments. - Subtle corrections in the handling of the version property in xmlconcat(). - Code refactoring for producing XML declarations.
2007-01-25 12:53:52 +01:00
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
/*
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
* Convert a text object to XML internal representation
*
* data is the source data (must not be toasted!), encoding is its encoding,
* and xmloption_arg and preserve_whitespace are options for the
* transformation.
*
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
* If parsed_xmloptiontype isn't NULL, *parsed_xmloptiontype is set to the
* XmlOptionType actually used to parse the input (typically the same as
* xmloption_arg, but a DOCTYPE node in the input can force DOCUMENT mode).
*
* If parsed_nodes isn't NULL and the input is not an XML document, the list
Revert "Add support for parsing of large XML data (>= 10MB)" This reverts commit 2197d06224a1, following a discussion over a Coverity report where issues like the "Billion laugh attack" could cause the backend to waste CPU and memory even if a client applied checks on the size of the data given in input, and libxml2 does not offer guarantees that input limits are respected under XML_PARSE_HUGE. Discussion: https://postgr.es/m/ZbHlgrPLtBZyr_QW@paquier.xyz
2024-01-26 02:15:32 +01:00
* of parsed nodes from the xmlParseBalancedChunkMemory call will be returned
* to *parsed_nodes.
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
*
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
* Errors normally result in ereport(ERROR), but if escontext is an
* ErrorSaveContext, then "safe" errors are reported there instead, and the
* caller must check SOFT_ERROR_OCCURRED() to see whether that happened.
Handle content and document options in xmlparse() correctly.
2006-12-28 04:17:38 +01:00
*
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
* Note: it is caller's responsibility to xmlFreeDoc() the result,
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
* else a permanent memory leak will ensue! But note the result could
* be NULL after a soft error.
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
*
* TODO maybe libxml2's xmlreader is better? (do not construct DOM,
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
* yet do not use SAX - see xmlreader.c)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
static xmlDocPtr
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
xml_parse(text *data, XmlOptionType xmloption_arg,
bool preserve_whitespace, int encoding,
XmlOptionType *parsed_xmloptiontype, xmlNodePtr *parsed_nodes,
Node *escontext)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
int32 len;
xmlChar *string;
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
xmlChar *utf8string;
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
PgXmlErrorContext *xmlerrcxt;
volatile xmlParserCtxtPtr ctxt = NULL;
volatile xmlDocPtr doc = NULL;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
/*
* This step looks annoyingly redundant, but we must do it to have a
* null-terminated string in case encoding conversion isn't required.
*/
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
len = VARSIZE_ANY_EXHDR(data); /* will be useful later */
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
string = xml_text2xmlChar(data);
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
/*
* If the data isn't UTF8, we must translate before giving it to libxml.
*
* XXX ideally, we'd catch any encoding conversion failure and return a
* soft error. However, failure to convert to UTF8 should be pretty darn
* rare, so for now this is left undone.
*/
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
utf8string = pg_do_encoding_conversion(string,
len,
Fix encoding handling in xml binary input function. If the XML header didn't specify an encoding explicitly, we used to treat it as being in database encoding when we parsed it, but then perform a UTF-8 -> database encoding conversion on it, which was completely bogus. It's now consistently treated as UTF-8.
2009-09-04 12:49:29 +02:00
encoding,
Clean up encoding issues in the xml type: In text mode, encoding declarations are ignored and removed, in binary mode they are honored as specified by the XML standard.
2007-01-18 14:59:11 +01:00
PG_UTF8);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Start up libxml and its parser */
xmlerrcxt = pg_xml_init(PG_XML_STRICTNESS_WELLFORMED);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Use a TRY block to ensure we clean up correctly */
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_TRY();
{
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
bool parse_as_document = false;
int res_code;
size_t count = 0;
xmlChar *version = NULL;
int standalone = 0;
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
/* Any errors here are reported as hard ereport's */
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlInitParser();
ctxt = xmlNewParserCtxt();
if (ctxt == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate parser context");
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
/* Decide whether to parse as document or content */
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
if (xmloption_arg == XMLOPTION_DOCUMENT)
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
parse_as_document = true;
else
{
/* Parse and skip over the XML declaration, if any */
res_code = parse_xml_decl(utf8string,
&count, &version, NULL, &standalone);
if (res_code != 0)
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
{
errsave(escontext,
errcode(ERRCODE_INVALID_XML_CONTENT),
errmsg_internal("invalid XML content: invalid XML declaration"),
errdetail_for_xml_code(res_code));
goto fail;
}
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
/* Is there a DOCTYPE element? */
if (xml_doctype_in_content(utf8string + count))
parse_as_document = true;
}
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
/* initialize output parameters */
if (parsed_xmloptiontype != NULL)
*parsed_xmloptiontype = parse_as_document ? XMLOPTION_DOCUMENT :
XMLOPTION_CONTENT;
if (parsed_nodes != NULL)
*parsed_nodes = NULL;
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
if (parse_as_document)
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
/*
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
* Note, that here we try to apply DTD defaults
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* (XML_PARSE_DTDATTR) according to SQL/XML:2008 GR 10.16.7.d:
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
* 'Default values defined by internal DTD are applied'. As for
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* external DTDs, we try to support them too, (see SQL/XML:2008 GR
* 10.16.7.e)
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
*/
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
doc = xmlCtxtReadDoc(ctxt, utf8string,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
NULL,
"UTF-8",
Revert "Add support for parsing of large XML data (>= 10MB)" This reverts commit 2197d06224a1, following a discussion over a Coverity report where issues like the "Billion laugh attack" could cause the backend to waste CPU and memory even if a client applied checks on the size of the data given in input, and libxml2 does not offer guarantees that input limits are respected under XML_PARSE_HUGE. Discussion: https://postgr.es/m/ZbHlgrPLtBZyr_QW@paquier.xyz
2024-01-26 02:15:32 +01:00
XML_PARSE_NOENT | XML_PARSE_DTDATTR
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
| (preserve_whitespace ? 0 : XML_PARSE_NOBLANKS));
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (doc == NULL || xmlerrcxt->err_occurred)
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
{
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
/* Use original option to decide which error code to report */
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
if (xmloption_arg == XMLOPTION_DOCUMENT)
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
xml_errsave(escontext, xmlerrcxt,
ERRCODE_INVALID_XML_DOCUMENT,
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
"invalid XML document");
else
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
xml_errsave(escontext, xmlerrcxt,
ERRCODE_INVALID_XML_CONTENT,
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
"invalid XML content");
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
goto fail;
Accept XML documents when xmloption = content, as required by SQL:2006+. Previously we were using the SQL:2003 definition, which doesn't allow this, but that creates a serious dump/restore gotcha: there is no setting of xmloption that will allow all valid XML data. Hence, switch to the 2006 definition. Since libxml doesn't accept <!DOCTYPE> directives in the mode we use for CONTENT parsing, the implementation is to detect <!DOCTYPE> in the input and switch to DOCUMENT parsing mode. This should not cost much, because <!DOCTYPE> should be close to the front of the input if it's there at all. It's possible that this causes the error messages for malformed input to be slightly different than they were before, if said input includes <!DOCTYPE>; but that does not seem like a big problem. In passing, buy back a few cycles in parsing of large XML documents by not doing strlen() of the whole input in parse_xml_decl(). Back-patch because dump/restore failures are not nice. This change shouldn't break any cases that worked before, so it seems safe to back-patch. Chapman Flack (revised a bit by me) Discussion: https://postgr.es/m/CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com
2019-03-23 21:24:30 +01:00
}
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
}
else
{
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
doc = xmlNewDoc(version);
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
if (doc == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate XML document");
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
Assert(doc->encoding == NULL);
doc->encoding = xmlStrdup((const xmlChar *) "UTF-8");
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
if (doc->encoding == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate XML document");
Fix intratransaction memory leaks in xml_recv, xmlconcat, xmlroot, and xml_parse, all arising from the same sloppy usage of parse_xml_decl. The original coding had that function returning its output string parameters in the libxml context, which is long-lived, and all but one of its callers neglected to free the strings afterwards. The easiest and most bulletproof fix is to return the strings in the local palloc context instead, since that's short-lived. This was only costing a dozen or two bytes per function call, but that adds up fast if the function is called repeatedly ... Noted while poking at the more general problem of what to do with our libxml memory allocation hooks. Back-patch to 8.3, which has the identical coding.
2009-05-12 22:17:40 +02:00
doc->standalone = standalone;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Allow empty content in xml type The xml type previously rejected "content" that is empty or consists only of spaces. But the SQL/XML standard allows that, so change that. The accepted values for XML "documents" are not changed. Reviewed-by: Ali Akbar <the.apaan@gmail.com>
2014-09-09 17:34:52 +02:00
/* allow empty content */
if (*(utf8string + count))
{
Revert "Add support for parsing of large XML data (>= 10MB)" This reverts commit 2197d06224a1, following a discussion over a Coverity report where issues like the "Billion laugh attack" could cause the backend to waste CPU and memory even if a client applied checks on the size of the data given in input, and libxml2 does not offer guarantees that input limits are respected under XML_PARSE_HUGE. Discussion: https://postgr.es/m/ZbHlgrPLtBZyr_QW@paquier.xyz
2024-01-26 02:15:32 +01:00
res_code = xmlParseBalancedChunkMemory(doc, NULL, NULL, 0,
utf8string + count,
parsed_nodes);
if (res_code != 0 || xmlerrcxt->err_occurred)
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
{
xml_errsave(escontext, xmlerrcxt,
ERRCODE_INVALID_XML_CONTENT,
Allow empty content in xml type The xml type previously rejected "content" that is empty or consists only of spaces. But the SQL/XML standard allows that, so change that. The accepted values for XML "documents" are not changed. Reviewed-by: Ali Akbar <the.apaan@gmail.com>
2014-09-09 17:34:52 +02:00
"invalid XML content");
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
goto fail;
}
Allow empty content in xml type The xml type previously rejected "content" that is empty or consists only of spaces. But the SQL/XML standard allows that, so change that. The accepted values for XML "documents" are not changed. Reviewed-by: Ali Akbar <the.apaan@gmail.com>
2014-09-09 17:34:52 +02:00
}
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
}
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
fail:
;
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
}
PG_CATCH();
{
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (doc != NULL)
xmlFreeDoc(doc);
if (ctxt != NULL)
xmlFreeParserCtxt(ctxt);
pg_xml_done(xmlerrcxt, true);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_RE_THROW();
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
}
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_END_TRY();
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
xmlFreeParserCtxt(ctxt);
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_done(xmlerrcxt, false);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
return doc;
}
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
/*
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
* xmlChar<->text conversions
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
static xmlChar *
xml_text2xmlChar(text *in)
{
Use new cstring/text conversion functions in some additional places. These changes assume that the varchar and xml data types are represented the same as text. (I did not, however, accept the portions of the proposed patch that wanted to assume bytea is the same as text --- tgl.) Brendan Jurd
2008-05-04 18:42:41 +02:00
return (xmlChar *) text_to_cstring(in);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
#ifdef USE_LIBXMLCONTEXT
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
/*
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
* Manage the special context used for all libxml allocations (but only
* in special debug builds; see notes at top of file)
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
*/
static void
xml_memory_init(void)
{
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
/* Create memory context if not there already */
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
if (LibxmlContext == NULL)
LibxmlContext = AllocSetContextCreate(TopMemoryContext,
Add macros to make AllocSetContextCreate() calls simpler and safer. I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <21072.1472321324@sss.pgh.pa.us>
2016-08-27 23:50:38 +02:00
"Libxml context",
ALLOCSET_DEFAULT_SIZES);
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
/* Re-establish the callbacks even if already set */
xmlMemSetup(xml_pfree, xml_palloc, xml_repalloc, xml_pstrdup);
}
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
/*
* Wrappers for memory management functions
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
static void *
xml_palloc(size_t size)
{
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
return MemoryContextAlloc(LibxmlContext, size);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
static void *
xml_repalloc(void *ptr, size_t size)
{
return repalloc(ptr, size);
}
static void
xml_pfree(void *ptr)
{
Ensure xmlFree(NULL) is a no-op instead of a core dump. Per report from Sergey Burladyan, there are at least some dank corners of libxml2 that assume this behavior, even though their published documentation suggests they shouldn't. This is only really a live problem in 8.3, but the code is still there for possible debugging use in HEAD, so patch both branches.
2009-06-10 05:44:35 +02:00
/* At least some parts of libxml assume xmlFree(NULL) is allowed */
if (ptr)
pfree(ptr);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
static char *
xml_pstrdup(const char *string)
{
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
return MemoryContextStrdup(LibxmlContext, string);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
#endif /* USE_LIBXMLCONTEXT */
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
/*
* xmlPgEntityLoader --- entity loader callback function
*
* Silently prevent any external entity URL from being loaded. We don't want
* to throw an error, so instead make the entity appear to expand to an empty
* string.
*
* We would prefer to allow loading entities that exist in the system's
* global XML catalog; but the available libxml2 APIs make that a complex
* and fragile task. For now, just shut down all external access.
*/
static xmlParserInputPtr
xmlPgEntityLoader(const char *URL, const char *ID,
xmlParserCtxtPtr ctxt)
{
return xmlNewStringInputStream(ctxt, (const xmlChar *) "");
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
/*
Export xml.c's libxml-error-handling support so that contrib/xml2 can use it too, instead of duplicating the functionality (badly). I renamed xml_init to pg_xml_init, because the former seemed just a bit too generic to be safe as a global symbol. I considered likewise renaming xml_ereport to pg_xml_ereport, but felt that the reference to ereport probably made it sufficiently PG-centric already.
2010-03-03 18:29:45 +01:00
* xml_ereport --- report an XML-related error
*
* The "msg" is the SQL-level message; some can be adopted from the SQL/XML
* standard. This function adds libxml's native error message, if any, as
* detail.
*
* This is exported for modules that want to share the core libxml error
* handler. Note that pg_xml_init() *must* have been called previously.
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
Export xml.c's libxml-error-handling support so that contrib/xml2 can use it too, instead of duplicating the functionality (badly). I renamed xml_init to pg_xml_init, because the former seemed just a bit too generic to be safe as a global symbol. I considered likewise renaming xml_ereport to pg_xml_ereport, but felt that the reference to ereport probably made it sufficiently PG-centric already.
2010-03-03 18:29:45 +01:00
void
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xml_ereport(PgXmlErrorContext *errcxt, int level, int sqlcode, const char *msg)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
char *detail;
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Defend against someone passing us a bogus context struct */
if (errcxt->magic != ERRCXT_MAGIC)
elog(ERROR, "xml_ereport called with invalid PgXmlErrorContext");
/* Flag that the current libxml error has been reported */
errcxt->err_occurred = false;
/* Include detail only if we have some text from libxml */
if (errcxt->err_buf.len > 0)
detail = errcxt->err_buf.data;
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
else
detail = NULL;
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
ereport(level,
(errcode(sqlcode),
errmsg_internal("%s", msg),
detail ? errdetail_internal("%s", detail) : 0));
}
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
/*
* xml_errsave --- save an XML-related error
*
* If escontext is an ErrorSaveContext, error details are saved into it,
* and control returns normally.
*
* Otherwise, the error is thrown, so that this is equivalent to
* xml_ereport() with level == ERROR.
*
* This should be used only for errors that we're sure we do not need
* a transaction abort to clean up after.
*/
static void
xml_errsave(Node *escontext, PgXmlErrorContext *errcxt,
int sqlcode, const char *msg)
{
char *detail;
/* Defend against someone passing us a bogus context struct */
if (errcxt->magic != ERRCXT_MAGIC)
elog(ERROR, "xml_errsave called with invalid PgXmlErrorContext");
/* Flag that the current libxml error has been reported */
errcxt->err_occurred = false;
/* Include detail only if we have some text from libxml */
if (errcxt->err_buf.len > 0)
detail = errcxt->err_buf.data;
else
detail = NULL;
errsave(escontext,
(errcode(sqlcode),
errmsg_internal("%s", msg),
detail ? errdetail_internal("%s", detail) : 0));
}
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* Error handler for libxml errors and warnings
*/
static void
Fix incompatibilities with libxml2 >= 2.12.0. libxml2 changed the required signature of error handler callbacks to make the passed xmlError struct "const". This is causing build failures on buildfarm member caiman, and no doubt will start showing up in the field quite soon. Add a version check to adjust the declaration of xml_errorHandler() according to LIBXML_VERSION. 2.12.x also produces deprecation warnings for contrib/xml2/xpath.c's assignment to xmlLoadExtDtdDefaultValue. I see no good reason for that to still be there, seeing that we disabled external DTDs (at a lower level) years ago for security reasons. Let's just remove it. Back-patch to all supported branches, since they might all get built with newer libxml2 once it gets a bit more popular. (The back branches produce another deprecation warning about xpath.c's use of xmlSubstituteEntitiesDefault(). We ought to consider whether to back-patch all or part of commit 65c5864d7 to silence that. It's less urgent though, since it won't break the buildfarm.) Discussion: https://postgr.es/m/1389505.1706382262@sss.pgh.pa.us
2024-01-29 18:06:07 +01:00
xml_errorHandler(void *data, PgXmlErrorPtr error)
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
{
PgXmlErrorContext *xmlerrcxt = (PgXmlErrorContext *) data;
xmlParserCtxtPtr ctxt = (xmlParserCtxtPtr) error->ctxt;
xmlParserInputPtr input = (ctxt != NULL) ? ctxt->input : NULL;
xmlNodePtr node = error->node;
const xmlChar *name = (node != NULL &&
node->type == XML_ELEMENT_NODE) ? node->name : NULL;
int domain = error->domain;
int level = error->level;
StringInfo errorBuf;
Check to see whether libxml2 handles error context the way we expect. It turns out to be possible to link against a libxml2.so that does this differently than the version we configured and built against, so we need a runtime check to avoid bizarre behavior. Per report from Bernd Helmle. Patch by Florian Pflug.
2011-07-26 22:29:53 +02:00
/*
* Defend against someone passing us a bogus context struct.
*
* We force a backend exit if this check fails because longjmp'ing out of
* libxml would likely render it unsafe to use further.
*/
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (xmlerrcxt->magic != ERRCXT_MAGIC)
Check to see whether libxml2 handles error context the way we expect. It turns out to be possible to link against a libxml2.so that does this differently than the version we configured and built against, so we need a runtime check to avoid bizarre behavior. Per report from Bernd Helmle. Patch by Florian Pflug.
2011-07-26 22:29:53 +02:00
elog(FATAL, "xml_errorHandler called with invalid PgXmlErrorContext");
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*----------
* Older libxml versions report some errors differently.
* First, some errors were previously reported as coming from the parser
* domain but are now reported as coming from the namespace domain.
* Second, some warnings were upgraded to errors.
* We attempt to compensate for that here.
*----------
*/
switch (error->code)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
case XML_WAR_NS_URI:
level = XML_ERR_ERROR;
domain = XML_FROM_NAMESPACE;
break;
case XML_ERR_NS_DECL_ERROR:
case XML_WAR_NS_URI_RELATIVE:
case XML_WAR_NS_COLUMN:
case XML_NS_ERR_XML_NAMESPACE:
case XML_NS_ERR_UNDEFINED_NAMESPACE:
case XML_NS_ERR_QNAME:
case XML_NS_ERR_ATTRIBUTE_REDEFINED:
case XML_NS_ERR_EMPTY:
domain = XML_FROM_NAMESPACE;
break;
}
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Decide whether to act on the error or not */
switch (domain)
{
case XML_FROM_PARSER:
case XML_FROM_NONE:
case XML_FROM_MEMORY:
case XML_FROM_IO:
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
Prevent access to external files/URLs via XML entity references. xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn't get returned directly to the user, portions of it could be exposed in error messages if the data didn't parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. The ideal solution to this would still allow fetching of references that are listed in the host system's XML catalogs, so that documents can be validated according to installed DTDs. However, doing that with the available libxml2 APIs appears complex and error-prone, so we're not going to risk it in a security patch that necessarily hasn't gotten wide review. So this patch merely shuts off all access, causing any external fetch to silently expand to an empty string. A future patch may improve this. In HEAD and 9.2, also suppress warnings about undefined entities, which would otherwise occur as a result of not loading referenced DTDs. Previous branches don't show such warnings anyway, due to different error handling arrangements. Credit to Noah Misch for first reporting the problem, and for much work towards a solution, though this simplistic approach was not his preference. Also thanks to Daniel Veillard for consultation. Security: CVE-2012-3489
2012-08-15 00:25:29 +02:00
/*
* Suppress warnings about undeclared entities. We need to do
* this to avoid problems due to not loading DTD definitions.
*/
if (error->code == XML_WAR_UNDECLARED_ENTITY)
return;
/* Otherwise, accept error regardless of the parsing purpose */
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
break;
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
default:
/* Ignore error if only doing well-formedness check */
if (xmlerrcxt->strictness == PG_XML_STRICTNESS_WELLFORMED)
return;
break;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Prepare error message in errorBuf */
errorBuf = makeStringInfo();
if (error->line > 0)
appendStringInfo(errorBuf, "line %d: ", error->line);
if (name != NULL)
appendStringInfo(errorBuf, "element %s: ", name);
Defend against null error message reported by libxml2. While this isn't really supposed to happen, it can occur in OOM situations and perhaps others. Instead of crashing, substitute "(no message provided)". I didn't worry about localizing this text, since we aren't localizing anything else here; besides, if we're on the edge of OOM, it's unlikely gettext() would work. Report and fix by Sergio Conde Gómez in bug #15624. Discussion: https://postgr.es/m/15624-4dea54091a2864e6@postgresql.org
2019-02-08 19:30:42 +01:00
if (error->message != NULL)
appendStringInfoString(errorBuf, error->message);
else
appendStringInfoString(errorBuf, "(no message provided)");
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* Append context information to errorBuf.
*
* xmlParserPrintFileContext() uses libxml's "generic" error handler to
* write the context. Since we don't want to duplicate libxml
* functionality here, we set up a generic error handler temporarily.
*
* We use appendStringInfo() directly as libxml's generic error handler.
* This should work because it has essentially the same signature as
* libxml expects, namely (void *ptr, const char *msg, ...).
*/
if (input != NULL)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlGenericErrorFunc errFuncSaved = xmlGenericError;
void *errCtxSaved = xmlGenericErrorContext;
xmlSetGenericErrorFunc((void *) errorBuf,
(xmlGenericErrorFunc) appendStringInfo);
/* Add context information to errorBuf */
appendStringInfoLineSeparator(errorBuf);
xmlParserPrintFileContext(input);
/* Restore generic error func */
xmlSetGenericErrorFunc(errCtxSaved, errFuncSaved);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/* Get rid of any trailing newlines in errorBuf */
chopStringInfoNewlines(errorBuf);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* Legacy error handling mode. err_occurred is never set, we just add the
* message to err_buf. This mode exists because the xml2 contrib module
* uses our error-handling infrastructure, but we don't want to change its
* behaviour since it's deprecated anyway. This is also why we don't
* distinguish between notices, warnings and errors here --- the old-style
* generic error handler wouldn't have done that either.
*/
if (xmlerrcxt->strictness == PG_XML_STRICTNESS_LEGACY)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
appendStringInfoLineSeparator(&xmlerrcxt->err_buf);
Use appendBinaryStringInfo in more places where the length is known When we already know the length that we're going to append, then it makes sense to use appendBinaryStringInfo instead of appendStringInfoString so that the append can be performed with a simple memcpy() using a known length rather than having to first perform a strlen() call to obtain the length. Discussion: https://postgr.es/m/CAKJS1f8+FRAM1s5+mAa3isajeEoAaicJ=4e0WzrH3tAusbbiMQ@mail.gmail.com
2019-07-22 14:14:11 +02:00
appendBinaryStringInfo(&xmlerrcxt->err_buf, errorBuf->data,
errorBuf->len);
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pfree(errorBuf->data);
pfree(errorBuf);
return;
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* We don't want to ereport() here because that'd probably leave libxml in
* an inconsistent state. Instead, we remember the error and ereport()
* from xml_ereport().
*
* Warnings and notices can be reported immediately since they won't cause
* a longjmp() out of libxml.
*/
if (level >= XML_ERR_ERROR)
{
appendStringInfoLineSeparator(&xmlerrcxt->err_buf);
Use appendBinaryStringInfo in more places where the length is known When we already know the length that we're going to append, then it makes sense to use appendBinaryStringInfo instead of appendStringInfoString so that the append can be performed with a simple memcpy() using a known length rather than having to first perform a strlen() call to obtain the length. Discussion: https://postgr.es/m/CAKJS1f8+FRAM1s5+mAa3isajeEoAaicJ=4e0WzrH3tAusbbiMQ@mail.gmail.com
2019-07-22 14:14:11 +02:00
appendBinaryStringInfo(&xmlerrcxt->err_buf, errorBuf->data,
errorBuf->len);
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlerrcxt->err_occurred = true;
}
else if (level >= XML_ERR_WARNING)
{
ereport(WARNING,
(errmsg_internal("%s", errorBuf->data)));
}
else
{
ereport(NOTICE,
(errmsg_internal("%s", errorBuf->data)));
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pfree(errorBuf->data);
pfree(errorBuf);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
/*
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
* Convert libxml error codes into textual errdetail messages.
*
* This should be called within an ereport or errsave invocation,
* just as errdetail would be.
*
* At the moment, we only need to cover those codes that we
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
* may raise in this file.
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
static int
errdetail_for_xml_code(int code)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
const char *det;
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
switch (code)
{
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
case XML_ERR_INVALID_CHAR:
Fix various infelicities that have snuck into usage of errdetail() and friends. Avoid double translation of some messages, ensure other messages are exposed for translation (and make them follow the style guidelines), avoid unsafe passing of an unpredictable message text as a format string.
2008-03-24 20:12:49 +01:00
det = gettext_noop("Invalid character value.");
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
break;
case XML_ERR_SPACE_REQUIRED:
Fix various infelicities that have snuck into usage of errdetail() and friends. Avoid double translation of some messages, ensure other messages are exposed for translation (and make them follow the style guidelines), avoid unsafe passing of an unpredictable message text as a format string.
2008-03-24 20:12:49 +01:00
det = gettext_noop("Space required.");
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
break;
case XML_ERR_STANDALONE_VALUE:
Fix various infelicities that have snuck into usage of errdetail() and friends. Avoid double translation of some messages, ensure other messages are exposed for translation (and make them follow the style guidelines), avoid unsafe passing of an unpredictable message text as a format string.
2008-03-24 20:12:49 +01:00
det = gettext_noop("standalone accepts only 'yes' or 'no'.");
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
break;
case XML_ERR_VERSION_MISSING:
Fix various infelicities that have snuck into usage of errdetail() and friends. Avoid double translation of some messages, ensure other messages are exposed for translation (and make them follow the style guidelines), avoid unsafe passing of an unpredictable message text as a format string.
2008-03-24 20:12:49 +01:00
det = gettext_noop("Malformed declaration: missing version.");
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
break;
case XML_ERR_MISSING_ENCODING:
Fix various infelicities that have snuck into usage of errdetail() and friends. Avoid double translation of some messages, ensure other messages are exposed for translation (and make them follow the style guidelines), avoid unsafe passing of an unpredictable message text as a format string.
2008-03-24 20:12:49 +01:00
det = gettext_noop("Missing encoding in text declaration.");
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
break;
case XML_ERR_XMLDECL_NOT_FINISHED:
Fix various infelicities that have snuck into usage of errdetail() and friends. Avoid double translation of some messages, ensure other messages are exposed for translation (and make them follow the style guidelines), avoid unsafe passing of an unpredictable message text as a format string.
2008-03-24 20:12:49 +01:00
det = gettext_noop("Parsing XML declaration: '?>' expected.");
Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per <http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>. For converting error codes to messages, we only need to cover those codes that we raise ourselves now.
2007-01-24 00:39:16 +01:00
break;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
default:
Fix various infelicities that have snuck into usage of errdetail() and friends. Avoid double translation of some messages, ensure other messages are exposed for translation (and make them follow the style guidelines), avoid unsafe passing of an unpredictable message text as a format string.
2008-03-24 20:12:49 +01:00
det = gettext_noop("Unrecognized libxml error code: %d.");
Bring some order and sanity to error handling in the xml patch. Use a TRY block instead of (inadequate) ad-hoc coding to ensure that libxml is cleaned up after a failure. Report the intended SQLCODE instead of defaulting to XX000. Avoid risking use of a dangling pointer by keeping the persistent error buffer in TopMemoryContext. Be less trusting that error messages don't contain %. This patch doesn't do anything about changing the way the messages are put together --- this is just about mechanism.
2006-12-24 19:25:58 +01:00
break;
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Convert xml_in to report errors softly. The key idea here is that xml_parse must distinguish hard errors from soft errors. We want to throw a hard error for libxml initialization failures: those might be out-of-memory, or something else, but in any case they are not the fault of the input string. If we get to the point of parsing the input, and something goes wrong, we can fairly consider that to mean bad input. One thing that arguably does mean bad input, but I didn't trouble to handle softly, is encoding conversion failure while converting the server encoding to UTF8. This might be something to improve later, but it seems like a pretty low-probability scenario. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 16:58:49 +01:00
return errdetail(det, code);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
/*
* Remove all trailing newlines from a StringInfo string
*/
static void
chopStringInfoNewlines(StringInfo str)
{
while (str->len > 0 && str->data[str->len - 1] == '\n')
str->data[--str->len] = '\0';
}
/*
* Append a newline after removing any existing trailing newlines
*/
static void
appendStringInfoLineSeparator(StringInfo str)
{
chopStringInfoNewlines(str);
if (str->len > 0)
appendStringInfoChar(str, '\n');
}
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
/*
Fix multiple breakages in last XML patch.
2006-12-29 17:44:28 +01:00
* Convert one char in the current server encoding to a Unicode codepoint.
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
static pg_wchar
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
sqlchar_to_unicode(const char *s)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
char *utf8string;
Fix machine-dependent crash in sqlchar_to_unicode(). Get rid of bletcherous and unsafe manipulation of global encoding setting. Clean up libxml reporting mechanism a bit (it still looks like a dangling-pointer crash waiting to happen, though, not to mention being far less than sane from a localization standpoint).
2006-12-24 01:57:48 +01:00
pg_wchar ret[2]; /* need space for trailing zero */
Suppress various compiler warnings in new xml code.
2006-12-23 05:56:50 +01:00
Prefer pg_any_to_server/pg_server_to_any over pg_do_encoding_conversion. A large majority of the callers of pg_do_encoding_conversion were specifying the database encoding as either source or target of the conversion, meaning that we can use the less general functions pg_any_to_server/pg_server_to_any instead. The main advantage of using the latter functions is that they can make use of a cached conversion-function lookup in the common case that the other encoding is the current client_encoding. It's notationally cleaner too in most cases, not least because of the historical artifact that the latter functions use "char *" rather than "unsigned char *" in their APIs. Note that pg_any_to_server will apply an encoding verification step in some cases where pg_do_encoding_conversion would have just done nothing. This seems to me to be a good idea at most of these call sites, though it partially negates the performance benefit. Per discussion of bug #9210.
2014-02-23 22:59:05 +01:00
/* note we're not assuming s is null-terminated */
utf8string = pg_server_to_any(s, pg_mblen(s), PG_UTF8);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Fix bugs in sqlchar_to_unicode and unicode_to_sqlchar: both were measuring the length of a UTF8 character with pg_mblen (wrong if DB encoding isn't UTF8), and the latter was blithely assuming that a static buffer would somehow revert to all zeroes for each use.
2008-11-10 19:02:20 +01:00
pg_encoding_mb2wchar_with_len(PG_UTF8, utf8string, ret,
pg_encoding_mblen(PG_UTF8, utf8string));
if (utf8string != s)
pfree(utf8string);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
Fix machine-dependent crash in sqlchar_to_unicode(). Get rid of bletcherous and unsafe manipulation of global encoding setting. Clean up libxml reporting mechanism a bit (it still looks like a dangling-pointer crash waiting to happen, though, not to mention being far less than sane from a localization standpoint).
2006-12-24 01:57:48 +01:00
return ret[0];
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
static bool
is_valid_xml_namefirst(pg_wchar c)
{
/* (Letter | '_' | ':') */
return (xmlIsBaseCharQ(c) || xmlIsIdeographicQ(c)
|| c == '_' || c == ':');
}
static bool
is_valid_xml_namechar(pg_wchar c)
{
/* Letter | Digit | '.' | '-' | '_' | ':' | CombiningChar | Extender */
return (xmlIsBaseCharQ(c) || xmlIsIdeographicQ(c)
|| xmlIsDigitQ(c)
|| c == '.' || c == '-' || c == '_' || c == ':'
|| xmlIsCombiningQ(c)
|| xmlIsExtenderQ(c));
}
#endif /* USE_LIBXML */
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map SQL identifier to XML name; see SQL/XML:2008 section 9.1.
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
*/
char *
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
map_sql_identifier_to_xml_name(const char *ident, bool fully_escaped,
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
bool escape_period)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
Un-break build on ANSI compilers (like msvc) by moving Assert to position after variable declarations.
2007-02-13 16:56:12 +01:00
#ifdef USE_LIBXML
StringInfoData buf;
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
const char *p;
Un-break build on ANSI compilers (like msvc) by moving Assert to position after variable declarations.
2007-02-13 16:56:12 +01:00
Add support for optionally escaping periods when converting SQL identifiers to XML names, which will be required for supporting XML export.
2007-02-11 23:18:16 +01:00
/*
* SQL/XML doesn't make use of this case anywhere, so it's probably a
* mistake.
*/
Assert(fully_escaped || !escape_period);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
initStringInfo(&buf);
for (p = ident; *p; p += pg_mblen(p))
{
if (*p == ':' && (p == ident || fully_escaped))
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&buf, "_x003A_");
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
else if (*p == '_' && *(p + 1) == 'x')
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&buf, "_x005F_");
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
else if (fully_escaped && p == ident &&
pg_strncasecmp(p, "xml", 3) == 0)
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
{
if (*p == 'x')
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&buf, "_x0078_");
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
else
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&buf, "_x0058_");
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
}
Add support for optionally escaping periods when converting SQL identifiers to XML names, which will be required for supporting XML export.
2007-02-11 23:18:16 +01:00
else if (escape_period && *p == '.')
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&buf, "_x002E_");
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
else
{
pg_wchar u = sqlchar_to_unicode(p);
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
if ((p == ident)
? !is_valid_xml_namefirst(u)
: !is_valid_xml_namechar(u))
appendStringInfo(&buf, "_x%04X_", (unsigned int) u);
Initial SQL/XML support: xml data type and initial set of functions.
2006-12-21 17:05:16 +01:00
else
appendBinaryStringInfo(&buf, p, pg_mblen(p));
}
}
return buf.data;
#else /* not USE_LIBXML */
NO_XML_SUPPORT();
return NULL;
#endif /* not USE_LIBXML */
}
De-escape XML names when reverse-compiling XML expressions.
2006-12-29 11:50:22 +01:00
Fix multiple breakages in last XML patch.
2006-12-29 17:44:28 +01:00
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map XML name to SQL identifier; see SQL/XML:2008 section 9.3.
De-escape XML names when reverse-compiling XML expressions.
2006-12-29 11:50:22 +01:00
*/
char *
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
map_xml_name_to_sql_identifier(const char *name)
De-escape XML names when reverse-compiling XML expressions.
2006-12-29 11:50:22 +01:00
{
StringInfoData buf;
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
const char *p;
De-escape XML names when reverse-compiling XML expressions.
2006-12-29 11:50:22 +01:00
initStringInfo(&buf);
for (p = name; *p; p += pg_mblen(p))
{
if (*p == '_' && *(p + 1) == 'x'
&& isxdigit((unsigned char) *(p + 2))
&& isxdigit((unsigned char) *(p + 3))
&& isxdigit((unsigned char) *(p + 4))
&& isxdigit((unsigned char) *(p + 5))
&& *(p + 6) == '_')
{
Allow Unicode escapes in any server encoding, not only UTF-8. SQL includes provisions for numeric Unicode escapes in string literals and identifiers. Previously we only accepted those if they represented ASCII characters or the server encoding was UTF-8, making the conversion to internal form trivial. This patch adjusts things so that we'll call the appropriate encoding conversion function in less-trivial cases, allowing the escape sequence to be accepted so long as it corresponds to some character available in the server encoding. This also applies to processing of Unicode escapes in JSONB. However, the old restriction still applies to client-side JSON processing, since that hasn't got access to the server's encoding conversion infrastructure. This patch includes some lexer infrastructure that simplifies throwing errors with error cursors pointing into the middle of a string (or other complex token). For the moment I only used it for errors relating to Unicode escapes, but we might later expand the usage to some other cases. Patch by me, reviewed by John Naylor. Discussion: https://postgr.es/m/2393.1578958316@sss.pgh.pa.us
2020-03-06 20:17:43 +01:00
char cbuf[MAX_UNICODE_EQUIVALENT_STRING + 1];
De-escape XML names when reverse-compiling XML expressions.
2006-12-29 11:50:22 +01:00
unsigned int u;
sscanf(p + 2, "%X", &u);
Allow Unicode escapes in any server encoding, not only UTF-8. SQL includes provisions for numeric Unicode escapes in string literals and identifiers. Previously we only accepted those if they represented ASCII characters or the server encoding was UTF-8, making the conversion to internal form trivial. This patch adjusts things so that we'll call the appropriate encoding conversion function in less-trivial cases, allowing the escape sequence to be accepted so long as it corresponds to some character available in the server encoding. This also applies to processing of Unicode escapes in JSONB. However, the old restriction still applies to client-side JSON processing, since that hasn't got access to the server's encoding conversion infrastructure. This patch includes some lexer infrastructure that simplifies throwing errors with error cursors pointing into the middle of a string (or other complex token). For the moment I only used it for errors relating to Unicode escapes, but we might later expand the usage to some other cases. Patch by me, reviewed by John Naylor. Discussion: https://postgr.es/m/2393.1578958316@sss.pgh.pa.us
2020-03-06 20:17:43 +01:00
pg_unicode_to_server(u, (unsigned char *) cbuf);
appendStringInfoString(&buf, cbuf);
De-escape XML names when reverse-compiling XML expressions.
2006-12-29 11:50:22 +01:00
p += 6;
}
else
appendBinaryStringInfo(&buf, p, pg_mblen(p));
}
return buf.data;
}
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map SQL value to XML value; see SQL/XML:2008 section 9.8.
Fix xmlattribute escaping XML special characters twice (bug #4822). Author: Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>
2009-06-10 00:00:57 +02:00
*
* When xml_escape_strings is true, then certain characters in string
* values are replaced by entity references (&lt; etc.), as specified
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* in SQL/XML:2008 section 9.8 GR 9) a) iii). This is normally what is
Fix xmlattribute escaping XML special characters twice (bug #4822). Author: Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>
2009-06-10 00:00:57 +02:00
* wanted. The false case is mainly useful when the resulting value
* is used with xmlTextWriterWriteAttribute() to write out an
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* attribute, because that function does the escaping itself.
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
*/
Use XML output escaping also in XMLFOREST.
2007-01-12 22:47:27 +01:00
char *
Fix xmlattribute escaping XML special characters twice (bug #4822). Author: Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>
2009-06-10 00:00:57 +02:00
map_sql_value_to_xml_value(Datum value, Oid type, bool xml_escape_strings)
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
{
Improve handling of domains over arrays. This patch eliminates various bizarre behaviors caused by sloppy thinking about the difference between a domain type and its underlying array type. In particular, the operation of updating one element of such an array has to be considered as yielding a value of the underlying array type, *not* a value of the domain, because there's no assurance that the domain's CHECK constraints are still satisfied. If we're intending to store the result back into a domain column, we have to re-cast to the domain type so that constraints are re-checked. For similar reasons, such a domain can't be blindly matched to an ANYARRAY polymorphic parameter, because the polymorphic function is likely to apply array-ish operations that could invalidate the domain constraints. For the moment, we just forbid such matching. We might later wish to insert an automatic downcast to the underlying array type, but such a change should also change matching of domains to ANYELEMENT for consistency. To ensure that all such logic is rechecked, this patch removes the original hack of setting a domain's pg_type.typelem field to match its base type; the typelem will always be zero instead. In those places where it's really okay to look through the domain type with no other logic changes, use the newly added get_base_element_type function in place of get_element_type. catversion bumped due to change in pg_type contents. Per bug #5717 from Richard Huxton and subsequent discussion.
2010-10-21 22:07:17 +02:00
if (type_is_array_domain(type))
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
{
ArrayType *array;
Oid elmtype;
int16 elmlen;
bool elmbyval;
char elmalign;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
int num_elems;
Datum *elem_values;
bool *elem_nulls;
Extend EXPLAIN to support output in XML or JSON format. There are probably still some adjustments to be made in the details of the output, but this gets the basic structure in place. Robert Haas
2009-08-10 07:46:50 +02:00
StringInfoData buf;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
int i;
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
array = DatumGetArrayTypeP(value);
elmtype = ARR_ELEMTYPE(array);
get_typlenbyvalalign(elmtype, &elmlen, &elmbyval, &elmalign);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
deconstruct_array(array, elmtype,
elmlen, elmbyval, elmalign,
&elem_values, &elem_nulls,
&num_elems);
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
initStringInfo(&buf);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
for (i = 0; i < num_elems; i++)
{
if (elem_nulls[i])
continue;
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
appendStringInfoString(&buf, "<element>");
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
appendStringInfoString(&buf,
map_sql_value_to_xml_value(elem_values[i],
Fix xmlattribute escaping XML special characters twice (bug #4822). Author: Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>
2009-06-10 00:00:57 +02:00
elmtype, true));
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
appendStringInfoString(&buf, "</element>");
}
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
pfree(elem_values);
pfree(elem_nulls);
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
return buf.data;
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
}
else
{
Oid typeOut;
bool isvarlena;
Extend EXPLAIN to support output in XML or JSON format. There are probably still some adjustments to be made in the details of the output, but this gets the basic structure in place. Robert Haas
2009-08-10 07:46:50 +02:00
char *str;
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
Fix map_sql_value_to_xml_value() to treat domains like their base types. This was already the case for domains over arrays, but not for domains over certain built-in types such as boolean. The special formatting rules for those types should apply to domains over them as well. Per discussion. While this is a bug fix, it's also a behavioral change that seems likely to trip up some applications. So no back-patch. Pavel Stehule
2013-03-04 01:32:12 +01:00
/*
* Flatten domains; the special-case treatments below should apply to,
* eg, domains over boolean not just boolean.
*/
type = getBaseType(type);
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
/*
* Special XSD formatting for some data types
*/
switch (type)
Add proper mapping of boolean type data to XML Schema.
2007-02-10 19:47:41 +01:00
{
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
case BOOLOID:
if (DatumGetBool(value))
return "true";
else
return "false";
case DATEOID:
{
DateADT date;
struct pg_tm tm;
char buf[MAXDATELEN + 1];
date = DatumGetDateADT(value);
Extend the date type to support infinity and -infinity, analogously to the timestamp types. Turns out this doesn't even reduce the available range of dates, since the restriction to dates that work for Julian-date arithmetic is much tighter than the int32 range anyway. Per a longstanding TODO item.
2008-10-14 19:12:33 +02:00
/* XSD doesn't support infinite values */
if (DATE_NOT_FINITE(date))
ereport(ERROR,
(errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
Add an errdetail explaining why we reject infinite dates and timestamps while converting to XML. Bernd Helmle
2009-03-27 19:56:57 +01:00
errmsg("date out of range"),
errdetail("XML does not support infinite date values.")));
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
j2date(date + POSTGRES_EPOCH_JDATE,
&(tm.tm_year), &(tm.tm_mon), &(tm.tm_mday));
EncodeDateOnly(&tm, USE_XSD_DATES, buf);
return pstrdup(buf);
}
case TIMESTAMPOID:
{
Timestamp timestamp;
struct pg_tm tm;
fsec_t fsec;
char buf[MAXDATELEN + 1];
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
timestamp = DatumGetTimestamp(value);
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
/* XSD doesn't support infinite values */
if (TIMESTAMP_NOT_FINITE(timestamp))
ereport(ERROR,
(errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
Add an errdetail explaining why we reject infinite dates and timestamps while converting to XML. Bernd Helmle
2009-03-27 19:56:57 +01:00
errmsg("timestamp out of range"),
errdetail("XML does not support infinite timestamp values.")));
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
else if (timestamp2tm(timestamp, NULL, &tm, &fsec, NULL, NULL) == 0)
Improve EncodeDateTime and EncodeTimeOnly APIs Use an explicit argument to tell whether to include the time zone in the output, rather than using some undocumented pointer magic.
2012-03-14 22:03:34 +01:00
EncodeDateTime(&tm, fsec, false, 0, NULL, USE_XSD_DATES, buf);
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
else
ereport(ERROR,
(errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
errmsg("timestamp out of range")));
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
return pstrdup(buf);
}
case TIMESTAMPTZOID:
{
TimestampTz timestamp;
struct pg_tm tm;
int tz;
fsec_t fsec;
Add const qualifier to tzn returned by timestamp2tm() The tzn value might come from tm->tm_zone, which libc declares as const, so it's prudent that the upper layers know about this as well.
2012-03-15 20:17:19 +01:00
const char *tzn = NULL;
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
char buf[MAXDATELEN + 1];
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
timestamp = DatumGetTimestamp(value);
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
/* XSD doesn't support infinite values */
if (TIMESTAMP_NOT_FINITE(timestamp))
ereport(ERROR,
(errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
Add an errdetail explaining why we reject infinite dates and timestamps while converting to XML. Bernd Helmle
2009-03-27 19:56:57 +01:00
errmsg("timestamp out of range"),
errdetail("XML does not support infinite timestamp values.")));
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
else if (timestamp2tm(timestamp, &tz, &tm, &fsec, &tzn, NULL) == 0)
Improve EncodeDateTime and EncodeTimeOnly APIs Use an explicit argument to tell whether to include the time zone in the output, rather than using some undocumented pointer magic.
2012-03-14 22:03:34 +01:00
EncodeDateTime(&tm, fsec, true, tz, tzn, USE_XSD_DATES, buf);
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
else
ereport(ERROR,
(errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
errmsg("timestamp out of range")));
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
return pstrdup(buf);
}
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
#ifdef USE_LIBXML
case BYTEAOID:
{
bytea *bstr = DatumGetByteaPP(value);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
PgXmlErrorContext *xmlerrcxt;
volatile xmlBufferPtr buf = NULL;
volatile xmlTextWriterPtr writer = NULL;
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
char *result;
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlerrcxt = pg_xml_init(PG_XML_STRICTNESS_ALL);
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_TRY();
{
buf = xmlBufferCreate();
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (buf == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
"could not allocate xmlBuffer");
writer = xmlNewTextWriterMemory(buf, 0);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (writer == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
"could not allocate xmlTextWriter");
if (xmlbinary == XMLBINARY_BASE64)
xmlTextWriterWriteBase64(writer, VARDATA_ANY(bstr),
0, VARSIZE_ANY_EXHDR(bstr));
else
xmlTextWriterWriteBinHex(writer, VARDATA_ANY(bstr),
0, VARSIZE_ANY_EXHDR(bstr));
/* we MUST do this now to flush data out to the buffer */
xmlFreeTextWriter(writer);
writer = NULL;
result = pstrdup((const char *) xmlBufferContent(buf));
}
PG_CATCH();
{
if (writer)
xmlFreeTextWriter(writer);
if (buf)
xmlBufferFree(buf);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_done(xmlerrcxt, true);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_RE_THROW();
}
PG_END_TRY();
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
xmlBufferFree(buf);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_done(xmlerrcxt, false);
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
return result;
}
#endif /* USE_LIBXML */
Add proper mapping of boolean type data to XML Schema.
2007-02-10 19:47:41 +01:00
}
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
/*
* otherwise, just use the type's native text representation
*/
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
getTypeOutputInfo(type, &typeOut, &isvarlena);
str = OidOutputFunctionCall(typeOut, value);
Fix xmlattribute escaping XML special characters twice (bug #4822). Author: Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>
2009-06-10 00:00:57 +02:00
/* ... exactly as-is for XML, and when escaping is not wanted */
if (type == XMLOID || !xml_escape_strings)
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
return str;
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
/* otherwise, translate special characters as needed */
Extend EXPLAIN to support output in XML or JSON format. There are probably still some adjustments to be made in the details of the output, but this gets the basic structure in place. Robert Haas
2009-08-10 07:46:50 +02:00
return escape_xml(str);
}
}
Add support for converting binary values (i.e. bytea) into xml values, with new GUC parameter "xmlbinary" that controls the output encoding, as per SQL/XML standard.
2007-01-19 17:58:46 +01:00
Extend EXPLAIN to support output in XML or JSON format. There are probably still some adjustments to be made in the details of the output, but this gets the basic structure in place. Robert Haas
2009-08-10 07:46:50 +02:00
/*
* Escape characters in text that have special meanings in XML.
*
* Returns a palloc'd string.
*
* NB: this is intentionally not dependent on libxml.
*/
char *
escape_xml(const char *str)
{
StringInfoData buf;
const char *p;
initStringInfo(&buf);
for (p = str; *p; p++)
{
switch (*p)
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
{
Extend EXPLAIN to support output in XML or JSON format. There are probably still some adjustments to be made in the details of the output, but this gets the basic structure in place. Robert Haas
2009-08-10 07:46:50 +02:00
case '&':
appendStringInfoString(&buf, "&amp;");
break;
case '<':
appendStringInfoString(&buf, "&lt;");
break;
case '>':
appendStringInfoString(&buf, "&gt;");
break;
case '\r':
appendStringInfoString(&buf, "&#x0d;");
break;
default:
appendStringInfoCharMacro(&buf, *p);
break;
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
}
Fix crash in bytea-to-XML mapping when the source value is toasted. Report and fix by Michael McMaster. Some minor code beautification by me, also avoid memory leaks in the special-case paths.
2008-10-09 17:49:04 +02:00
}
Extend EXPLAIN to support output in XML or JSON format. There are probably still some adjustments to be made in the details of the output, but this gets the basic structure in place. Robert Haas
2009-08-10 07:46:50 +02:00
return buf.data;
Allow for arbitrary data types as content in XMLELEMENT. The original coercion to type xml was a mistake. Escape values so they are valid XML character data.
2007-01-12 17:29:24 +01:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
static char *
_SPI_strdup(const char *s)
{
Minor perf tweak for _SPI_strdup(): if we're going to call strlen() anyway, it is faster to memcpy() than to strcpy().
2008-01-12 11:38:32 +01:00
size_t len = strlen(s) + 1;
char *ret = SPI_palloc(len);
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Minor perf tweak for _SPI_strdup(): if we're going to call strlen() anyway, it is faster to memcpy() than to strcpy().
2008-01-12 11:38:32 +01:00
memcpy(ret, s, len);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
return ret;
}
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
/*
* SQL to XML mapping functions
*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* What follows below was at one point intentionally organized so that
* you can read along in the SQL/XML standard. The functions are
* mostly split up the way the clauses lay out in the standards
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
* document, and the identifiers are also aligned with the standard
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* text. Unfortunately, SQL/XML:2006 reordered the clauses
* differently than SQL/XML:2003, so the order below doesn't make much
* sense anymore.
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
*
* There are many things going on there:
*
* There are two kinds of mappings: Mapping SQL data (table contents)
* to XML documents, and mapping SQL structure (the "schema") to XML
* Schema. And there are functions that do both at the same time.
*
* Then you can map a database, a schema, or a table, each in both
* ways. This breaks down recursively: Mapping a database invokes
* mapping schemas, which invokes mapping tables, which invokes
* mapping rows, which invokes mapping columns, although you can't
* call the last two from the outside. Because of this, there are a
* number of xyz_internal() functions which are to be called both from
* the function manager wrapper and from some upper layer in a
* recursive call.
*
* See the documentation about what the common function arguments
* nulls, tableforest, and targetns mean.
*
* Some style guidelines for XML output: Use double quotes for quoting
* XML attributes. Indent XML elements by two spaces, but remember
* that a lot of code is called recursively at different levels, so
* it's better not to indent rather than create output that indents
* and outdents weirdly. Add newlines to make the output look nice.
*/
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Visibility of objects for XML mappings; see SQL/XML:2008 section
* 4.10.8.
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
*/
/*
* Given a query, which must return type oid as first column, produce
* a list of Oids with the query results.
*/
static List *
query_to_oid_list(const char *query)
{
Widen query numbers-of-tuples-processed counters to uint64. This patch widens SPI_processed, EState's es_processed field, PortalData's portalPos field, FuncCallContext's call_cntr and max_calls fields, ExecutorRun's count argument, PortalRunFetch's result, and the max number of rows in a SPITupleTable to uint64, and deals with (I hope) all the ensuing fallout. Some of these values were declared uint32 before, and others "long". I also removed PortalData's posOverflow field, since that logic seems pretty useless given that portalPos is now always 64 bits. The user-visible results are that command tags for SELECT etc will correctly report tuple counts larger than 4G, as will plpgsql's GET GET DIAGNOSTICS ... ROW_COUNT command. Queries processing more tuples than that are still not exactly the norm, but they're becoming more common. Most values associated with FETCH/MOVE distances, such as PortalRun's count argument and the count argument of most SPI functions that have one, remain declared as "long". It's not clear whether it would be worth promoting those to int64; but it would definitely be a large dollop of additional API churn on top of this, and it would only help 32-bit platforms which seem relatively less likely to see any benefit. Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me
2016-03-12 22:05:10 +01:00
uint64 i;
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
List *list = NIL;
Check after errors of SPI_execute() in xml.c SPI gets used to build a list of relation OIDs for XML object generation, and one code path building a list uses SPI_execute() without looking at errors it produces. So fix that. Author: Mark Dilger Reviewed-by: Michael Paquier, Pavel Stehule Discussion: https://postgr.es/m/17d30445-4862-7917-170f-84328dcd292d@gmail.com
2019-11-07 03:13:31 +01:00
int spi_result;
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Check after errors of SPI_execute() in xml.c SPI gets used to build a list of relation OIDs for XML object generation, and one code path building a list uses SPI_execute() without looking at errors it produces. So fix that. Author: Mark Dilger Reviewed-by: Michael Paquier, Pavel Stehule Discussion: https://postgr.es/m/17d30445-4862-7917-170f-84328dcd292d@gmail.com
2019-11-07 03:13:31 +01:00
spi_result = SPI_execute(query, true, 0);
if (spi_result != SPI_OK_SELECT)
elog(ERROR, "SPI_execute returned %s for %s",
SPI_result_code_string(spi_result), query);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
for (i = 0; i < SPI_processed; i++)
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
Datum oid;
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
bool isnull;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
oid = SPI_getbinval(SPI_tuptable->vals[i],
SPI_tuptable->tupdesc,
1,
&isnull);
if (!isnull)
list = lappend_oid(list, DatumGetObjectId(oid));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
return list;
}
static List *
schema_get_xml_visible_tables(Oid nspid)
{
StringInfoData query;
initStringInfo(&query);
Fix hard-coded relkind constants in assorted other files. Although it's reasonable to expect that most of these constants will never change, that does not make it good programming style to hard-code the value rather than using the RELKIND_FOO macros. I think I've now gotten all the hard-coded references in C code. Unfortunately there's no equally convenient way to parameterize SQL files ... Discussion: https://postgr.es/m/11145.1488931324@sss.pgh.pa.us
2017-03-10 05:36:44 +01:00
appendStringInfo(&query, "SELECT oid FROM pg_catalog.pg_class"
" WHERE relnamespace = %u AND relkind IN ("
CppAsString2(RELKIND_RELATION) ","
CppAsString2(RELKIND_MATVIEW) ","
CppAsString2(RELKIND_VIEW) ")"
" AND pg_catalog.has_table_privilege (oid, 'SELECT')"
" ORDER BY relname;", nspid);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
return query_to_oid_list(query.data);
}
/*
* Including the system schemas is probably not useful for a database
* mapping.
*/
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
#define XML_VISIBLE_SCHEMAS_EXCLUDE "(nspname ~ '^pg_' OR nspname = 'information_schema')"
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
#define XML_VISIBLE_SCHEMAS "SELECT oid FROM pg_catalog.pg_namespace WHERE pg_catalog.has_schema_privilege (oid, 'USAGE') AND NOT " XML_VISIBLE_SCHEMAS_EXCLUDE
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
static List *
database_get_xml_visible_schemas(void)
{
return query_to_oid_list(XML_VISIBLE_SCHEMAS " ORDER BY nspname;");
}
static List *
database_get_xml_visible_tables(void)
{
/* At the moment there is no order required here. */
Fix hard-coded relkind constants in assorted other files. Although it's reasonable to expect that most of these constants will never change, that does not make it good programming style to hard-code the value rather than using the RELKIND_FOO macros. I think I've now gotten all the hard-coded references in C code. Unfortunately there's no equally convenient way to parameterize SQL files ... Discussion: https://postgr.es/m/11145.1488931324@sss.pgh.pa.us
2017-03-10 05:36:44 +01:00
return query_to_oid_list("SELECT oid FROM pg_catalog.pg_class"
" WHERE relkind IN ("
CppAsString2(RELKIND_RELATION) ","
CppAsString2(RELKIND_MATVIEW) ","
CppAsString2(RELKIND_VIEW) ")"
" AND pg_catalog.has_table_privilege(pg_class.oid, 'SELECT')"
" AND relnamespace IN (" XML_VISIBLE_SCHEMAS ");");
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map SQL table to XML and/or XML Schema document; see SQL/XML:2008
* section 9.11.
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*/
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
static StringInfo
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
table_to_xml_internal(Oid relid,
const char *xmlschema, bool nulls, bool tableforest,
const char *targetns, bool top_level)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
StringInfoData query;
initStringInfo(&query);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
appendStringInfo(&query, "SELECT * FROM %s",
DatumGetCString(DirectFunctionCall1(regclassout,
ObjectIdGetDatum(relid))));
return query_to_xml_internal(query.data, get_rel_name(relid),
xmlschema, nulls, tableforest,
targetns, top_level);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Datum
table_to_xml(PG_FUNCTION_ARGS)
{
Oid relid = PG_GETARG_OID(0);
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(table_to_xml_internal(relid, NULL,
nulls, tableforest,
targetns, true)));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
Datum
query_to_xml(PG_FUNCTION_ARGS)
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
char *query = text_to_cstring(PG_GETARG_TEXT_PP(0));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(query_to_xml_internal(query, NULL,
NULL, nulls, tableforest,
targetns, true)));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
Datum
cursor_to_xml(PG_FUNCTION_ARGS)
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
char *name = text_to_cstring(PG_GETARG_TEXT_PP(0));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
int32 count = PG_GETARG_INT32(1);
bool nulls = PG_GETARG_BOOL(2);
bool tableforest = PG_GETARG_BOOL(3);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(4));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
StringInfoData result;
Portal portal;
Widen query numbers-of-tuples-processed counters to uint64. This patch widens SPI_processed, EState's es_processed field, PortalData's portalPos field, FuncCallContext's call_cntr and max_calls fields, ExecutorRun's count argument, PortalRunFetch's result, and the max number of rows in a SPITupleTable to uint64, and deals with (I hope) all the ensuing fallout. Some of these values were declared uint32 before, and others "long". I also removed PortalData's posOverflow field, since that logic seems pretty useless given that portalPos is now always 64 bits. The user-visible results are that command tags for SELECT etc will correctly report tuple counts larger than 4G, as will plpgsql's GET GET DIAGNOSTICS ... ROW_COUNT command. Queries processing more tuples than that are still not exactly the norm, but they're becoming more common. Most values associated with FETCH/MOVE distances, such as PortalRun's count argument and the count argument of most SPI functions that have one, remain declared as "long". It's not clear whether it would be worth promoting those to int64; but it would definitely be a large dollop of additional API churn on top of this, and it would only help 32-bit platforms which seem relatively less likely to see any benefit. Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me
2016-03-12 22:05:10 +01:00
uint64 i;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
initStringInfo(&result);
Fix cursor_to_xml in tableforest false mode It only produced <row> elements but no wrapping <table> element. By contrast, cursor_to_xmlschema produced a schema that is now correct but did not previously match the XML data produced by cursor_to_xml. In passing, also fix a minor misunderstanding about moving cursors in the tests related to this. Reported-by: filip@jirsak.org Based-on-patch-by: Thomas Munro <thomas.munro@enterprisedb.com>
2017-05-04 03:25:01 +02:00
if (!tableforest)
{
xmldata_root_element_start(&result, "table", NULL, targetns, true);
appendStringInfoChar(&result, '\n');
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
SPI_connect();
portal = SPI_cursor_find(name);
if (portal == NULL)
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_CURSOR),
errmsg("cursor \"%s\" does not exist", name)));
SPI_cursor_fetch(portal, true, count);
for (i = 0; i < SPI_processed; i++)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
SPI_sql_row_to_xmlelement(i, &result, NULL, nulls,
tableforest, targetns, true);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
SPI_finish();
Fix cursor_to_xml in tableforest false mode It only produced <row> elements but no wrapping <table> element. By contrast, cursor_to_xmlschema produced a schema that is now correct but did not previously match the XML data produced by cursor_to_xml. In passing, also fix a minor misunderstanding about moving cursors in the tests related to this. Reported-by: filip@jirsak.org Based-on-patch-by: Thomas Munro <thomas.munro@enterprisedb.com>
2017-05-04 03:25:01 +02:00
if (!tableforest)
xmldata_root_element_end(&result, "table");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(&result));
}
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
/*
* Write the start tag of the root element of a data mapping.
*
* top_level means that this is the very top level of the eventual
* output. For example, when the user calls table_to_xml, then a call
* with a table name to this function is the top level. When the user
* calls database_to_xml, then a call with a schema name to this
* function is not the top level. If top_level is false, then the XML
* namespace declarations are omitted, because they supposedly already
* appeared earlier in the output. Repeating them is not wrong, but
* it looks ugly.
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
*/
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
static void
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmldata_root_element_start(StringInfo result, const char *eltname,
const char *xmlschema, const char *targetns,
bool top_level)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
/* This isn't really wrong but currently makes no sense. */
Assert(top_level || !xmlschema);
appendStringInfo(result, "<%s", eltname);
if (top_level)
{
appendStringInfoString(result, " xmlns:xsi=\"" NAMESPACE_XSI "\"");
if (strlen(targetns) > 0)
appendStringInfo(result, " xmlns=\"%s\"", targetns);
}
if (xmlschema)
{
/* FIXME: better targets */
if (strlen(targetns) > 0)
appendStringInfo(result, " xsi:schemaLocation=\"%s #\"", targetns);
else
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(result, " xsi:noNamespaceSchemaLocation=\"#\"");
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(result, ">\n");
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
static void
xmldata_root_element_end(StringInfo result, const char *eltname)
{
appendStringInfo(result, "</%s>\n", eltname);
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
static StringInfo
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
query_to_xml_internal(const char *query, char *tablename,
const char *xmlschema, bool nulls, bool tableforest,
const char *targetns, bool top_level)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
StringInfo result;
char *xmltn;
Widen query numbers-of-tuples-processed counters to uint64. This patch widens SPI_processed, EState's es_processed field, PortalData's portalPos field, FuncCallContext's call_cntr and max_calls fields, ExecutorRun's count argument, PortalRunFetch's result, and the max number of rows in a SPITupleTable to uint64, and deals with (I hope) all the ensuing fallout. Some of these values were declared uint32 before, and others "long". I also removed PortalData's posOverflow field, since that logic seems pretty useless given that portalPos is now always 64 bits. The user-visible results are that command tags for SELECT etc will correctly report tuple counts larger than 4G, as will plpgsql's GET GET DIAGNOSTICS ... ROW_COUNT command. Queries processing more tuples than that are still not exactly the norm, but they're becoming more common. Most values associated with FETCH/MOVE distances, such as PortalRun's count argument and the count argument of most SPI functions that have one, remain declared as "long". It's not clear whether it would be worth promoting those to int64; but it would definitely be a large dollop of additional API churn on top of this, and it would only help 32-bit platforms which seem relatively less likely to see any benefit. Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me
2016-03-12 22:05:10 +01:00
uint64 i;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (tablename)
xmltn = map_sql_identifier_to_xml_name(tablename, true, false);
else
xmltn = "table";
result = makeStringInfo();
SPI_connect();
if (SPI_execute(query, true, 0) != SPI_OK_SELECT)
ereport(ERROR,
(errcode(ERRCODE_DATA_EXCEPTION),
errmsg("invalid query")));
if (!tableforest)
Avoid extra newlines in XML mapping in table forest mode found by P. Broennimann
2012-07-12 22:47:33 +02:00
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmldata_root_element_start(result, xmltn, xmlschema,
targetns, top_level);
Use appendStringInfoString/Char et al where appropriate. Patch by David Rowley. Backpatch to 9.5, as some of the calls were new in 9.5, and keeping the code in sync with master makes future backpatching easier.
2015-07-02 11:32:48 +02:00
appendStringInfoChar(result, '\n');
Avoid extra newlines in XML mapping in table forest mode found by P. Broennimann
2012-07-12 22:47:33 +02:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (xmlschema)
appendStringInfo(result, "%s\n\n", xmlschema);
for (i = 0; i < SPI_processed; i++)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
SPI_sql_row_to_xmlelement(i, result, tablename, nulls,
tableforest, targetns, top_level);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (!tableforest)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xmldata_root_element_end(result, xmltn);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
SPI_finish();
return result;
}
Datum
table_to_xmlschema(PG_FUNCTION_ARGS)
{
Oid relid = PG_GETARG_OID(0);
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
const char *result;
Relation rel;
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
rel = table_open(relid, AccessShareLock);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
result = map_sql_table_to_xmlschema(rel->rd_att, relid, nulls,
tableforest, targetns);
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
table_close(rel, NoLock);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
PG_RETURN_XML_P(cstring_to_xmltype(result));
}
Datum
query_to_xmlschema(PG_FUNCTION_ARGS)
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
char *query = text_to_cstring(PG_GETARG_TEXT_PP(0));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
const char *result;
Make use of plancache module for SPI plans. In particular, since plpgsql uses SPI plans, this finally fixes the ancient gotcha that you can't drop and recreate a temp table used by a plpgsql function. Along the way, clean up SPI's API a little bit by declaring SPI plan pointers as "SPIPlanPtr" instead of "void *". This is cosmetic but helps to forestall simple programming mistakes. (I have changed some but not all of the callers to match; there are still some "void *"'s in contrib and the PL's. This is intentional so that we can see if anyone's compiler complains about it.)
2007-03-16 00:12:07 +01:00
SPIPlanPtr plan;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Portal portal;
SPI_connect();
Fix two places in xml.c that neglected to check the return values of SPI_prepare() and SPI_cursor_open(), to silence a Coverity warning.
2008-01-12 11:50:03 +01:00
if ((plan = SPI_prepare(query, 0, NULL)) == NULL)
elog(ERROR, "SPI_prepare(\"%s\") failed", query);
if ((portal = SPI_cursor_open(NULL, plan, NULL, NULL, true)) == NULL)
elog(ERROR, "SPI_cursor_open(\"%s\") failed", query);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
result = _SPI_strdup(map_sql_table_to_xmlschema(portal->tupDesc,
InvalidOid, nulls,
tableforest, targetns));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
SPI_cursor_close(portal);
SPI_finish();
PG_RETURN_XML_P(cstring_to_xmltype(result));
}
Datum
cursor_to_xmlschema(PG_FUNCTION_ARGS)
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
char *name = text_to_cstring(PG_GETARG_TEXT_PP(0));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
const char *xmlschema;
Portal portal;
SPI_connect();
portal = SPI_cursor_find(name);
if (portal == NULL)
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_CURSOR),
errmsg("cursor \"%s\" does not exist", name)));
Don't crash if cursor_to_xmlschema is used on a non-data-returning Portal. cursor_to_xmlschema() assumed that any Portal must have a tupDesc, which is not so. Add a defensive check. It's plausible that this mistake occurred because of the rather poorly chosen name of the lookup function SPI_cursor_find(), which in such cases is returning something that isn't very much like a cursor. Add some documentation to try to forestall future errors of the same ilk. Report and patch by Boyu Yang (docs changes by me). Back-patch to all supported branches. Discussion: https://postgr.es/m/dd343010-c637-434c-a8cb-418f53bda3b8.yangboyu.yby@alibaba-inc.com
2023-09-18 20:27:47 +02:00
if (portal->tupDesc == NULL)
ereport(ERROR,
(errcode(ERRCODE_INVALID_CURSOR_STATE),
errmsg("portal \"%s\" does not return tuples", name)));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlschema = _SPI_strdup(map_sql_table_to_xmlschema(portal->tupDesc,
InvalidOid, nulls,
tableforest, targetns));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
SPI_finish();
PG_RETURN_XML_P(cstring_to_xmltype(xmlschema));
}
Datum
table_to_xml_and_xmlschema(PG_FUNCTION_ARGS)
{
Oid relid = PG_GETARG_OID(0);
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Relation rel;
const char *xmlschema;
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
rel = table_open(relid, AccessShareLock);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlschema = map_sql_table_to_xmlschema(rel->rd_att, relid, nulls,
tableforest, targetns);
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
table_close(rel, NoLock);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(table_to_xml_internal(relid,
xmlschema, nulls, tableforest,
targetns, true)));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
Datum
query_to_xml_and_xmlschema(PG_FUNCTION_ARGS)
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
char *query = text_to_cstring(PG_GETARG_TEXT_PP(0));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
const char *xmlschema;
Make use of plancache module for SPI plans. In particular, since plpgsql uses SPI plans, this finally fixes the ancient gotcha that you can't drop and recreate a temp table used by a plpgsql function. Along the way, clean up SPI's API a little bit by declaring SPI plan pointers as "SPIPlanPtr" instead of "void *". This is cosmetic but helps to forestall simple programming mistakes. (I have changed some but not all of the callers to match; there are still some "void *"'s in contrib and the PL's. This is intentional so that we can see if anyone's compiler complains about it.)
2007-03-16 00:12:07 +01:00
SPIPlanPtr plan;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Portal portal;
SPI_connect();
Fix two places in xml.c that neglected to check the return values of SPI_prepare() and SPI_cursor_open(), to silence a Coverity warning.
2008-01-12 11:50:03 +01:00
if ((plan = SPI_prepare(query, 0, NULL)) == NULL)
elog(ERROR, "SPI_prepare(\"%s\") failed", query);
if ((portal = SPI_cursor_open(NULL, plan, NULL, NULL, true)) == NULL)
elog(ERROR, "SPI_cursor_open(\"%s\") failed", query);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlschema = _SPI_strdup(map_sql_table_to_xmlschema(portal->tupDesc,
InvalidOid, nulls, tableforest, targetns));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
SPI_cursor_close(portal);
SPI_finish();
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(query_to_xml_internal(query, NULL,
xmlschema, nulls, tableforest,
targetns, true)));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map SQL schema to XML and/or XML Schema document; see SQL/XML:2008
* sections 9.13, 9.14.
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
*/
static StringInfo
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
schema_to_xml_internal(Oid nspid, const char *xmlschema, bool nulls,
bool tableforest, const char *targetns, bool top_level)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
StringInfo result;
char *xmlsn;
List *relid_list;
ListCell *cell;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlsn = map_sql_identifier_to_xml_name(get_namespace_name(nspid),
true, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
result = makeStringInfo();
xmldata_root_element_start(result, xmlsn, xmlschema, targetns, top_level);
Use appendStringInfoString/Char et al where appropriate. Patch by David Rowley. Backpatch to 9.5, as some of the calls were new in 9.5, and keeping the code in sync with master makes future backpatching easier.
2015-07-02 11:32:48 +02:00
appendStringInfoChar(result, '\n');
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
if (xmlschema)
appendStringInfo(result, "%s\n\n", xmlschema);
SPI_connect();
relid_list = schema_get_xml_visible_tables(nspid);
foreach(cell, relid_list)
{
Oid relid = lfirst_oid(cell);
StringInfo subres;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
subres = table_to_xml_internal(relid, NULL, nulls, tableforest,
targetns, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Use appendBinaryStringInfo in more places where the length is known When we already know the length that we're going to append, then it makes sense to use appendBinaryStringInfo instead of appendStringInfoString so that the append can be performed with a simple memcpy() using a known length rather than having to first perform a strlen() call to obtain the length. Discussion: https://postgr.es/m/CAKJS1f8+FRAM1s5+mAa3isajeEoAaicJ=4e0WzrH3tAusbbiMQ@mail.gmail.com
2019-07-22 14:14:11 +02:00
appendBinaryStringInfo(result, subres->data, subres->len);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
appendStringInfoChar(result, '\n');
}
SPI_finish();
xmldata_root_element_end(result, xmlsn);
return result;
}
Datum
schema_to_xml(PG_FUNCTION_ARGS)
{
Name name = PG_GETARG_NAME(0);
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
char *schemaname;
Oid nspid;
schemaname = NameStr(*name);
Allow CREATE TABLE IF EXIST so succeed if the schema is nonexistent Previously, CREATE TABLE IF EXIST threw an error if the schema was nonexistent. This was done by passing 'missing_ok' to the function that looks up the schema oid.
2013-01-26 19:24:50 +01:00
nspid = LookupExplicitNamespace(schemaname, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(schema_to_xml_internal(nspid, NULL,
nulls, tableforest, targetns, true)));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
/*
* Write the start element of the root element of an XML Schema mapping.
*/
static void
xsd_schema_element_start(StringInfo result, const char *targetns)
{
appendStringInfoString(result,
"<xsd:schema\n"
" xmlns:xsd=\"" NAMESPACE_XSD "\"");
if (strlen(targetns) > 0)
appendStringInfo(result,
"\n"
" targetNamespace=\"%s\"\n"
" elementFormDefault=\"qualified\"",
targetns);
appendStringInfoString(result,
">\n\n");
}
static void
xsd_schema_element_end(StringInfo result)
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
appendStringInfoString(result, "</xsd:schema>");
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
static StringInfo
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
schema_to_xmlschema_internal(const char *schemaname, bool nulls,
bool tableforest, const char *targetns)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
Oid nspid;
List *relid_list;
List *tupdesc_list;
ListCell *cell;
StringInfo result;
result = makeStringInfo();
Allow CREATE TABLE IF EXIST so succeed if the schema is nonexistent Previously, CREATE TABLE IF EXIST threw an error if the schema was nonexistent. This was done by passing 'missing_ok' to the function that looks up the schema oid.
2013-01-26 19:24:50 +01:00
nspid = LookupExplicitNamespace(schemaname, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xsd_schema_element_start(result, targetns);
SPI_connect();
relid_list = schema_get_xml_visible_tables(nspid);
tupdesc_list = NIL;
foreach(cell, relid_list)
{
Relation rel;
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
rel = table_open(lfirst_oid(cell), AccessShareLock);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
tupdesc_list = lappend(tupdesc_list, CreateTupleDescCopy(rel->rd_att));
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
table_close(rel, NoLock);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
appendStringInfoString(result,
map_sql_typecoll_to_xmlschema_types(tupdesc_list));
appendStringInfoString(result,
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
map_sql_schema_to_xmlschema_types(nspid, relid_list,
nulls, tableforest, targetns));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xsd_schema_element_end(result);
SPI_finish();
return result;
}
Datum
schema_to_xmlschema(PG_FUNCTION_ARGS)
{
Name name = PG_GETARG_NAME(0);
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(schema_to_xmlschema_internal(NameStr(*name),
nulls, tableforest, targetns)));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
Datum
schema_to_xml_and_xmlschema(PG_FUNCTION_ARGS)
{
Name name = PG_GETARG_NAME(0);
bool nulls = PG_GETARG_BOOL(1);
bool tableforest = PG_GETARG_BOOL(2);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(3));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
char *schemaname;
Oid nspid;
StringInfo xmlschema;
schemaname = NameStr(*name);
Allow CREATE TABLE IF EXIST so succeed if the schema is nonexistent Previously, CREATE TABLE IF EXIST threw an error if the schema was nonexistent. This was done by passing 'missing_ok' to the function that looks up the schema oid.
2013-01-26 19:24:50 +01:00
nspid = LookupExplicitNamespace(schemaname, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlschema = schema_to_xmlschema_internal(schemaname, nulls,
tableforest, targetns);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(schema_to_xml_internal(nspid,
xmlschema->data, nulls,
tableforest, targetns, true)));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map SQL database to XML and/or XML Schema document; see SQL/XML:2008
* sections 9.16, 9.17.
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
*/
static StringInfo
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
database_to_xml_internal(const char *xmlschema, bool nulls,
bool tableforest, const char *targetns)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
StringInfo result;
List *nspid_list;
ListCell *cell;
char *xmlcn;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlcn = map_sql_identifier_to_xml_name(get_database_name(MyDatabaseId),
true, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
result = makeStringInfo();
xmldata_root_element_start(result, xmlcn, xmlschema, targetns, true);
Use appendStringInfoString/Char et al where appropriate. Patch by David Rowley. Backpatch to 9.5, as some of the calls were new in 9.5, and keeping the code in sync with master makes future backpatching easier.
2015-07-02 11:32:48 +02:00
appendStringInfoChar(result, '\n');
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
if (xmlschema)
appendStringInfo(result, "%s\n\n", xmlschema);
SPI_connect();
nspid_list = database_get_xml_visible_schemas();
foreach(cell, nspid_list)
{
Oid nspid = lfirst_oid(cell);
StringInfo subres;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
subres = schema_to_xml_internal(nspid, NULL, nulls,
tableforest, targetns, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Use appendBinaryStringInfo in more places where the length is known When we already know the length that we're going to append, then it makes sense to use appendBinaryStringInfo instead of appendStringInfoString so that the append can be performed with a simple memcpy() using a known length rather than having to first perform a strlen() call to obtain the length. Discussion: https://postgr.es/m/CAKJS1f8+FRAM1s5+mAa3isajeEoAaicJ=4e0WzrH3tAusbbiMQ@mail.gmail.com
2019-07-22 14:14:11 +02:00
appendBinaryStringInfo(result, subres->data, subres->len);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
appendStringInfoChar(result, '\n');
}
SPI_finish();
xmldata_root_element_end(result, xmlcn);
return result;
}
Datum
database_to_xml(PG_FUNCTION_ARGS)
{
bool nulls = PG_GETARG_BOOL(0);
bool tableforest = PG_GETARG_BOOL(1);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(2));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(database_to_xml_internal(NULL, nulls,
tableforest, targetns)));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
static StringInfo
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
database_to_xmlschema_internal(bool nulls, bool tableforest,
const char *targetns)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
List *relid_list;
List *nspid_list;
List *tupdesc_list;
ListCell *cell;
StringInfo result;
result = makeStringInfo();
xsd_schema_element_start(result, targetns);
SPI_connect();
relid_list = database_get_xml_visible_tables();
nspid_list = database_get_xml_visible_schemas();
tupdesc_list = NIL;
foreach(cell, relid_list)
{
Relation rel;
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
rel = table_open(lfirst_oid(cell), AccessShareLock);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
tupdesc_list = lappend(tupdesc_list, CreateTupleDescCopy(rel->rd_att));
Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-21 19:32:19 +01:00
table_close(rel, NoLock);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
appendStringInfoString(result,
map_sql_typecoll_to_xmlschema_types(tupdesc_list));
appendStringInfoString(result,
map_sql_catalog_to_xmlschema_types(nspid_list, nulls, tableforest, targetns));
xsd_schema_element_end(result);
SPI_finish();
return result;
}
Datum
database_to_xmlschema(PG_FUNCTION_ARGS)
{
bool nulls = PG_GETARG_BOOL(0);
bool tableforest = PG_GETARG_BOOL(1);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(2));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(database_to_xmlschema_internal(nulls,
tableforest, targetns)));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
Datum
database_to_xml_and_xmlschema(PG_FUNCTION_ARGS)
{
bool nulls = PG_GETARG_BOOL(0);
bool tableforest = PG_GETARG_BOOL(1);
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
const char *targetns = text_to_cstring(PG_GETARG_TEXT_PP(2));
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
StringInfo xmlschema;
xmlschema = database_to_xmlschema_internal(nulls, tableforest, targetns);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
PG_RETURN_XML_P(stringinfo_to_xmltype(database_to_xml_internal(xmlschema->data,
nulls, tableforest, targetns)));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map a multi-part SQL name to an XML name; see SQL/XML:2008 section
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
* 9.2.
*/
static char *
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
map_multipart_sql_identifier_to_xml_name(const char *a, const char *b, const char *c, const char *d)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
StringInfoData result;
initStringInfo(&result);
if (a)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result,
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
map_sql_identifier_to_xml_name(a, true, true));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (b)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
appendStringInfo(&result, ".%s",
map_sql_identifier_to_xml_name(b, true, true));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (c)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
appendStringInfo(&result, ".%s",
map_sql_identifier_to_xml_name(c, true, true));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (d)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
appendStringInfo(&result, ".%s",
map_sql_identifier_to_xml_name(d, true, true));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
return result.data;
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map an SQL table to an XML Schema document; see SQL/XML:2008
* section 9.11.
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map an SQL table to XML Schema data types; see SQL/XML:2008 section
* 9.9.
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*/
static const char *
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
map_sql_table_to_xmlschema(TupleDesc tupdesc, Oid relid, bool nulls,
bool tableforest, const char *targetns)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
int i;
char *xmltn;
char *tabletypename;
char *rowtypename;
StringInfoData result;
initStringInfo(&result);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
if (OidIsValid(relid))
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
HeapTuple tuple;
Form_pg_class reltuple;
Wrap calls to SearchSysCache and related functions using macros. The purpose of this change is to eliminate the need for every caller of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists, GetSysCacheOid, and SearchSysCacheList to know the maximum number of allowable keys for a syscache entry (currently 4). This will make it far easier to increase the maximum number of keys in a future release should we choose to do so, and it makes the code shorter, too. Design and review by Tom Lane.
2010-02-14 19:42:19 +01:00
tuple = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for relation %u", relid);
reltuple = (Form_pg_class) GETSTRUCT(tuple);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmltn = map_sql_identifier_to_xml_name(NameStr(reltuple->relname),
true, false);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
tabletypename = map_multipart_sql_identifier_to_xml_name("TableType",
get_database_name(MyDatabaseId),
get_namespace_name(reltuple->relnamespace),
NameStr(reltuple->relname));
rowtypename = map_multipart_sql_identifier_to_xml_name("RowType",
get_database_name(MyDatabaseId),
get_namespace_name(reltuple->relnamespace),
NameStr(reltuple->relname));
ReleaseSysCache(tuple);
}
else
{
if (tableforest)
xmltn = "row";
else
xmltn = "table";
tabletypename = "TableType";
rowtypename = "RowType";
}
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xsd_schema_element_start(&result, targetns);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
appendStringInfoString(&result,
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
map_sql_typecoll_to_xmlschema_types(list_make1(tupdesc)));
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
appendStringInfo(&result,
"<xsd:complexType name=\"%s\">\n"
" <xsd:sequence>\n",
rowtypename);
for (i = 0; i < tupdesc->natts; i++)
Fix map_sql_table_to_xmlschema() with dropped attributes. also backpatched to 8.3
2009-06-08 23:32:33 +02:00
{
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
Form_pg_attribute att = TupleDescAttr(tupdesc, i);
if (att->attisdropped)
Fix map_sql_table_to_xmlschema() with dropped attributes. also backpatched to 8.3
2009-06-08 23:32:33 +02:00
continue;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
appendStringInfo(&result,
" <xsd:element name=\"%s\" type=\"%s\"%s></xsd:element>\n",
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
map_sql_identifier_to_xml_name(NameStr(att->attname),
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
true, false),
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
map_sql_type_to_xml_name(att->atttypid, -1),
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
nulls ? " nillable=\"true\"" : " minOccurs=\"0\"");
Fix map_sql_table_to_xmlschema() with dropped attributes. also backpatched to 8.3
2009-06-08 23:32:33 +02:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
appendStringInfoString(&result,
" </xsd:sequence>\n"
"</xsd:complexType>\n\n");
if (!tableforest)
{
appendStringInfo(&result,
"<xsd:complexType name=\"%s\">\n"
" <xsd:sequence>\n"
" <xsd:element name=\"row\" type=\"%s\" minOccurs=\"0\" maxOccurs=\"unbounded\"/>\n"
" </xsd:sequence>\n"
"</xsd:complexType>\n\n",
tabletypename, rowtypename);
appendStringInfo(&result,
"<xsd:element name=\"%s\" type=\"%s\"/>\n\n",
xmltn, tabletypename);
}
else
appendStringInfo(&result,
"<xsd:element name=\"%s\" type=\"%s\"/>\n\n",
xmltn, rowtypename);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xsd_schema_element_end(&result);
return result.data;
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map an SQL schema to XML Schema data types; see SQL/XML:2008
* section 9.12.
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
*/
static const char *
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
map_sql_schema_to_xmlschema_types(Oid nspid, List *relid_list, bool nulls,
bool tableforest, const char *targetns)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
char *dbname;
char *nspname;
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
char *xmlsn;
char *schematypename;
StringInfoData result;
ListCell *cell;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
dbname = get_database_name(MyDatabaseId);
nspname = get_namespace_name(nspid);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
initStringInfo(&result);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlsn = map_sql_identifier_to_xml_name(nspname, true, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
schematypename = map_multipart_sql_identifier_to_xml_name("SchemaType",
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
dbname,
nspname,
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
NULL);
appendStringInfo(&result,
"<xsd:complexType name=\"%s\">\n", schematypename);
if (!tableforest)
appendStringInfoString(&result,
" <xsd:all>\n");
else
appendStringInfoString(&result,
" <xsd:sequence>\n");
foreach(cell, relid_list)
{
Oid relid = lfirst_oid(cell);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
char *relname = get_rel_name(relid);
char *xmltn = map_sql_identifier_to_xml_name(relname, true, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
char *tabletypename = map_multipart_sql_identifier_to_xml_name(tableforest ? "RowType" : "TableType",
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
dbname,
nspname,
relname);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
if (!tableforest)
appendStringInfo(&result,
Add missing closing / in xsd:restriction, and remove some unnecessary spaces for consistency. Per bug #3734 from Ben Leslie; fix by Euler Taveira de Oliveira.
2007-11-10 20:29:54 +01:00
" <xsd:element name=\"%s\" type=\"%s\"/>\n",
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xmltn, tabletypename);
else
appendStringInfo(&result,
Add missing closing / in xsd:restriction, and remove some unnecessary spaces for consistency. Per bug #3734 from Ben Leslie; fix by Euler Taveira de Oliveira.
2007-11-10 20:29:54 +01:00
" <xsd:element name=\"%s\" type=\"%s\" minOccurs=\"0\" maxOccurs=\"unbounded\"/>\n",
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xmltn, tabletypename);
}
if (!tableforest)
appendStringInfoString(&result,
" </xsd:all>\n");
else
appendStringInfoString(&result,
" </xsd:sequence>\n");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
appendStringInfoString(&result,
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
"</xsd:complexType>\n\n");
appendStringInfo(&result,
"<xsd:element name=\"%s\" type=\"%s\"/>\n\n",
xmlsn, schematypename);
return result.data;
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map an SQL catalog to XML Schema data types; see SQL/XML:2008
* section 9.15.
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
*/
static const char *
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
map_sql_catalog_to_xmlschema_types(List *nspid_list, bool nulls,
bool tableforest, const char *targetns)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
char *dbname;
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
char *xmlcn;
char *catalogtypename;
StringInfoData result;
ListCell *cell;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
dbname = get_database_name(MyDatabaseId);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
initStringInfo(&result);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
xmlcn = map_sql_identifier_to_xml_name(dbname, true, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
catalogtypename = map_multipart_sql_identifier_to_xml_name("CatalogType",
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
dbname,
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
NULL,
NULL);
appendStringInfo(&result,
"<xsd:complexType name=\"%s\">\n", catalogtypename);
appendStringInfoString(&result,
" <xsd:all>\n");
foreach(cell, nspid_list)
{
Oid nspid = lfirst_oid(cell);
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
char *nspname = get_namespace_name(nspid);
char *xmlsn = map_sql_identifier_to_xml_name(nspname, true, false);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
char *schematypename = map_multipart_sql_identifier_to_xml_name("SchemaType",
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
dbname,
nspname,
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
NULL);
appendStringInfo(&result,
Add missing closing / in xsd:restriction, and remove some unnecessary spaces for consistency. Per bug #3734 from Ben Leslie; fix by Euler Taveira de Oliveira.
2007-11-10 20:29:54 +01:00
" <xsd:element name=\"%s\" type=\"%s\"/>\n",
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xmlsn, schematypename);
}
appendStringInfoString(&result,
" </xsd:all>\n");
appendStringInfoString(&result,
"</xsd:complexType>\n\n");
appendStringInfo(&result,
"<xsd:element name=\"%s\" type=\"%s\"/>\n\n",
xmlcn, catalogtypename);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
return result.data;
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map an SQL data type to an XML name; see SQL/XML:2008 section 9.4.
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*/
static const char *
map_sql_type_to_xml_name(Oid typeoid, int typmod)
{
StringInfoData result;
initStringInfo(&result);
switch (typeoid)
{
case BPCHAROID:
if (typmod == -1)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "CHAR");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfo(&result, "CHAR_%d", typmod - VARHDRSZ);
break;
case VARCHAROID:
if (typmod == -1)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "VARCHAR");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfo(&result, "VARCHAR_%d", typmod - VARHDRSZ);
break;
case NUMERICOID:
if (typmod == -1)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "NUMERIC");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfo(&result, "NUMERIC_%d_%d",
((typmod - VARHDRSZ) >> 16) & 0xffff,
(typmod - VARHDRSZ) & 0xffff);
break;
case INT4OID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "INTEGER");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case INT2OID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "SMALLINT");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case INT8OID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "BIGINT");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case FLOAT4OID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "REAL");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case FLOAT8OID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "DOUBLE");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case BOOLOID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "BOOLEAN");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case TIMEOID:
if (typmod == -1)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "TIME");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfo(&result, "TIME_%d", typmod);
break;
case TIMETZOID:
if (typmod == -1)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "TIME_WTZ");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfo(&result, "TIME_WTZ_%d", typmod);
break;
case TIMESTAMPOID:
if (typmod == -1)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "TIMESTAMP");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfo(&result, "TIMESTAMP_%d", typmod);
break;
case TIMESTAMPTZOID:
if (typmod == -1)
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "TIMESTAMP_WTZ");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfo(&result, "TIMESTAMP_WTZ_%d", typmod);
break;
case DATEOID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "DATE");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case XMLOID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "XML");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
default:
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
HeapTuple tuple;
Form_pg_type typtuple;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Wrap calls to SearchSysCache and related functions using macros. The purpose of this change is to eliminate the need for every caller of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists, GetSysCacheOid, and SearchSysCacheList to know the maximum number of allowable keys for a syscache entry (currently 4). This will make it far easier to increase the maximum number of keys in a future release should we choose to do so, and it makes the code shorter, too. Design and review by Tom Lane.
2010-02-14 19:42:19 +01:00
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for type %u", typeoid);
typtuple = (Form_pg_type) GETSTRUCT(tuple);
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
appendStringInfoString(&result,
Support enum data types. Along the way, use macros for the values of pg_type.typtype whereever practical. Tom Dunstan, with some kibitzing from Tom Lane.
2007-04-02 05:49:42 +02:00
map_multipart_sql_identifier_to_xml_name((typtuple->typtype == TYPTYPE_DOMAIN) ? "Domain" : "UDT",
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
get_database_name(MyDatabaseId),
get_namespace_name(typtuple->typnamespace),
NameStr(typtuple->typname)));
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
ReleaseSysCache(tuple);
}
}
return result.data;
}
/*
* Map a collection of SQL data types to XML Schema data types; see
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* SQL/XML:2008 section 9.7.
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*/
static const char *
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
map_sql_typecoll_to_xmlschema_types(List *tupdesc_list)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
List *uniquetypes = NIL;
int i;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
StringInfoData result;
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
ListCell *cell0;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
/* extract all column types used in the set of TupleDescs */
foreach(cell0, tupdesc_list)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
TupleDesc tupdesc = (TupleDesc) lfirst(cell0);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
for (i = 0; i < tupdesc->natts; i++)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
Form_pg_attribute att = TupleDescAttr(tupdesc, i);
if (att->attisdropped)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
continue;
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
uniquetypes = list_append_unique_oid(uniquetypes, att->atttypid);
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
/* add base types of domains */
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
foreach(cell0, uniquetypes)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
Oid typid = lfirst_oid(cell0);
Oid basetypid = getBaseType(typid);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
if (basetypid != typid)
uniquetypes = list_append_unique_oid(uniquetypes, basetypid);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
/* Convert to textual form */
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
initStringInfo(&result);
Fix map_sql_typecoll_to_xmlschema_types() to not fail on dropped columns, per my gripe earlier today. Make it look a bit less like someone's first effort at backend coding.
2007-07-13 05:43:23 +02:00
foreach(cell0, uniquetypes)
{
appendStringInfo(&result, "%s\n",
map_sql_type_to_xmlschema_type(lfirst_oid(cell0),
-1));
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
return result.data;
}
/*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* Map an SQL data type to a named XML Schema data type; see
* SQL/XML:2008 sections 9.5 and 9.6.
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* (The distinction between 9.5 and 9.6 is basically that 9.6 adds
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
* a name attribute, which this function does. The name-less version
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* 9.5 doesn't appear to be required anywhere.)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*/
static const char *
map_sql_type_to_xmlschema_type(Oid typeoid, int typmod)
{
StringInfoData result;
const char *typename = map_sql_type_to_xml_name(typeoid, typmod);
initStringInfo(&result);
if (typeoid == XMLOID)
{
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result,
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
"<xsd:complexType mixed=\"true\">\n"
" <xsd:sequence>\n"
" <xsd:any name=\"element\" minOccurs=\"0\" maxOccurs=\"unbounded\" processContents=\"skip\"/>\n"
" </xsd:sequence>\n"
"</xsd:complexType>\n");
}
else
{
appendStringInfo(&result,
"<xsd:simpleType name=\"%s\">\n", typename);
switch (typeoid)
{
case BPCHAROID:
case VARCHAROID:
case TEXTOID:
Fix up some misusage of appendStringInfo() and friends Change to appendStringInfoChar() or appendStringInfoString() where those can be used. Author: David Rowley <david.rowley@2ndquadrant.com> Reviewed-by: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>
2017-08-16 05:34:39 +02:00
appendStringInfoString(&result,
" <xsd:restriction base=\"xsd:string\">\n");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (typmod != -1)
appendStringInfo(&result,
Fix XML Schema structure for char types without length (bug #3782)
2007-11-28 15:01:51 +01:00
" <xsd:maxLength value=\"%d\"/>\n",
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
typmod - VARHDRSZ);
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, " </xsd:restriction>\n");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case BYTEAOID:
appendStringInfo(&result,
" <xsd:restriction base=\"xsd:%s\">\n"
" </xsd:restriction>\n",
xmlbinary == XMLBINARY_BASE64 ? "base64Binary" : "hexBinary");
Fix XML Schema structure for char types without length (bug #3782)
2007-11-28 15:01:51 +01:00
break;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
case NUMERICOID:
if (typmod != -1)
appendStringInfo(&result,
" <xsd:restriction base=\"xsd:decimal\">\n"
" <xsd:totalDigits value=\"%d\"/>\n"
" <xsd:fractionDigits value=\"%d\"/>\n"
" </xsd:restriction>\n",
((typmod - VARHDRSZ) >> 16) & 0xffff,
(typmod - VARHDRSZ) & 0xffff);
break;
case INT2OID:
appendStringInfo(&result,
" <xsd:restriction base=\"xsd:short\">\n"
" <xsd:maxInclusive value=\"%d\"/>\n"
" <xsd:minInclusive value=\"%d\"/>\n"
" </xsd:restriction>\n",
SHRT_MAX, SHRT_MIN);
break;
case INT4OID:
appendStringInfo(&result,
Use double quotes for quoting xml attributes.
2007-11-25 13:08:11 +01:00
" <xsd:restriction base=\"xsd:int\">\n"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:maxInclusive value=\"%d\"/>\n"
" <xsd:minInclusive value=\"%d\"/>\n"
" </xsd:restriction>\n",
INT_MAX, INT_MIN);
break;
case INT8OID:
appendStringInfo(&result,
" <xsd:restriction base=\"xsd:long\">\n"
" <xsd:maxInclusive value=\"" INT64_FORMAT "\"/>\n"
" <xsd:minInclusive value=\"" INT64_FORMAT "\"/>\n"
" </xsd:restriction>\n",
Make use of PG_INT64_MAX/PG_INT64_MIN This code was written before those symbols were introduced, but now we can simplify it.
2021-09-22 07:31:05 +02:00
PG_INT64_MAX,
PG_INT64_MIN);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
break;
case FLOAT4OID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result,
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:restriction base=\"xsd:float\"></xsd:restriction>\n");
break;
case FLOAT8OID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result,
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:restriction base=\"xsd:double\"></xsd:restriction>\n");
break;
case BOOLOID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result,
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:restriction base=\"xsd:boolean\"></xsd:restriction>\n");
break;
case TIMEOID:
case TIMETZOID:
{
Fix incorrect xmlschema output for types timetz and timestamptz. The output of table_to_xmlschema() and allied functions includes a regex describing valid values for these types ... but the regex was itself invalid, as it failed to escape a literal "+" sign. Report and fix by Renan Soares Lopes. Back-patch to all supported branches. Discussion: https://postgr.es/m/7f6fabaa-3f8f-49ab-89ca-59fbfe633105@me.com
2022-03-18 21:01:42 +01:00
const char *tz = (typeoid == TIMETZOID ? "(\\+|-)\\p{Nd}{2}:\\p{Nd}{2}" : "");
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (typmod == -1)
appendStringInfo(&result,
" <xsd:restriction base=\"xsd:time\">\n"
" <xsd:pattern value=\"\\p{Nd}{2}:\\p{Nd}{2}:\\p{Nd}{2}(.\\p{Nd}+)?%s\"/>\n"
" </xsd:restriction>\n", tz);
else if (typmod == 0)
appendStringInfo(&result,
" <xsd:restriction base=\"xsd:time\">\n"
" <xsd:pattern value=\"\\p{Nd}{2}:\\p{Nd}{2}:\\p{Nd}{2}%s\"/>\n"
" </xsd:restriction>\n", tz);
else
appendStringInfo(&result,
" <xsd:restriction base=\"xsd:time\">\n"
" <xsd:pattern value=\"\\p{Nd}{2}:\\p{Nd}{2}:\\p{Nd}{2}.\\p{Nd}{%d}%s\"/>\n"
" </xsd:restriction>\n", typmod - VARHDRSZ, tz);
break;
}
case TIMESTAMPOID:
case TIMESTAMPTZOID:
{
Fix incorrect xmlschema output for types timetz and timestamptz. The output of table_to_xmlschema() and allied functions includes a regex describing valid values for these types ... but the regex was itself invalid, as it failed to escape a literal "+" sign. Report and fix by Renan Soares Lopes. Back-patch to all supported branches. Discussion: https://postgr.es/m/7f6fabaa-3f8f-49ab-89ca-59fbfe633105@me.com
2022-03-18 21:01:42 +01:00
const char *tz = (typeoid == TIMESTAMPTZOID ? "(\\+|-)\\p{Nd}{2}:\\p{Nd}{2}" : "");
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (typmod == -1)
appendStringInfo(&result,
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
" <xsd:restriction base=\"xsd:dateTime\">\n"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:pattern value=\"\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}T\\p{Nd}{2}:\\p{Nd}{2}:\\p{Nd}{2}(.\\p{Nd}+)?%s\"/>\n"
" </xsd:restriction>\n", tz);
else if (typmod == 0)
appendStringInfo(&result,
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
" <xsd:restriction base=\"xsd:dateTime\">\n"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:pattern value=\"\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}T\\p{Nd}{2}:\\p{Nd}{2}:\\p{Nd}{2}%s\"/>\n"
" </xsd:restriction>\n", tz);
else
appendStringInfo(&result,
Fix date/time formats for XML Schema output. Pavel Stehule
2007-03-01 15:52:04 +01:00
" <xsd:restriction base=\"xsd:dateTime\">\n"
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:pattern value=\"\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}T\\p{Nd}{2}:\\p{Nd}{2}:\\p{Nd}{2}.\\p{Nd}{%d}%s\"/>\n"
" </xsd:restriction>\n", typmod - VARHDRSZ, tz);
break;
}
case DATEOID:
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result,
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
" <xsd:restriction base=\"xsd:date\">\n"
" <xsd:pattern value=\"\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}\"/>\n"
" </xsd:restriction>\n");
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
break;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
default:
Support enum data types. Along the way, use macros for the values of pg_type.typtype whereever practical. Tom Dunstan, with some kibitzing from Tom Lane.
2007-04-02 05:49:42 +02:00
if (get_typtype(typeoid) == TYPTYPE_DOMAIN)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
Oid base_typeoid;
int32 base_typmod = -1;
base_typeoid = getBaseTypeAndTypmod(typeoid, &base_typmod);
appendStringInfo(&result,
Add missing closing / in xsd:restriction, and remove some unnecessary spaces for consistency. Per bug #3734 from Ben Leslie; fix by Euler Taveira de Oliveira.
2007-11-10 20:29:54 +01:00
" <xsd:restriction base=\"%s\"/>\n",
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
map_sql_type_to_xml_name(base_typeoid, base_typmod));
}
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
break;
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
Use appendStringInfoString instead of appendStringInfo where possible. This shaves a few cycles, and generally seems like good programming practice. David Rowley
2013-10-31 15:55:59 +01:00
appendStringInfoString(&result, "</xsd:simpleType>\n");
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
return result.data;
}
/*
* Map an SQL row to an XML element, taking the row from the active
Change references to SQL/XML:2003 to :2008 and renumber sections accordingly
2010-10-15 20:18:47 +02:00
* SPI cursor. See also SQL/XML:2008 section 9.10.
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
*/
static void
Widen query numbers-of-tuples-processed counters to uint64. This patch widens SPI_processed, EState's es_processed field, PortalData's portalPos field, FuncCallContext's call_cntr and max_calls fields, ExecutorRun's count argument, PortalRunFetch's result, and the max number of rows in a SPITupleTable to uint64, and deals with (I hope) all the ensuing fallout. Some of these values were declared uint32 before, and others "long". I also removed PortalData's posOverflow field, since that logic seems pretty useless given that portalPos is now always 64 bits. The user-visible results are that command tags for SELECT etc will correctly report tuple counts larger than 4G, as will plpgsql's GET GET DIAGNOSTICS ... ROW_COUNT command. Queries processing more tuples than that are still not exactly the norm, but they're becoming more common. Most values associated with FETCH/MOVE distances, such as PortalRun's count argument and the count argument of most SPI functions that have one, remain declared as "long". It's not clear whether it would be worth promoting those to int64; but it would definitely be a large dollop of additional API churn on top of this, and it would only help 32-bit platforms which seem relatively less likely to see any benefit. Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me
2016-03-12 22:05:10 +01:00
SPI_sql_row_to_xmlelement(uint64 rownum, StringInfo result, char *tablename,
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
bool nulls, bool tableforest,
const char *targetns, bool top_level)
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
{
int i;
char *xmltn;
if (tablename)
xmltn = map_sql_identifier_to_xml_name(tablename, true, false);
else
{
if (tableforest)
xmltn = "row";
else
xmltn = "table";
}
if (tableforest)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
xmldata_root_element_start(result, xmltn, NULL, targetns, top_level);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfoString(result, "<row>\n");
for (i = 1; i <= SPI_tuptable->tupdesc->natts; i++)
{
char *colname;
Datum colval;
bool isnull;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
colname = map_sql_identifier_to_xml_name(SPI_fname(SPI_tuptable->tupdesc, i),
true, false);
colval = SPI_getbinval(SPI_tuptable->vals[rownum],
SPI_tuptable->tupdesc,
i,
&isnull);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
if (isnull)
{
if (nulls)
Use double quotes for quoting xml attributes.
2007-11-25 13:08:11 +01:00
appendStringInfo(result, " <%s xsi:nil=\"true\"/>\n", colname);
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
}
else
appendStringInfo(result, " <%s>%s</%s>\n",
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
colname,
map_sql_value_to_xml_value(colval,
Fix xmlattribute escaping XML special characters twice (bug #4822). Author: Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>
2009-06-10 00:00:57 +02:00
SPI_gettypeid(SPI_tuptable->tupdesc, i), true),
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
colname);
}
if (tableforest)
Mapping schemas and databases to XML and XML Schema. Refactor and document the remaining mapping code.
2007-04-01 11:00:26 +02:00
{
xmldata_root_element_end(result, xmltn);
appendStringInfoChar(result, '\n');
}
Functions for mapping table data and table schemas to XML (a.k.a. XML export)
2007-02-16 08:46:55 +01:00
else
appendStringInfoString(result, "</row>\n\n");
}
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
/*
* XPath related functions
*/
#ifdef USE_LIBXML
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
/*
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
* Convert XML node to text.
*
* For attribute and text nodes, return the escaped text. For anything else,
* dump the whole subtree.
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
*/
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
static text *
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
xml_xmlnodetoxmltype(xmlNodePtr cur, PgXmlErrorContext *xmlerrcxt)
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
{
Fix some compiler warnings on older compilers Some older compilers appear to not understand the recently introduced PG_FINALLY code structure that well in some circumstances and complain about possibly uninitialized variables. So to fix, initialize the variables explicitly in the cases complained about. Discussion: https://www.postgresql.org/message-id/flat/95a822c3-728b-af0e-d7e5-71890507ae0c%402ndquadrant.com
2019-11-04 11:07:32 +01:00
xmltype *result = NULL;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
if (cur->type != XML_ATTRIBUTE_NODE && cur->type != XML_TEXT_NODE)
Commit newest version of xmlpath(). Nikolay Samokhvalov
2007-03-22 21:26:30 +01:00
{
Add volatile qualifier missed in commit 2e616dee9. Noted by Pavel Stehule Discussion: https://postgr.es/m/CAFj8pRAaGO5FX7bnP3E=mRssoK8y5T78x7jKy-vDiyS68L888Q@mail.gmail.com
2019-04-01 20:37:25 +02:00
void (*volatile nodefree) (xmlNodePtr) = NULL;
Fix crash with old libxml2 Certain libxml2 versions (such as the 2.7.6 commonly seen in older distributions, but apparently only on x86_64) contain a bug that causes xmlCopyNode, when called on a XML_DOCUMENT_NODE, to return a node that xmlFreeNode crashes on. Arrange to call xmlFreeDoc instead of xmlFreeNode for those nodes. Per buildfarm members lapwing and grison. Author: Pavel Stehule, light editing by Álvaro. Discussion: https://postgr.es/m/20190308024436.GA2374@alvherre.pgsql
2019-03-08 23:13:25 +01:00
volatile xmlBufferPtr buf = NULL;
volatile xmlNodePtr cur_copy = NULL;
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
Fix crash with old libxml2 Certain libxml2 versions (such as the 2.7.6 commonly seen in older distributions, but apparently only on x86_64) contain a bug that causes xmlCopyNode, when called on a XML_DOCUMENT_NODE, to return a node that xmlFreeNode crashes on. Arrange to call xmlFreeDoc instead of xmlFreeNode for those nodes. Per buildfarm members lapwing and grison. Author: Pavel Stehule, light editing by Álvaro. Discussion: https://postgr.es/m/20190308024436.GA2374@alvherre.pgsql
2019-03-08 23:13:25 +01:00
PG_TRY();
{
int bytes;
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
Fix crash with old libxml2 Certain libxml2 versions (such as the 2.7.6 commonly seen in older distributions, but apparently only on x86_64) contain a bug that causes xmlCopyNode, when called on a XML_DOCUMENT_NODE, to return a node that xmlFreeNode crashes on. Arrange to call xmlFreeDoc instead of xmlFreeNode for those nodes. Per buildfarm members lapwing and grison. Author: Pavel Stehule, light editing by Álvaro. Discussion: https://postgr.es/m/20190308024436.GA2374@alvherre.pgsql
2019-03-08 23:13:25 +01:00
buf = xmlBufferCreate();
if (buf == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate xmlBuffer");
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
Fix crash with old libxml2 Certain libxml2 versions (such as the 2.7.6 commonly seen in older distributions, but apparently only on x86_64) contain a bug that causes xmlCopyNode, when called on a XML_DOCUMENT_NODE, to return a node that xmlFreeNode crashes on. Arrange to call xmlFreeDoc instead of xmlFreeNode for those nodes. Per buildfarm members lapwing and grison. Author: Pavel Stehule, light editing by Álvaro. Discussion: https://postgr.es/m/20190308024436.GA2374@alvherre.pgsql
2019-03-08 23:13:25 +01:00
/*
* Produce a dump of the node that we can serialize. xmlNodeDump
* does that, but the result of that function won't contain
* namespace definitions from ancestor nodes, so we first do a
* xmlCopyNode() which duplicates the node along with its required
* namespace definitions.
*
* Some old libxml2 versions such as 2.7.6 produce partially
* broken XML_DOCUMENT_NODE nodes (unset content field) when
* copying them. xmlNodeDump of such a node works fine, but
* xmlFreeNode crashes; set us up to call xmlFreeDoc instead.
*/
cur_copy = xmlCopyNode(cur, 1);
if (cur_copy == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not copy node");
nodefree = (cur_copy->type == XML_DOCUMENT_NODE) ?
(void (*) (xmlNodePtr)) xmlFreeDoc : xmlFreeNode;
Don't request pretty-printed output from xmlNodeDump(). xml.c passed format = 1 to xmlNodeDump(), resulting in sometimes getting extra whitespace (newlines + spaces) in the output. We don't really want that, first because whitespace might be semantically significant in some XML uses, and second because it happens only very inconsistently. Only one case in our regression tests is affected. This potentially affects the results of xpath() and the XMLTABLE construct, when emitting nodeset values. Note that the older code in contrib/xml2 doesn't do this; it seems to have been an aboriginal bad decision in commit ea3b212fe. While this definitely seems like a bug to me, the small number of complaints to date argues against back-patching a behavioral change. Hence, fix in HEAD only, at least for now. Per report from Jean-Marc Voillequin. Discussion: https://postgr.es/m/1EC8157EB499BF459A516ADCF135ADCE3A23A9CA@LON-WGMSX712.ad.moodys.net
2019-04-23 16:51:07 +02:00
bytes = xmlNodeDump(buf, NULL, cur_copy, 0, 0);
Fix crash with old libxml2 Certain libxml2 versions (such as the 2.7.6 commonly seen in older distributions, but apparently only on x86_64) contain a bug that causes xmlCopyNode, when called on a XML_DOCUMENT_NODE, to return a node that xmlFreeNode crashes on. Arrange to call xmlFreeDoc instead of xmlFreeNode for those nodes. Per buildfarm members lapwing and grison. Author: Pavel Stehule, light editing by Álvaro. Discussion: https://postgr.es/m/20190308024436.GA2374@alvherre.pgsql
2019-03-08 23:13:25 +01:00
if (bytes == -1 || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not dump node");
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
result = xmlBuffer_to_xmltype(buf);
}
Fix nested error handling in PG_FINALLY We need to pop the error stack before running the user-supplied PG_FINALLY code. Otherwise an error in the cleanup code would end up at the same sigsetjmp() invocation and result in an infinite error handling loop. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/95a822c3-728b-af0e-d7e5-71890507ae0c%402ndquadrant.com
2019-11-07 09:54:09 +01:00
PG_FINALLY();
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
{
Fix crash with old libxml2 Certain libxml2 versions (such as the 2.7.6 commonly seen in older distributions, but apparently only on x86_64) contain a bug that causes xmlCopyNode, when called on a XML_DOCUMENT_NODE, to return a node that xmlFreeNode crashes on. Arrange to call xmlFreeDoc instead of xmlFreeNode for those nodes. Per buildfarm members lapwing and grison. Author: Pavel Stehule, light editing by Álvaro. Discussion: https://postgr.es/m/20190308024436.GA2374@alvherre.pgsql
2019-03-08 23:13:25 +01:00
if (nodefree)
nodefree(cur_copy);
if (buf)
xmlBufferFree(buf);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
}
PG_END_TRY();
Commit newest version of xmlpath(). Nikolay Samokhvalov
2007-03-22 21:26:30 +01:00
}
else
{
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
xmlChar *str;
Commit newest version of xmlpath(). Nikolay Samokhvalov
2007-03-22 21:26:30 +01:00
str = xmlXPathCastNodeToString(cur);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_TRY();
{
Ensure that xpath() escapes special characters in string values. Without this it's possible for the output to not be legal XML, as illustrated by the added regression test cases. NB: this change will need to be called out as an incompatibility in the 9.2 release notes, since it's possible somebody was relying on the old behavior, even though it's clearly wrong. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 00:44:09 +02:00
/* Here we rely on XML having the same representation as TEXT */
char *escaped = escape_xml((char *) str);
result = (xmltype *) cstring_to_text(escaped);
pfree(escaped);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
}
PG_FINALLY This gives an alternative way of catching exceptions, for the common case where the cleanup code is the same in the error and non-error cases. So instead of PG_TRY(); { ... code that might throw ereport(ERROR) ... } PG_CATCH(); { cleanup(); PG_RE_THROW(); } PG_END_TRY(); cleanup(); one can write PG_TRY(); { ... code that might throw ereport(ERROR) ... } PG_FINALLY(); { cleanup(); } PG_END_TRY(); Discussion: https://www.postgresql.org/message-id/flat/95a822c3-728b-af0e-d7e5-71890507ae0c%402ndquadrant.com
2019-11-01 11:09:52 +01:00
PG_FINALLY();
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
{
xmlFree(str);
}
PG_END_TRY();
Commit newest version of xmlpath(). Nikolay Samokhvalov
2007-03-22 21:26:30 +01:00
}
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
return result;
}
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
/*
* Convert an XML XPath object (the result of evaluating an XPath expression)
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
* to an array of xml values, which are appended to astate. The function
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
* result value is the number of elements in the array.
*
* If "astate" is NULL then we don't generate the array value, but we still
* return the number of elements it would have had.
*
* Nodesets are converted to an array containing the nodes' textual
* representations. Primitive values (float, double, string) are converted
* to a single-element array containing the value's string representation.
*/
static int
xml_xpathobjtoxmlarray(xmlXPathObjectPtr xpathobj,
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
ArrayBuildState *astate,
PgXmlErrorContext *xmlerrcxt)
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
{
int result = 0;
Datum datum;
Oid datumtype;
char *result_str;
switch (xpathobj->type)
{
case XPATH_NODESET:
if (xpathobj->nodesetval != NULL)
{
result = xpathobj->nodesetval->nodeNr;
if (astate != NULL)
{
int i;
for (i = 0; i < result; i++)
{
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
datum = PointerGetDatum(xml_xmlnodetoxmltype(xpathobj->nodesetval->nodeTab[i],
xmlerrcxt));
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
(void) accumArrayResult(astate, datum, false,
XMLOID, CurrentMemoryContext);
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
}
}
}
return result;
case XPATH_BOOLEAN:
if (astate == NULL)
return 1;
datum = BoolGetDatum(xpathobj->boolval);
datumtype = BOOLOID;
break;
case XPATH_NUMBER:
if (astate == NULL)
return 1;
datum = Float8GetDatum(xpathobj->floatval);
datumtype = FLOAT8OID;
break;
case XPATH_STRING:
if (astate == NULL)
return 1;
datum = CStringGetDatum((char *) xpathobj->stringval);
datumtype = CSTRINGOID;
break;
default:
elog(ERROR, "xpath expression result type %d is unsupported",
xpathobj->type);
return 0; /* keep compiler quiet */
}
/* Common code for scalar-value cases */
result_str = map_sql_value_to_xml_value(datum, datumtype, true);
datum = PointerGetDatum(cstring_to_xmltype(result_str));
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
(void) accumArrayResult(astate, datum, false,
XMLOID, CurrentMemoryContext);
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
return 1;
}
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
/*
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
* Common code for xpath() and xmlexists()
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
*
Fix inconsistencies and typos in the tree, take 10 This addresses some issues with unnecessary code comments, fixes various typos in docs and comments, and removes some orphaned structures and definitions. Author: Alexander Lakhin Discussion: https://postgr.es/m/9aabc775-5494-b372-8bcb-4dfc0bd37c68@gmail.com
2019-08-13 06:53:41 +02:00
* Evaluate XPath expression and return number of nodes in res_nitems
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
* and array of XML values in astate. Either of those pointers can be
* NULL if the corresponding result isn't wanted.
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
*
Remove munging of xml and xpath params to xpath(). The XML must now be a well formed XML document.
2009-03-23 22:00:39 +01:00
* It is up to the user to ensure that the XML passed is in fact
* an XML document - XPath doesn't work easily on fragments without
* a context node being known.
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
*/
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
static void
xpath_internal(text *xpath_expr_text, xmltype *data, ArrayType *namespaces,
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
int *res_nitems, ArrayBuildState *astate)
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
{
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
PgXmlErrorContext *xmlerrcxt;
volatile xmlParserCtxtPtr ctxt = NULL;
volatile xmlDocPtr doc = NULL;
volatile xmlXPathContextPtr xpathctx = NULL;
volatile xmlXPathCompExprPtr xpathcomp = NULL;
volatile xmlXPathObjectPtr xpathobj = NULL;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
char *datastr;
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
int32 len;
int32 xpath_len;
xmlChar *string;
xmlChar *xpath_expr;
Ignore XML declaration in xpath_internal(), for UTF8 databases. When a value contained an XML declaration naming some other encoding, this function interpreted UTF8 bytes as the named encoding, yielding mojibake. xml_parse() already has similar logic. This would be necessary but not sufficient for non-UTF8 databases, so preserve behavior there until the xpath facility can support such databases comprehensively. Back-patch to 9.3 (all supported versions). Pavel Stehule and Noah Misch Discussion: https://postgr.es/m/CAFj8pRC-dM=tT=QkGi+Achkm+gwPmjyOayGuUfXVumCxkDgYWg@mail.gmail.com
2017-11-11 20:10:53 +01:00
size_t xmldecl_len = 0;
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
int i;
int ndim;
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
Datum *ns_names_uris;
bool *ns_names_uris_nulls;
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
int ns_count;
/*
* Namespace mappings are passed as text[]. If an empty array is passed
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
* (ndim = 0, "0-dimensional"), then there are no namespace mappings.
* Else, a 2-dimensional array with length of the second axis being equal
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
* to 2 should be passed, i.e., every subarray contains 2 elements, the
* first element defining the name, the second one the URI. Example:
* ARRAY[ARRAY['myns', 'http://example.com'], ARRAY['myns2',
* 'http://example2.com']].
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
*/
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
ndim = namespaces ? ARR_NDIM(namespaces) : 0;
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
if (ndim != 0)
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
{
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
int *dims;
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
dims = ARR_DIMS(namespaces);
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
if (ndim != 2 || dims[1] != 2)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
ereport(ERROR,
(errcode(ERRCODE_DATA_EXCEPTION),
errmsg("invalid array for XML namespace mapping"),
errdetail("The array must be two-dimensional with length of the second axis equal to 2.")));
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
Assert(ARR_ELEMTYPE(namespaces) == TEXTOID);
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
Add construct_array_builtin, deconstruct_array_builtin There were many calls to construct_array() and deconstruct_array() for built-in types, for example, when dealing with system catalog columns. These all hardcoded the type attributes necessary to pass to these functions. To simplify this a bit, add construct_array_builtin(), deconstruct_array_builtin() as wrappers that centralize this hardcoded knowledge. This simplifies many call sites and reduces the amount of hardcoded stuff that is spread around. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/2914356f-9e5f-8c59-2995-5997fc48bcba%40enterprisedb.com
2022-07-01 10:51:45 +02:00
deconstruct_array_builtin(namespaces, TEXTOID,
&ns_names_uris, &ns_names_uris_nulls,
&ns_count);
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
Assert((ns_count % 2) == 0); /* checked above */
ns_count /= 2; /* count pairs only */
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
}
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
else
{
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
ns_names_uris = NULL;
ns_names_uris_nulls = NULL;
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
ns_count = 0;
}
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
datastr = VARDATA(data);
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
len = VARSIZE(data) - VARHDRSZ;
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
xpath_len = VARSIZE_ANY_EXHDR(xpath_expr_text);
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
if (xpath_len == 0)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
ereport(ERROR,
(errcode(ERRCODE_DATA_EXCEPTION),
errmsg("empty XPath expression")));
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
string = pg_xmlCharStrndup(datastr, len);
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
xpath_expr = pg_xmlCharStrndup(VARDATA_ANY(xpath_expr_text), xpath_len);
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
Ignore XML declaration in xpath_internal(), for UTF8 databases. When a value contained an XML declaration naming some other encoding, this function interpreted UTF8 bytes as the named encoding, yielding mojibake. xml_parse() already has similar logic. This would be necessary but not sufficient for non-UTF8 databases, so preserve behavior there until the xpath facility can support such databases comprehensively. Back-patch to 9.3 (all supported versions). Pavel Stehule and Noah Misch Discussion: https://postgr.es/m/CAFj8pRC-dM=tT=QkGi+Achkm+gwPmjyOayGuUfXVumCxkDgYWg@mail.gmail.com
2017-11-11 20:10:53 +01:00
/*
* In a UTF8 database, skip any xml declaration, which might assert
* another encoding. Ignore parse_xml_decl() failure, letting
* xmlCtxtReadMemory() report parse errors. Documentation disclaims
* xpath() support for non-ASCII data in non-UTF8 databases, so leave
* those scenarios bug-compatible with historical behavior.
*/
if (GetDatabaseEncoding() == PG_UTF8)
parse_xml_decl(string, &xmldecl_len, NULL, NULL, NULL);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlerrcxt = pg_xml_init(PG_XML_STRICTNESS_ALL);
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_TRY();
{
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
xmlInitParser();
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
/*
* redundant XML parsing (two parsings for the same value during one
* command execution are possible)
*/
ctxt = xmlNewParserCtxt();
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (ctxt == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
"could not allocate parser context");
Ignore XML declaration in xpath_internal(), for UTF8 databases. When a value contained an XML declaration naming some other encoding, this function interpreted UTF8 bytes as the named encoding, yielding mojibake. xml_parse() already has similar logic. This would be necessary but not sufficient for non-UTF8 databases, so preserve behavior there until the xpath facility can support such databases comprehensively. Back-patch to 9.3 (all supported versions). Pavel Stehule and Noah Misch Discussion: https://postgr.es/m/CAFj8pRC-dM=tT=QkGi+Achkm+gwPmjyOayGuUfXVumCxkDgYWg@mail.gmail.com
2017-11-11 20:10:53 +01:00
doc = xmlCtxtReadMemory(ctxt, (char *) string + xmldecl_len,
Revert "Add support for parsing of large XML data (>= 10MB)" This reverts commit 2197d06224a1, following a discussion over a Coverity report where issues like the "Billion laugh attack" could cause the backend to waste CPU and memory even if a client applied checks on the size of the data given in input, and libxml2 does not offer guarantees that input limits are respected under XML_PARSE_HUGE. Discussion: https://postgr.es/m/ZbHlgrPLtBZyr_QW@paquier.xyz
2024-01-26 02:15:32 +01:00
len - xmldecl_len, NULL, NULL, 0);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (doc == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_INVALID_XML_DOCUMENT,
Remove munging of xml and xpath params to xpath(). The XML must now be a well formed XML document.
2009-03-23 22:00:39 +01:00
"could not parse XML document");
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
xpathctx = xmlXPathNewContext(doc);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (xpathctx == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
"could not allocate XPath context");
Set correct context for XPath evaluation According to the SQL standard, the context of XMLTABLE's XPath row_expression is the document node of the XML input document, not the root node. This becomes visible when a relative path rather than absolute is used as row expression. Absolute paths is what was used in original tests and docs (and the most common form used in examples throughout the interwebs), which explains why this wasn't noticed before. Other functions such as xpath() and xpath_exists() also have this problem. While not specified by the SQL standard, it would be pretty odd to leave those functions to behave differently than XMLTABLE, so change them too. However, this is a backwards-incompatible change. No backpatch, out of fear of breaking code depending on the original broken behavior. Author: Markus Winand Reported-By: Markus Winand Reviewed-by: Álvaro Herrera Discussion: https://postgr.es/m/0684A598-002C-42A2-AE12-F024A324EAE4@winand.at
2018-06-21 21:56:11 +02:00
xpathctx->node = (xmlNodePtr) doc;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
/* register namespaces, if any */
if (ns_count > 0)
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
{
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
for (i = 0; i < ns_count; i++)
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
{
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
char *ns_name;
char *ns_uri;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
if (ns_names_uris_nulls[i * 2] ||
ns_names_uris_nulls[i * 2 + 1])
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("neither namespace name nor URI may be null")));
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
ns_name = TextDatumGetCString(ns_names_uris[i * 2]);
ns_uri = TextDatumGetCString(ns_names_uris[i * 2 + 1]);
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
if (xmlXPathRegisterNs(xpathctx,
(xmlChar *) ns_name,
(xmlChar *) ns_uri) != 0)
ereport(ERROR, /* is this an internal error??? */
(errmsg("could not register XML namespace with name \"%s\" and URI \"%s\"",
ns_name, ns_uri)));
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
}
Some code review for xml.c: Add some more xml_init() calls that might not be necessary, but seem like a good idea to avoid possible problems like we saw in xmlelement(). Fix unsafe assumption that you can keep using the tupledesc of a relcache entry you don't have open. Add missing error checks for SearchSysCache failure. Get rid of handwritten array traversal in xpath() and O(N^2), broken-for-nulls array access code in map_sql_value_to_xml_value(), in favor of using deconstruct_array. Manually adjust a lot of line breaks in places where the code is otherwise gonna look pretty awful after pg_indent hacks it up (original author seems to have liked to lay out code for a 200-column window).
2007-11-06 04:06:28 +01:00
}
XPath fixes: - Function renamed to "xpath". - Function is now strict, per discussion. - Return empty array in case when XPath expression detects nothing (previously, NULL was returned in such case), per discussion. - (bugfix) Work with fragments with prologue: select xpath('/a', '<?xml version="1.0"?><a /><b />'); // now XML datum is always wrapped with dummy <x>...</x>, XML prologue simply goes away (if any). - Some cleanup. Nikolay Samokhvalov Some code cleanup and documentation work by myself.
2007-05-21 19:10:29 +02:00
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
xpathcomp = xmlXPathCompile(xpath_expr);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (xpathcomp == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
"invalid XPath expression");
Guard against possible double free during error escape from XML functions. Patch for the reported issue from Kris Jurka, some other potential trouble spots plugged by Tom.
2007-10-13 22:46:47 +02:00
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
/*
* Version 2.6.27 introduces a function named
* xmlXPathCompiledEvalToBoolean, which would be enough for xmlexists,
* but we can derive the existence by whether any nodes are returned,
* thereby preventing a library version upgrade and keeping the code
* the same.
*/
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
xpathobj = xmlXPathCompiledEval(xpathcomp, xpathctx);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
if (xpathobj == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
"could not create XPath object");
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Make xpath() do something useful with XPath expressions that return scalars. Previously, xpath() simply returned an empty array if the expression did not yield a node set. This is useless for expressions that return scalars, such as one with name() at the top level. Arrange to return the scalar value as a single-element xml array, instead. (String values will be suitably escaped.) This change will also cause xpath_exists() to return true, not false, for such expressions. Florian Pflug, reviewed by Radoslaw Smogura
2011-07-21 17:32:46 +02:00
/*
* Extract the results as requested.
*/
if (res_nitems != NULL)
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
*res_nitems = xml_xpathobjtoxmlarray(xpathobj, astate, xmlerrcxt);
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
else
Fix namespace handling in xpath function Previously, the xml value resulting from an xpath query would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value. That means the output value was not correct XML. Fix that by running the result value through xmlCopyNode(), which produces the correct namespace declarations. Author: Ali Akbar <the.apaan@gmail.com>
2015-01-07 05:06:13 +01:00
(void) xml_xpathobjtoxmlarray(xpathobj, astate, xmlerrcxt);
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
}
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_CATCH();
{
if (xpathobj)
xmlXPathFreeObject(xpathobj);
if (xpathcomp)
xmlXPathFreeCompExpr(xpathcomp);
if (xpathctx)
xmlXPathFreeContext(xpathctx);
if (doc)
xmlFreeDoc(doc);
if (ctxt)
xmlFreeParserCtxt(ctxt);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_done(xmlerrcxt, true);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
PG_RE_THROW();
}
PG_END_TRY();
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
xmlXPathFreeObject(xpathobj);
Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2. I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix. I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it. This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-13 22:27:17 +02:00
xmlXPathFreeCompExpr(xpathcomp);
Revise memory management for libxml calls. Instead of keeping libxml's data in whichever context happens to be current during a call of an xml.c function, use a dedicated context that will not go away until we explicitly delete it (which we do at transaction end or subtransaction abort). This makes recovery after an error much simpler --- we don't have to individually delete the data structures created by libxml. Also, we need to initialize and cleanup libxml only once per transaction (if there's no error) instead of once per function call, so it should be a bit faster. We'll need to keep an eye out for intra-transaction memory leaks, though. Alvaro and Tom.
2008-01-15 19:57:00 +01:00
xmlXPathFreeContext(xpathctx);
xmlFreeDoc(doc);
xmlFreeParserCtxt(ctxt);
Rewrite libxml error handling to be more robust. libxml reports some errors (like invalid xmlns attributes) via the error handler hook, but still returns a success indicator to the library caller. This causes us to miss some errors that are important to report. Since the "generic" error handler hook doesn't know whether the message it's getting is for an error, warning, or notice, stop using that and instead start using the "structured" error handler hook, which gets enough information to be useful. While at it, arrange to save and restore the error handler hook setting in each libxml-using function, rather than assuming we can set and forget the hook. This should improve the odds of working nicely with third-party libraries that also use libxml. In passing, volatile-ize some local variables that get modified within PG_TRY blocks. I noticed this while testing with an older gcc version than I'd previously tried to compile xml.c with. Florian Pflug and Tom Lane, with extensive review/testing by Noah Misch
2011-07-20 19:03:12 +02:00
pg_xml_done(xmlerrcxt, false);
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
}
#endif /* USE_LIBXML */
/*
* Evaluate XPath expression and return array of XML values.
*
* As we have no support of XQuery sequences yet, this function seems
* to be the most useful one (array of XML functions plays a role of
* some kind of substitution for XQuery sequences).
*/
Datum
xpath(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *xpath_expr_text = PG_GETARG_TEXT_PP(0);
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
xmltype *data = PG_GETARG_XML_P(1);
ArrayType *namespaces = PG_GETARG_ARRAYTYPE_P(2);
ArrayBuildState *astate;
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
astate = initArrayResult(XMLOID, CurrentMemoryContext, true);
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
xpath_internal(xpath_expr_text, data, namespaces,
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
NULL, astate);
Fix incorrect uses of Datum conversion macros Since these macros just cast whatever you give them to the designated output type, and many normal uses also cast the output type further, a number of incorrect uses go undiscovered. The fixes in this patch have been discovered by changing these macros to inline functions, which is the subject of a future patch. Reviewed-by: Aleksander Alekseev <aleksander@timescale.com> Discussion: https://www.postgresql.org/message-id/flat/8528fb7e-0aa2-6b54-85fb-0c0886dbd6ed%40enterprisedb.com
2022-08-28 10:47:10 +02:00
PG_RETURN_DATUM(makeArrayResult(astate, CurrentMemoryContext));
Add xmlpath() to evaluate XPath expressions, with namespaces support. Nikolay Samokhvalov
2007-03-22 21:14:58 +01:00
#else
NO_XML_SUPPORT();
return 0;
#endif
}
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
/*
* Determines if the node specified by the supplied XPath exists
* in a given XML document, returning a boolean.
*/
Datum
xmlexists(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *xpath_expr_text = PG_GETARG_TEXT_PP(0);
Add xmlexists function by Mike Fowler, reviewed by Peter Eisentraut
2010-08-05 06:21:54 +02:00
xmltype *data = PG_GETARG_XML_P(1);
int res_nitems;
xpath_internal(xpath_expr_text, data, NULL,
&res_nitems, NULL);
PG_RETURN_BOOL(res_nitems > 0);
#else
NO_XML_SUPPORT();
return 0;
#endif
}
Add an xpath_exists() function. This is equivalent to XMLEXISTS except that it offers support for namespace mapping. Mike Fowler, reviewed by David Fetter
2010-08-08 21:15:27 +02:00
/*
* Determines if the node specified by the supplied XPath exists
* in a given XML document, returning a boolean. Differs from
* xmlexists as it supports namespaces and is not defined in SQL/XML.
*/
Datum
xpath_exists(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *xpath_expr_text = PG_GETARG_TEXT_PP(0);
Add an xpath_exists() function. This is equivalent to XMLEXISTS except that it offers support for namespace mapping. Mike Fowler, reviewed by David Fetter
2010-08-08 21:15:27 +02:00
xmltype *data = PG_GETARG_XML_P(1);
ArrayType *namespaces = PG_GETARG_ARRAYTYPE_P(2);
int res_nitems;
xpath_internal(xpath_expr_text, data, namespaces,
&res_nitems, NULL);
PG_RETURN_BOOL(res_nitems > 0);
#else
NO_XML_SUPPORT();
return 0;
#endif
}
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content functions to the core XML code. Per discussion, the former depends on XMLOPTION while the others do not. These supersede a version previously offered by contrib/xml2. Mike Fowler, reviewed by Pavel Stehule
2010-08-13 20:36:26 +02:00
/*
* Functions for checking well-formed-ness
*/
#ifdef USE_LIBXML
static bool
wellformed_xml(text *data, XmlOptionType xmloption_arg)
{
Clean up dubious error handling in wellformed_xml(). This ancient bit of code was summarily trapping any ereport longjmp whatsoever and assuming that it must represent an invalid-XML report. It's not really appropriate to handle OOM-like situations that way: maybe the input is valid or maybe not, but we couldn't find out. And it'd be a seriously bad idea to ignore, say, a query cancel error that way. (Perhaps that can't happen because there is no CHECK_FOR_INTERRUPTS anywhere within xml_parse, but even if that's true today it's obviously a very fragile assumption.) But in the wake of the previous commit, we can drop the PG_TRY here altogether, and use the soft error mechanism to catch only the kinds of errors that are legitimate to treat as invalid-XML. (This is our first use of the soft error mechanism for something not directly related to a datatype input function. It won't be the last.) xml_is_document can be converted in the same way. That one is not actively broken, because it was checking specifically for ERRCODE_INVALID_XML_DOCUMENT rather than trapping everything; but the code is still shorter and probably faster this way. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 17:10:36 +01:00
xmlDocPtr doc;
ErrorSaveContext escontext = {T_ErrorSaveContext};
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content functions to the core XML code. Per discussion, the former depends on XMLOPTION while the others do not. These supersede a version previously offered by contrib/xml2. Mike Fowler, reviewed by Pavel Stehule
2010-08-13 20:36:26 +02:00
Clean up dubious error handling in wellformed_xml(). This ancient bit of code was summarily trapping any ereport longjmp whatsoever and assuming that it must represent an invalid-XML report. It's not really appropriate to handle OOM-like situations that way: maybe the input is valid or maybe not, but we couldn't find out. And it'd be a seriously bad idea to ignore, say, a query cancel error that way. (Perhaps that can't happen because there is no CHECK_FOR_INTERRUPTS anywhere within xml_parse, but even if that's true today it's obviously a very fragile assumption.) But in the wake of the previous commit, we can drop the PG_TRY here altogether, and use the soft error mechanism to catch only the kinds of errors that are legitimate to treat as invalid-XML. (This is our first use of the soft error mechanism for something not directly related to a datatype input function. It won't be the last.) xml_is_document can be converted in the same way. That one is not actively broken, because it was checking specifically for ERRCODE_INVALID_XML_DOCUMENT rather than trapping everything; but the code is still shorter and probably faster this way. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 17:10:36 +01:00
/*
* We'll report "true" if no soft error is reported by xml_parse().
*/
doc = xml_parse(data, xmloption_arg, true,
Support [NO] INDENT option in XMLSERIALIZE(). This adds the ability to pretty-print XML documents ... according to libxml's somewhat idiosyncratic notions of what's pretty, anyway. One notable divergence from a strict reading of the spec is that libxml is willing to collapse empty nodes "<node></node>" to just "<node/>", whereas SQL and the underlying XML spec say that this option should only result in whitespace tweaks. Nonetheless, it seems close enough to justify using the SQL-standard syntax. Jim Jones, reviewed by Peter Smith and myself Discussion: https://postgr.es/m/2f5df461-dad8-6d7d-4568-08e10608a69b@uni-muenster.de
2023-03-15 21:58:59 +01:00
GetDatabaseEncoding(), NULL, NULL, (Node *) &escontext);
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content functions to the core XML code. Per discussion, the former depends on XMLOPTION while the others do not. These supersede a version previously offered by contrib/xml2. Mike Fowler, reviewed by Pavel Stehule
2010-08-13 20:36:26 +02:00
if (doc)
xmlFreeDoc(doc);
Clean up dubious error handling in wellformed_xml(). This ancient bit of code was summarily trapping any ereport longjmp whatsoever and assuming that it must represent an invalid-XML report. It's not really appropriate to handle OOM-like situations that way: maybe the input is valid or maybe not, but we couldn't find out. And it'd be a seriously bad idea to ignore, say, a query cancel error that way. (Perhaps that can't happen because there is no CHECK_FOR_INTERRUPTS anywhere within xml_parse, but even if that's true today it's obviously a very fragile assumption.) But in the wake of the previous commit, we can drop the PG_TRY here altogether, and use the soft error mechanism to catch only the kinds of errors that are legitimate to treat as invalid-XML. (This is our first use of the soft error mechanism for something not directly related to a datatype input function. It won't be the last.) xml_is_document can be converted in the same way. That one is not actively broken, because it was checking specifically for ERRCODE_INVALID_XML_DOCUMENT rather than trapping everything; but the code is still shorter and probably faster this way. Discussion: https://postgr.es/m/3564577.1671142683@sss.pgh.pa.us
2022-12-16 17:10:36 +01:00
return !escontext.error_occurred;
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content functions to the core XML code. Per discussion, the former depends on XMLOPTION while the others do not. These supersede a version previously offered by contrib/xml2. Mike Fowler, reviewed by Pavel Stehule
2010-08-13 20:36:26 +02:00
}
#endif
Datum
xml_is_well_formed(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *data = PG_GETARG_TEXT_PP(0);
Remove useless whitespace at end of lines
2010-11-23 21:27:50 +01:00
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content functions to the core XML code. Per discussion, the former depends on XMLOPTION while the others do not. These supersede a version previously offered by contrib/xml2. Mike Fowler, reviewed by Pavel Stehule
2010-08-13 20:36:26 +02:00
PG_RETURN_BOOL(wellformed_xml(data, xmloption));
#else
NO_XML_SUPPORT();
return 0;
#endif /* not USE_LIBXML */
}
Datum
xml_is_well_formed_document(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *data = PG_GETARG_TEXT_PP(0);
Remove useless whitespace at end of lines
2010-11-23 21:27:50 +01:00
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content functions to the core XML code. Per discussion, the former depends on XMLOPTION while the others do not. These supersede a version previously offered by contrib/xml2. Mike Fowler, reviewed by Pavel Stehule
2010-08-13 20:36:26 +02:00
PG_RETURN_BOOL(wellformed_xml(data, XMLOPTION_DOCUMENT));
#else
NO_XML_SUPPORT();
return 0;
#endif /* not USE_LIBXML */
}
Datum
xml_is_well_formed_content(PG_FUNCTION_ARGS)
{
#ifdef USE_LIBXML
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 00:35:34 +01:00
text *data = PG_GETARG_TEXT_PP(0);
Remove useless whitespace at end of lines
2010-11-23 21:27:50 +01:00
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content functions to the core XML code. Per discussion, the former depends on XMLOPTION while the others do not. These supersede a version previously offered by contrib/xml2. Mike Fowler, reviewed by Pavel Stehule
2010-08-13 20:36:26 +02:00
PG_RETURN_BOOL(wellformed_xml(data, XMLOPTION_CONTENT));
#else
NO_XML_SUPPORT();
return 0;
#endif /* not USE_LIBXML */
}
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
/*
* support functions for XMLTABLE
*
*/
#ifdef USE_LIBXML
/*
* Returns private data from executor state. Ensure validity by check with
* MAGIC number.
*/
static inline XmlTableBuilderData *
GetXmlTableBuilderPrivateData(TableFuncScanState *state, const char *fname)
{
XmlTableBuilderData *result;
if (!IsA(state, TableFuncScanState))
elog(ERROR, "%s called with invalid TableFuncScanState", fname);
result = (XmlTableBuilderData *) state->opaque;
if (result->magic != XMLTABLE_CONTEXT_MAGIC)
elog(ERROR, "%s called with invalid TableFuncScanState", fname);
return result;
}
#endif
/*
* XmlTableInitOpaque
* Fill in TableFuncScanState->opaque for XmlTable processor; initialize
* the XML parser.
*
* Note: Because we call pg_xml_init() here and pg_xml_done() in
* XmlTableDestroyOpaque, it is critical for robustness that no other
* executor nodes run until this node is processed to completion. Caller
* must execute this to completion (probably filling a tuplestore to exhaust
* this node in a single pass) instead of using row-per-call mode.
*/
static void
XmlTableInitOpaque(TableFuncScanState *state, int natts)
{
#ifdef USE_LIBXML
volatile xmlParserCtxtPtr ctxt = NULL;
XmlTableBuilderData *xtCxt;
PgXmlErrorContext *xmlerrcxt;
xtCxt = palloc0(sizeof(XmlTableBuilderData));
xtCxt->magic = XMLTABLE_CONTEXT_MAGIC;
xtCxt->natts = natts;
xtCxt->xpathscomp = palloc0(sizeof(xmlXPathCompExprPtr) * natts);
xmlerrcxt = pg_xml_init(PG_XML_STRICTNESS_ALL);
PG_TRY();
{
xmlInitParser();
ctxt = xmlNewParserCtxt();
if (ctxt == NULL || xmlerrcxt->err_occurred)
xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate parser context");
}
PG_CATCH();
{
if (ctxt != NULL)
xmlFreeParserCtxt(ctxt);
pg_xml_done(xmlerrcxt, true);
PG_RE_THROW();
}
PG_END_TRY();
xtCxt->xmlerrcxt = xmlerrcxt;
xtCxt->ctxt = ctxt;
state->opaque = xtCxt;
#else
NO_XML_SUPPORT();
#endif /* not USE_LIBXML */
}
/*
* XmlTableSetDocument
* Install the input document
*/
static void
XmlTableSetDocument(TableFuncScanState *state, Datum value)
{
#ifdef USE_LIBXML
XmlTableBuilderData *xtCxt;
xmltype *xmlval = DatumGetXmlP(value);
char *str;
xmlChar *xstr;
int length;
volatile xmlDocPtr doc = NULL;
volatile xmlXPathContextPtr xpathcxt = NULL;
xtCxt = GetXmlTableBuilderPrivateData(state, "XmlTableSetDocument");
/*
* Use out function for casting to string (remove encoding property). See
* comment in xml_out.
*/
str = xml_out_internal(xmlval, 0);
length = strlen(str);
xstr = pg_xmlCharStrndup(str, length);
PG_TRY();
{
Revert "Add support for parsing of large XML data (>= 10MB)" This reverts commit 2197d06224a1, following a discussion over a Coverity report where issues like the "Billion laugh attack" could cause the backend to waste CPU and memory even if a client applied checks on the size of the data given in input, and libxml2 does not offer guarantees that input limits are respected under XML_PARSE_HUGE. Discussion: https://postgr.es/m/ZbHlgrPLtBZyr_QW@paquier.xyz
2024-01-26 02:15:32 +01:00
doc = xmlCtxtReadMemory(xtCxt->ctxt, (char *) xstr, length, NULL, NULL, 0);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
if (doc == NULL || xtCxt->xmlerrcxt->err_occurred)
xml_ereport(xtCxt->xmlerrcxt, ERROR, ERRCODE_INVALID_XML_DOCUMENT,
"could not parse XML document");
xpathcxt = xmlXPathNewContext(doc);
if (xpathcxt == NULL || xtCxt->xmlerrcxt->err_occurred)
xml_ereport(xtCxt->xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
"could not allocate XPath context");
Set correct context for XPath evaluation According to the SQL standard, the context of XMLTABLE's XPath row_expression is the document node of the XML input document, not the root node. This becomes visible when a relative path rather than absolute is used as row expression. Absolute paths is what was used in original tests and docs (and the most common form used in examples throughout the interwebs), which explains why this wasn't noticed before. Other functions such as xpath() and xpath_exists() also have this problem. While not specified by the SQL standard, it would be pretty odd to leave those functions to behave differently than XMLTABLE, so change them too. However, this is a backwards-incompatible change. No backpatch, out of fear of breaking code depending on the original broken behavior. Author: Markus Winand Reported-By: Markus Winand Reviewed-by: Álvaro Herrera Discussion: https://postgr.es/m/0684A598-002C-42A2-AE12-F024A324EAE4@winand.at
2018-06-21 21:56:11 +02:00
xpathcxt->node = (xmlNodePtr) doc;
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
}
PG_CATCH();
{
if (xpathcxt != NULL)
xmlXPathFreeContext(xpathcxt);
if (doc != NULL)
xmlFreeDoc(doc);
PG_RE_THROW();
}
PG_END_TRY();
xtCxt->doc = doc;
xtCxt->xpathcxt = xpathcxt;
#else
NO_XML_SUPPORT();
#endif /* not USE_LIBXML */
}
/*
* XmlTableSetNamespace
* Add a namespace declaration
*/
static void
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
XmlTableSetNamespace(TableFuncScanState *state, const char *name, const char *uri)
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
#ifdef USE_LIBXML
XmlTableBuilderData *xtCxt;
if (name == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("DEFAULT namespace is not supported")));
xtCxt = GetXmlTableBuilderPrivateData(state, "XmlTableSetNamespace");
if (xmlXPathRegisterNs(xtCxt->xpathcxt,
pg_xmlCharStrndup(name, strlen(name)),
pg_xmlCharStrndup(uri, strlen(uri))))
xml_ereport(xtCxt->xmlerrcxt, ERROR, ERRCODE_DATA_EXCEPTION,
"could not set XML namespace");
#else
NO_XML_SUPPORT();
#endif /* not USE_LIBXML */
}
/*
* XmlTableSetRowFilter
* Install the row-filter Xpath expression.
*/
static void
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
XmlTableSetRowFilter(TableFuncScanState *state, const char *path)
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
#ifdef USE_LIBXML
XmlTableBuilderData *xtCxt;
xmlChar *xstr;
xtCxt = GetXmlTableBuilderPrivateData(state, "XmlTableSetRowFilter");
if (*path == '\0')
ereport(ERROR,
(errcode(ERRCODE_DATA_EXCEPTION),
errmsg("row path filter must not be empty string")));
xstr = pg_xmlCharStrndup(path, strlen(path));
xtCxt->xpathcomp = xmlXPathCompile(xstr);
if (xtCxt->xpathcomp == NULL || xtCxt->xmlerrcxt->err_occurred)
xml_ereport(xtCxt->xmlerrcxt, ERROR, ERRCODE_SYNTAX_ERROR,
"invalid XPath expression");
#else
NO_XML_SUPPORT();
#endif /* not USE_LIBXML */
}
/*
* XmlTableSetColumnFilter
* Install the column-filter Xpath expression, for the given column.
*/
static void
Add some const decorations to prototypes Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-31 15:34:31 +01:00
XmlTableSetColumnFilter(TableFuncScanState *state, const char *path, int colnum)
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
#ifdef USE_LIBXML
XmlTableBuilderData *xtCxt;
xmlChar *xstr;
Remove AssertArg and AssertState These don't offer anything over plain Assert, and their usage had already been declared obsolescent. Author: Nathan Bossart <nathandbossart@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/20221009210148.GA900071@nathanxps13
2022-10-28 09:19:06 +02:00
Assert(PointerIsValid(path));
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
xtCxt = GetXmlTableBuilderPrivateData(state, "XmlTableSetColumnFilter");
if (*path == '\0')
ereport(ERROR,
(errcode(ERRCODE_DATA_EXCEPTION),
errmsg("column path filter must not be empty string")));
xstr = pg_xmlCharStrndup(path, strlen(path));
xtCxt->xpathscomp[colnum] = xmlXPathCompile(xstr);
if (xtCxt->xpathscomp[colnum] == NULL || xtCxt->xmlerrcxt->err_occurred)
xml_ereport(xtCxt->xmlerrcxt, ERROR, ERRCODE_DATA_EXCEPTION,
"invalid XPath expression");
#else
NO_XML_SUPPORT();
#endif /* not USE_LIBXML */
}
/*
* XmlTableFetchRow
* Prepare the next "current" tuple for upcoming GetValue calls.
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 06:22:32 +02:00
* Returns false if the row-filter expression returned no more rows.
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
*/
static bool
XmlTableFetchRow(TableFuncScanState *state)
{
#ifdef USE_LIBXML
XmlTableBuilderData *xtCxt;
xtCxt = GetXmlTableBuilderPrivateData(state, "XmlTableFetchRow");
Simplify some comments in xml.c Author: Justin Pryzby Discussion: https://postgr.es/m/X/Ff7jfnvJUab013@paquier.xyz
2021-01-04 11:47:58 +01:00
/* Propagate our own error context to libxml2 */
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
xmlSetStructuredErrorFunc((void *) xtCxt->xmlerrcxt, xml_errorHandler);
if (xtCxt->xpathobj == NULL)
{
xtCxt->xpathobj = xmlXPathCompiledEval(xtCxt->xpathcomp, xtCxt->xpathcxt);
if (xtCxt->xpathobj == NULL || xtCxt->xmlerrcxt->err_occurred)
xml_ereport(xtCxt->xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
"could not create XPath object");
xtCxt->row_count = 0;
}
if (xtCxt->xpathobj->type == XPATH_NODESET)
{
if (xtCxt->xpathobj->nodesetval != NULL)
{
if (xtCxt->row_count++ < xtCxt->xpathobj->nodesetval->nodeNr)
return true;
}
}
return false;
#else
NO_XML_SUPPORT();
return false;
Suppress compiler warning in non-USE_LIBXML builds. Compilers that don't realize that ereport(ERROR) doesn't return complained that XmlTableGetValue() failed to return a value. Also, make XmlTableFetchRow's non-USE_LIBXML case look more like the other ones. As coded, it could lead to "unreachable code" warnings with USE_LIBXML enabled. Oversights in commit fcec6caaf. Per buildfarm.
2017-03-08 22:10:00 +01:00
#endif /* not USE_LIBXML */
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
}
/*
* XmlTableGetValue
* Return the value for column number 'colnum' for the current row. If
* column -1 is requested, return representation of the whole row.
*
* This leaks memory, so be sure to reset often the context in which it's
* called.
*/
static Datum
XmlTableGetValue(TableFuncScanState *state, int colnum,
Oid typid, int32 typmod, bool *isnull)
{
#ifdef USE_LIBXML
XmlTableBuilderData *xtCxt;
Datum result = (Datum) 0;
xmlNodePtr cur;
char *cstr = NULL;
volatile xmlXPathObjectPtr xpathobj = NULL;
xtCxt = GetXmlTableBuilderPrivateData(state, "XmlTableGetValue");
Assert(xtCxt->xpathobj &&
xtCxt->xpathobj->type == XPATH_NODESET &&
xtCxt->xpathobj->nodesetval != NULL);
Simplify some comments in xml.c Author: Justin Pryzby Discussion: https://postgr.es/m/X/Ff7jfnvJUab013@paquier.xyz
2021-01-04 11:47:58 +01:00
/* Propagate our own error context to libxml2 */
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
xmlSetStructuredErrorFunc((void *) xtCxt->xmlerrcxt, xml_errorHandler);
*isnull = false;
cur = xtCxt->xpathobj->nodesetval->nodeTab[xtCxt->row_count - 1];
Assert(xtCxt->xpathscomp[colnum] != NULL);
PG_TRY();
{
/* Set current node as entry point for XPath evaluation */
Fix XMLTABLE on older libxml2 libxml2 older than 2.9.1 does not have xmlXPathSetContextNode (released in 2013, so reasonable platforms have trouble). That function is fairly trivial, so I have inlined it in the one added caller. This passes tests on my machine; let's see what the buildfarm thinks about it. Per joint complaint from Tom Lane and buildfarm.
2017-03-08 17:29:48 +01:00
xtCxt->xpathcxt->node = cur;
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
/* Evaluate column path */
xpathobj = xmlXPathCompiledEval(xtCxt->xpathscomp[colnum], xtCxt->xpathcxt);
if (xpathobj == NULL || xtCxt->xmlerrcxt->err_occurred)
xml_ereport(xtCxt->xmlerrcxt, ERROR, ERRCODE_INTERNAL_ERROR,
"could not create XPath object");
/*
* There are four possible cases, depending on the number of nodes
* returned by the XPath expression and the type of the target column:
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
* a) XPath returns no nodes. b) The target type is XML (return all
* as XML). For non-XML return types: c) One node (return content).
* d) Multiple nodes (error).
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
*/
if (xpathobj->type == XPATH_NODESET)
{
int count = 0;
if (xpathobj->nodesetval != NULL)
count = xpathobj->nodesetval->nodeNr;
if (xpathobj->nodesetval == NULL || count == 0)
{
*isnull = true;
}
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
else
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
if (typid == XMLOID)
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
text *textstr;
StringInfoData str;
/* Concatenate serialized values */
initStringInfo(&str);
for (int i = 0; i < count; i++)
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
textstr =
xml_xmlnodetoxmltype(xpathobj->nodesetval->nodeTab[i],
xtCxt->xmlerrcxt);
appendStringInfoText(&str, textstr);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
}
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
cstr = str.data;
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
}
else
{
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
xmlChar *str;
if (count > 1)
ereport(ERROR,
(errcode(ERRCODE_CARDINALITY_VIOLATION),
errmsg("more than one value returned by column XPath expression")));
str = xmlXPathCastNodeSetToString(xpathobj->nodesetval);
cstr = str ? xml_pstrdup_and_free(str) : "";
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
}
}
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
}
else if (xpathobj->type == XPATH_STRING)
{
/* Content should be escaped when target will be XML */
if (typid == XMLOID)
cstr = escape_xml((char *) xpathobj->stringval);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
else
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
cstr = (char *) xpathobj->stringval;
}
else if (xpathobj->type == XPATH_BOOLEAN)
{
char typcategory;
bool typispreferred;
xmlChar *str;
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
/* Allow implicit casting from boolean to numbers */
get_type_category_preferred(typid, &typcategory, &typispreferred);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
if (typcategory != TYPCATEGORY_NUMERIC)
str = xmlXPathCastBooleanToString(xpathobj->boolval);
else
str = xmlXPathCastNumberToString(xmlXPathCastBooleanToNumber(xpathobj->boolval));
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
cstr = xml_pstrdup_and_free(str);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
}
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
else if (xpathobj->type == XPATH_NUMBER)
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
Fix minor deficiencies in XMLTABLE, xpath(), xmlexists() Correctly process nodes of more types than previously. In some cases, nodes were being ignored (nothing was output); in other cases, trying to return them resulted in errors about unrecognized nodes. In yet other cases, necessary escaping (of XML special characters) was not being done. Fix all those (as far as the authors could find) and add regression tests cases verifying the new behavior. I (Álvaro) was of two minds about backpatching these changes. They do seem bugfixes that would benefit most users of the affected functions; but on the other hand it would change established behavior in minor releases, so it seems prudent not to. Authors: Pavel Stehule, Markus Winand, Chapman Flack Discussion: https://postgr.es/m/CAFj8pRA6J25CtAZ2TuRvxK3gat7-bBUYh0rfE2yM7Hj9GD14Dg@mail.gmail.com https://postgr.es/m/8BDB0627-2105-4564-AA76-7849F028B96E@winand.at The elephant in the room as pointed out by Chapman Flack, not fixed in this commit, is that we still have XMLTABLE operating on XPath 1.0 instead of the standard-mandated XQuery (or even its subset XPath 2.0). Fixing that is a major undertaking, however.
2019-03-07 19:21:56 +01:00
xmlChar *str;
str = xmlXPathCastNumberToString(xpathobj->floatval);
cstr = xml_pstrdup_and_free(str);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
}
else
elog(ERROR, "unexpected XPath object type %u", xpathobj->type);
/*
* By here, either cstr contains the result value, or the isnull flag
* has been set.
*/
Assert(cstr || *isnull);
if (!*isnull)
result = InputFunctionCall(&state->in_functions[colnum],
cstr,
state->typioparams[colnum],
typmod);
}
PG_FINALLY This gives an alternative way of catching exceptions, for the common case where the cleanup code is the same in the error and non-error cases. So instead of PG_TRY(); { ... code that might throw ereport(ERROR) ... } PG_CATCH(); { cleanup(); PG_RE_THROW(); } PG_END_TRY(); cleanup(); one can write PG_TRY(); { ... code that might throw ereport(ERROR) ... } PG_FINALLY(); { cleanup(); } PG_END_TRY(); Discussion: https://www.postgresql.org/message-id/flat/95a822c3-728b-af0e-d7e5-71890507ae0c%402ndquadrant.com
2019-11-01 11:09:52 +01:00
PG_FINALLY();
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
{
if (xpathobj != NULL)
xmlXPathFreeObject(xpathobj);
}
PG_END_TRY();
return result;
#else
NO_XML_SUPPORT();
Suppress compiler warning in non-USE_LIBXML builds. Compilers that don't realize that ereport(ERROR) doesn't return complained that XmlTableGetValue() failed to return a value. Also, make XmlTableFetchRow's non-USE_LIBXML case look more like the other ones. As coded, it could lead to "unreachable code" warnings with USE_LIBXML enabled. Oversights in commit fcec6caaf. Per buildfarm.
2017-03-08 22:10:00 +01:00
return 0;
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
#endif /* not USE_LIBXML */
}
/*
* XmlTableDestroyOpaque
* Release all libxml2 resources
*/
static void
XmlTableDestroyOpaque(TableFuncScanState *state)
{
#ifdef USE_LIBXML
XmlTableBuilderData *xtCxt;
xtCxt = GetXmlTableBuilderPrivateData(state, "XmlTableDestroyOpaque");
Simplify some comments in xml.c Author: Justin Pryzby Discussion: https://postgr.es/m/X/Ff7jfnvJUab013@paquier.xyz
2021-01-04 11:47:58 +01:00
/* Propagate our own error context to libxml2 */
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
xmlSetStructuredErrorFunc((void *) xtCxt->xmlerrcxt, xml_errorHandler);
if (xtCxt->xpathscomp != NULL)
{
int i;
for (i = 0; i < xtCxt->natts; i++)
if (xtCxt->xpathscomp[i] != NULL)
xmlXPathFreeCompExpr(xtCxt->xpathscomp[i]);
}
if (xtCxt->xpathobj != NULL)
xmlXPathFreeObject(xtCxt->xpathobj);
if (xtCxt->xpathcomp != NULL)
xmlXPathFreeCompExpr(xtCxt->xpathcomp);
if (xtCxt->xpathcxt != NULL)
xmlXPathFreeContext(xtCxt->xpathcxt);
if (xtCxt->doc != NULL)
xmlFreeDoc(xtCxt->doc);
if (xtCxt->ctxt != NULL)
xmlFreeParserCtxt(xtCxt->ctxt);
pg_xml_done(xtCxt->xmlerrcxt, true);
/* not valid anymore */
xtCxt->magic = 0;
state->opaque = NULL;
#else
NO_XML_SUPPORT();
#endif /* not USE_LIBXML */
}