Commit Graph

115 Commits

Author SHA1 Message Date
Tom Lane
41b9c8452b Replace libpq's "row processor" API with a "single row" mode.
After taking awhile to digest the row-processor feature that was added to
libpq in commit 92785dac2e, we've concluded
it is over-complicated and too hard to use.  Leave the core infrastructure
changes in place (that is, there's still a row processor function inside
libpq), but remove the exposed API pieces, and instead provide a "single
row" mode switch that causes PQgetResult to return one row at a time in
separate PGresult objects.

This approach incurs more overhead than proper use of a row processor
callback would, since construction of a PGresult per row adds extra cycles.
However, it is far easier to use and harder to break.  The single-row mode
still affords applications the primary benefit that the row processor API
was meant to provide, namely not having to accumulate large result sets in
memory before processing them.  Preliminary testing suggests that we can
probably buy back most of the extra cycles by micro-optimizing construction
of the extra results, but that task will be left for another day.

Marko Kreen
2012-08-02 13:10:30 -04:00
Bruce Momjian
927d61eeff Run pgindent on 9.2 source tree in preparation for first 9.3
commit-fest.
2012-06-10 15:20:04 -04:00
Robert Haas
e4f06b70c9 Another typographical correction.
Noted by Guillaume Smet.
2012-04-24 08:15:45 -04:00
Robert Haas
5d4b60f2f2 Lots of doc corrections.
Josh Kupershmidt
2012-04-23 22:43:09 -04:00
Tom Lane
6f922ef88e Improve efficiency of dblink by using libpq's new row processor API.
This patch provides a test case for libpq's row processor API.
contrib/dblink can deal with very large result sets by dumping them into
a tuplestore (which can spill to disk) --- but until now, the intermediate
storage of the query result in a PGresult meant memory bloat for any large
result.  Now we use a row processor to convert the data to tuple form and
dump it directly into the tuplestore.

A limitation is that this only works for plain dblink() queries, not
dblink_send_query() followed by dblink_get_result().  In the latter
case we don't know the desired tuple rowtype soon enough.  While hack
solutions to that are possible, a different user-level API would
probably be a better answer.

Kyotaro Horiguchi, reviewed by Marko Kreen and Tom Lane
2012-04-04 18:39:08 -04:00
Tom Lane
d843ed2116 Fix a couple of contrib/dblink bugs.
dblink_exec leaked temporary database connections if any error occurred
after connection setup, for example
	SELECT dblink_exec('...connect string...', 'select 1/0');
Add a PG_TRY block to ensure PQfinish gets done when it is needed.
(dblink_record_internal is on the hairy edge of needing similar treatment,
but seems not to be actively broken at the moment.)

Also, in 9.0 and up, only one of the three functions using tuplestore
return mode was properly checking that the query context would allow
a tuplestore result.

Noted while reviewing dblink patch.  Back-patch to all supported branches.
2012-04-03 20:43:15 -04:00
Tom Lane
b75fbe9191 Fix dblink's failure to report correct connection name in error messages.
The DBLINK_GET_CONN and DBLINK_GET_NAMED_CONN macros did not set the
surrounding function's conname variable, causing errors to be incorrectly
reported as having occurred on the "unnamed" connection in some cases.
This bug was actually visible in two cases in the regression tests,
but apparently whoever added those cases wasn't paying attention.

Noted by Kyotaro Horiguchi, though this is different from his proposed
patch.

Back-patch to 8.4; 8.3 does not have the same type of error reporting
so the patch is not relevant.
2012-03-29 17:52:28 -04:00
Bruce Momjian
e126958c2e Update copyright notices for year 2012. 2012-01-01 18:01:58 -05:00
Bruce Momjian
6416a82a62 Remove unnecessary #include references, per pgrminclude script. 2011-09-01 10:04:27 -04:00
Tom Lane
1af37ec96d Replace errdetail("%s", ...) with errdetail_internal("%s", ...).
There may be some other places where we should use errdetail_internal,
but they'll have to be evaluated case-by-case.  This commit just hits
a bunch of places where invoking gettext is obviously a waste of cycles.
2011-07-16 14:22:18 -04:00
Alvaro Herrera
b93f5a5673 Move Trigger and TriggerDesc structs out of rel.h into a new reltrigger.h
This lets us stop including rel.h into execnodes.h, which is a widely
used header.
2011-07-04 14:35:58 -04:00
Joe Conway
8af3596d6b Async dblink functions require a named connection, and therefore should
use DBLINK_GET_NAMED_CONN rather than DBLINK_GET_CONN.
Problem found by Peter Eisentraut and patch by Fujii Masao.
2011-06-25 15:58:07 -07:00
Bruce Momjian
5d950e3b0c Stamp copyrights for year 2011. 2011-01-01 13:18:15 -05:00
Itagaki Takahiro
d3c1265443 Don't raise "identifier will be truncated" messages in dblink
except creating new connections.
2010-11-25 19:40:58 +09:00
Robert Haas
4343c0e546 Expose quote_literal_cstr() from core.
This eliminates the need for inefficient implementions of this
functionality in both contrib/dblink and contrib/tablefunc, so remove
them.  The upcoming patch implementing an in-core format() function
will also require this functionality.

In passing, add some regression tests.
2010-11-20 10:04:48 -05:00
Magnus Hagander
9f2e211386 Remove cvs keywords from all files. 2010-09-20 22:08:53 +02:00
Bruce Momjian
239d769e7e pgindent run for 9.0, second run 2010-07-06 19:19:02 +00:00
Tom Lane
c0989c67fa Change the interpretation of the primary_key_attnums parameter of
dblink_build_sql_insert() and related functions.  Now the column numbers
are treated as logical not physical column numbers.  This will provide saner
behavior in the presence of dropped columns; furthermore, if we ever get
around to allowing rearrangement of logical column ordering, the original
definition would become nearly untenable from a usability standpoint.
Per recent discussion of dblink's handling of dropped columns.
Not back-patched for fear of breaking existing applications.
2010-06-15 20:29:01 +00:00
Tom Lane
3b3706d2cf Fix dblink_build_sql_insert() and related functions to handle dropped
columns correctly.  In passing, get rid of some dead logic in the
underlying get_sql_insert() etc functions --- there is no caller that
will pass null value-arrays to them.

Per bug report from Robert Voinea.
2010-06-15 19:04:15 +00:00
Tom Lane
48a7245391 Consolidate and improve checking of key-column-attnum arguments for
dblink_build_sql_insert() and related functions.  In particular, be sure to
reject references to dropped and out-of-range column numbers.  The numbers
are still interpreted as physical column numbers, though, for backward
compatibility.

This patch replaces Joe's patch of 2010-02-03, which handled only some aspects
of the problem.
2010-06-15 16:22:19 +00:00
Tom Lane
6bbaa3148d Rearrange dblink's dblink_build_sql_insert() and related routines to open and
lock the target relation just once per SQL function call.  The original coding
obtained and released lock several times per call.  Aside from saving a
not-insignificant number of cycles, this eliminates possible race conditions
if someone tries to modify the relation's schema concurrently.  Also
centralize locking and permission-checking logic.

Problem noted while investigating a trouble report from Robert Voinea --- his
problem is still to be fixed, though.
2010-06-14 20:49:33 +00:00
Itagaki Takahiro
5a41c317c5 Avoid "identifier will be truncated" warning in dblink
when connection string is longer than NAMEDATALEN.
The previous fix for long connection name broke the behavior.
2010-06-09 03:39:26 +00:00
Itagaki Takahiro
a624356072 Fix connection leak in dblink when dblink_connect() or dblink_connect_u()
end with "duplicate connection name" errors.

Backported to release 7.4.
2010-06-09 00:56:02 +00:00
Itagaki Takahiro
7dff7260af Fix dblink to treat connection names longer than NAMEDATALEN-2 (62 bytes).
Now long names are adjusted with truncate_identifier() and NOTICE messages
are raised if names are actually truncated.

Backported to release 8.0.
2010-06-03 09:38:33 +00:00
Bruce Momjian
65e806cba1 pgindent run for 9.0 2010-02-26 02:01:40 +00:00
Itagaki Takahiro
912eb88c7d Remove useless codes to initialize TupleDesc from dblink_exec. 2010-02-24 05:20:49 +00:00
Robert Haas
e26c539e9f Wrap calls to SearchSysCache and related functions using macros.
The purpose of this change is to eliminate the need for every caller
of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists,
GetSysCacheOid, and SearchSysCacheList to know the maximum number
of allowable keys for a syscache entry (currently 4).  This will
make it far easier to increase the maximum number of keys in a
future release should we choose to do so, and it makes the code
shorter, too.

Design and review by Tom Lane.
2010-02-14 18:42:19 +00:00
Joe Conway
fdac8cf998 Check to ensure the number of primary key fields supplied does not
exceed the total number of non-dropped source table fields for
dblink_build_sql_*(). Addresses bug report from Rushabh Lathia.

Backpatch all the way to the 7.3 branch.
2010-02-03 23:01:11 +00:00
Joe Conway
56cbb611ec Rewrite dblink_record_internal() and dblink_fetch() to use a tuplestore
(SFRM_Materialize mode) to return tuples. Since we don't return from the
dblink function in tuplestore mode, release the PGresult with a PG_CATCH
block on error. Also rearrange to share the same code to materialize the
tuplestore. Patch by Takahiro Itagaki.
2010-01-24 22:19:38 +00:00
Bruce Momjian
0239800893 Update copyright for the year 2010. 2010-01-02 16:58:17 +00:00
Heikki Linnakangas
84d723b6ce Previous fix for temporary file management broke returning a set from
PL/pgSQL function within an exception handler. Make sure we use the right
resource owner when we create the tuplestore to hold returned tuples.

Simplify tuplestore API so that the caller doesn't need to be in the right
memory context when calling tuplestore_put* functions. tuplestore.c
automatically switches to the memory context used when the tuplestore was
created. Tuplesort was already modified like this earlier. This patch also
removes the now useless MemoryContextSwitch calls from callers.

Report by Aleksei on pgsql-bugs on Dec 22 2009. Backpatch to 8.1, like
the previous patch that broke this.
2009-12-29 17:40:59 +00:00
Joe Conway
d6119d8091 plug dblink resource leak
dblink generates orphaned connections when called with a connection string,
fail_on_error = true, and an ERROR occurs. Discovery and patch by
Tatsuhito Kasahara. Introduced in 8.4.
2009-09-12 23:20:52 +00:00
Joe Conway
f4095b4c4b Implement dblink_get_notify().
Adds the ability to retrieve async notifications using dblink,
via the addition of the function dblink_get_notify(). Original patch
by Marcus Kempe, suggestions by Tom Lane and Alvaro Herrera, patch
review and adjustments by Joe Conway.
2009-08-05 16:11:07 +00:00
Bruce Momjian
d747140279 8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list
provided by Andrew.
2009-06-11 14:49:15 +00:00
Tom Lane
f3e122fcdf Restore dblink_current_query() to being a C-language function, so as to not
create an ABI break between 8.3 and 8.4.  It is still just a wrapper around
the built-in current_query() function, but at a different implementation
level.  Per my proposal.

Note: this change doesn't break 8.4beta installations, since their
SQL-language definition of the function still works fine.
2009-06-09 17:41:02 +00:00
Joe Conway
e5de601267 Default client encoding to server encoding for dblink connections. Addresses
issue raised by Ruzsinszky Attila and confirmed by others.

 ----------------------------------------------------------------------
2009-06-09 16:35:36 +00:00
Joe Conway
4334695b30 Add support for using SQL/MED compliant FOREIGN DATA WRAPPER, SERVER,
and USER MAPPING as method to supply dblink connect parameters. Per
mailing list and PGCon discussions.
2009-06-06 21:27:56 +00:00
Joe Conway
db02073305 Fix dblink_get_result() as reported by Oleksiy Shchukin. Refactor a bit
while we're at it per request by Tom Lane. Specifically, don't try to
perform dblink_send_query() via dblink_record_internal() -- it was
inappropriate and ugly.
2009-06-02 03:21:56 +00:00
Bruce Momjian
511db38ace Update copyright for 2009. 2009-01-01 17:24:05 +00:00
Tom Lane
76cc2fe6a1 Fix dblink and tablefunc to not return with the wrong CurrentMemoryContext.
Per buildfarm results.
2008-11-30 23:23:52 +00:00
Tom Lane
cae7ad906a Fix dblink_connect() so that it verifies that a password is supplied in the
conninfo string *before* trying to connect to the remote server, not after.
As pointed out by Marko Kreen, in certain not-very-plausible situations
this could result in sending a password from the postgres user's .pgpass file,
or other places that non-superusers shouldn't have access to, to an
untrustworthy remote server.  The cleanest fix seems to be to expose libpq's
conninfo-string-parsing code so that dblink can check for a password option
without duplicating the parsing logic.

Joe Conway, with a little cleanup by Tom Lane
2008-09-22 13:55:14 +00:00
Joe Conway
c0241b9573 When an ERROR happens on a dblink remote connection, take
pains to pass the ERROR message components locally, including
using the passed SQLSTATE. Also wrap the passed info in an
appropriate CONTEXT message. Addresses complaint by Henry
Combrinck. Joe Conway, with much good advice from Tom Lane.
2008-07-03 03:56:57 +00:00
Bruce Momjian
4f22db501e Remove unneed #include now that current_query() has moved to the backend. 2008-04-04 17:02:56 +00:00
Bruce Momjian
f96928fde9 Implement current_query(), that shows the currently executing query.
At the same time remove dblink/dblink_current_query() as it is no longer
necessary
*BACKWARD COMPATIBILITY ISSUE* for dblink

Tomas Doran
2008-04-04 16:57:21 +00:00
Alvaro Herrera
73b0300b2a Move the HTSU_Result enum definition into snapshot.h, to avoid including
tqual.h into heapam.h.  This makes all inclusion of tqual.h explicit.

I also sorted alphabetically the includes on some source files.
2008-03-26 21:10:39 +00:00
Tom Lane
220db7ccd8 Simplify and standardize conversions between TEXT datums and ordinary C
strings.  This patch introduces four support functions cstring_to_text,
cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and
two macros CStringGetTextDatum and TextDatumGetCString.  A number of
existing macros that provided variants on these themes were removed.

Most of the places that need to make such conversions now require just one
function or macro call, in place of the multiple notational layers that used
to be needed.  There are no longer any direct calls of textout or textin,
and we got most of the places that were using handmade conversions via
memcpy (there may be a few still lurking, though).

This commit doesn't make any serious effort to eliminate transient memory
leaks caused by detoasting toasted text objects before they reach
text_to_cstring.  We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few
places where it was easy, but much more could be done.

Brendan Jurd and Tom Lane
2008-03-25 22:42:46 +00:00
Tom Lane
66661858ae Use an indexscan not a heapscan to search pg_index in get_pkey_attnames.
Noted while looking for heapscans that might need to start from block
zero.
2008-01-14 02:49:47 +00:00
Tom Lane
919c9f6cce The original patch to disallow non-passworded connections to non-superusers
failed to cover all the ways in which a connection can be initiated in dblink.
Plug the remaining holes.  Also, disallow transient connections in functions
for which that feature makes no sense (because they are only sensible as
part of a sequence of operations on the same connection).  Joe Conway

Security: CVE-2007-6601
2008-01-03 21:27:59 +00:00
Bruce Momjian
9098ab9e32 Update copyrights in source tree to 2008. 2008-01-01 19:46:01 +00:00
Bruce Momjian
fdf5a5efb7 pgindent run for 8.3. 2007-11-15 21:14:46 +00:00