Commit Graph

30 Commits

Author SHA1 Message Date
Tom Lane 08361cea2b Fix null-pointer-deref crash while doing COPY IN with check constraints.
In commit bf7ca15875 I introduced an
assumption that an RTE referenced by a whole-row Var must have a valid eref
field.  This is false for RTEs constructed by DoCopy, and there are other
places taking similar shortcuts.  Perhaps we should make all those places
go through addRangeTableEntryForRelation or its siblings instead of having
ad-hoc logic, but the most reliable fix seems to be to make the new code in
ExecEvalWholeRowVar cope if there's no eref.  We can reasonably assume that
there's no need to insert column aliases if no aliases were provided.

Add a regression test case covering this, and also verifying that a sane
column name is in fact available in this situation.

Although the known case only crashes in 9.4 and HEAD, it seems prudent to
back-patch the code change to 9.2, since all the ingredients for a similar
failure exist in the variant patch applied to 9.3 and 9.2.

Per report from Jean-Pierre Pelletier.
2015-02-15 23:26:45 -05:00
Bruce Momjian 7ec73783d8 copy: update docs for FORCE_NULL and FORCE_NOT_NULL combination
Also update regression tests

Patch by Michael Paquier
2014-04-22 16:06:37 -04:00
Andrew Dunstan 3b5e03dca2 Provide a FORCE NULL option to COPY in CSV mode.
This forces an input field containing the quoted null string to be
returned as a NULL. Without this option, only unquoted null strings
behave this way. This helps where some CSV producers insist on quoting
every field, whether or not it is needed. The option takes a list of
fields, and only applies to those columns. There is an equivalent
column-level option added to file_fdw.

Ian Barwick, with some tweaking by Andrew Dunstan, reviewed by Payal
Singh.
2014-03-04 17:31:59 -05:00
Bruce Momjian 4deb57de7d Issue ERROR if FREEZE mode can't be honored by COPY
Previously non-honored FREEZE mode was ignored.  This also issues an
appropriate error message based on the cause of the failure, per
suggestion from Tom.  Additional regression test case added.
2013-01-26 13:33:24 -05:00
Simon Riggs 1f023f9297 Optimize COPY FREEZE with CREATE TABLE also.
Jeff Davis, additional test by me
2012-12-07 13:26:52 +00:00
Simon Riggs 5457a130d3 Reduce scope of changes for COPY FREEZE.
Allow support only for freezing tuples by explicit
command. Previous coding mistakenly extended
slightly beyond what was agreed as correct on -hackers.
So essentially a partial revoke of earlier work,
leaving just the COPY FREEZE command.
2012-12-02 20:52:52 +00:00
Simon Riggs ddf509eb4a Tweak tests in COPY FREEZE 2012-12-01 13:46:41 +00:00
Simon Riggs 8de72b66a2 COPY FREEZE and mark committed on fresh tables.
When a relfilenode is created in this subtransaction or
a committed child transaction and it cannot otherwise
be seen by our own process, mark tuples committed ahead
of transaction commit for all COPY commands in same
transaction. If FREEZE specified on COPY
and pre-conditions met then rows will also be frozen.
Both options designed to avoid revisiting rows after commit,
increasing performance of subsequent commands after
data load and upgrade. pg_restore changes later.

Simon Riggs, review comments from Heikki Linnakangas, Noah Misch and design
input from Tom Lane, Robert Haas and Kevin Grittner
2012-12-01 12:54:20 +00:00
Tom Lane e8476f46fc Fix COPY FROM for null marker strings that correspond to invalid encoding.
The COPY documentation says "COPY FROM matches the input against the null
string before removing backslashes".  It is therefore reasonable to presume
that null markers like E'\\0' will work ... and they did, until someone put
the tests in the wrong order during microoptimization-driven rewrites.
Since then, we've been failing if the null marker is something that would
de-escape to an invalidly-encoded string.  Since null markers generally
need to be something that can't appear in the data, this represents a
nontrivial loss of functionality; surprising nobody noticed it earlier.

Per report from Jeff Davis.  Backpatch to 8.4 where this got broken.
2012-03-25 23:17:22 -04:00
Itagaki Takahiro 3cba8240a1 Add ENCODING option to COPY TO/FROM and file_fdw.
File encodings can be specified separately from client encoding.
If not specified, client encoding is used for backward compatibility.

Cases when the encoding doesn't match client encoding are slower
than matched cases because we don't have conversion procs for other
encodings. Performance improvement would be be a future work.

Original patch by Hitoshi Harada, and modified by me.
2011-02-21 14:32:40 +09:00
Tom Lane 50a90fac40 Stamp HEAD as 9.0devel, and update various places that were referring to 8.5
(hope I got 'em all).  Per discussion, this release will be 9.0 not 8.5.
2010-02-17 04:19:41 +00:00
Tom Lane 923413ac6d Define a new, more extensible syntax for COPY options.
This is intentionally similar to the recently revised syntax for EXPLAIN
options, ie, (name value, ...).  The old syntax is still supported for
backwards compatibility, but we intend that any options added in future
will be provided only in the new syntax.

Robert Haas, Emmanuel Cecchet
2009-09-21 20:10:21 +00:00
Andrew Dunstan de7531a971 Allow * as parameter for FORCE QUOTE for COPY CSV. Itagaki Takahiro. 2009-07-25 00:07:14 +00:00
Tom Lane f51c9a6665 Change test tables in copy2 regression test to be temporary tables.
This prevents autovacuum from reclaiming free space in them and causing
the test's output row order to change, which is causing intermittent
bogus failure reports in the buildfarm.

Backpatch to 8.3.  The issue exists further back, but since autovacuum was
not on by default before 8.3, it's not a problem for buildfarm testing.
2009-06-14 00:00:24 +00:00
Peter Eisentraut 7f4f42fa10 Clean up CREATE FUNCTION syntax usage in contrib and elsewhere, in
particular get rid of single quotes around language names and old WITH ()
construct.
2006-02-27 16:09:50 +00:00
Bruce Momjian 54e5caa440 Add COPY CSV test that tests CSV output of \. 2005-12-28 15:07:02 +00:00
Bruce Momjian 87289ff35c Add regression tests for CSV and \., and add automatic quoting of a
single column dump that has a \. value, so the load works properly.  I
also added documentation describing this issue.
2005-12-28 03:25:32 +00:00
Bruce Momjian bb3cce4ec9 Add E'' syntax so eventually normal strings can treat backslashes
literally.

Add GUC variables:

        "escape_string_warning" - warn about backslashes in non-E strings
        "escape_string_syntax" - supports E'' syntax?
        "standard_compliant_strings" - treats backslashes literally in ''

Update code to use E'' when escapes are used.
2005-06-26 03:04:37 +00:00
Neil Conway 167bc6c621 Add regression test for consecutive newlines in COPY CSV mode. (There is
no bug related to this functionality in HEAD, but it's worth adding a test
for anyway.) From Andrew Dunstan.
2005-05-13 06:33:40 +00:00
Neil Conway 5df3fc67a7 This patch updates the regression tests to allow "make installcheck" to
pass if "default_with_oids" is set to false. I took the approach of
explicitly adding WITH OIDS to the CREATE TABLEs where necessary, rather
than tweaking the default_with_oids GUC var.
2005-01-22 05:12:33 +00:00
Tom Lane 45616f5bbb Clean up generation of default names for constraints, indexes, and serial
sequences, as per recent discussion.  All these names are now of the
form table_column_type, with digits added if needed to make them unique.
Default constraint names are chosen to be unique across their whole schema,
not just within the parent object, so as to be more SQL-spec-compatible
and make the information schema views more useful.
2004-06-10 17:56:03 +00:00
Bruce Momjian 659660c116 Add regression tests for CSV COPY format. 2004-06-09 23:39:24 +00:00
Bruce Momjian 962987a1c2 Add NULL regression tests.
Manfred Koizar
2003-10-10 03:46:51 +00:00
Tom Lane f85f43dfb5 Backend support for autocommit removed, per recent discussions. The
only remnant of this failed experiment is that the server will take
SET AUTOCOMMIT TO ON.  Still TODO: provide some client-side autocommit
logic in libpq.
2003-05-14 03:26:03 +00:00
Bruce Momjian 5c6a5fe18b Make regression tests safe for autocommit = 'off'. 2002-10-19 01:35:43 +00:00
Tom Lane b663f3443b Add a bunch of pseudo-types to replace the behavior formerly associated
with OPAQUE, as per recent pghackers discussion.  I still want to do some
more work on the 'cstring' pseudo-type, but I'm going to commit the bulk
of the changes now before the tree starts shifting under me ...
2002-08-22 00:01:51 +00:00
Tom Lane 38bb77a5d1 ALTER TABLE DROP COLUMN works. Patch by Christopher Kings-Lynne,
code review by Tom Lane.  Remaining issues: functions that take or
return tuple types are likely to break if one drops (or adds!)
a column in the table defining the type.  Need to think about what
to do here.

Along the way: some code review for recent COPY changes; mark system
columns attnotnull = true where appropriate, per discussion a month ago.
2002-08-02 18:15:10 +00:00
Bruce Momjian 874148fe34 IMPROVED VERSION APPLIED:
The attached patch completes the following TODO item:

    * Generate failure on short COPY lines rather than pad NULLs

I also restructed a lot of the existing COPY code, did some code
review on the column list patch sent in by Brent Verner a little
while ago, and added some regression tests. I also added an
explicit check (and resultant error) for extra data before
the end-of-line.

Neil Conway
2002-07-30 16:55:06 +00:00
Bruce Momjian 776fc8cc5c New depend code doesn't need sequence drop. 2002-07-18 22:31:44 +00:00
Bruce Momjian 2c48b3db95 here are the copy2.sql and copy2.out files for the new regression
tests

Brent Verner
2002-07-18 04:45:51 +00:00