mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-10-01 11:31:21 +02:00
3b5e03dca2
This forces an input field containing the quoted null string to be returned as a NULL. Without this option, only unquoted null strings behave this way. This helps where some CSV producers insist on quoting every field, whether or not it is needed. The option takes a list of fields, and only applies to those columns. There is an equivalent column-level option added to file_fdw. Ian Barwick, with some tweaking by Andrew Dunstan, reviewed by Payal Singh.
250 lines
6.5 KiB
Plaintext
250 lines
6.5 KiB
Plaintext
<!-- doc/src/sgml/file-fdw.sgml -->
|
|
|
|
<sect1 id="file-fdw" xreflabel="file_fdw">
|
|
<title>file_fdw</title>
|
|
|
|
<indexterm zone="file-fdw">
|
|
<primary>file_fdw</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
The <filename>file_fdw</> module provides the foreign-data wrapper
|
|
<function>file_fdw</function>, which can be used to access data
|
|
files in the server's file system. Data files must be in a format
|
|
that can be read by <command>COPY FROM</command>;
|
|
see <xref linkend="sql-copy"> for details.
|
|
Access to such data files is currently read-only.
|
|
</para>
|
|
|
|
<para>
|
|
A foreign table created using this wrapper can have the following options:
|
|
</para>
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term><literal>filename</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies the file to be read. Required. Must be an absolute path name.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>format</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies the file's format,
|
|
the same as <command>COPY</>'s <literal>FORMAT</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>header</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies whether the file has a header line,
|
|
the same as <command>COPY</>'s <literal>HEADER</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>delimiter</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies the file's delimiter character,
|
|
the same as <command>COPY</>'s <literal>DELIMITER</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>quote</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies the file's quote character,
|
|
the same as <command>COPY</>'s <literal>QUOTE</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>escape</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies the file's escape character,
|
|
the same as <command>COPY</>'s <literal>ESCAPE</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>null</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies the file's null string,
|
|
the same as <command>COPY</>'s <literal>NULL</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>encoding</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
Specifies the file's encoding,
|
|
the same as <command>COPY</>'s <literal>ENCODING</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
<para>
|
|
Note that while <command>COPY</> allows options such as OIDS and HEADER
|
|
to be specified without a corresponding value, the foreign data wrapper
|
|
syntax requires a value to be present in all cases. To activate
|
|
<command>COPY</> options normally supplied without a value, you can
|
|
instead pass the value TRUE.
|
|
</para>
|
|
|
|
<para>
|
|
A column of a foreign table created using this wrapper can have the
|
|
following options:
|
|
</para>
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term><literal>force_not_null</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
This is a Boolean option. If true, it specifies that values of the
|
|
column should not be matched against the null string (that is, the
|
|
file-level <literal>null</literal> option). This has the same effect
|
|
as listing the column in <command>COPY</>'s
|
|
<literal>FORCE_NOT_NULL</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><literal>force_null</literal></term>
|
|
|
|
<listitem>
|
|
<para>
|
|
This is a Boolean option. If true, it specifies that values of the
|
|
column which match the null string are returned as <literal>NULL</>
|
|
even if the value is quoted. Without this option, only unquoted
|
|
values matching the null string are returned as <literal>NULL</>.
|
|
This has the same effect as listing the column in
|
|
<command>COPY</>'s <literal>FORCE_NULL</literal> option.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
<para>
|
|
<command>COPY</>'s <literal>OIDS</literal> and
|
|
<literal>FORCE_QUOTE</literal> options are currently not supported by
|
|
<literal>file_fdw</>.
|
|
</para>
|
|
|
|
<para>
|
|
These options can only be specified for a foreign table or its columns, not
|
|
in the options of the <literal>file_fdw</> foreign-data wrapper, nor in the
|
|
options of a server or user mapping using the wrapper.
|
|
</para>
|
|
|
|
<para>
|
|
Changing table-level options requires superuser privileges, for security
|
|
reasons: only a superuser should be able to determine which file is read.
|
|
In principle non-superusers could be allowed to change the other options,
|
|
but that's not supported at present.
|
|
</para>
|
|
|
|
<para>
|
|
For a foreign table using <literal>file_fdw</>, <command>EXPLAIN</> shows
|
|
the name of the file to be read. Unless <literal>COSTS OFF</> is
|
|
specified, the file size (in bytes) is shown as well.
|
|
</para>
|
|
|
|
<example>
|
|
<title id="csvlog-fdw">Create a Foreign Table for PostgreSQL CSV Logs</title>
|
|
|
|
<para>
|
|
One of the obvious uses for the <literal>file_fdw</> is to make
|
|
the PostgreSQL activity log available as a table for querying. To
|
|
do this, first you must be logging to a CSV file, which here we
|
|
will call <literal>pglog.csv</>. First, install <literal>file_fdw</>
|
|
as an extension:
|
|
</para>
|
|
|
|
<programlisting>
|
|
CREATE EXTENSION file_fdw;
|
|
</programlisting>
|
|
|
|
<para>
|
|
Then create a foreign server:
|
|
|
|
<programlisting>
|
|
CREATE SERVER pglog FOREIGN DATA WRAPPER file_fdw;
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
Now you are ready to create the foreign data table. Using the
|
|
<command>CREATE FOREIGN TABLE</> command, you will need to define
|
|
the columns for the table, the CSV file name, and its format:
|
|
|
|
<programlisting>
|
|
CREATE FOREIGN TABLE pglog (
|
|
log_time timestamp(3) with time zone,
|
|
user_name text,
|
|
database_name text,
|
|
process_id integer,
|
|
connection_from text,
|
|
session_id text,
|
|
session_line_num bigint,
|
|
command_tag text,
|
|
session_start_time timestamp with time zone,
|
|
virtual_transaction_id text,
|
|
transaction_id bigint,
|
|
error_severity text,
|
|
sql_state_code text,
|
|
message text,
|
|
detail text,
|
|
hint text,
|
|
internal_query text,
|
|
internal_query_pos integer,
|
|
context text,
|
|
query text,
|
|
query_pos integer,
|
|
location text,
|
|
application_name text
|
|
) SERVER pglog
|
|
OPTIONS ( filename '/home/josh/9.1/data/pg_log/pglog.csv', format 'csv' );
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
That's it — now you can query your log directly. In production, of
|
|
course, you would need to define some way to deal with log rotation.
|
|
</para>
|
|
</example>
|
|
|
|
</sect1>
|