file_fdw
file_fdw
The file_fdw> module provides the foreign-data wrapper
file_fdw, which can be used to access data
files in the server's file system. Data files must be in a format
that can be read by COPY FROM;
see for details.
A foreign table created using this wrapper can have the following options:
filename
Specifies the file to be read. Required. Must be an absolute path name.
format
Specifies the file's format,
the same as COPY>'s FORMAT option.
header
Specifies whether the file has a header line,
the same as COPY>'s HEADER option.
delimiter
Specifies the file's delimiter character,
the same as COPY>'s DELIMITER option.
quote
Specifies the file's quote character,
the same as COPY>'s QUOTE option.
escape
Specifies the file's escape character,
the same as COPY>'s ESCAPE option.
null
Specifies the file's null string,
the same as COPY>'s NULL option.
encoding
Specifies the file's encoding,
the same as COPY>'s ENCODING option.
A column of a foreign table created using this wrapper can have the
following options:
force_not_null
This is a boolean option. If true, it specifies that values of the
column should not be matched against the null string (that is, the
file-level null option). This has the same effect
as listing the column in COPY>'s
FORCE_NOT_NULL option.
COPY>'s OIDS and
FORCE_QUOTE options are currently not supported by
file_fdw>.
These options can only be specified for a foreign table or its columns, not
in the options of the file_fdw> foreign-data wrapper, nor in the
options of a server or user mapping using the wrapper.
Changing table-level options requires superuser privileges, for security
reasons: only a superuser should be able to determine which file is read.
In principle non-superusers could be allowed to change the other options,
but that's not supported at present.
For a foreign table using file_fdw>, EXPLAIN> shows
the name of the file to be read. Unless COSTS OFF> is
specified, the file size (in bytes) is shown as well.
Create a Foreign Table for PostgreSQL CSV Logs
One of the obvious uses for the file_fdw> is to make
the PostgreSQL activity log available as a table for querying. To
do this, first you must be logging to a CSV file, which here we
will call pglog.csv>. First, install file_fdw>
as an extension:
CREATE EXTENSION file_fdw;
Then create a foreign data server:
CREATE SERVER pglog FOREIGN DATA WRAPPER file_fdw;
Now you are ready to create the foreign data table. Using the
CREATE FOREIGN TABLE> command, you will need to define
the columns for the table, the CSV filename, and its format:
CREATE FOREIGN TABLE pglog (
log_time timestamp(3) with time zone,
user_name text,
database_name text,
process_id integer,
connection_from text,
session_id text,
session_line_num bigint,
command_tag text,
session_start_time timestamp with time zone,
virtual_transaction_id text,
transaction_id bigint,
error_severity text,
sql_state_code text,
message text,
detail text,
hint text,
internal_query text,
internal_query_pos integer,
context text,
query text,
query_pos integer,
location text,
application_name text
) SERVER pglog
OPTIONS ( filename '/home/josh/9.1/data/pg_log/pglog.csv', format 'csv' );
That's it — now you can query your log directly. In production, of course,
you would need to define some way to adjust to log rotation.