Server Runtime Environment
This chapter discusses how to set up and run the database server
and the interactions with the operating system.
The PostgreSQL user accountpostgres user
As with any other server daemon that is connected to the world at
large, it is advisable to run PostgreSQL under a separate user
account. This user account should only own the data itself that is
being managed by the server, and should not be shared with other
daemons. (Thus, using the user nobody is a bad
idea.) It is not advisable to install the executables as owned by
this user account because that runs the risk of user-defined
functions gone astray or any other exploits compromising the
executable programs.
To add a user account to your system, look for a command
useradd or adduser. The user
name postgres is often used but by no means
required.
Creating a database clusterdatabase clusterdata areadatabase cluster
Before you can do anything, you must initialize a database storage
area on disk. We call this a database cluster.
(SQL speaks of a catalog cluster instead.) A
database cluster is a collection of databases that will be accessible
through a single instance of a running database server. After
initialization, a database cluster will contain one database named
template1. As the name suggests, this will be used
as a template for subsequently created databases; it should not be
used for actual work.
In file system terms, a database cluster will be a single directory
under which all data will be stored. We call this the data
directory or data area. It is
completely up to you where you choose to store your data. There is no
default, although locations such as
/usr/local/pgsql/data or
/var/lib/pgsql/data are popular. To initialize a
database cluster, use the command initdb, which is
installed with PostgreSQL. The desired
file system location of your database system is indicated by the
option, for example
$> initdb -D /usr/local/pgsql/data
Note that you must execute this command while being logged into
the PostgreSQL user account, which is described in the previous
section.
PGDATA
As an alternative to the option, you can set
the environment variable PGDATA.
initdb will attempt to create the directory you
specify if it does not already exist. It is likely that it won't
have the permission to do so (if you followed our advice and
created an unprivileged account). In that case you should create the
directory yourself (as root) and transfer ownership of it to the
PostgreSQL user account. Here is how this might work:
root# mkdir /usr/local/pgsql/data
root# chown postgres /usr/local/pgsql/data
root# su postgres
postgres$ initdb -D /usr/local/pgsql/datainitdb will refuse to run if the data directory
looks like it belongs to an already initialized installation.
Because the data directory contains all the data stored in the
database, it is essential that it be well secured from unauthorized
access. initdb therefore revokes access
permissions from everyone but the PostgreSQL user account.
However, while the directory contents are secure, the default
pg_hba.conf authentication method of
trust allows any local user to connect to the database
and even become the database superuser. If you don't trust other local
users, we recommend you use initdb's option
or to assign a
password to the database superuser. After initdb,
modify pg_hba.conf to use md5> or
password>, instead of trust>, authentication
before> you first start the postmaster. (Other, possibly
more convenient approaches include using ident
authentication or filesystem permissions to restrict connections. See
for more information.)
LC_COLLATE>>
One surprise you might encounter while running initdb is
a notice similar to this one:
NOTICE: Initializing database with en_US collation order.
This locale setting will prevent use of index optimization for
LIKE and regexp searches. If you are concerned about speed of
such queries, you may wish to set LC_COLLATE to "C" and
re-initdb. For more information see the Administrator's Guide.
This notice is intended to warn you that the currently selected locale
will cause indexes to be sorted in an order that prevents them from
being used for LIKE and regular-expression searches. If you need
good performance of such searches, you should set your current locale
to C> and re-run initdb. On most systems, setting the
current locale is done by changing the value of the environment variable
LC_ALL or LANG. The sort order used
within a particular database cluster is set by initdb
and cannot be changed later, short of dumping all data, rerunning initdb,
and reloading the data. So it's important to make this choice correctly now.
Starting the database serverpostmaster
Before anyone can access the database you must start the database
server. The database server is called
postmaster.
The postmaster must know where to find the data it is supposed
to work on. This is done with the option. Thus,
the simplest way to start the server is, for example,
$ postmaster -D /usr/local/pgsql/data
which will leave the server running in the foreground. This must
again be done while logged into the PostgreSQL user account. Without
a , the server will try to use the data
directory in the environment variable PGDATA; if
neither of these works it will fail.
To start the postmaster in the
background, use the usual shell syntax:
$ postmaster -D /usr/local/pgsql/data > logfile 2>&1 &
It is an extremely good idea to keep the server's stdout and stderr
output around somewhere, as suggested here. It will help both for auditing
purposes and to diagnose problems.
(See for a more thorough discussion
of log file handling.)
TCP/IP
The postmaster also takes a number of other command line options.
For more information see the reference page and below.
In particular, in order for the server to accept
TCP/IP connections (rather than just Unix domain socket ones), you
must also specify the option.
pg_ctl
This shell syntax can get tedious quickly. Therefore the shell
script wrapper pg_ctl is provided that
encapsulates some of the tasks. E.g.,
pg_ctl start -l logfile
will start the server in the background and put the output into the
named log file. The option has the same
meaning as when invoking postmaster directly.
pg_ctl also implements a symmetric
stop operation.
Normally, you will want to start the database server when the
computer boots up. This is not required; the
PostgreSQL server can be run
successfully from non-privileged accounts without root
intervention.
Different systems have different conventions for starting up
daemons at boot time, so you are advised to familiarize yourself
with them. Many systems have a file
/etc/rc.local or
/etc/rc.d/rc.local which is almost certainly
no bad place to put such a command. Whatever you do, the server
must be run by the PostgreSQL user account
and not by root or any other user. Therefore
you probably always want to form your command lines along the lines
of su -c '...' postgres, for example:
su -c 'pg_ctl start -D /usr/local/pgsql/data -l serverlog' postgres
Here are a few more operating system specific suggestions. (Always
replace the proper installation directory and the user name you
chose.)
For FreeBSD, take a look at the file
contrib/start-scripts/freebsd in the
PostgreSQL source distribution.
FreeBSD>>
On OpenBSD, add the following lines
to the file /etc/rc.local:
OpenBSD>>
if [ -x /usr/local/pgsql/bin/pg_ctl -a -x /usr/local/pgsql/bin/postmaster ]; then
su - -c '/usr/local/pgsql/bin/pg_ctl start -l /var/postgresql/log -s' postgres
echo -n ' postgresql'
fi
On Linux systems either add
Linux>>
/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data
to /etc/rc.d/rc.local or look into the file
contrib/start-scripts/linux in the
PostgreSQL source distribution to
integrate the start and shutdown into the run level system.
On NetBSD, either use the
FreeBSD or
Linux start scripts, depending on
preference, as an example and place the file at
/usr/local/etc/rc.d/postgresql.
NetBSD>>
On Solaris, create a file called
/etc/init.d/postgresql to contain the following
single line:
Solaris>>
su - postgres -c "/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data"
Then, create a symbolic link to it in /etc/rc3.d> as
S99postgresql>.
While the postmaster is running, its
PID is in the file postmaster.pid in the data
directory. This is used as an interlock against multiple postmasters
running in the same data directory, and can also be used for
shutting down the postmaster.
Server Start-up Failures
There are several common reasons for the postmaster to fail to
start up. Check the postmaster's log file, or start it by hand
(without redirecting standard output or standard error) to see
what complaint messages appear. Some of the possible error
messages are reasonably self-explanatory, but here are some that
are not.
FATAL: StreamServerPort: bind() failed: Address already in use
Is another postmaster already running on that port?
This usually means just what it suggests: you tried to
start a second postmaster on the same port where one is already
running. However, if the kernel error message is not
Address already in use or some
variant of that wording, there may be a different problem. For
example, trying to start a postmaster on a reserved port number
may draw something like
$ postmaster -i -p 666
FATAL: StreamServerPort: bind() failed: Permission denied
Is another postmaster already running on that port?
A message like
IpcMemoryCreate: shmget(key=5440001, size=83918612, 01600) failed: Invalid argument
FATAL 1: ShmemCreate: cannot create region
probably means that your kernel's limit on the size of shared
memory areas is smaller than the buffer area that PostgreSQL is
trying to create (83918612 bytes in this example). Or it could
mean that you don't have System-V-style shared memory support
configured into your kernel at all. As a temporary workaround,
you can try starting the postmaster with a smaller-than-normal
number of buffers ( switch). You will
eventually want to reconfigure your kernel to increase the
allowed shared memory size, however. You may see this message
when trying to start multiple postmasters on the same machine, if
their total space requests exceed the kernel limit.
An error like
IpcSemaphoreCreate: semget(key=5440026, num=16, 01600) failed: No space left on device
does not mean that you've run out of disk
space; it means that your kernel's limit on the number of System
V semaphores is smaller than the number
PostgreSQL wants to create. As above,
you may be able to work around the problem by starting the
postmaster with a reduced number of backend processes
( switch), but you'll eventually want to
increase the kernel limit.
If you get an illegal system call> error, then it is likely that
shared memory or semaphores are not supported at all in your kernel. In
that case your only option is to re-configure the kernel to turn on these
features.
Details about configuring System V> IPC> facilities are given in
.
Client Connection Problems
Although the possible error conditions on the client side are
both virtually infinite and application-dependent, a few of them
might be directly related to how the server was started up.
Conditions other than those shown below should be documented with
the respective client application.
psql: could not connect to server: Connection refused
Is the server running on host server.joe.com and accepting
TCP/IP connections on port 5432?
This is the generic I couldn't find a server to talk
to failure. It looks like the above when TCP/IP
communication is attempted. A common mistake is to forget the
option to allow the postmaster to accept TCP/IP
connections.
Alternatively, you'll get this when attempting
Unix-socket communication to a local postmaster:
psql: could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
The last line is useful in verifying that the client is trying to
connect where it is supposed to. If there is in fact no
postmaster running there, the kernel error message will typically
be either Connection refused or
No such file or directory, as
illustrated. (It is particularly important to realize that
Connection refused in this
context does not mean that the postmaster
got your connection request and rejected it -- that case will
produce a different message, as shown in .) Other error messages
such as Connection timed out may
indicate more fundamental problems, like lack of network
connectivity.
Run-time configurationconfigurationserver
There are a lot of configuration parameters that affect the
behavior of the database system in some way or other. Here we
describe how to set them and the following subsections will
discuss each of them.
All parameter names are case-insensitive. Every parameter takes a
value of one of the four types Boolean, integer, floating point,
string as described below. Boolean values are
ON, OFF,
TRUE, FALSE,
YES, NO,
1, 0 (case-insensitive) or
any non-ambiguous prefix of these.
One way to set these options is to edit the file
postgresql.conf in the data directory.
(A default file is installed there.) An example of what
this file could look like is:
# This is a comment
log_connections = yes
syslog = 2
As you see, options are one per line. The equal sign between name
and value is optional. White space is insignificant, blank lines
are ignored. Hash marks (#) introduce comments
anywhere.
SIGHUP
The configuration file is reread whenever the postmaster receives
a SIGHUP> signal (which is most easily sent by means
of pg_ctl reload>). The postmaster also propagates
this signal to all already-running backend processes, so that
existing sessions also get the new default.
Alternatively, you can send the signal to only one backend process
directly.
A second way to set these configuration parameters is to give them
as a command line option to the postmaster, such as
postmaster -c log_connections=yes -c syslog=2
which would have the same effect as the previous example.
Command-line options override any conflicting settings in
postgresql.conf.
Occasionally it is also useful to give a command line option to
one particular backend session only. The environment variable
PGOPTIONS can be used for this purpose on the
client side:
env PGOPTIONS='-c geqo=off' psql
(This works for any client application, not just
psql.) Note that this won't work for
options that are necessarily fixed once the server is started,
such as the port number.
Finally, some options can be changed in individual SQL sessions
with the SET command, for example
=> SET ENABLE_SEQSCAN TO OFF;
See the SQL command language reference for details on the syntax.
Planner and Optimizer TuningCPU_INDEX_TUPLE_COST (floating point)
Sets the query optimizer's estimate of the cost of processing
each index tuple during an index scan. This is measured as a
fraction of the cost of a sequential page fetch.
CPU_OPERATOR_COST (floating point)
Sets the optimizer's estimate of the cost of processing each
operator in a WHERE clause. This is measured as a fraction of
the cost of a sequential page fetch.
CPU_TUPLE_COST (floating point)
Sets the query optimizer's estimate of the cost of processing
each tuple during a query. This is measured as a fraction of
the cost of a sequential page fetch.
EFFECTIVE_CACHE_SIZE (floating point)
Sets the optimizer's assumption about the effective size of
the disk cache (that is, the portion of the kernel's disk
cache that will be used for
PostgreSQL data files). This is
measured in disk pages, which are normally 8 kB apiece.
ENABLE_HASHJOIN (boolean)
Enables or disables the query planner's use of hash-join plan
types. The default is on. This is mostly useful to debug the
query planner.
index scanENABLE_INDEXSCAN (boolean)
Enables or disables the query planner's use of index-scan plan
types. The default is on. This is mostly useful to debug the
query planner.
ENABLE_MERGEJOIN (boolean)
Enables or disables the query planner's use of merge-join plan
types. The default is on. This is mostly useful to debug the
query planner.
ENABLE_NESTLOOP (boolean)
Enables or disables the query planner's use of nested-loop
join plans. It's not possible to suppress nested-loop joins
entirely, but turning this variable off discourages the
planner from using one if there is any other method available.
The default is on. This is mostly useful to debug the query
planner.
sequential scanENABLE_SEQSCAN (boolean)
Enables or disables the query planner's use of sequential scan
plan types. It's not possible to suppress sequential scans
entirely, but turning this variable off discourages the
planner from using one if there is any other method available.
The default is on. This is mostly useful to debug the query
planner.
ENABLE_SORT (boolean)
Enables or disables the query planner's use of explicit sort
steps. It's not possible to suppress explicit sorts entirely,
but turning this variable off discourages the planner from
using one if there is any other method available. The default
is on. This is mostly useful to debug the query planner.
ENABLE_TIDSCAN (boolean)
Enables or disables the query planner's use of TID> scan plan
types. The default is on. This is mostly useful to debug the
query planner.
genetic query optimizationGEQOgenetic query optimizationGEQO (boolean)
Enables or disables genetic query optimization, which is an
algorithm that attempts to do query planning without
exhaustive search. This is on by default. See also the various
other GEQO_ settings.
GEQO_EFFORT (integer)GEQO_GENERATIONS (integer)GEQO_POOL_SIZE (integer)GEQO_RANDOM_SEED (integer)GEQO_SELECTION_BIAS (floating point)
Various tuning parameters for the genetic query optimization
algorithm: The pool size is the number of individuals in one
population. Valid values are between 128 and 1024. If it is
set to 0 (the default) a pool size of 2^(QS+1), where QS
is the number of FROM items in the query, is taken. The effort
is used to calculate a default for generations. Valid values
are between 1 and 80, 40 being the default. Generations
specifies the number of iterations in the algorithm. The
number must be a positive integer. If 0 is specified then
Effort * Log2(PoolSize) is used. The run time of the algorithm
is roughly proportional to the sum of pool size and
generations. The selection bias is the selective pressure
within the population. Values can be from 1.50 to 2.00; the
latter is the default. The random seed can be set to get
reproducible results from the algorithm. If it is set to -1
then the algorithm behaves non-deterministically.
GEQO_THRESHOLD (integer)
Use genetic query optimization to plan queries with at least
this many FROM> items involved. (Note that a JOIN> construct
counts as only one FROM> item.) The default is 11. For simpler
queries it is usually best to use the
deterministic, exhaustive planner. This parameter also controls
how hard the optimizer will try to merge subquery
FROM clauses into the upper query.
KSQO (boolean)
The Key Set Query Optimizer
(KSQO) causes the query planner to convert
queries whose WHERE> clause contains many OR'ed AND clauses
(such as WHERE (a=1 AND b=2) OR (a=2 AND b=3)
...) into a union query. This method can be faster
than the default implementation, but it doesn't necessarily
give exactly the same results, since UNION> implicitly adds a
SELECT DISTINCT> clause to eliminate identical output rows.
KSQO is commonly used when working with products like
Microsoft Access, which tend to
generate queries of this form.
The KSQO algorithm used to be absolutely essential for queries
with many OR'ed AND clauses, but in
PostgreSQL 7.0 and later the standard
planner handles these queries fairly successfully. Hence the
default is off.
RANDOM_PAGE_COST (floating point)
Sets the query optimizer's estimate of the cost of a
nonsequentially fetched disk page. This is measured as a
multiple of the cost of a sequential page fetch.
Unfortunately, there is no well-defined method of determining
ideal values for the family of COST variables that
were just described. You are encouraged to experiment and share
your findings.
Logging and DebuggingDEBUG_ASSERTIONS (boolean)
Turns on various assertion checks. This is a debugging aid. If
you are experiencing strange problems or crashes you might
want to turn this on, as it might expose programming mistakes.
To use this option, the macro USE_ASSERT_CHECKING
must be defined when PostgreSQL is built (see the configure option
--enable-cassert). Note that
DEBUG_ASSERTIONS defaults to on if PostgreSQL
has been built this way.
DEBUG_LEVEL (integer)
The higher this value is set, the more
debugging output of various sorts is generated
in the server log during operation. This option is 0 by
default, which means no debugging output. Values up to about 4
currently make sense.
DEBUG_PRINT_QUERY (boolean)DEBUG_PRINT_PARSE (boolean)DEBUG_PRINT_REWRITTEN (boolean)DEBUG_PRINT_PLAN (boolean)DEBUG_PRETTY_PRINT (boolean)
These flags enable various debugging output to be sent to the
server log. For each executed query, prints either the query text,
the resulting parse tree, the query rewriter output, or the execution
plan. indents these displays
to produce a more readable but much longer output format.
Setting above zero implicitly turns
on some of these flags.
HOSTNAME_LOOKUP (boolean)
By default, connection logs only show the IP address of the
connecting host. If you want it to show the host name you can
turn this on, but depending on your host name resolution setup
it might impose a non-negligible performance penalty. This
option can only be set at server start.
LOG_CONNECTIONS (boolean)
Prints a line informing about each successful connection in
the server log. This is off by default, although it is
probably very useful. This option can only be set at server
start or in the postgresql.conf
configuration file.
LOG_PID (boolean)
Prefixes each server log message with the process id of the
backend process. This is useful to sort out which messages
pertain to which connection. The default is off.
LOG_TIMESTAMP (boolean)
Prefixes each server log message with a time stamp. The default
is off.
SHOW_QUERY_STATS (boolean)SHOW_PARSER_STATS (boolean)SHOW_PLANNER_STATS (boolean)SHOW_EXECUTOR_STATS (boolean)
For each query, write performance statistics of the respective
module to the server log. This is a crude profiling
instrument.
SHOW_SOURCE_PORT (boolean)
Shows the outgoing port number of the connecting host in the
connection log messages. You could trace back the port number
to find out what user initiated the connection. Other than
that it's pretty useless and therefore off by default. This
option can only be set at server start.
STATS_COMMAND_STRING (boolean)STATS_BLOCK_LEVEL (boolean)STATS_ROW_LEVEL (boolean)
These flags determine what information backends send to the statistics
collector process: current commands, block-level activity statistics,
or row-level activity statistics. All default to off. Enabling
statistics collection costs a small amount of time per query, but
is invaluable for debugging and performance tuning.
STATS_RESET_ON_SERVER_START (boolean)
If on, collected statistics are zeroed out whenever the server
is restarted. If off, statistics are accumulated across server
restarts. The default is on. This option
can only be set at server start.
STATS_START_COLLECTOR (boolean)
Controls whether the server should start the statistics-collection
subprocess. This is on by default, but may be turned off if you
know you have no interest in collecting statistics. This option
can only be set at server start.
SYSLOG (integer)PostgreSQL allows the use of
syslog for logging. If this option
is set to 1, messages go both to syslog> and the standard
output. A setting of 2 sends output only to syslog>. (Some
messages will still go to the standard output/error.) The
default is 0, which means syslog> is off. This option must be
set at server start.
To use syslog>, the build of
PostgreSQL must be configured with
the option.
SYSLOG_FACILITY (string)
This option determines the syslogfacility to be used when syslog is enabled.
You may choose from LOCAL0, LOCAL1, LOCAL2, LOCAL3, LOCAL4,
LOCAL5, LOCAL6, LOCAL7; the default is LOCAL0. See also the
documentation of your system's
syslog.
SYSLOG_IDENT (string)
If logging to syslog> is enabled, this option determines the
program name used to identify
PostgreSQL messages in
syslog log messages. The default
is postgres.
TRACE_NOTIFY (boolean)
Generates a great amount of debugging output for the
LISTEN and NOTIFY
commands.
General operationAUSTRALIAN_TIMEZONES (bool)Australian time zones>>
If set to true, CST, EST,
and SAT are interpreted as Australian
time zones rather than as North American Central/Eastern
time zones and Saturday. The default is false.
AUTHENTICATION_TIMEOUT (integer)timeout>authentication>
Maximum time to complete client authentication, in seconds.
If a would-be client has not completed the authentication protocol
in this much time, the server unceremoniously breaks the connection.
This prevents hung clients from occupying a connection indefinitely.
This option can only be set at server start or in the
postgresql.conf file.
deadlocktimeouttimeoutdeadlockDEADLOCK_TIMEOUT (integer)
This is the amount of time, in milliseconds, to wait on a lock
before checking to see if there is a deadlock condition or not.
The check for deadlock is relatively slow, so we don't want to
run it every time we wait for a lock. We (optimistically?)
assume that deadlocks are not common in production applications,
and just wait on the lock for awhile before starting to ask
questions about whether it can ever get unlocked.
Increasing this value reduces the amount of time wasted in
needless deadlock checks, but slows down reporting of real deadlock
errors. The default is 1000 (i.e., one second), which is probably
about the smallest value you would want in practice. On a heavily
loaded server you might want to raise it. Ideally the setting
should exceed your typical transaction time, so as to improve the
odds that the lock will be released before the waiter decides to
check for deadlock.
This option can only be set at server start.
transaction isolation levelDEFAULT_TRANSACTION_ISOLATION (string)
Each SQL transaction has an isolation level, which can be
either read committed or
serializable. This parameter controls what the
isolation level of each new transaction is set to. The
default is read committed.
Consult the PostgreSQL User's Guide and
the command SET TRANSACTION for more
information.
DYNAMIC_LIBRARY_PATH (string)dynamic_library_path>>
dynamic loading>>
If a dynamically loadable module needs to be opened and the
specified name does not have a directory component (i.e., the
name does not contain a slash), the system will search this
path for the specified file. (The name that is used is the
name specified in the CREATE FUNCTION or
LOAD command.)
The value for dynamic_library_path has to be a colon-separated
list of absolute directory names. If a directory name starts
with the special value $libdir, the
compiled-in PostgreSQL package library directory, which is where the
modules provided by the PostgreSQL distribution are installed,
is substituted. (Use pg_config --pkglibdir
to print the name of this directory.) An example value:
dynamic_library_path = '/usr/local/lib/postgresql:/home/my_project/lib:$libdir'
The default value for this parameter is
$libdir. If the value is set to the empty
string, the automatic path search is turned off.
This parameter can be changed at run time by superusers, but
note that a setting done that way will only persist till the
end of the client connection, so this method should be
reserved for development purposes. The recommended way to set
this parameter is in the postgresql.conf
configuration file.
fsyncFSYNC (boolean)
If this option is on, the PostgreSQL> backend
will use the fsync()> system call in several
places to make sure that updates are physically written to
disk and do not hang around in the kernel buffer cache. This
increases the chance by a large amount that a database
installation will still be usable after an operating system or
hardware crash. (Crashes of the database server itself do
not> affect this consideration.)
However, this operation slows down PostgreSQL>,
because at all those points it has
to block and wait for the operating system to flush the
buffers. Without fsync>, the operating system is
allowed to do its best in buffering, sorting, and delaying
writes, which can make for a considerable performance
increase. However, if the system crashes, the results of the
last few committed transactions may be lost in part or whole;
in the worst case, unrecoverable data corruption may occur.
This option is the subject of an eternal debate in the
PostgreSQL> user and developer communities. Some
always leave it off, some turn it off only for bulk loads,
where there is a clear restart point if something goes wrong,
some leave it on just to be on the safe side. Because it is
the safe side, on is also the default. If you trust your
operating system, your hardware, and your utility company (or
better your UPS), you might want to disable fsync.
It should be noted that the performance penalty from doing
fsyncs is considerably less in PostgreSQL> version
7.1 than it was in prior releases. If you previously suppressed
fsyncs because of performance problems, you may wish to reconsider
your choice.
This option can only be set at server start or in the
postgresql.conf file.
KRB_SERVER_KEYFILE (string)
Sets the location of the Kerberos server key file. See
for details.
MAX_CONNECTIONS (integer)
Determines how many concurrent connections the database server
will allow. The default is 32 (unless altered while building
the server). This parameter can only be set at server start.
MAX_EXPR_DEPTH (integer)
Sets the maximum expression nesting depth that the parser will
accept. The default value is high enough for any normal query,
but you can raise it if you need to. (But if you raise it too
high, you run the risk of backend crashes due to stack
overflow.)
MAX_FILES_PER_PROCESS (integer)
Sets the maximum number of simultaneously open files in each server
subprocess. The default is 1000. The limit actually used by the code
is the smaller of this setting and the result of
sysconf(_SC_OPEN_MAX).
Therefore, on systems where sysconf returns a reasonable limit,
you don't need to worry about this setting. But on some platforms
(notably, most BSD systems), sysconf returns a value that is much
larger than the system can really support when a large number of
processes all try to open that many files. If you find yourself
seeing Too many open files> failures, try reducing this
setting.
This option can only be set at server start or in the
postgresql.conf configuration file;
if changed in the configuration file, it only affects
subsequently-started server subprocesses.
MAX_FSM_RELATIONS (integer)
Sets the maximum number of relations (tables) for which free space
will be tracked in the shared free-space map.
The default is 100. This option can only be set at server start.
MAX_FSM_PAGES (integer)
Sets the maximum number of disk pages for which free space
will be tracked in the shared free-space map.
The default is 10000. This option can only be set at server start.
MAX_LOCKS_PER_TRANSACTION (integer)
The shared lock table is sized on the assumption that at most
max_locks_per_transaction> * max_connections distinct objects will need
to be locked at any one time. The default, 64, has historically
proven sufficient, but you might need to raise this value if you
have clients that touch many different tables in a single transaction.
This option can only be set at server start.
PASSWORD_ENCRYPTION (boolean)
When a password is specified in CREATE USER> or
ALTER USER> without writing either ENCRYPTED or
UNENCRYPTED, this flag determines whether the password is to be
encrypted.
The default is off (do not encrypt the password), but this choice
may change in a future release.
PORT (integer)port>>
The TCP port the server listens on; 5432 by default. This
option can only be set at server start.
SHARED_BUFFERS (integer)
Sets the number of shared memory buffers the database server
will use. The default is 64. Each buffer is typically 8192
bytes. This option can only be set at server start.
SILENT_MODE (bool)
Runs postmaster silently. If this option is set, postmaster
will automatically run in background and any controlling ttys
are disassociated, thus no messages are written to standard output or
standard error (same effect as postmaster's -S option). Unless some
logging system such as syslog> is enabled, using this option is
discouraged since it makes it impossible to see error
messages.
SORT_MEM (integer)
Specifies the amount of memory to be used by internal sorts
and hashes before switching to temporary disk files. The value
is specified in kilobytes, and defaults to 512 kilobytes. Note
that for a complex query, several sorts and/or hashes might be
running in parallel, and each one will be allowed to use as
much memory as this value specifies before it starts to put
data into temporary files. And don't forget that each running
backend could be doing one or more sorts. So the total memory
space needed could be many times the value of SORT_MEM.
SQL_INHERITANCE (bool)inheritance>>
This controls the inheritance semantics, in particular whether
subtables are included into the consideration of various
commands by default. This was not the case in versions prior
to 7.1. If you need the old behavior you can set this
variable to off, but in the long run you are encouraged to
change your applications to use the ONLY
keyword to exclude subtables. See the SQL language reference
and the User's Guide for more
information about inheritance.
SSLSSL (boolean)
Enables SSL> connections. Please read
before using this. The default
is off.
TCPIP_SOCKET (boolean)
If this is true, then the server will accept TCP/IP
connections. Otherwise only local Unix domain socket
connections are accepted. It is off by default. This option
can only be set at server start.
TRANSFORM_NULL_EQUALS (boolean)IS NULL>>
When turned on, expressions of the form
expr> = NULL (or
NULL = expr>) are treated as
expr> IS NULL, that is, they
return true if expr> evaluates to the NULL
value, and false otherwise. The correct behavior of
expr> = NULL is to always
return NULL (unknown). Therefore this option defaults to off.
However, filtered forms in Microsoft
Access generate queries that appear to use
expr> = NULL to test for
NULLs, so if you use that interface to access the database you
might want to turn this option on. Since expressions of the
form expr> = NULL always
return NULL (using the correct interpretation) they are not
very useful and do not appear often in normal applications, so
this option does little harm in practice. But new users are
frequently confused about the semantics of expressions
involving NULL, so we do not turn this option on by default.
Note that this option only affects the literal =>
operator, not other comparison operators or other expressions
that are computationally equivalent to some expression
involving the equals operator (such as IN).
Thus, this option is not a general fix for bad programming.
Refer to the User's Guide for related
information.
UNIX_SOCKET_DIRECTORY (string)
Specifies the directory of the Unix-domain socket on which the
postmaster is to listen for
connections from client applications. The default is normally
/tmp, but can be changed at build time.
UNIX_SOCKET_GROUP (string)
Sets the group owner of the Unix domain socket. (The owning
user of the socket is always the user that starts the
postmaster.) In combination with the option
this can be used as
an additional access control mechanism for this socket type.
By default this is the empty string, which uses the default
group for the current user. This option can only be set at
server start.
UNIX_SOCKET_PERMISSIONS (integer)
Sets the access permissions of the Unix domain socket. Unix
domain sockets use the usual Unix file system permission set.
The option value is expected to be an numeric mode
specification in the form accepted by the
chmod and umask
system calls. (To use the customary octal format the number
must start with a 0 (zero).)
The default permissions are 0777, meaning
anyone can connect. Reasonable alternatives would be
0770 (only user and group, see also under
) and
0700 (only user). (Note that actually for
a Unix socket, only write permission matters and there is no
point in setting or revoking read or execute permissions.)
This access control mechanism is independent from the one
described in .
This option can only be set at server start.
VACUUM_MEM (integer)
Specifies the maximum amount of memory to be used by
VACUUM to keep track of to-be-reclaimed tuples.
The value is specified in kilobytes, and defaults to 8192 kilobytes.
Larger settings may improve the speed of vacuuming large tables
that have many deleted tuples.
VIRTUAL_HOST (string)
Specifies the TCP/IP host name or address on which the
postmaster is to listen for
connections from client applications. Defaults to
listening on all configured addresses (including localhost>).
WAL
See also for details on WAL
tuning.
CHECKPOINT_SEGMENTS (integer)
Maximum distance between automatic WAL checkpoints, in log file
segments (each segment is normally 16 megabytes).
This option can only be set at server start or in the
postgresql.conf file.
CHECKPOINT_TIMEOUT (integer)
Maximum time between automatic WAL checkpoints, in seconds.
This option can only be set at server start or in the
postgresql.conf file.
COMMIT_DELAY (integer)
Time delay between writing a commit record to the WAL buffer and
flushing the buffer out to disk, in microseconds. A nonzero delay
allows multiple transactions to be committed with only one fsync,
if system load is high enough that additional transactions become
ready to commit within the given interval. But the delay is just
wasted time if no other transactions become ready to commit.
Therefore, the delay is only performed if at least COMMIT_SIBLINGS
other transactions are active at the instant that a backend has
written its commit record.
COMMIT_SIBLINGS (integer)
Minimum number of concurrent open transactions to require before
performing the COMMIT_DELAY> delay. A larger value makes it more
probable that at least one other transaction will become ready to
commit during the delay interval.
WAL_BUFFERS (integer)
Number of disk-page buffers in shared memory for WAL log.
This option can only be set at server start.
WAL_DEBUG (integer)
If non-zero, turn on WAL-related debugging output on standard
error.
WAL_FILES (integer)
Number of log files that are created in advance at checkpoint
time. This option can only be set at server start or in the
postgresql.conf file.
WAL_SYNC_METHOD (string)
Method used for forcing WAL updates out to disk. Possible
values are
FSYNC> (call fsync() at each commit),
FDATASYNC> (call fdatasync() at each commit),
OPEN_SYNC> (write WAL files with open() option O_SYNC), or
OPEN_DATASYNC> (write WAL files with open() option O_DSYNC).
Not all of these choices are available on all platforms.
This option can only be set at server start or in the
postgresql.conf file.
Short options
For convenience there are also single letter option switches
available for many parameters. They are described in the following
table.
For historical reasons, options marked * must be
passed to the individual backend process via the
postmaster option, for example,
$ postmaster -o '-S 1024 -s'
or via PGOPTIONS from the client side, as explained
above.
Managing Kernel Resources
A large PostgreSQL> installation can quickly hit
various operating system resource limits. (On some systems, the
factory defaults are so low that you don't even need a really
large> installation.) If you have encountered this kind of
problem then keep reading.
Shared Memory and Semaphoresshared memorysemaphores
Shared memory and semaphores are collectively referred to as
System V> IPC> (together with message queues, which are
not relevant for PostgreSQL>). Almost all modern
operating systems provide these features, but not all of them have
them turned on or sufficiently sized by default, especially
systems with BSD heritage. (For the QNX> and BeOS> ports,
PostgreSQL> provides its own replacement
implementation of these facilities.)
The complete lack of these facilities is usually manifested by an
Illegal system call> error upon postmaster start. In
that case there's nothing left to do but to reconfigure your
kernel -- PostgreSQL> won't work without them.
When PostgreSQL> exceeds one of the various hard
limits of the IPC> resources then the postmaster will refuse to
start up and should leave a marginally instructive error message
about which problem was encountered and what needs to be done
about it. (See also .)
The relevant kernel parameters are named
consistently across different systems; gives an overview. The methods to
set them, however, vary; suggestions for some platforms are given
below. Be warned that it is often necessary to reboot your
machine at least, possibly even recompile the kernel, to change these
settings.
System V> IPC> parameters>
Name>
Description>
Reasonable values>
SHMMAX>>
Maximum size of shared memory segment (bytes)>
250kB + 8.2kB * shared_buffers> + 14.2kB * max_connections> or infinitySHMMIN>>
Minimum size of shared memory segment (bytes)>
1>
SHMALL>>
Total amount of shared memory available (bytes or pages)>
if bytes, same as SHMMAX; if pages, ceil(SHMMAX/PAGE_SIZE)>
SHMSEG>>
Maximum number of shared memory segments per process>
only 1 segment is needed, but the default is much higher>
SHMMNI>>
Maximum number of shared memory segments system-wide>
like SHMSEG> plus room for other applications>
SEMMNI>>
Maximum number of semaphore identifiers (i.e., sets)>
>= ceil(max_connections / 16)>
SEMMNS>>
Maximum number of semaphores system-wide>
ceil(max_connections / 16) * 17 + room for other applications>
SEMMSL>>
Maximum number of semaphores per set>
>= 17>
SEMMAP>>
Number of entries in semaphore map>
see text>
SEMVMX>>
Maximum value of semaphore>
>= 255 (The default is often 32767, don't change unless asked to.)>
SHMMAX
The most important shared memory parameter is SHMMAX>,
the maximum size, in bytes, that a shared memory segment can have.
If you get an error message from shmget> along the
lines of Invalid argument> then it is possible that
this limit has been exceeded. The size of the required shared
memory segments varies both with the number of requested buffers
(
Less likely to cause problems is the minimum size for shared
memory segments (SHMMIN>), which should be at most
somewhere around 256 kB for PostgreSQL> (it is
usually just 1). The maximum number of segments system-wide
(SHMMNI>) or per-process (SHMSEG>) should
not cause a problem unless your system has them set to zero. Some
systems also have a limit on the total amount of shared memory in
the system; see the platform-specific instructions below.
PostgreSQL> uses one semaphore per allowed connection
(
In some cases it might also turn out to be necessary to increase
SEMMAP> to be at least on the order of
SEMMNS>. This parameter defines the size of the
semaphore resource map, in which each contiguous block of available
semaphores needs an entry. When a semaphore set is freed it is
either added to an existing entry that is adjacent to the freed
block or it is registered under a new map entry. If the map is
full, the freed semaphores get lost (until reboot). Fragmentation
of the semaphore space could therefore over time lead to less
available semaphores than there should be.
The SEMMSL> parameter, which determines how many
semaphores can be in a set, must be at least 17 for
PostgreSQL>.
Various other settings related to semaphore undo>, such as
SEMMNU> and SEMUME>, are not of concern
for PostgreSQL>.
BSD/OS>BSD/OS>>
Shared Memory>
By default, only 4 MB of shared memory is supported. Keep in
mind that shared memory is not pageable; it is locked in RAM.
To increase the number of shared buffers supported by the
postmaster, add the following to your kernel configuration file. A
SHMALL> value of 1024 represents 4MB of shared
memory. The following increases the maximum shared memory area
to 32 MB:
options "SHMALL=8192"
options "SHMMAX=\(SHMALL*PAGE_SIZE\)"
For those running 4.1 or later, just make the above changes,
recompile the kernel, and reboot. For those running earlier
releases, use bpatch> to find the
sysptsize> value in the current kernel. This is
computed dynamically at boot time.
$ bpatch -r sysptsize>
0x9 = 9>
Next, add SYSPTSIZE> as a hard-coded value in the
kernel configuration file. Increase the value you found using
bpatch>. Add 1 for every additional 4 MB of
shared memory you desire.
options "SYSPTSIZE=16"
sysptsize> cannot be changed by sysctl.
Semaphores>
You may need to increase the number of semaphores. By
default, PostgreSQL> allocates 34 semaphores,
which is over half the default system total of 60.
Set the values you want in your kernel configuration file, e.g.:
options "SEMMNI=40"
options "SEMMNS=240"
options "SEMUME=40"
options "SEMMNU=120"
FreeBSD>NetBSD>OpenBSD>FreeBSD>>
NetBSD>>
OpenBSD>>
The options SYSVSHM> and SYSVSEM> need
to be enabled when the kernel is compiled. (They are by
default.) The maximum size of shared memory is determined by
the option SHMMAXPGS> (in pages). The following
shows an example of how to set the various parameters:
options SYSVSHM
options SHMMAXPGS=4096
options SHMSEG=256
options SYSVSEM
options SEMMNI=256
options SEMMNS=512
options SEMMNU=256
options SEMMAP=256
(On NetBSD> and OpenBSD> the key word is actually
option singular.)
HP-UX>HP-UX>>
The default settings tend to suffice for normal installations.
On HP-UX> 10, the factory default for
SEMMNS> is 128, which might be too low for larger
database sites.
IPC> parameters can be set in the System
Administration Manager> (SAM>) under
Kernel
Configuration>Configurable Parameters>>.
Hit Create A New Kernel> when you're done.
Linux>Linux>>
The default shared memory limit (both
SHMMAX and SHMALL) is 32
MB in 2.2 kernels, but it can be changed in the
proc file system (without reboot). For
example, to allow 128 MB:
$echo 134217728 >/proc/sys/kernel/shmall$echo 134217728 >/proc/sys/kernel/shmmax
You could put these commands into a script run at boot-time.
Alternatively, you can use
sysctl8, if available, to
control these parameters. Look for a file called
/etc/sysctl.conf and add lines like the
following to it:
kernel.shmall = 134217728
kernel.shmmax = 134217728
This file is usually processed at boot time, but
sysctl can also be called
explicitly later.
Other parameters are sufficiently sized for any application.
If you want to see for yourself look into
/usr/src/linux/include/asm-xxx>/shmparam.h>
and /usr/src/linux/include/linux/sem.h>.
SCO OpenServer>SCO OpenServer>>
In the default configuration, only 512 kB of shared memory per
segment is allowed, which is about enough for Solaris>Solaris>>
At least in version 2.6, the maximum size of a shared memory
segment is set too low for PostgreSQL>. The
relevant settings can be changed in /etc/system>,
for example:
set shmsys:shminfo_shmmax=0x2000000
set shmsys:shminfo_shmmin=1
set shmsys:shminfo_shmmni=256
set shmsys:shminfo_shmseg=256
set semsys:seminfo_semmap=256
set semsys:seminfo_semmni=512
set semsys:seminfo_semmns=512
set semsys:seminfo_semmsl=32
You need to reboot to make the changes effective.
See also >
for information on shared memory under
Solaris>.
UnixWare>Unixware>>
On UnixWare> 7, the maximum size for shared
memory segments is 512 kB in the default configuration. This
is enough for about Resource Limits
Unix-like operating systems enforce various kinds of resource
limits that might interfere with the operation of your
PostgreSQL server. Of importance are
especially the limits on the number of processes per user, the
number of open files per process, and the amount of memory
available to a process. Each of these have a hard
and a soft limit. The soft limit is what actually
counts but it can be changed by the user up to the hard limit.
The hard limit can only be changed by the root user. The system
call setrlimit is responsible for setting
these parameters. The shell's built-in command
ulimit (Bourne shells) or
limit (csh>) is used to control the resource
limits from the command line. On BSD-derived systems the file
/etc/login.conf controls what values the
various resource limits are set to upon login. See
login.conf5 for details. The relevant
parameters are maxproc,
openfiles, and datasize.
For example:
default:\
...
:datasize-cur=256M:\
:maxproc-cur=256:\
:openfiles-cur=256:\
...
(-cur is the soft limit. Append
-max to set the hard limit.)
Kernels generally also have an implementation-dependent
system-wide limit on some resources.
On Linux/proc/sys/fs/file-max determines the
maximum number of open files that the kernel will support. It can
be changed by writing a different number into the file or by
adding an assignment in /etc/sysctl.conf.
The maximum limit of files per process is fixed at the time the
kernel is compiled; see
/usr/src/linux/Documentation/proc.txt for
more information.
The PostgreSQL server uses one process
per connection so you should provide for at least as many processes
as allowed connections, in addition to what you need for the rest
of your system. This is usually not a problem but if you run
several servers on one machine things might get tight.
The factory default limit on open files is often set to
socially friendly values that allow many users to
coexist on a machine without using an inappropriate fraction of
the system resources. If you run many servers on a machine this
is perhaps what you want, but on dedicated servers you may want to
raise this limit.
On the other side of the coin, some systems allow individual
processes to open large numbers of files; if more than a few processes
do so then the system-wide limit can easily be exceeded. If you find
this happening, and don't want to alter the system-wide limit, you
can set PostgreSQL's
max_files_per_process configuration parameter
to limit its consumption of open files.
Shutting down the server
Depending on your needs, there are several ways to shut down the
database server when your work is done. The differentiation is
done by what signal you send to the server process.
SIGTERM
After receiving SIGTERM, the postmaster disallows new
connections, but lets existing backends end their work normally.
It shuts down only after all of the backends terminate by client
request.
This is the Smart Shutdown.
SIGINT
The postmaster disallows new connections and sends all existing
backends SIGTERM, which will cause them to abort their current
transactions and exit promptly. It then waits for the backends to exit
and finally shuts down the data base.
This is the Fast Shutdown.
SIGQUIT
This is the Immediate Shutdown which
will cause the postmaster to send a SIGQUIT to all backends and
exit immediately (without properly shutting down the database
system). The backends likewise exit immediately upon receiving
SIGQUIT. This will lead to recovery (by replaying the WAL log)
upon next start-up. This is recommended only in emergencies.
It is best not to use SIGKILL to shut down the postmaster. This
will prevent the postmaster from releasing shared memory and
semaphores, which you may then have to do by hand.
The PID> of the postmaster process can be found using the
ps program, or from the file
postmaster.pid in the data directory. So for
example, to do a fast shutdown:
$ kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`
The program pg_ctl is a shell script
that provides a more convenient interface for shutting down the
postmaster.
Secure TCP/IP Connections with SSLSSLPostgreSQL> has native support for connections over
SSL> to encrypt
client/server communications for increased security. This requires
OpenSSL to be installed on both client
and server systems and support enabled at build time (see ).
With SSL support compiled in, the PostgreSQL> server
can be started with the argument
The server will listen for both standard and SSL connections
on the same TCP/IP port, and will negotiate with any connecting
client whether or not to use SSL.
See
about how to force on the server side the use of SSL for certain
connections.
For details on how to create your server private key and certificate,
refer to the OpenSSL> documentation. A simple self-signed
certificate can be used to get started for testing, but a certificate signed
by a CA> (either one of the global CAs> or a local one) should be used in
production so the client can verify the server's identity. To create
a quick self-signed certificate, use the following OpenSSL command:
openssl req -new -text -out cert.req
Fill out the information that openssl> asks for. Make sure that you enter
the local host name as Common Name; the challenge password can be
left blank. The script will generate a key that is passphrase protected;
it will not accept a pass phrase that is less than four characters long.
To remove the passphrase (as you must if you want automatic start-up of
the server), run the commands
openssl rsa -in privkey.pem -out cert.pem
Enter the old passphrase to unlock the existing key. Now do
openssl req -x509 -in cert.req -text -key cert.pem -out cert.cert
cp cert.pem $PGDATA/server.key
cp cert.cert $PGDATA/server.crt
to turn the certificate into a self-signed certificate and to copy the
key and certificate to where the server will look for them.
Secure TCP/IP Connections with SSH tunnelssshAcknowledgement
Idea taken from an email by Gene Selkov, Jr.
(selkovjr@mcs.anl.gov>) written on 1999-09-08 in response
to a question from Eric Marsden.
One can use ssh to encrypt the network
connection between clients and a
PostgreSQL server. Done properly, this
should lead to an adequately secure network connection.
First make sure that an ssh server is
running properly on the same machine as
PostgreSQL and that you can log in using
ssh as some user. Then you can establish a secure tunnel with a
command like this from the client machine:
$ ssh -L 3333:foo.com:5432 joe@foo.com
The first number in the argument, 3333, is the
port number of your end of the tunnel; it can be chosen freely. The
second number, 5432, is the remote end of the tunnel -- the port
number your server is using. The name or the address in between
the port numbers is the host with the database server you are going
to connect to. In order to connect to the database server using
this tunnel, you connect to port 3333 on the local machine:
psql -h localhost -p 3333 template1
To the database server it will then look as though you are really
user joe@foo.com and it will use whatever
authentication procedure was set up for this user. In order for the
tunnel setup to succeed you must be allowed to connect via ssh as
joe@foo.com, just as if you had attempted to use ssh to set up a
terminal session.
Several other products exist that can provide secure tunnels using
a procedure similar in concept to the one just described.