mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-08-22 12:53:23 +02:00
1624 lines
62 KiB
Plaintext
1624 lines
62 KiB
Plaintext
<!--
|
|
$PostgreSQL: pgsql/doc/src/sgml/runtime.sgml,v 1.356 2005/10/26 12:55:07 momjian Exp $
|
|
-->
|
|
|
|
<chapter Id="runtime">
|
|
<title>Operating System Environment</title>
|
|
|
|
<para>
|
|
This chapter discusses how to set up and run the database server
|
|
and its interactions with the operating system.
|
|
</para>
|
|
|
|
<sect1 id="postgres-user">
|
|
<title>The <productname>PostgreSQL</productname> User Account</title>
|
|
|
|
<indexterm>
|
|
<primary>postgres user</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
As with any other server daemon that is accessible to the outside world,
|
|
it is advisable to run <productname>PostgreSQL</productname> under a
|
|
separate user account. This user account should only own the data
|
|
that is managed by the server, and should not be shared with other
|
|
daemons. (For example, using the user <literal>nobody</literal> is a bad
|
|
idea.) It is not advisable to install executables owned by this
|
|
user because compromised systems could then modify their own
|
|
binaries.
|
|
</para>
|
|
|
|
<para>
|
|
To add a Unix user account to your system, look for a command
|
|
<command>useradd</command> or <command>adduser</command>. The user
|
|
name <systemitem>postgres</systemitem> is often used, and is assumed
|
|
throughout this book, but you can use another name if you like.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="creating-cluster">
|
|
<title>Creating a Database Cluster</title>
|
|
|
|
<indexterm>
|
|
<primary>database cluster</primary>
|
|
</indexterm>
|
|
|
|
<indexterm>
|
|
<primary>data area</primary>
|
|
<see>database cluster</see>
|
|
</indexterm>
|
|
|
|
<para>
|
|
Before you can do anything, you must initialize a database storage
|
|
area on disk. We call this a <firstterm>database cluster</firstterm>.
|
|
(<acronym>SQL</acronym> uses the term catalog cluster.) A
|
|
database cluster is a collection of databases that is managed by a
|
|
single instance of a running database server. After initialization, a
|
|
database cluster will contain a database named <literal>postgres</literal>,
|
|
which is meant as a default database for use by utilities, users and third
|
|
party applications. The database server itself does not require the
|
|
<literal>postgres</literal> database to exist, but many external utility
|
|
programs assume it exists. Another database created within each cluster
|
|
during initialization is called
|
|
<literal>template1</literal>. As the name suggests, this will be used
|
|
as a template for subsequently created databases; it should not be
|
|
used for actual work. (See <xref linkend="managing-databases"> for
|
|
information about creating new databases within a cluster.)
|
|
</para>
|
|
|
|
<para>
|
|
In file system terms, a database cluster will be a single directory
|
|
under which all data will be stored. We call this the <firstterm>data
|
|
directory</firstterm> or <firstterm>data area</firstterm>. It is
|
|
completely up to you where you choose to store your data. There is no
|
|
default, although locations such as
|
|
<filename>/usr/local/pgsql/data</filename> or
|
|
<filename>/var/lib/pgsql/data</filename> are popular. To initialize a
|
|
database cluster, use the command <xref
|
|
linkend="app-initdb">,<indexterm><primary>initdb</></> which is
|
|
installed with <productname>PostgreSQL</productname>. The desired
|
|
file system location of your database cluster is indicated by the
|
|
<option>-D</option> option, for example
|
|
<screen>
|
|
<prompt>$</> <userinput>initdb -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
Note that you must execute this command while logged into the
|
|
<productname>PostgreSQL</productname> user account, which is
|
|
described in the previous section.
|
|
</para>
|
|
|
|
<tip>
|
|
<para>
|
|
As an alternative to the <option>-D</option> option, you can set
|
|
the environment variable <envar>PGDATA</envar>.
|
|
<indexterm><primary><envar>PGDATA</envar></primary></indexterm>
|
|
</para>
|
|
</tip>
|
|
|
|
<para>
|
|
<command>initdb</command> will attempt to create the directory you
|
|
specify if it does not already exist. It is likely that it will not
|
|
have the permission to do so (if you followed our advice and created
|
|
an unprivileged account). In that case you should create the
|
|
directory yourself (as root) and change the owner to be the
|
|
<productname>PostgreSQL</productname> user. Here is how this might
|
|
be done:
|
|
<screen>
|
|
root# <userinput>mkdir /usr/local/pgsql/data</userinput>
|
|
root# <userinput>chown postgres /usr/local/pgsql/data</userinput>
|
|
root# <userinput>su postgres</userinput>
|
|
postgres$ <userinput>initdb -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> will refuse to run if the data directory
|
|
looks like it has already been initialized.</para>
|
|
|
|
<para>
|
|
Because the data directory contains all the data stored in the
|
|
database, it is essential that it be secured from unauthorized
|
|
access. <command>initdb</command> therefore revokes access
|
|
permissions from everyone but the
|
|
<productname>PostgreSQL</productname> user.
|
|
</para>
|
|
|
|
<para>
|
|
However, while the directory contents are secure, the default
|
|
client authentication setup allows any local user to connect to the
|
|
database and even become the database superuser. If you do not
|
|
trust other local users, we recommend you use one of
|
|
<command>initdb</command>'s <option>-W</option>, <option>--pwprompt</option>
|
|
or <option>--pwfile</option> options to assign a password to the
|
|
database superuser.<indexterm><primary>password</><secondary>of the
|
|
superuser</></indexterm> Also, specify <option>-A md5</> or
|
|
<option>-A password</> so that the default <literal>trust</> authentication
|
|
mode is not used; or modify the generated <filename>pg_hba.conf</filename>
|
|
file after running <command>initdb</command>,
|
|
<emphasis>before</> you start the server for the first time. (Other
|
|
reasonable approaches include using <literal>ident</literal> authentication
|
|
or file system permissions to restrict connections. See <xref
|
|
linkend="client-authentication"> for more information.)
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> also initializes the default
|
|
locale<indexterm><primary>locale</></> for the database cluster.
|
|
Normally, it will just take the locale settings in the environment
|
|
and apply them to the initialized database. It is possible to
|
|
specify a different locale for the database; more information about
|
|
that can be found in <xref linkend="locale">. The sort order used
|
|
within a particular database cluster is set by
|
|
<command>initdb</command> and cannot be changed later, short of
|
|
dumping all data, rerunning <command>initdb</command>, and reloading
|
|
the data. There is also a performance impact for using locales
|
|
other than <literal>C</> or <literal>POSIX</>. Therefore, it is
|
|
important to make this choice correctly the first time.
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> also sets the default character set encoding
|
|
for the database cluster. Normally this should be chosen to match the
|
|
locale setting. For details see <xref linkend="multibyte">.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="postmaster-start">
|
|
<title>Starting the Database Server</title>
|
|
|
|
<para>
|
|
Before anyone can access the database, you must start the database
|
|
server. The database server program is called
|
|
<command>postmaster</command>.<indexterm><primary>postmaster</></>
|
|
The <command>postmaster</command> must know where to
|
|
find the data it is supposed to use. This is done with the
|
|
<option>-D</option> option. Thus, the simplest way to start the
|
|
server is:
|
|
<screen>
|
|
$ <userinput>postmaster -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
which will leave the server running in the foreground. This must be
|
|
done while logged into the <productname>PostgreSQL</productname> user
|
|
account. Without <option>-D</option>, the server will try to use
|
|
the data directory named by the environment variable <envar>PGDATA</envar>.
|
|
If that variable is not provided either, it will fail.
|
|
</para>
|
|
|
|
<para>
|
|
Normally it is better to start the <command>postmaster</command> in the
|
|
background. For this, use the usual shell syntax:
|
|
<screen>
|
|
$ <userinput>postmaster -D /usr/local/pgsql/data >logfile 2>&1 &</userinput>
|
|
</screen>
|
|
It is important to store the server's <systemitem>stdout</> and
|
|
<systemitem>stderr</> output somewhere, as shown above. It will help
|
|
for auditing purposes and to diagnose problems. (See <xref
|
|
linkend="logfile-maintenance"> for a more thorough discussion of log
|
|
file handling.)
|
|
</para>
|
|
|
|
<para>
|
|
The <command>postmaster</command> also takes a number of other
|
|
command line options. For more information, see the
|
|
<xref linkend="app-postmaster"> reference page
|
|
and <xref linkend="runtime-config"> below.
|
|
</para>
|
|
|
|
<para>
|
|
This shell syntax can get tedious quickly. Therefore the wrapper
|
|
program
|
|
<xref linkend="app-pg-ctl"><indexterm><primary>pg_ctl</primary></indexterm>
|
|
is provided to simplify some tasks. For example:
|
|
<programlisting>
|
|
pg_ctl start -l logfile
|
|
</programlisting>
|
|
will start the server in the background and put the output into the
|
|
named log file. The <option>-D</option> option has the same meaning
|
|
here as in the <command>postmaster</command>. <command>pg_ctl</command>
|
|
is also capable of stopping the server.
|
|
</para>
|
|
|
|
<para>
|
|
Normally, you will want to start the database server when the
|
|
computer boots.<indexterm><primary>booting</><secondary>starting
|
|
the server during</></> Autostart scripts are operating-system-specific.
|
|
There are a few distributed with
|
|
<productname>PostgreSQL</productname> in the
|
|
<filename>contrib/start-scripts</> directory. Installing one will require
|
|
root privileges.
|
|
</para>
|
|
|
|
<para>
|
|
Different systems have different conventions for starting up daemons
|
|
at boot time. Many systems have a file
|
|
<filename>/etc/rc.local</filename> or
|
|
<filename>/etc/rc.d/rc.local</filename>. Others use
|
|
<filename>rc.d</> directories. Whatever you do, the server must be
|
|
run by the <productname>PostgreSQL</productname> user account
|
|
<emphasis>and not by root</emphasis> or any other user. Therefore you
|
|
probably should form your commands using <literal>su -c '...'
|
|
postgres</literal>. For example:
|
|
<programlisting>
|
|
su -c 'pg_ctl start -D /usr/local/pgsql/data -l serverlog' postgres
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
Here are a few more operating-system-specific suggestions. (In each
|
|
case be sure to use the proper installation directory and user
|
|
name where we show generic values.)
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
For <productname>FreeBSD</productname>, look at the file
|
|
<filename>contrib/start-scripts/freebsd</filename> in the
|
|
<productname>PostgreSQL</productname> source distribution.
|
|
<indexterm><primary>FreeBSD</><secondary>start script</secondary></>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>OpenBSD</productname>, add the following lines
|
|
to the file <filename>/etc/rc.local</filename>:
|
|
<indexterm><primary>OpenBSD</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
if [ -x /usr/local/pgsql/bin/pg_ctl -a -x /usr/local/pgsql/bin/postmaster ]; then
|
|
su - -c '/usr/local/pgsql/bin/pg_ctl start -l /var/postgresql/log -s' postgres
|
|
echo -n ' postgresql'
|
|
fi
|
|
</programlisting>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>Linux</productname> systems either add
|
|
<indexterm><primary>Linux</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data
|
|
</programlisting>
|
|
to <filename>/etc/rc.d/rc.local</filename> or look at the file
|
|
<filename>contrib/start-scripts/linux</filename> in the
|
|
<productname>PostgreSQL</productname> source distribution.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>NetBSD</productname>, either use the
|
|
<productname>FreeBSD</productname> or
|
|
<productname>Linux</productname> start scripts, depending on
|
|
preference. <indexterm><primary>NetBSD</><secondary>start script</secondary></>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>Solaris</productname>, create a file called
|
|
<filename>/etc/init.d/postgresql</filename> that contains
|
|
the following line:
|
|
<indexterm><primary>Solaris</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
su - postgres -c "/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data"
|
|
</programlisting>
|
|
Then, create a symbolic link to it in <filename>/etc/rc3.d</> as
|
|
<filename>S99postgresql</>.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</para>
|
|
|
|
<para>
|
|
While the <command>postmaster</command> is running, its
|
|
<acronym>PID</acronym> is stored in the file
|
|
<filename>postmaster.pid</filename> in the data directory. This is
|
|
used to prevent multiple <command>postmaster</command> processes
|
|
running in the same data directory and can also be used for
|
|
shutting down the <command>postmaster</command> process.
|
|
</para>
|
|
|
|
<sect2 id="postmaster-start-failures">
|
|
<title>Server Start-up Failures</title>
|
|
|
|
<para>
|
|
There are several common reasons the server might fail to
|
|
start. Check the server's log file, or start it by hand (without
|
|
redirecting standard output or standard error) and see what error
|
|
messages appear. Below we explain some of the most common error
|
|
messages in more detail.
|
|
</para>
|
|
|
|
<para>
|
|
<screen>
|
|
LOG: could not bind IPv4 socket: Address already in use
|
|
HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
|
|
FATAL: could not create TCP/IP listen socket
|
|
</screen>
|
|
This usually means just what it suggests: you tried to start
|
|
another <command>postmaster</command> on the same port where one is already running.
|
|
However, if the kernel error message is not <computeroutput>Address
|
|
already in use</computeroutput> or some variant of that, there may
|
|
be a different problem. For example, trying to start a <command>postmaster</command>
|
|
on a reserved port number may draw something like:
|
|
<screen>
|
|
$ <userinput>postmaster -p 666</userinput>
|
|
LOG: could not bind IPv4 socket: Permission denied
|
|
HINT: Is another postmaster already running on port 666? If not, wait a few seconds and retry.
|
|
FATAL: could not create TCP/IP listen socket
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
A message like
|
|
<screen>
|
|
FATAL: could not create shared memory segment: Invalid argument
|
|
DETAIL: Failed system call was shmget(key=5440001, size=4011376640, 03600).
|
|
</screen>
|
|
probably means your kernel's limit on the size of shared memory is
|
|
smaller than the work area <productname>PostgreSQL</productname>
|
|
is trying to create (4011376640 bytes in this example). Or it could
|
|
mean that you do not have System-V-style shared memory support
|
|
configured into your kernel at all. As a temporary workaround, you
|
|
can try starting the server with a smaller-than-normal number of
|
|
buffers (<xref linkend="guc-shared-buffers">). You will eventually want
|
|
to reconfigure your kernel to increase the allowed shared memory
|
|
size. You may also see this message when trying to start multiple
|
|
servers on the same machine, if their total space requested
|
|
exceeds the kernel limit.
|
|
</para>
|
|
|
|
<para>
|
|
An error like
|
|
<screen>
|
|
FATAL: could not create semaphores: No space left on device
|
|
DETAIL: Failed system call was semget(5440126, 17, 03600).
|
|
</screen>
|
|
does <emphasis>not</emphasis> mean you've run out of disk
|
|
space. It means your kernel's limit on the number of <systemitem
|
|
class="osname">System V</> semaphores is smaller than the number
|
|
<productname>PostgreSQL</productname> wants to create. As above,
|
|
you may be able to work around the problem by starting the
|
|
server with a reduced number of allowed connections
|
|
(<xref linkend="guc-max-connections">), but you'll eventually want to
|
|
increase the kernel limit.
|
|
</para>
|
|
|
|
<para>
|
|
If you get an <quote>illegal system call</> error, it is likely that
|
|
shared memory or semaphores are not supported in your kernel at
|
|
all. In that case your only option is to reconfigure the kernel to
|
|
enable these features.
|
|
</para>
|
|
|
|
<para>
|
|
Details about configuring <systemitem class="osname">System V</>
|
|
<acronym>IPC</> facilities are given in <xref linkend="sysvipc">.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="client-connection-problems">
|
|
<title>Client Connection Problems</title>
|
|
|
|
<para>
|
|
Although the error conditions possible on the client side are quite
|
|
varied and application-dependent, a few of them might be directly
|
|
related to how the server was started up. Conditions other than
|
|
those shown below should be documented with the respective client
|
|
application.
|
|
</para>
|
|
|
|
<para>
|
|
<screen>
|
|
psql: could not connect to server: Connection refused
|
|
Is the server running on host "server.joe.com" and accepting
|
|
TCP/IP connections on port 5432?
|
|
</screen>
|
|
This is the generic <quote>I couldn't find a server to talk
|
|
to</quote> failure. It looks like the above when TCP/IP
|
|
communication is attempted. A common mistake is to forget to
|
|
configure the server to allow TCP/IP connections.
|
|
</para>
|
|
|
|
<para>
|
|
Alternatively, you'll get this when attempting Unix-domain socket
|
|
communication to a local server:
|
|
<screen>
|
|
psql: could not connect to server: No such file or directory
|
|
Is the server running locally and accepting
|
|
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
The last line is useful in verifying that the client is trying to
|
|
connect to the right place. If there is in fact no server
|
|
running there, the kernel error message will typically be either
|
|
<computeroutput>Connection refused</computeroutput> or
|
|
<computeroutput>No such file or directory</computeroutput>, as
|
|
illustrated. (It is important to realize that
|
|
<computeroutput>Connection refused</computeroutput> in this context
|
|
does <emphasis>not</emphasis> mean that the server got your
|
|
connection request and rejected it. That case will produce a
|
|
different message, as shown in <xref
|
|
linkend="client-authentication-problems">.) Other error messages
|
|
such as <computeroutput>Connection timed out</computeroutput> may
|
|
indicate more fundamental problems, like lack of network
|
|
connectivity.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="kernel-resources">
|
|
<title>Managing Kernel Resources</title>
|
|
|
|
<para>
|
|
A large <productname>PostgreSQL</> installation can quickly exhaust
|
|
various operating system resource limits. (On some systems, the
|
|
factory defaults are so low that you don't even need a really
|
|
<quote>large</> installation.) If you have encountered this kind of
|
|
problem, keep reading.
|
|
</para>
|
|
|
|
<sect2 id="sysvipc">
|
|
<title>Shared Memory and Semaphores</title>
|
|
|
|
<indexterm zone="sysvipc">
|
|
<primary>shared memory</primary>
|
|
</indexterm>
|
|
|
|
<indexterm zone="sysvipc">
|
|
<primary>semaphores</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
Shared memory and semaphores are collectively referred to as
|
|
<quote><systemitem class="osname">System V</>
|
|
<acronym>IPC</></quote> (together with message queues, which are not
|
|
relevant for <productname>PostgreSQL</>). Almost all modern
|
|
operating systems provide these features, but not all of them have
|
|
them turned on or sufficiently sized by default, especially systems
|
|
with BSD heritage. (For the <systemitem class="osname">Windows</>,
|
|
<systemitem class="osname">QNX</> and <systemitem class="osname">BeOS</>
|
|
ports, <productname>PostgreSQL</> provides its own replacement
|
|
implementation of these facilities.)
|
|
</para>
|
|
|
|
<para>
|
|
The complete lack of these facilities is usually manifested by an
|
|
<errorname>Illegal system call</> error upon server start. In
|
|
that case there's nothing left to do but to reconfigure your
|
|
kernel. <productname>PostgreSQL</> won't work without them.
|
|
</para>
|
|
|
|
<para>
|
|
When <productname>PostgreSQL</> exceeds one of the various hard
|
|
<acronym>IPC</> limits, the server will refuse to start and
|
|
should leave an instructive error message describing the problem
|
|
encountered and what to do about it. (See also <xref
|
|
linkend="postmaster-start-failures">.) The relevant kernel
|
|
parameters are named consistently across different systems; <xref
|
|
linkend="sysvipc-parameters"> gives an overview. The methods to set
|
|
them, however, vary. Suggestions for some platforms are given below.
|
|
Be warned that it is often necessary to reboot your machine, and
|
|
possibly even recompile the kernel, to change these settings.
|
|
</para>
|
|
|
|
|
|
<table id="sysvipc-parameters">
|
|
<title><systemitem class="osname">System V</> <acronym>IPC</> parameters</>
|
|
|
|
<tgroup cols="3">
|
|
<thead>
|
|
<row>
|
|
<entry>Name</>
|
|
<entry>Description</>
|
|
<entry>Reasonable values</>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody>
|
|
<row>
|
|
<entry><varname>SHMMAX</></>
|
|
<entry>Maximum size of shared memory segment (bytes)</>
|
|
<entry>at least several megabytes (see text)</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMMIN</></>
|
|
<entry>Minimum size of shared memory segment (bytes)</>
|
|
<entry>1</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMALL</></>
|
|
<entry>Total amount of shared memory available (bytes or pages)</>
|
|
<entry>if bytes, same as <varname>SHMMAX</varname>; if pages, <literal>ceil(SHMMAX/PAGE_SIZE)</literal></>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMSEG</></>
|
|
<entry>Maximum number of shared memory segments per process</>
|
|
<entry>only 1 segment is needed, but the default is much higher</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMMNI</></>
|
|
<entry>Maximum number of shared memory segments system-wide</>
|
|
<entry>like <varname>SHMSEG</> plus room for other applications</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMNI</></>
|
|
<entry>Maximum number of semaphore identifiers (i.e., sets)</>
|
|
<entry>at least <literal>ceil(max_connections / 16)</literal></>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMNS</></>
|
|
<entry>Maximum number of semaphores system-wide</>
|
|
<entry><literal>ceil(max_connections / 16) * 17</literal> plus room for other applications</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMSL</></>
|
|
<entry>Maximum number of semaphores per set</>
|
|
<entry>at least 17</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMAP</></>
|
|
<entry>Number of entries in semaphore map</>
|
|
<entry>see text</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMVMX</></>
|
|
<entry>Maximum value of semaphore</>
|
|
<entry>at least 1000 (The default is often 32767, don't change unless forced to)</>
|
|
</row>
|
|
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
|
|
|
|
<para>
|
|
<indexterm><primary>SHMMAX</primary></indexterm> The most important
|
|
shared memory parameter is <varname>SHMMAX</>, the maximum size, in
|
|
bytes, of a shared memory segment. If you get an error message from
|
|
<function>shmget</> like <errorname>Invalid argument</>, it is
|
|
likely that this limit has been exceeded. The size of the required
|
|
shared memory segment varies depending on several
|
|
<productname>PostgreSQL</> configuration parameters, as shown in
|
|
<xref linkend="shared-memory-parameters">.
|
|
You can, as a temporary solution, lower some of those settings to
|
|
avoid the failure. As a rough approximation, you can estimate the
|
|
required segment size as 500 kB plus the variable amounts shown in
|
|
the table. (Any error message you might get will include the exact
|
|
size of the failed allocation request.) While it is possible to get
|
|
<productname>PostgreSQL</> to run with <varname>SHMMAX</> as small as
|
|
1 MB, you need at least 4 MB for acceptable performance, and desirable
|
|
settings are in the tens of megabytes.
|
|
</para>
|
|
|
|
<para>
|
|
Some systems also have a limit on the total amount of shared memory in
|
|
the system (<varname>SHMALL</>). Make sure this is large enough
|
|
for <productname>PostgreSQL</> plus any other applications that
|
|
are using shared memory segments. (Caution: <varname>SHMALL</>
|
|
is measured in pages rather than bytes on many systems.)
|
|
</para>
|
|
|
|
<para>
|
|
Less likely to cause problems is the minimum size for shared
|
|
memory segments (<varname>SHMMIN</>), which should be at most
|
|
approximately 500 kB for <productname>PostgreSQL</> (it is
|
|
usually just 1). The maximum number of segments system-wide
|
|
(<varname>SHMMNI</>) or per-process (<varname>SHMSEG</>) are unlikely
|
|
to cause a problem unless your system has them set to zero.
|
|
</para>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</> uses one semaphore per allowed connection
|
|
(<xref linkend="guc-max-connections">), in sets of 16. Each such set will
|
|
also contain a 17th semaphore which contains a <quote>magic
|
|
number</quote>, to detect collision with semaphore sets used by
|
|
other applications. The maximum number of semaphores in the system
|
|
is set by <varname>SEMMNS</>, which consequently must be at least
|
|
as high as <varname>max_connections</> plus one extra for each 16
|
|
allowed connections (see the formula in <xref
|
|
linkend="sysvipc-parameters">). The parameter <varname>SEMMNI</>
|
|
determines the limit on the number of semaphore sets that can
|
|
exist on the system at one time. Hence this parameter must be at
|
|
least <literal>ceil(max_connections / 16)</>. Lowering the number
|
|
of allowed connections is a temporary workaround for failures,
|
|
which are usually confusingly worded <errorname>No space
|
|
left on device</>, from the function <function>semget</>.
|
|
</para>
|
|
|
|
<para>
|
|
In some cases it might also be necessary to increase
|
|
<varname>SEMMAP</> to be at least on the order of
|
|
<varname>SEMMNS</>. This parameter defines the size of the semaphore
|
|
resource map, in which each contiguous block of available semaphores
|
|
needs an entry. When a semaphore set is freed it is either added to
|
|
an existing entry that is adjacent to the freed block or it is
|
|
registered under a new map entry. If the map is full, the freed
|
|
semaphores get lost (until reboot). Fragmentation of the semaphore
|
|
space could over time lead to fewer available semaphores than there
|
|
should be.
|
|
</para>
|
|
|
|
<para>
|
|
The <varname>SEMMSL</> parameter, which determines how many
|
|
semaphores can be in a set, must be at least 17 for
|
|
<productname>PostgreSQL</>.
|
|
</para>
|
|
|
|
<para>
|
|
Various other settings related to <quote>semaphore undo</>, such as
|
|
<varname>SEMMNU</> and <varname>SEMUME</>, are not of concern
|
|
for <productname>PostgreSQL</>.
|
|
</para>
|
|
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">BSD/OS</></term>
|
|
<indexterm><primary>BSD/OS</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<formalpara>
|
|
<title>Shared Memory</>
|
|
<para>
|
|
By default, only 4 MB of shared memory is supported. Keep in
|
|
mind that shared memory is not pageable; it is locked in RAM.
|
|
To increase the amount of shared memory supported by your
|
|
system, add something like the following to your kernel configuration
|
|
file:
|
|
<programlisting>
|
|
options "SHMALL=8192"
|
|
options "SHMMAX=\(SHMALL*PAGE_SIZE\)"
|
|
</programlisting>
|
|
<varname>SHMALL</> is measured in 4KB pages, so a value of
|
|
1024 represents 4 MB of shared memory. Therefore the above increases
|
|
the maximum shared memory area to 32 MB.
|
|
For those running 4.3 or later, you will probably also need to increase
|
|
<varname>KERNEL_VIRTUAL_MB</> above the default <literal>248</>.
|
|
Once all changes have been made, recompile the kernel, and reboot.
|
|
</para>
|
|
</formalpara>
|
|
|
|
<para>
|
|
For those running 4.0 and earlier releases, use <command>bpatch</>
|
|
to find the <varname>sysptsize</> value in the current
|
|
kernel. This is computed dynamically at boot time.
|
|
<screen>
|
|
$ <userinput>bpatch -r sysptsize</>
|
|
<computeroutput>0x9 = 9</>
|
|
</screen>
|
|
Next, add <varname>SYSPTSIZE</> as a hard-coded value in the
|
|
kernel configuration file. Increase the value you found using
|
|
<command>bpatch</>. Add 1 for every additional 4 MB of
|
|
shared memory you desire.
|
|
<programlisting>
|
|
options "SYSPTSIZE=16"
|
|
</programlisting>
|
|
<varname>sysptsize</> cannot be changed by <command>sysctl</command>.
|
|
</para>
|
|
|
|
<formalpara>
|
|
<title>Semaphores</>
|
|
<para>
|
|
You will probably want to increase the number of semaphores
|
|
as well; the default system total of 60 will only allow about
|
|
50 <productname>PostgreSQL</productname> connections. Set the
|
|
values you want in your kernel configuration file, e.g.:
|
|
<programlisting>
|
|
options "SEMMNI=40"
|
|
options "SEMMNS=240"
|
|
</programlisting>
|
|
</para>
|
|
</formalpara>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">FreeBSD</></term>
|
|
<indexterm><primary>FreeBSD</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings are only suitable for small installations
|
|
(for example, default <varname>SHMMAX</varname> is 32
|
|
MB). Changes can be made via the <command>sysctl</command> or
|
|
<command>loader</command> interfaces. The following
|
|
parameters can be set using <command>sysctl</command>:
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmall=32768</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmmax=134217728</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.semmap=256</userinput>
|
|
</screen>
|
|
To have these settings persist over reboots, modify
|
|
<filename>/etc/sysctl.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
The remaining semaphore settings are read-only as far as
|
|
<command>sysctl</command> is concerned, but can be changed
|
|
before boot using the <command>loader</command> prompt:
|
|
<screen>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmni=256</userinput>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmns=512</userinput>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmnu=256</userinput>
|
|
</screen>
|
|
Similarly these can be saved between reboots in
|
|
<filename>/boot/loader.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
You might also want to configure your kernel to lock shared
|
|
memory into RAM and prevent it from being paged out to swap.
|
|
This can be accomplished using the <command>sysctl</command>
|
|
setting <literal>kern.ipc.shm_use_phys</literal>.
|
|
</para>
|
|
|
|
<para>
|
|
<systemitem class="osname">FreeBSD</> versions before 4.0 work like
|
|
<systemitem class="osname">NetBSD</> and <systemitem class="osname">
|
|
OpenBSD</> (see below).
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">NetBSD</></term>
|
|
<term><systemitem class="osname">OpenBSD</></term>
|
|
<indexterm><primary>NetBSD</><secondary>IPC configuration</></>
|
|
<indexterm><primary>OpenBSD</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The options <varname>SYSVSHM</> and <varname>SYSVSEM</> need
|
|
to be enabled when the kernel is compiled. (They are by
|
|
default.) The maximum size of shared memory is determined by
|
|
the option <varname>SHMMAXPGS</> (in pages). The following
|
|
shows an example of how to set the various parameters
|
|
(<systemitem class="osname">OpenBSD</> uses <literal>option</> instead):
|
|
<programlisting>
|
|
options SYSVSHM
|
|
options SHMMAXPGS=4096
|
|
options SHMSEG=256
|
|
|
|
options SYSVSEM
|
|
options SEMMNI=256
|
|
options SEMMNS=512
|
|
options SEMMNU=256
|
|
options SEMMAP=256
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
You might also want to configure your kernel to lock shared
|
|
memory into RAM and prevent it from being paged out to swap.
|
|
This can be accomplished using the <command>sysctl</command>
|
|
setting <literal>kern.ipc.shm_use_phys</literal>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">HP-UX</></term>
|
|
<indexterm><primary>HP-UX</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings tend to suffice for normal installations.
|
|
On <productname>HP-UX</> 10, the factory default for
|
|
<varname>SEMMNS</> is 128, which might be too low for larger
|
|
database sites.
|
|
</para>
|
|
<para>
|
|
<acronym>IPC</> parameters can be set in the <application>System
|
|
Administration Manager</> (<acronym>SAM</>) under
|
|
<menuchoice><guimenu>Kernel
|
|
Configuration</><guimenuitem>Configurable Parameters</></>. Hit
|
|
<guibutton>Create A New Kernel</> when you're done.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Linux</></term>
|
|
<indexterm><primary>Linux</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings are only suitable for small installations
|
|
(the default max segment size is 32 MB). However the remaining
|
|
defaults are quite generously sized, and usually do not require
|
|
changes. The max segment size can be changed via the
|
|
<command>sysctl</command> interface. For example, to allow 128 MB,
|
|
and explicitly set the maximum total shared memory size to 2097152
|
|
pages (the default):
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>sysctl -w kernel.shmmax=134217728</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kernel.shmall=2097152</userinput>
|
|
</screen>
|
|
In addition these settings can be saved between reboots in
|
|
<filename>/etc/sysctl.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
Older distributions may not have the <command>sysctl</command> program,
|
|
but equivalent changes can be made by manipulating the
|
|
<filename>/proc</filename> filesystem:
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>echo 134217728 >/proc/sys/kernel/shmmax</userinput>
|
|
<prompt>$</prompt> <userinput>echo 2097152 >/proc/sys/kernel/shmall</userinput>
|
|
</screen>
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">MacOS X</></term>
|
|
<indexterm><primary>MacOS X</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
In OS X 10.2 and earlier, edit the file
|
|
<filename>/System/Library/StartupItems/SystemTuning/SystemTuning</>
|
|
and change the values in the following commands:
|
|
<programlisting>
|
|
sysctl -w kern.sysv.shmmax
|
|
sysctl -w kern.sysv.shmmin
|
|
sysctl -w kern.sysv.shmmni
|
|
sysctl -w kern.sysv.shmseg
|
|
sysctl -w kern.sysv.shmall
|
|
</programlisting>
|
|
In OS X 10.3 and later, these commands have been moved to
|
|
<filename>/etc/rc</> and must be edited there. Note that
|
|
<filename>/etc/rc</> is usually overwritten by OS X updates (such as
|
|
10.3.6 to 10.3.7) so you should expect to have to redo your editing
|
|
after each update. In all versions, you'll need to reboot to make
|
|
changes take effect.
|
|
</para>
|
|
|
|
<!--
|
|
It might appear that we could recommend putting sysctl settings into
|
|
/etc/sysctl.conf so that they aren't destroyed by OS version updates.
|
|
This does not work as of OS X 10.4.2, however, because /etc/rc
|
|
promptly overrides whatever SHM settings are read from sysctl.conf :-(
|
|
Perhaps someday we can recommend this ...
|
|
-->
|
|
|
|
<para>
|
|
<varname>SHMALL</> is measured in 4KB pages on this platform.
|
|
Also note that some releases of OS X will reject attempts to
|
|
set <varname>SHMMAX</> to a value that isn't an exact multiple
|
|
of 4096.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">SCO OpenServer</></term>
|
|
<indexterm><primary>SCO OpenServer</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
In the default configuration, only 512 kB of shared memory per
|
|
segment is allowed. To increase the setting, first change to the
|
|
directory <filename>/etc/conf/cf.d</>. To display the current value of
|
|
<varname>SHMMAX</>, run
|
|
<programlisting>
|
|
./configure -y SHMMAX
|
|
</programlisting>
|
|
To set a new value for <varname>SHMMAX</>, run
|
|
<programlisting>
|
|
./configure SHMMAX=<replaceable>value</>
|
|
</programlisting>
|
|
where <replaceable>value</> is the new value you want to use
|
|
(in bytes). After setting <varname>SHMMAX</>, rebuild the kernel:
|
|
<programlisting>
|
|
./link_unix
|
|
</programlisting>
|
|
and reboot.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">AIX</></term>
|
|
<indexterm><primary>AIX</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
At least as of version 5.1, it should not be necessary to do
|
|
any special configuration for such parameters as
|
|
<varname>SHMMAX</varname>, as it appears this is configured to
|
|
allow all memory to be used as shared memory. That is the
|
|
sort of configuration commonly used for other databases such
|
|
as <application>DB/2</application>.</para>
|
|
|
|
<para> It may, however, be necessary to modify the global
|
|
<command>ulimit</command> information in
|
|
<filename>/etc/security/limits</filename>, as the default hard
|
|
limits for file sizes (<varname>fsize</varname>) and numbers of
|
|
files (<varname>nofiles</varname>) may be too low.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Solaris</></term>
|
|
<indexterm><primary>Solaris</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
At least in version 2.6, the default maximum size of a shared
|
|
memory segments is too low for <productname>PostgreSQL</>. The
|
|
relevant settings can be changed in <filename>/etc/system</>,
|
|
for example:
|
|
<programlisting>
|
|
set shmsys:shminfo_shmmax=0x2000000
|
|
set shmsys:shminfo_shmmin=1
|
|
set shmsys:shminfo_shmmni=256
|
|
set shmsys:shminfo_shmseg=256
|
|
|
|
set semsys:seminfo_semmap=256
|
|
set semsys:seminfo_semmni=512
|
|
set semsys:seminfo_semmns=512
|
|
set semsys:seminfo_semmsl=32
|
|
</programlisting>
|
|
You need to reboot for the changes to take effect.
|
|
</para>
|
|
|
|
<para>
|
|
See also <ulink
|
|
url="http://sunsite.uakom.sk/sunworldonline/swol-09-1997/swol-09-insidesolaris.html"></>
|
|
for information on shared memory under
|
|
<productname>Solaris</>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">UnixWare</></term>
|
|
<indexterm><primary>UnixWare</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
On <productname>UnixWare</> 7, the maximum size for shared
|
|
memory segments is only 512 kB in the default configuration.
|
|
To display the current value of <varname>SHMMAX</>, run
|
|
<programlisting>
|
|
/etc/conf/bin/idtune -g SHMMAX
|
|
</programlisting>
|
|
which displays the current, default, minimum, and maximum
|
|
values. To set a new value for <varname>SHMMAX</>,
|
|
run
|
|
<programlisting>
|
|
/etc/conf/bin/idtune SHMMAX <replaceable>value</>
|
|
</programlisting>
|
|
where <replaceable>value</> is the new value you want to use
|
|
(in bytes). After setting <varname>SHMMAX</>, rebuild the
|
|
kernel:
|
|
<programlisting>
|
|
/etc/conf/bin/idbuild -B
|
|
</programlisting>
|
|
and reboot.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
|
|
<table id="shared-memory-parameters">
|
|
<title>Configuration parameters affecting
|
|
<productname>PostgreSQL</productname>'s shared memory usage</>
|
|
|
|
<tgroup cols="2">
|
|
<thead>
|
|
<row>
|
|
<entry>Name</>
|
|
<entry>Approximate multiplier (bytes per increment)</>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody>
|
|
<row>
|
|
<entry><xref linkend="guc-max-connections"></>
|
|
<entry>400 + 220 * <xref linkend="guc-max-locks-per-transaction"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-max-prepared-transactions"></>
|
|
<entry>600 + 220 * <xref linkend="guc-max-locks-per-transaction"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-shared-buffers"></>
|
|
<entry>8300 (assuming 8K <symbol>BLCKSZ</>)</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-wal-buffers"></>
|
|
<entry>8200 (assuming 8K <symbol>BLCKSZ</>)</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-max-fsm-relations"></>
|
|
<entry>70</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-max-fsm-pages"></>
|
|
<entry>6</>
|
|
</row>
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
</sect2>
|
|
|
|
|
|
<sect2>
|
|
<title>Resource Limits</title>
|
|
|
|
<para>
|
|
Unix-like operating systems enforce various kinds of resource limits
|
|
that might interfere with the operation of your
|
|
<productname>PostgreSQL</productname> server. Of particular
|
|
importance are limits on the number of processes per user, the
|
|
number of open files per process, and the amount of memory available
|
|
to each process. Each of these have a <quote>hard</quote> and a
|
|
<quote>soft</quote> limit. The soft limit is what actually counts
|
|
but it can be changed by the user up to the hard limit. The hard
|
|
limit can only be changed by the root user. The system call
|
|
<function>setrlimit</function> is responsible for setting these
|
|
parameters. The shell's built-in command <command>ulimit</command>
|
|
(Bourne shells) or <command>limit</command> (<application>csh</>) is
|
|
used to control the resource limits from the command line. On
|
|
BSD-derived systems the file <filename>/etc/login.conf</filename>
|
|
controls the various resource limits set during login. See the
|
|
operating system documentation for details. The relevant
|
|
parameters are <varname>maxproc</varname>,
|
|
<varname>openfiles</varname>, and <varname>datasize</varname>. For
|
|
example:
|
|
<programlisting>
|
|
default:\
|
|
...
|
|
:datasize-cur=256M:\
|
|
:maxproc-cur=256:\
|
|
:openfiles-cur=256:\
|
|
...
|
|
</programlisting>
|
|
(<literal>-cur</literal> is the soft limit. Append
|
|
<literal>-max</literal> to set the hard limit.)
|
|
</para>
|
|
|
|
<para>
|
|
Kernels can also have system-wide limits on some resources.
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
On <productname>Linux</productname>
|
|
<filename>/proc/sys/fs/file-max</filename> determines the
|
|
maximum number of open files that the kernel will support. It can
|
|
be changed by writing a different number into the file or by
|
|
adding an assignment in <filename>/etc/sysctl.conf</filename>.
|
|
The maximum limit of files per process is fixed at the time the
|
|
kernel is compiled; see
|
|
<filename>/usr/src/linux/Documentation/proc.txt</filename> for
|
|
more information.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
The <productname>PostgreSQL</productname> server uses one process
|
|
per connection so you should provide for at least as many processes
|
|
as allowed connections, in addition to what you need for the rest
|
|
of your system. This is usually not a problem but if you run
|
|
several servers on one machine things might get tight.
|
|
</para>
|
|
|
|
<para>
|
|
The factory default limit on open files is often set to
|
|
<quote>socially friendly</quote> values that allow many users to
|
|
coexist on a machine without using an inappropriate fraction of
|
|
the system resources. If you run many servers on a machine this
|
|
is perhaps what you want, but on dedicated servers you may want to
|
|
raise this limit.
|
|
</para>
|
|
|
|
<para>
|
|
On the other side of the coin, some systems allow individual
|
|
processes to open large numbers of files; if more than a few
|
|
processes do so then the system-wide limit can easily be exceeded.
|
|
If you find this happening, and you do not want to alter the
|
|
system-wide limit, you can set <productname>PostgreSQL</>'s <xref
|
|
linkend="guc-max-files-per-process"> configuration parameter to
|
|
limit the consumption of open files.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Linux Memory Overcommit</title>
|
|
|
|
<para>
|
|
In Linux 2.4 and later, the default virtual memory behavior is not
|
|
optimal for <productname>PostgreSQL</productname>. Because of the
|
|
way that the kernel implements memory overcommit, the kernel may
|
|
terminate the <productname>PostgreSQL</productname> server (the
|
|
<filename>postmaster</filename> process) if the memory demands of
|
|
another process cause the system to run out of virtual memory.
|
|
</para>
|
|
|
|
<para>
|
|
If this happens, you will see a kernel message that looks like
|
|
this (consult your system documentation and configuration on where
|
|
to look for such a message):
|
|
<programlisting>
|
|
Out of Memory: Killed process 12345 (postmaster).
|
|
</programlisting>
|
|
This indicates that the <filename>postmaster</filename> process
|
|
has been terminated due to memory pressure.
|
|
Although existing database connections will continue to function
|
|
normally, no new connections will be accepted. To recover,
|
|
<productname>PostgreSQL</productname> will need to be restarted.
|
|
</para>
|
|
|
|
<para>
|
|
One way to avoid this problem is to run
|
|
<productname>PostgreSQL</productname>
|
|
on a machine where you can be sure that other processes will not
|
|
run the machine out of memory.
|
|
</para>
|
|
|
|
<para>
|
|
On Linux 2.6 and later, a better solution is to modify the kernel's
|
|
behavior so that it will not <quote>overcommit</> memory. This is
|
|
done by selecting strict overcommit mode via <command>sysctl</command>:
|
|
<programlisting>
|
|
sysctl -w vm.overcommit_memory=2
|
|
</programlisting>
|
|
or placing an equivalent entry in <filename>/etc/sysctl.conf</>.
|
|
You may also wish to modify the related setting
|
|
<literal>vm.overcommit_ratio</>. For details see the kernel documentation
|
|
file <filename>Documentation/vm/overcommit-accounting</>.
|
|
</para>
|
|
|
|
<para>
|
|
Some vendors' Linux 2.4 kernels are reported to have early versions
|
|
of the 2.6 overcommit <command>sysctl</command> parameter. However, setting
|
|
<literal>vm.overcommit_memory</> to 2
|
|
on a kernel that does not have the relevant code will make
|
|
things worse not better. It is recommended that you inspect
|
|
the actual kernel source code (see the function
|
|
<function>vm_enough_memory</> in the file <filename>mm/mmap.c</>)
|
|
to verify what is supported in your copy before you try this in a 2.4
|
|
installation. The presence of the <filename>overcommit-accounting</>
|
|
documentation file should <emphasis>not</> be taken as evidence that the
|
|
feature is there. If in any doubt, consult a kernel expert or your
|
|
kernel vendor.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
|
|
<sect1 id="postmaster-shutdown">
|
|
<title>Shutting Down the Server</title>
|
|
|
|
<indexterm zone="postmaster-shutdown">
|
|
<primary>shutdown</>
|
|
</indexterm>
|
|
|
|
<para>
|
|
There are several ways to shut down the database server. You control
|
|
the type of shutdown by sending different signals to the
|
|
<command>postmaster</command> process.
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><systemitem>SIGTERM</systemitem><indexterm><primary>SIGTERM</></></term>
|
|
<listitem>
|
|
<para>
|
|
After receiving <systemitem>SIGTERM</systemitem>, the server
|
|
disallows new connections, but lets existing sessions end their
|
|
work normally. It shuts down only after all of the sessions
|
|
terminate normally. This is the <firstterm>Smart
|
|
Shutdown</firstterm>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem>SIGINT</systemitem><indexterm><primary>SIGINT</></></term>
|
|
<listitem>
|
|
<para>
|
|
The server disallows new connections and sends all existing
|
|
server processes <systemitem>SIGTERM</systemitem>, which will cause them
|
|
to abort their current transactions and exit promptly. It then
|
|
waits for the server processes to exit and finally shuts down. This is the
|
|
<firstterm>Fast Shutdown</firstterm>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem>SIGQUIT</systemitem><indexterm><primary>SIGQUIT</></></term>
|
|
<listitem>
|
|
<para>
|
|
This is the <firstterm>Immediate Shutdown</firstterm>, which
|
|
will cause the <command>postmaster</command> process to send a
|
|
<systemitem>SIGQUIT</systemitem> to all child processes and exit
|
|
immediately, without properly shutting itself down. The child processes
|
|
likewise exit immediately upon receiving
|
|
<systemitem>SIGQUIT</systemitem>. This will lead to recovery (by
|
|
replaying the WAL log) upon next start-up. This is recommended
|
|
only in emergencies.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
|
|
<para>
|
|
The <xref linkend="app-pg-ctl"> program provides a convenient
|
|
interface for sending these signals to shut down the server.
|
|
</para>
|
|
|
|
<para>
|
|
Alternatively, you can send the signal directly using <command>kill</>.
|
|
The <acronym>PID</> of the <command>postmaster</command> process can be
|
|
found using the <command>ps</command> program, or from the file
|
|
<filename>postmaster.pid</filename> in the data directory. For
|
|
example, to do a fast shutdown:
|
|
<screen>
|
|
$ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput>
|
|
</screen>
|
|
</para>
|
|
|
|
<important>
|
|
<para>
|
|
It is best not to use <systemitem>SIGKILL</systemitem> to shut down
|
|
the server. Doing so will prevent the server from releasing
|
|
shared memory and semaphores, which may then have to be done
|
|
manually before a new server can be started. Furthermore,
|
|
<systemitem>SIGKILL</systemitem> kills the <command>postmaster</command>
|
|
process without letting it relay the signal to its subprocesses,
|
|
so it will be necessary to kill the individual subprocesses by hand as
|
|
well.
|
|
</para>
|
|
</important>
|
|
</sect1>
|
|
|
|
<sect1 id="encryption-options">
|
|
<title>Encryption Options</title>
|
|
|
|
<indexterm zone="encryption-options">
|
|
<primary>encryption</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</productname> offers encryption at several
|
|
levels, and provides flexibility in protecting data from disclosure
|
|
due to database server theft, unscrupulous administrators, and
|
|
insecure networks. Encryption might also be required to secure
|
|
sensitive data such as medical records or financial transactions.
|
|
</para>
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term>Password Storage Encryption</term>
|
|
<listitem>
|
|
|
|
<para>
|
|
By default, database user passwords are stored as MD5 hashes, so
|
|
the administrator cannot determine the actual password assigned
|
|
to the user. If MD5 encryption is used for client authentication,
|
|
the unencrypted password is never even temporarily present on the
|
|
server because the client MD5 encrypts it before being sent
|
|
across the network.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encryption For Specific Columns</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
The <filename>/contrib</> function library
|
|
<function>pgcrypto</function> allows certain fields to be stored
|
|
encrypted. This is useful if only some of the data is sensitive.
|
|
The client supplies the decryption key and the data is decrypted
|
|
on the server and then sent to the client.
|
|
</para>
|
|
|
|
<para>
|
|
The decrypted data and the decryption key are present on the
|
|
server for a brief time while it is being decrypted and
|
|
communicated between the client and server. This presents a brief
|
|
moment where the data and keys can be intercepted by someone with
|
|
complete access to the database server, such as the system
|
|
administrator.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Data Partition Encryption</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
On Linux, encryption can be layered on top of a filesystem mount
|
|
using a <quote>loopback device</quote>. This allows an entire
|
|
filesystem partition be encrypted on disk, and decrypted by the
|
|
operating system. On FreeBSD, the equivalent facility is called
|
|
GEOM Based Disk Encryption, or <acronym>gbde</acronym>.
|
|
</para>
|
|
|
|
<para>
|
|
This mechanism prevents unecrypted data from being read from the
|
|
drives if the drives or the entire computer is stolen. This does
|
|
not protect against attacks while the filesystem is mounted,
|
|
because when mounted, the operating system provides an unencrypted
|
|
view of the data. However, to mount the filesystem, you need some
|
|
way for the encryption key to be passed to the operating system,
|
|
and sometimes the key is stored somewhere on the host that mounts
|
|
the disk.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encrypting Passwords Across A Network</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
The <literal>MD5</> authentication method double-encrypts the
|
|
password on the client before sending it to the server. It first
|
|
MD5 encrypts it based on the user name, and then encrypts it
|
|
based on a random salt sent by the server when the database
|
|
connection was made. It is this double-encrypted value that is
|
|
sent over the network to the server. Double-encryption not only
|
|
prevents the password from being discovered, it also prevents
|
|
another connection from using the same encrypted password to
|
|
connect to the database server at a later time.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encrypting Data Across A Network</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
SSL connections encrypt all data sent across the network: the
|
|
password, the queries, and the data returned. The
|
|
<filename>pg_hba.conf</> file allows administrators to specify
|
|
which hosts can use non-encrypted connections (<literal>host</>)
|
|
and which require SSL-encrypted connections
|
|
(<literal>hostssl</>). Also, clients can specify that they
|
|
connect to servers only via SSL. <application>Stunnel</> or
|
|
<application>SSH</> can also be used to encrypt transmissions.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>SSL Host Authentication</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
It is possible for both the client and server to provide SSL keys
|
|
or certificates to each other. It takes some extra configuration
|
|
on each side, but this provides stronger verification of identity
|
|
than the mere use of passwords. It prevents a computer from
|
|
pretending to be the server just long enough to read the password
|
|
send by the client. It also helps prevent "man in the middle"
|
|
attacks where a computer between the client and server pretends to
|
|
be the server and reads and passes all data between the client and
|
|
server.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Client-Side Encryption</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
If the system administrator cannot be trusted, it is necessary
|
|
for the client to encrypt the data; this way, unencrypted data
|
|
never appears on the database server. Data is encrypted on the
|
|
client before being sent to the server, and database results have
|
|
to be decrypted on the client before being used.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="ssl-tcp">
|
|
<title>Secure TCP/IP Connections with SSL</title>
|
|
|
|
<indexterm zone="ssl-tcp">
|
|
<primary>SSL</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</> has native support for using
|
|
<acronym>SSL</> connections to encrypt client/server communications
|
|
for increased security. This requires that
|
|
<productname>OpenSSL</productname> is installed on both client and
|
|
server systems and that support in <productname>PostgreSQL</> is
|
|
enabled at build time (see <xref linkend="installation">).
|
|
</para>
|
|
|
|
<para>
|
|
With <acronym>SSL</> support compiled in, the
|
|
<productname>PostgreSQL</> server can be started with
|
|
<acronym>SSL</> enabled by setting the parameter
|
|
<xref linkend="guc-ssl"> to <literal>on</> in
|
|
<filename>postgresql.conf</>. When
|
|
starting in <acronym>SSL</> mode, the server will look for the
|
|
files <filename>server.key</> and <filename>server.crt</> in the
|
|
data directory, which must contain the server private key
|
|
and certificate, respectively. These files must be set up correctly
|
|
before an <acronym>SSL</>-enabled server can start. If the private key is
|
|
protected with a passphrase, the server will prompt for the
|
|
passphrase and will not start until it has been entered.
|
|
</para>
|
|
|
|
<para>
|
|
The server will listen for both standard and <acronym>SSL</>
|
|
connections on the same TCP port, and will negotiate with any
|
|
connecting client on whether to use <acronym>SSL</>. By default,
|
|
this is at the client's option; see <xref
|
|
linkend="auth-pg-hba-conf"> about how to set up the server to
|
|
require use of <acronym>SSL</> for some or all connections.
|
|
</para>
|
|
|
|
<para>
|
|
For details on how to create your server private key and certificate,
|
|
refer to the <productname>OpenSSL</> documentation. A
|
|
self-signed certificate can be used for testing, but a
|
|
certificate signed by a certificate authority (<acronym>CA</>)
|
|
(either one of the global
|
|
<acronym>CAs</> or a local one) should be used in production so the
|
|
client can verify the server's identity. To create a quick
|
|
self-signed certificate, use the following
|
|
<productname>OpenSSL</productname> command:
|
|
<programlisting>
|
|
openssl req -new -text -out server.req
|
|
</programlisting>
|
|
Fill out the information that <command>openssl</> asks for. Make sure
|
|
that you enter the local host name as <quote>Common Name</>; the challenge
|
|
password can be left blank. The program will generate a key that is
|
|
passphrase protected; it will not accept a passphrase that is less
|
|
than four characters long. To remove the passphrase (as you must if
|
|
you want automatic start-up of the server), run the commands
|
|
<programlisting>
|
|
openssl rsa -in privkey.pem -out server.key
|
|
rm privkey.pem
|
|
</programlisting>
|
|
Enter the old passphrase to unlock the existing key. Now do
|
|
<programlisting>
|
|
openssl req -x509 -in server.req -text -key server.key -out server.crt
|
|
chmod og-rwx server.key
|
|
</programlisting>
|
|
to turn the certificate into a self-signed certificate and to copy the
|
|
key and certificate to where the server will look for them.
|
|
</para>
|
|
|
|
<para>
|
|
If verification of client certificates is required, place the
|
|
certificates of the <acronym>CA</acronym>(s) you wish to check for in
|
|
the file <filename>root.crt</filename> in the data directory. When
|
|
present, a client certificate will be requested from the client
|
|
during SSL connection startup, and it must have been signed by one of the
|
|
certificates present in <filename>root.crt</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
When the <filename>root.crt</filename> file is not present, client
|
|
certificates will not be requested or checked. In this mode, SSL
|
|
provides communication security but not authentication.
|
|
</para>
|
|
|
|
<para>
|
|
The files <filename>server.key</>, <filename>server.crt</>,
|
|
and <filename>root.crt</filename> are only examined during server
|
|
start; so you must restart the server to make changes in them take
|
|
effect.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="ssh-tunnels">
|
|
<title>Secure TCP/IP Connections with <application>SSH</application> Tunnels</title>
|
|
|
|
<indexterm zone="ssh-tunnels">
|
|
<primary>ssh</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
One can use <application>SSH</application> to encrypt the network
|
|
connection between clients and a
|
|
<productname>PostgreSQL</productname> server. Done properly, this
|
|
provides an adequately secure network connection, even for non-SSL-capable
|
|
clients.
|
|
</para>
|
|
|
|
<para>
|
|
First make sure that an <application>SSH</application> server is
|
|
running properly on the same machine as the
|
|
<productname>PostgreSQL</productname> server and that you can log in using
|
|
<command>ssh</command> as some user. Then you can establish a secure
|
|
tunnel with a command like this from the client machine:
|
|
<programlisting>
|
|
ssh -L 3333:foo.com:5432 joe@foo.com
|
|
</programlisting>
|
|
The first number in the <option>-L</option> argument, 3333, is the
|
|
port number of your end of the tunnel; it can be chosen freely. The
|
|
second number, 5432, is the remote end of the tunnel: the port
|
|
number your server is using. The name or IP address between
|
|
the port numbers is the host with the database server you are going
|
|
to connect to. In order to connect to the database server using
|
|
this tunnel, you connect to port 3333 on the local machine:
|
|
<programlisting>
|
|
psql -h localhost -p 3333 postgres
|
|
</programlisting>
|
|
To the database server it will then look as though you are really
|
|
user <literal>joe@foo.com</literal> and it will use whatever
|
|
authentication procedure was configured for connections from this
|
|
user and host. Note that the server will not think the connection is
|
|
SSL-encrypted, since in fact it is not encrypted between the
|
|
<application>SSH</application> server and the
|
|
<productname>PostgreSQL</productname> server. This should not pose any
|
|
extra security risk as long as they are on the same machine.
|
|
</para>
|
|
<para>
|
|
In order for the
|
|
tunnel setup to succeed you must be allowed to connect via
|
|
<command>ssh</command> as <literal>joe@foo.com</literal>, just
|
|
as if you had attempted to use <command>ssh</command> to set up a
|
|
terminal session.
|
|
</para>
|
|
|
|
<tip>
|
|
<para>
|
|
Several other applications exist that can provide secure tunnels using
|
|
a procedure similar in concept to the one just described.
|
|
</para>
|
|
</tip>
|
|
|
|
</sect1>
|
|
|
|
</chapter>
|
|
|
|
<!-- Keep this comment at the end of the file
|
|
Local variables:
|
|
mode:sgml
|
|
sgml-omittag:nil
|
|
sgml-shorttag:t
|
|
sgml-minimize-attributes:nil
|
|
sgml-always-quote-attributes:t
|
|
sgml-indent-step:1
|
|
sgml-indent-data:t
|
|
sgml-parent-document:nil
|
|
sgml-default-dtd-file:"./reference.ced"
|
|
sgml-exposed-tags:nil
|
|
sgml-local-catalogs:("/usr/lib/sgml/catalog")
|
|
sgml-local-ecat-files:nil
|
|
End:
|
|
-->
|