rdelaage/postgresql
1
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2024-09-28 00:01:51 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add shared_memory_type GUC.

Browse Source 
Since 9.3 we have used anonymous shared mmap for our main shared memory
region, except in EXEC_BACKEND builds.  Provide a GUC so that users
can opt for System V shared memory once again, like in 9.2 and earlier.

A later patch proposes to add huge/large page support for AIX, which
requires System V shared memory and provided the motivation to revive
this possibility.  It may also be useful on some BSDs.

Author: Andres Freund (revived and documented by Thomas Munro)
Discussion: https://postgr.es/m/HE1PR0202MB28126DB4E0B6621CC6A1A91286D90%40HE1PR0202MB2812.eurprd02.prod.outlook.com
Discussion: https://postgr.es/m/2AE143D2-87D3-4AD1-AC78-CE2258230C05%40FreeBSD.org
This commit is contained in:
Thomas Munro 2019-02-03 09:55:39 +01:00
parent 0d1fe9f74e
commit f1bebef60e
7 changed files with 101 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
doc/src/sgml/config.sgml
View File

@ -1694,6 +1694,31 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
<varlistentry id="guc-shared-memory-type" xreflabel="shared_memory_type">
<term><varname>shared_memory_type</varname> (<type>enum</type>)
<indexterm>
<primary><varname>shared_memory_type</varname> configuration parameter</primary>
</indexterm>
</term>
<listitem>
<para>
Specifies the shared memory implementation that the server
should use for the main shared memory region that holds
<productname>PostgreSQL</productname>'s shared buffers and other
shared data. Possible values are <literal>mmap</literal> (for
anonymous shared memory allocated using <function>mmap</function>),
<literal>sysv</literal> (for System V shared memory allocated via
<function>shmget</function>) and <literal>windows</literal> (for Windows
shared memory). Not all values are supported on all platforms; the
first supported option is the default for that platform. The use of
the <literal>sysv</literal> option, which is not the default on any
platform, is generally discouraged because it typically requires
non-default kernel settings to allow for large allocations (see <xref
linkend="sysvipc"/>).
</para>
</listitem>
</varlistentry>
<varlistentry id="guc-dynamic-shared-memory-type" xreflabel="dynamic_shared_memory_type">
<term><varname>dynamic_shared_memory_type</varname> (<type>enum</type>)
<indexterm>

17
doc/src/sgml/runtime.sgml
View File

@ -638,9 +638,12 @@ psql: could not connect to server: No such file or directory
</para>
<para>
Upon starting the server, <productname>PostgreSQL</productname> normally allocates
By default, <productname>PostgreSQL</productname> allocates
a very small amount of System V shared memory, as well as a much larger
amount of POSIX (<function>mmap</function>) shared memory.
amount of anonymous <function>mmap</function> shared memory.
Alternatively, a single large System V shared memory region can be used
(see <xref linkend="guc-shared-memory-type"/>).
In addition a significant number of semaphores, which can be either
System V or POSIX style, are created at server startup. Currently,
POSIX semaphores are used on Linux and FreeBSD systems while other
@ -752,8 +755,10 @@ psql: could not connect to server: No such file or directory
<productname>PostgreSQL</productname> requires a few bytes of System V shared memory
(typically 48 bytes, on 64-bit platforms) for each copy of the server.
On most modern operating systems, this amount can easily be allocated.
However, if you are running many copies of the server, or if other
applications are also using System V shared memory, it may be necessary to
However, if you are running many copies of the server or you explicitly
configure the server to use large amounts of System V shared memory (see
<xref linkend="guc-shared-memory-type"/> and <xref
linkend="guc-dynamic-shared-memory-type"/>), it may be necessary to
increase <varname>SHMALL</varname>, which is the total amount of System V shared
memory system-wide. Note that <varname>SHMALL</varname> is measured in pages
rather than bytes on many systems.
@ -879,7 +884,7 @@ kern.ipc.semmns=512
</para>
<para>
You might also want to configure your kernel to lock shared
You might also want to configure your kernel to lock System V shared
memory into RAM and prevent it from being paged out to swap.
This can be accomplished using the <command>sysctl</command>
setting <literal>kern.ipc.shm_use_phys</literal>.
@ -928,7 +933,7 @@ kern.ipc.semmns=512
</para>
<para>
You might also want to configure your kernel to lock shared
You might also want to configure your kernel to lock System V shared
memory into RAM and prevent it from being paged out to swap.
This can be accomplished using the <command>sysctl</command>
setting <literal>kern.ipc.shm_use_phys</literal>.

42
src/backend/port/sysv_shmem.c
View File

@ -62,10 +62,11 @@
* to a process after exec(). Since EXEC_BACKEND is intended only for
* developer use, this shouldn't be a big problem. Because of this, we do
* not worry about supporting anonymous shmem in the EXEC_BACKEND cases below.
*
* As of PostgreSQL 12, we regained the ability to use a large System V shared
* memory region even in non-EXEC_BACKEND builds, if shared_memory_type is set
* to sysv (though this is not the default).
*/
#ifndef EXEC_BACKEND
#define USE_ANONYMOUS_SHMEM
#endif
typedef key_t IpcMemoryKey; /* shared memory key passed to shmget(2) */
@ -75,10 +76,8 @@ typedef int IpcMemoryId; /* shared memory ID returned by shmget(2) */
unsigned long UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;
#ifdef USE_ANONYMOUS_SHMEM
static Size AnonymousShmemSize;
static void *AnonymousShmem = NULL;
#endif
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
@ -370,8 +369,6 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
return true;
}
#ifdef USE_ANONYMOUS_SHMEM
#ifdef MAP_HUGETLB
/*
@ -534,8 +531,6 @@ AnonymousShmemDetach(int status, Datum arg)
}
}
#endif /* USE_ANONYMOUS_SHMEM */
/*
* PGSharedMemoryCreate
*
@ -566,7 +561,7 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
Size sysvsize;
/* Complain if hugepages demanded but we can't possibly support them */
#if !defined(USE_ANONYMOUS_SHMEM) || !defined(MAP_HUGETLB)
#if !defined(MAP_HUGETLB)
if (huge_pages == HUGE_PAGES_ON)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
@ -576,18 +571,19 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
#ifdef USE_ANONYMOUS_SHMEM
AnonymousShmem = CreateAnonymousSegment(&size);
AnonymousShmemSize = size;
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
AnonymousShmem = CreateAnonymousSegment(&size);
AnonymousShmemSize = size;
/* Register on-exit routine to unmap the anonymous segment */
on_shmem_exit(AnonymousShmemDetach, (Datum) 0);
/* Register on-exit routine to unmap the anonymous segment */
on_shmem_exit(AnonymousShmemDetach, (Datum) 0);
/* Now we need only allocate a minimal-sized SysV shmem block. */
sysvsize = sizeof(PGShmemHeader);
#else
sysvsize = size;
#endif
/* Now we need only allocate a minimal-sized SysV shmem block. */
sysvsize = sizeof(PGShmemHeader);
}
else
sysvsize = size;
/* Make sure PGSharedMemoryAttach doesn't fail without need */
UsedShmemSegAddr = NULL;
@ -687,14 +683,10 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
* block. Otherwise, the System V shared memory block is only a shim, and
* we must return a pointer to the real block.
*/
#ifdef USE_ANONYMOUS_SHMEM
if (AnonymousShmem == NULL)
return hdr;
memcpy(AnonymousShmem, hdr, sizeof(PGShmemHeader));
return (PGShmemHeader *) AnonymousShmem;
#else
return hdr;
#endif
}
#ifdef EXEC_BACKEND
@ -801,7 +793,6 @@ PGSharedMemoryDetach(void)
UsedShmemSegAddr = NULL;
}
#ifdef USE_ANONYMOUS_SHMEM
if (AnonymousShmem != NULL)
{
if (munmap(AnonymousShmem, AnonymousShmemSize) < 0)
@ -809,7 +800,6 @@ PGSharedMemoryDetach(void)
AnonymousShmem, AnonymousShmemSize);
AnonymousShmem = NULL;
}
#endif
}

2
src/backend/storage/ipc/ipci.c
View File

@ -46,6 +46,8 @@
#include "storage/spin.h"
#include "utils/snapmgr.h"
/* GUCs */
int shared_memory_type = DEFAULT_SHARED_MEMORY_TYPE;
shmem_startup_hook_type shmem_startup_hook = NULL;

23
src/backend/utils/misc/guc.c
View File

@ -453,6 +453,19 @@ const struct config_enum_entry ssl_protocol_versions_info[] = {
{NULL, 0, false}
};
static struct config_enum_entry shared_memory_options[] = {
#ifndef WIN32
{ "sysv", SHMEM_TYPE_SYSV, false},
#endif
#ifndef EXEC_BACKEND
{ "mmap", SHMEM_TYPE_MMAP, false},
#endif
#ifdef WIN32
{ "windows", SHMEM_TYPE_WINDOWS, false},
#endif
{NULL, 0, false}
};
/*
* Options for enum values stored in other modules
*/
@ -4327,6 +4340,16 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
{
{"shared_memory_type", PGC_POSTMASTER, RESOURCES_MEM,
gettext_noop("Selects the shared memory implementation used for the main shared memory region."),
NULL
},
&shared_memory_type,
DEFAULT_SHARED_MEMORY_TYPE, shared_memory_options,
NULL, NULL, NULL
},
{
{"wal_sync_method", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Selects the method used for forcing WAL updates to disk."),

6
src/backend/utils/misc/postgresql.conf.sample
View File

@ -129,6 +129,12 @@
#maintenance_work_mem = 64MB # min 1MB
#autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
#max_stack_depth = 2MB # min 100kB
#shared_memory_type = mmap # the default is the first option
# supported by the operating system:
# mmap
# sysv
# windows
# (change requires restart)
#dynamic_shared_memory_type = posix # the default is the first option
# supported by the operating system:
# posix

19
src/include/storage/pg_shmem.h
View File

@ -41,7 +41,8 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
#endif
} PGShmemHeader;
/* GUC variable */
/* GUC variables */
extern int shared_memory_type;
extern int huge_pages;
/* Possible values for huge_pages */
@ -52,6 +53,14 @@ typedef enum
HUGE_PAGES_TRY
} HugePagesType;
/* Possible values for shared_memory_type */
typedef enum
{
SHMEM_TYPE_WINDOWS,
SHMEM_TYPE_SYSV,
SHMEM_TYPE_MMAP
} PGShmemType;
#ifndef WIN32
extern unsigned long UsedShmemSegID;
#else
@ -59,6 +68,14 @@ extern HANDLE UsedShmemSegID;
#endif
extern void *UsedShmemSegAddr;
#if !defined(WIN32) && !defined(EXEC_BACKEND)
#define DEFAULT_SHARED_MEMORY_TYPE SHMEM_TYPE_MMAP
#elif !defined(WIN32)
#define DEFAULT_SHARED_MEMORY_TYPE SHMEM_TYPE_SYSV
#else
#define DEFAULT_SHARED_MEMORY_TYPE SHMEM_TYPE_WINDOWS
#endif
#ifdef EXEC_BACKEND
extern void PGSharedMemoryReAttach(void);
extern void PGSharedMemoryNoReAttach(void);