mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-08-28 21:27:20 +02:00
255 lines
8.7 KiB
Plaintext
255 lines
8.7 KiB
Plaintext
DBMirror - PostgreSQL Database Mirroring
|
|
===================================================
|
|
|
|
|
|
DBMirror is a database mirroring system developed for the PostgreSQL
|
|
database Written and maintained by Steven Singer(ssinger@navtechinc.com)
|
|
|
|
|
|
(c) 2001-2004 Navtech Systems Support Inc.
|
|
ALL RIGHTS RESERVED
|
|
|
|
Permission to use, copy, modify, and distribute this software and its
|
|
documentation for any purpose, without fee, and without a written agreement
|
|
is hereby granted, provided that the above copyright notice and this
|
|
paragraph and the following two paragraphs appear in all copies.
|
|
|
|
IN NO EVENT SHALL THE AUTHOR OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR
|
|
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
|
|
LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
|
|
DOCUMENTATION, EVEN IF THE AUTHOR OR DISTRIBUTORS HAVE BEEN ADVISED OF THE
|
|
POSSIBILITY OF SUCH DAMAGE.
|
|
|
|
THE AUTHOR AND DISTRIBUTORS SPECIFICALLY DISCLAIMS ANY WARRANTIES,
|
|
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
|
|
AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS
|
|
ON AN "AS IS" BASIS, AND THE AUTHOR AND DISTRIBUTORS HAS NO OBLIGATIONS TO
|
|
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
|
|
|
|
|
|
|
|
Overview
|
|
--------------------------------------------------------------------
|
|
|
|
The mirroring system is trigger based and provides the following key features:
|
|
|
|
-Support for multiple mirror slaves
|
|
-Transactions are maintained
|
|
-Per table selection of what gets mirrored.
|
|
|
|
|
|
The system is based on the idea that a master database exists where all
|
|
edits are made to the tables being mirrored. A trigger attached to the
|
|
tables being mirrored runs logging information about the edit to
|
|
the Pending table and PendingData table.
|
|
|
|
A perl script(DBMirror.pl) runs continuously for each slave database(A database
|
|
that the change is supposed to be mirrored to) examining the Pending
|
|
table; searching for transactions that need to be sent to that particular slave
|
|
database. Those transactions are then mirrored to the slave database and
|
|
the MirroredTransaction table is updated to reflect that the transaction has
|
|
been sent.
|
|
|
|
If the transaction has been sent to all know slave hosts (All entries
|
|
in the MirrorHost table) then all records of it are purged from the
|
|
Pending tables.
|
|
|
|
Requirements:
|
|
---------------------------------
|
|
-PostgreSQL-7.4 (Older versions are no longer supported)
|
|
-Perl 5.6 or 5.8 (Other versions might work)
|
|
-PgPerl (http://gborg.postgresql.org/project/pgperl/projdisplay.php)
|
|
|
|
|
|
Upgrading from versions prior to 8.0
|
|
---------------------------------------
|
|
Users upgrading from a version of dbmirror prior to the one shipped with
|
|
Postgresql 8.0 will need to perform the following steps
|
|
|
|
1. Dump the database then drop it (dropdb no not use the -C option)
|
|
2. Create database with createdb.
|
|
3. Run psql databasename -f MirrorSetup.sql
|
|
4. Restore the database(do not use the -C option of pg_dump/pg_restore)
|
|
5. run the SQL commands: DROP "Pending";DROP "PendingData"; DROP "MirrorHost";
|
|
DROP "MirroredTransaction";
|
|
|
|
The above steps are needed A) Because the names of the tables used by dbmirror
|
|
to store data have changed and B) In order for sequences to be mirrored properly
|
|
all serial types must be recreated.
|
|
|
|
|
|
|
|
Installation Instructions
|
|
------------------------------------------------------------------------
|
|
|
|
|
|
|
|
1) Compile pending.c
|
|
|
|
The file pending.c contains the recordchange trigger. This runs every
|
|
time a row inside of a table being mirrored changes.
|
|
|
|
|
|
To build the trigger run make on the "Makefile" in the DBMirror directory.
|
|
|
|
PostgreSQL-8.0 Make Instructions:
|
|
|
|
If you have already run "configure" in the top (pgsql) directory
|
|
then run "make" in the dbmirror directory to compile the trigger.
|
|
|
|
|
|
You should now have a file named pending.so that contains the trigger.
|
|
|
|
Install this file in your Postgresql lib directory (/usr/local/pgsql/lib)
|
|
|
|
|
|
2) Run MirrorSetup.sql
|
|
|
|
This file contains SQL commands to setup the Mirroring environment.
|
|
This includes
|
|
|
|
-Telling PostgreSQL about the "recordchange" trigger function.
|
|
-Creating the dbmirror_Pending,dbmirror_PendingData,dbmirror_MirrorHost,
|
|
dbmirror_MirroredTransaction tables
|
|
|
|
|
|
To execute the script use psql as follows
|
|
|
|
"psql -f MirrorSetup.sql MyDatabaseName"
|
|
|
|
where MyDatabaseName is the name of the database you wish to install mirroring
|
|
on(Your master).
|
|
|
|
|
|
3) Create slaveDatabase.conf files.
|
|
|
|
Each slave database needs its own configuration file for the
|
|
DBMirror.pl script. See slaveDatabase.conf for a sample.
|
|
|
|
The master settings refer to the master database(The one that is
|
|
being mirrored).
|
|
|
|
The slave settings refer to the database that the data is being
|
|
mirrored to.
|
|
|
|
The slaveName setting in the configuration file must match the slave
|
|
name specified in the dbmirror_MirrorHost table.
|
|
|
|
DBMirror.pl can be run in two modes of operation:
|
|
|
|
A) It can connect directly to the slave database. To do this specify
|
|
a slave database name and optional host and port along with a username
|
|
and password. See slaveDatabase.conf for details.
|
|
|
|
|
|
The master user must have sufficient permissions to modify the Pending
|
|
tables and to read all of the tables being mirrored.
|
|
|
|
The slave user must have enough permissions on the slave database to
|
|
modify(INSERT,UPDATE,DELETE) any tables on the slave system that are being
|
|
mirrored.
|
|
|
|
B) The SQL statements that should be executed on the slave can be
|
|
written to files which can then be executed slave database through
|
|
psql. This would be suitable for setups where their is no direct
|
|
connection between the slave database and the master. A file is
|
|
generated for each transaction in the directory specified by
|
|
TransactionFileDirectory. The file name contains the date/time the
|
|
file was created along with the transaction id.
|
|
|
|
|
|
4) Add the trigger to tables.
|
|
|
|
Execute the SQL code in AddTrigger.sql once for each table that should
|
|
be mirrored. Replace MyTableName with the name of the table that should
|
|
be mirrored.
|
|
|
|
NOTE: DBMirror requires that every table being mirrored have a primary key
|
|
defined.
|
|
|
|
5) Create the slave database.
|
|
|
|
The DBMirror system keeps the contents of mirrored tables identical on the
|
|
master and slave databases. When you first install the mirror triggers the
|
|
master and slave databases must be the same.
|
|
|
|
If you are starting with an empty master database then the slave should
|
|
be empty as well. Otherwise use pg_dump to ensure that the slave database
|
|
tables are initially identical to the master.
|
|
|
|
6) Add entries in the MirrorHost table.
|
|
|
|
Each slave database must have an entry in the MirrorHost table.
|
|
|
|
The name of the host in the MirrorHost table must exactly match the
|
|
slaveHost variable for that slave in the configuration file.
|
|
|
|
For example
|
|
INSERT INTO "MirrorHost" ("SlaveName") VALUES ('backup_system');
|
|
|
|
|
|
6) Start DBMirror.pl
|
|
|
|
|
|
DBMirror.pl is the perl script that handles the mirroring.
|
|
|
|
It requires the Perl library Pg(See http://gborg.postgresql.org/project/pgperl/projdisplay.php)
|
|
|
|
It takes its configuration file as an argument(The one from step 3)
|
|
One instance of DBMirror.pl runs for each slave machine that is receiving
|
|
mirrored data.
|
|
|
|
Any errors are printed to standard out and emailed to the address specified in
|
|
the configuration file.
|
|
|
|
DBMirror can be run from the master, the slave, or a third machine as long
|
|
as it is able to access both the master and slave databases(not
|
|
required if SQL files are being generated)
|
|
|
|
7) Periodically run clean_pending.pl
|
|
clean_pending.pl cleans out any entries from the Pending tables that
|
|
have already been mirrored to all hosts in the MirrorHost table.
|
|
It uses the same configuration file as DBMirror.pl.
|
|
|
|
Normally DBMirror.pl will clean these tables as it goes but in some
|
|
circumstances this will not happen.
|
|
|
|
For example if a transaction has been mirrored to all slaves except for
|
|
one, then that host is removed from the MirrorHost table(It stops being
|
|
a mirror slave) the transactions that had already been mirrored to
|
|
all the other hosts will not be deleted from the Pending tables by
|
|
DBMirror.pl since DBMirror.pl will run against these transactions again
|
|
since they have already been sent to all the other hosts.
|
|
|
|
clean_pending.pl will remove these transactions.
|
|
|
|
TODO(Current Limitations)
|
|
----------
|
|
-Support for selective mirroring based on the content of data.
|
|
-Support for BLOB's.
|
|
-Support for multi-master mirroring with conflict resolution.
|
|
-Better support for dealing with Schema changes.
|
|
|
|
|
|
|
|
Significant Changes Since 7.4
|
|
----------------
|
|
-Support for mirroring SEQUENCE's
|
|
-Support for unix domain sockets
|
|
-Support for outputting slave SQL statements to a file
|
|
-Changed the names of replication tables are now named
|
|
dbmirror_pending etc..
|
|
|
|
|
|
|
|
Credits
|
|
-----------
|
|
Achilleus Mantzios <achill@matrix.gatewaynet.com>
|
|
|
|
|
|
|
|
|
|
Steven Singer
|
|
Navtech Systems Support Inc.
|
|
ssinger@navtechinc.com
|