mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-08-17 08:13:23 +02:00
245 lines
8.5 KiB
Plaintext
245 lines
8.5 KiB
Plaintext
DBMirror - Postgres Database Mirroring
|
|
===================================================
|
|
|
|
|
|
DBMirror is a database mirroring system developed for the Postgres
|
|
database Written and maintained by Steven Singer(ssinger@navtechinc.com)
|
|
|
|
|
|
(c) 2001-2002 Navtech Systems Support Inc.
|
|
ALL RIGHTS RESERVED
|
|
|
|
Permission to use, copy, modify, and distribute this software and its
|
|
documentation for any purpose, without fee, and without a written agreement
|
|
is hereby granted, provided that the above copyright notice and this
|
|
paragraph and the following two paragraphs appear in all copies.
|
|
|
|
IN NO EVENT SHALL THE AUTHOR OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR
|
|
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
|
|
LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
|
|
DOCUMENTATION, EVEN IF THE AUTHOR OR DISTRIBUTORS HAVE BEEN ADVISED OF THE
|
|
POSSIBILITY OF SUCH DAMAGE.
|
|
|
|
THE AUTHOR AND DISTRIBUTORS SPECIFICALLY DISCLAIMS ANY WARRANTIES,
|
|
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
|
|
AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS
|
|
ON AN "AS IS" BASIS, AND THE AUTHOR AND DISTRIBUTORS HAS NO OBLIGATIONS TO
|
|
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
|
|
|
|
|
|
|
|
Overview
|
|
--------------------------------------------------------------------
|
|
|
|
The mirroring system is trigger based and provides the following key features:
|
|
|
|
-Support for multiple mirror slaves
|
|
-Transactions are maintained
|
|
-Per table selection of what gets mirrored.
|
|
|
|
|
|
The system is based on the idea that a master database exists where all
|
|
edits are made to the tables being mirrored. A trigger attached to the
|
|
tables being mirrored runs logging information about the edit to
|
|
the Pending table and PendingData table.
|
|
|
|
A perl script(DBMirror.pl) runs continuously for each slave database(A database
|
|
that the change is supposed to be mirrored to) examining the Pending
|
|
table; searching for transactions that need to be sent to that particular slave
|
|
database. Those transactions are then mirrored to the slave database and
|
|
the MirroredTransaction table is updated to reflect that the transaction has
|
|
been sent.
|
|
|
|
If the transaction has been sent to all know slave hosts (All entries
|
|
in the MirrorHost table) then all records of it are purged from the
|
|
Pending tables.
|
|
|
|
Installation Instructions
|
|
------------------------------------------------------------------------
|
|
|
|
|
|
1) Compile pending.c
|
|
|
|
The file pending.c contains the recordchange trigger. This runs every
|
|
time a row inside of a table being mirrored changes.
|
|
|
|
|
|
To build the trigger run make on the "Makefile" in the DBMirror directory.
|
|
|
|
Postgres-7.3 Make Instructions:
|
|
|
|
If you have already run "configure" in the pgsql-server directory
|
|
then run "make" in the dbmirror directory to compile the trigger.
|
|
|
|
Postgres-7.1 & Postgres-7.2 Make Instructions:
|
|
|
|
The included Makefile is not compatible with postgres 7.1 and 7.2
|
|
The trigger will need to be built by hand.
|
|
|
|
Run the following commands
|
|
|
|
gcc -fpic -I/usr/local/pgsql/include/server -c pending.c -DNOSCHEMAS
|
|
ld -shared -o pending.so pending.o
|
|
|
|
Assuming the postgres include files are in /usr/local/pgsql/include/server.
|
|
|
|
Postgres-7.1.x installations should change this to
|
|
/usr/local/pgsql/include (The server part is for 7.2+)
|
|
|
|
If you have installed the postgres include files to another location then
|
|
modify the include path to reflect this.
|
|
|
|
Compiling the trigger by hand requires that all postgres headers be installed
|
|
,this is accomplished in postgresql(7.1 or 7.2) by running
|
|
"make install-all-headers" in the postgres source directory.
|
|
|
|
|
|
You should now have a file named pending.so that contains the trigger.
|
|
|
|
Install this file in /usr/local/pgsql/lib (or another suitable location).
|
|
|
|
If you choose a different location the MirrorSetup.sql script will need
|
|
to be modified to reflect your new location. The CREATE FUNCTION command
|
|
in the MirrorSetup.sql script associates the trigger function with the
|
|
pending.so shared library. Modify the arguments to this command if you
|
|
choose to install the trigger elsewhere.
|
|
|
|
2) Run MirrorSetup.sql
|
|
|
|
This file contains SQL commands to setup the Mirroring environment.
|
|
This includes
|
|
|
|
-Telling Postgres about the "recordchange" trigger function.
|
|
-Creating the Pending,PendingData, MirrorHost, MirroredTransaction tables
|
|
|
|
|
|
To execute the script use psql as follows
|
|
|
|
"psql -f MirrorSetup.sql MyDatabaseName"
|
|
|
|
where MyDatabaseName is the name of the database you wish to install mirroring
|
|
on(Your master).
|
|
|
|
Postgres-7.1 and 7.2 Notes:
|
|
-The syntax for creating a trigger function changed in Postgres-7.3.
|
|
Change the line in MirrorSetup.sql from
|
|
|
|
CREATE FUNCTION "recordchange" () RETURNS trigger AS
|
|
|
|
to
|
|
CREATE FUNCTION "recordchange" () RETURNS OPAQUE AS
|
|
|
|
|
|
3) Create slaveDatabase.conf files.
|
|
|
|
Each slave database needs its own configuration file for the
|
|
DBMirror.pl script. See slaveDatabase.conf for a sample.
|
|
|
|
The master settings refer to the master database(The one that is
|
|
being mirrored).
|
|
|
|
The slave settings refer to the database that the data is being mirrored to.
|
|
The slaveHost parameter must refer to the machine name of the slave (Either
|
|
a resolvable hostname or an IP address). The value for slave host
|
|
must match the Hostname field in the MirrorHost table(See step 6).
|
|
|
|
The master user must have sufficient permissions to modify the Pending
|
|
tables and to read all of the tables being mirrored.
|
|
|
|
The slave user must have enough permissions on the slave database to
|
|
modify(INSERT,UPDATE,DELETE) any tables on the slave system that are being
|
|
mirrored.
|
|
|
|
4) Add the trigger to tables.
|
|
|
|
Execute the SQL code in AddTrigger.sql once for each table that should
|
|
be mirrored. Replace MyTableName with the name of the table that should
|
|
be mirrored.
|
|
|
|
NOTE: DBMirror requires that every table being mirrored have a primary key
|
|
defined.
|
|
|
|
5) Create the slave database.
|
|
|
|
The DBMirror system keeps the contents of mirrored tables identical on the
|
|
master and slave databases. When you first install the mirror triggers the
|
|
master and slave databases must be the same.
|
|
|
|
If you are starting with an empty master database then the slave should
|
|
be empty as well. Otherwise use pg_dump to ensure that the slave database
|
|
tables are initially identical to the master.
|
|
|
|
6) Add entries in the MirrorHost table.
|
|
|
|
Each slave database must have an entry in the MirrorHost table.
|
|
|
|
The name of the host in the MirrorHost table must exactly match the
|
|
slaveHost variable for that slave in the configuration file.
|
|
|
|
For example
|
|
INSERT INTO "MirrorHost" ("HostName") VALUES ('mySlaveMachine.mycompany.com');
|
|
|
|
|
|
6) Start DBMirror.pl
|
|
|
|
|
|
DBMirror.pl is the perl script that handles the mirroring.
|
|
|
|
It requires the Perl library Pg(See src/interfaces/perl5 in the postgres
|
|
source distribution).
|
|
|
|
It takes its configuration file as an argument(The one from step 3)
|
|
One instance of DBMirror.pl runs for each slave machine that is receiving
|
|
mirrored data.
|
|
|
|
Any errors are printed to standard out and emailed to the address specified in
|
|
the configuration file.
|
|
|
|
DBMirror can be run from the master, the slave, or a third machine as long
|
|
as it is able to access both the master and slave databases.
|
|
|
|
7) Periodically run clean_pending.pl
|
|
clean_pending.pl cleans out any entries from the Pending tables that
|
|
have already been mirrored to all hosts in the MirrorHost table.
|
|
It uses the same configuration file as DBMirror.pl.
|
|
|
|
Normally DBMirror.pl will clean these tables as it goes but in some
|
|
circumstances this will not happen.
|
|
|
|
For example if a transaction has been mirrored to all slaves except for
|
|
one, then that host is removed from the MirrorHost table(It stops being
|
|
a mirror slave) the transactions that had already been mirrored to
|
|
all the other hosts will not be deleted from the Pending tables by
|
|
DBMirror.pl since DBMirror.pl will run against these transactions again
|
|
since they have already been sent to all the other hosts.
|
|
|
|
clean_pending.pl will remove these transactions.
|
|
|
|
TODO(Current Limitations)
|
|
----------
|
|
-Support for selective mirroring based on the content of data.
|
|
-Support for BLOB's.
|
|
-Support for conflict resolution.
|
|
-Batching SQL commands in DBMirror for better performance over WAN's.
|
|
-Better support for dealing with Schema changes.
|
|
|
|
Tested Platforms:
|
|
------------------
|
|
|
|
DBMirror has been tested on the following configurations but should
|
|
work on any platform with Postgres >= 7.1 and Perl 5.6.
|
|
|
|
RedHat Linux 7.1 & 6.2
|
|
-Postgres 7.1.2
|
|
-Perl 5.6
|
|
|
|
Mandrake Linux 8.0(Limited Testing)
|
|
-Postgres 7.2
|
|
-Postgres 7.3
|
|
-Perl 5.6
|
|
|
|
|
|
Steven Singer
|
|
Navtech Systems Support Inc.
|
|
ssinger@navtechinc.com
|