OpenSSL 0.9.6x. The DES functions use the older 'des_'
API, but the newer 3DES functions use the 0.9.7x-only
'DES_' API.
I think I just used /usr/include/openssl/des.h for reference
when implementing them, and had upgraded OpenSSL in the
meantime.
Following patch converts DES also to newer API and provides
compatibility functions for OpenSSL < 0.9.7.
I chose this route because:
- openssl.c uses few DES functions.
- compatibility for old 'des_' API is going away at some point
of time from OpenSSL.
- as seen from macros, new API is saner
- Thus pgcrypto supports any OpenSSL version from 0.9.5 to 1.0
Tested with OpenSSL 0.9.6c and 0.9.7e.
Marko Kreen
of password-based encryption from RFC2440 (OpenPGP).
The goal of this code is to be more featureful encryption solution
than current encrypt(), which only functionality is running cipher
over data.
Compared to encrypt(), pgp_encrypt() does following:
* It uses the equvialent of random Inital Vector to get cipher
into random state before it processes user data
* Stores SHA-1 of the data into result so any modification
will be detected.
* Remembers if data was text or binary - thus it can decrypt
to/from text data. This was a major nuisance for encrypt().
* Stores info about used algorithms with result, so user needs
not remember them - more user friendly!
* Uses String2Key algorithms (similar to crypt()) with random salt
to generate full-length binary key to be used for encrypting.
* Uses standard format for data - you can feed it to GnuPG, if needed.
Optional features (off by default):
* Can use separate session key - user data will be encrypted
with totally random key, which will be encrypted with S2K
generated key and attached to result.
* Data compression with zlib.
* Can convert between CRLF<->LF line-endings - to get fully
RFC2440-compliant behaviour. This is off by default as
pgcrypto does not know the line-endings of user data.
Interface is simple:
pgp_encrypt(data text, key text) returns bytea
pgp_decrypt(data text, key text) returns text
pgp_encrypt_bytea(data bytea, key text) returns bytea
pgp_decrypt_bytea(data bytea, key text) returns bytea
To change parameters (cipher, compression, mdc):
pgp_encrypt(data text, key text, parms text) returns bytea
pgp_decrypt(data text, key text, parms text) returns text
pgp_encrypt_bytea(data bytea, key text, parms text) returns bytea
pgp_decrypt_bytea(data bytea, key text, parms text) returns bytea
Parameter names I lifted from gpg:
pgp_encrypt('message', 'key', 'compress-algo=1,cipher-algo=aes256')
For text data, pgp_encrypt simply encrypts the PostgreSQL internal data.
This maps to RFC2440 data type 't' - 'extenally specified encoding'.
But this may cause problems if data is dumped and reloaded into database
which as different internal encoding. My next goal is to implement data
type 'u' - which means data is in UTF-8 encoding by converting internal
encoding to UTF-8 and back. And there wont be any compatibility
problems with current code, I think its ok to submit this without UTF-8
encoding by converting internal encoding to UTF-8 and back. And there
wont be any compatibility problems with current code, I think its ok to
submit this without UTF-8 support.
Here is v4 of PGP encrypt. This depends on previously sent
Fortuna-patch, as it uses the px_add_entropy function.
- New function: pgp_key_id() for finding key id's.
- Add SHA1 of user data and key into RNG pools. We need to get
randomness from somewhere, and it is in user best interests
to contribute.
- Regenerate pgp-armor test for SQL_ASCII database.
- Cleanup the key handling so that the pubkey support is less
hackish.
Marko Kreen
- Move openssl random provider to openssl.c and builtin provider
to internal.c
- Make px_random_bytes use Fortuna, instead of giving error.
- Retarget random.c to aquiring system randomness, for initial seeding
of Fortuna. There is ATM 2 functions for Windows,
reader from /dev/urandom and the regular time()/getpid() silliness.
Marko Kreen
functions as STRICT, and all functions except gen_salt() as IMMUTABLE.
gen_salt() is VOLATILE.
Although the functions are now STRICT, I left their PG_ARGISNULL()
checks in place as a protective measure for users who install the
new code but use old (non-STRICT) catalog entries (e.g., restored
from a dump). Per recent discussion in pgsql-hackers.
Patch from Michael Fuhr.
to make. We ship the table file in the tarball and so this dependency
just opens file timestamp skew problems without doing anything useful.
(Not that it should hurt, either ... except for cross-compile builds.)
chdir into PGDATA and subsequently use relative paths instead of absolute
paths to access all files under PGDATA. This seems to give a small
performance improvement, and it should make the system more robust
against naive DBAs doing things like moving a database directory that
has a live postmaster in it. Per recent discussion.
(currently in beta) when cryptolib = openssl. According to the
following checkin message from several years ago, OpenSSL application
developers should no longer rely on <openssl/evp.h> to include
everything they need:
http://cvs.openssl.org/chngview?cn=9888
This patch adds the necessary header files. It doesn't appear to
break anything when building against OpenSSL 0.9.7.
BTW, core appears to build and work fine with OpenSSL 0.9.8. I've
built 7.3 through HEAD against 0.9.8-beta6 without noticing any
problems.
Michael Fuhr
- Fix wrong index results on text, char, varchar for multibyte strings
- Fix some SIGFPE signals
- Add support for infinite timestamps
- Because of locale settings, btree_gist can not be a prefix index anymore (for text).
Each node holds now just the lower and upper boundary.
current time: provide a GetCurrentTimestamp() function that returns
current time in the form of a TimestampTz, instead of separate time_t
and microseconds fields. This is what all the callers really want
anyway, and it eliminates low-level dependencies on AbsoluteTime,
which is a deprecated datatype that will have to disappear eventually.
literally.
Add GUC variables:
"escape_string_warning" - warn about backslashes in non-E strings
"escape_string_syntax" - supports E'' syntax?
"standard_compliant_strings" - treats backslashes literally in ''
Update code to use E'' when escapes are used.
to the existing X-direction tests. An rtree class now includes 4 actual
2-D tests, 4 1-D X-direction tests, and 4 1-D Y-direction tests.
This involved adding four new Y-direction test operators for each of
box and polygon; I followed the PostGIS project's lead as to the names
of these operators.
NON BACKWARDS COMPATIBLE CHANGE: the poly_overleft (&<) and poly_overright
(&>) operators now have semantics comparable to box_overleft and box_overright.
This is necessary to make r-tree indexes work correctly on polygons.
Also, I changed circle_left and circle_right to agree with box_left and
box_right --- formerly they allowed the boundaries to touch. This isn't
actually essential given the lack of any r-tree opclass for circles, but
it seems best to sync all the definitions while we are at it.
polygon operators (<<, &<, >>, &>). Per ideas originally put forward
by andrew@supernews and later rediscovered by moi. This patch just
fixes the existing opclasses, and does not add any new behavior as I
proposed earlier; that can be sorted out later. In principle this
could be back-patched, since it changes only search behavior and not
system catalog entries nor rtree index contents. I'm not currently
planning to do that, though, since I think it could use more testing.
a physically separate type. Defining 'lo' as a domain over OID works
just fine and is more efficient. Improve documentation and fix up the
test script. (Would like to turn test script into a proper regression
test, but right now its output is not constant because of numeric OIDs;
plus it makes Unix-specific assumptions about files it can import.)
unlike template0 and template1 does not have any special status in
terms of backend functionality. However, all external utilities such
as createuser and createdb now connect to "postgres" instead of
template1, and the documentation is changed to encourage people to use
"postgres" instead of template1 as a play area. This should fix some
longstanding gotchas involving unexpected propagation of database
objects by createdb (when you used template1 without understanding
the implications), as well as ameliorating the problem that CREATE
DATABASE is unhappy if anyone else is connected to template1.
Patch by Dave Page, minor editing by Tom Lane. All per recent
pghackers discussions.
includes error checking and an appropriate ereport(ERROR) message.
This gets rid of rather tedious and error-prone manipulation of errno,
as well as a Windows-specific bug workaround, at more than a dozen
call sites. After an idea in a recent patch by Heikki Linnakangas.
it is sufficient to track whether a backend holds a lock or not, and
store information about transaction vs. session locks only in the
inside-the-backend LocalLockTable. Since there can now be but one
PROCLOCK per lock per backend, LockCountMyLocks() is no longer needed,
thus eliminating some O(N^2) behavior when a backend holds many locks.
Also simplify the LockAcquire/LockRelease API by passing just a
'sessionLock' boolean instead of a transaction ID. The previous API
was designed with the idea that per-transaction lock holding would be
important for subtransactions, but now that we have subtransactions we
know that this is unwanted. While at it, add an 'isTempObject' parameter
to LockAcquire to indicate whether the lock is being taken on a temp
table. This is not used just yet, but will be needed shortly for
two-phase commit.
and RelationNameGetTupleDesc() as deprecated; remove uses of the
latter in the contrib library. Along the way, clean up crosstab()
code and documentation a little.
are now reported via elog, eliminating the need to test the result code
at most call sites. Make it possible for the caller to distinguish a
freshly acquired lock from one already held in the current transaction.
Use that capability to avoid redundant AcceptInvalidationMessages() calls
in LockRelation().
spotted by Qingqing Zhou. The HASH_ENTER action now automatically
fails with elog(ERROR) on out-of-memory --- which incidentally lets
us eliminate duplicate error checks in quite a bunch of places. If
you really need the old return-NULL-on-out-of-memory behavior, you
can ask for HASH_ENTER_NULL. But there is now an Assert in that path
checking that you aren't hoping to get that behavior in a palloc-based
hash table.
Along the way, remove the old HASH_FIND_SAVE/HASH_REMOVE_SAVED actions,
which were not being used anywhere anymore, and were surely too ugly
and unsafe to want to see revived again.
Also, remove the rather useless return value of LockReleaseAll. Change
response to detection of corruption in the shared lock tables to PANIC,
since that is the only way of cleaning up fully.
Originally an idea of Heikki Linnakangas, variously hacked on by
Alvaro Herrera and Tom Lane.
for testing PLs and contrib_regression for testing contrib, instead of
overwriting the core system's regression database as formerly done.
Andrew Dunstan
external projects, we should be careful about what parts of the GiST
API are considered implementation details, and which are part of the
public API. Therefore, I've moved internal-only declarations into
gist_private.h -- future backward-incompatible changes to gist.h should
be made with care, to avoid needlessly breaking external GiST extensions.
Also did some related header cleanup: remove some unnecessary #includes
from gist.h, and remove some unused definitions: isAttByVal(), _gistdump(),
and GISTNStrategies.
which is neither needed by nor related to that header. Remove the bogus
inclusion and instead include the header in those C files that actually
need it. Also fix unnecessary inclusions and bad inclusion order in
tsearch2 files.
that return INTERNAL without also having INTERNAL arguments. Since the
functions in question aren't meant to be called by hand anyway, I just
redeclared them to take 'internal' instead of 'text'. Also add code
to ProcedureCreate() to enforce the restriction, as I should have done
to start with :-(
Essentially, we shoehorn in a lockable-object-type field by taking
a byte away from the lockmethodid, which can surely fit in one byte
instead of two. This allows less artificial definitions of all the
other fields of LOCKTAG; we can get rid of the special pg_xactlock
pseudo-relation, and also support locks on individual tuples and
general database objects (including shared objects). None of those
possibilities are actually exploited just yet, however.
I removed pg_xactlock from pg_class, but did not force initdb for
that change. At this point, relkind 's' (SPECIAL) is unused and
could be removed entirely.
command line. We find this useful because we frequently deal with
thousands of tables in an environment where neither the databases nor
the tables are updated frequently. This helps allow us to cut down on
the overhead of updating the list for every other primary loop of
pg_autovacuum.
I chose -i as the command-line argument and documented it briefly in
the README.
The patch was applied to the 7.4.7 version of pg_autovacuum in contrib.
Thomas F.O'Connell
indexes. Replace all heap_openr and index_openr calls by heap_open
and index_open. Remove runtime lookups of catalog OID numbers in
various places. Remove relcache's support for looking up system
catalogs by name. Bulky but mostly very boring patch ...
change saves a great deal of space in pg_proc and its primary index,
and it eliminates the former requirement that INDEX_MAX_KEYS and
FUNC_MAX_ARGS have the same value. INDEX_MAX_KEYS is still embedded
in the on-disk representation (because it affects index tuple header
size), but FUNC_MAX_ARGS is not. I believe it would now be possible
to increase FUNC_MAX_ARGS at little cost, but haven't experimented yet.
There are still a lot of vestigial references to FUNC_MAX_ARGS, which
I will clean up in a separate pass. However, getting rid of it
altogether would require changing the FunctionCallInfoData struct,
and I'm not sure I want to buy into that.
* test error handling
* add tests for des, 3des, cast5
* add some tests to blowfish, rijndael
* Makefile: ability to specify different tests for different crypto
libraries, so we can skip des, 3des and cast5 for builtin.
Marko Kreen
Reserve px_get_random_bytes() for strong randomness,
add new function px_get_pseudo_random_bytes() for
weak randomness and use it in gen_salt().
On openssl case, use RAND_pseudo_bytes() for
px_get_pseudo_random_bytes().
Final result is that is user has not configured random
souce but kept the 'silly' one, gen_salt() keeps
working, but pgp_encrypt() will throw error.
Marko Kreen
* openssl.c: Add 3des and AES support
* README.pgcrypto: list only supported ciphers for openssl
OpenSSL has pre-processor symbol OPENSSL_NO_AES, which
isn't that helpful for detecting if it _does_ exist.
Thus the hack with AES_ENCRYPT.
Marko Kreen
* Use error codes instead of -1
* px_strerror for new error codes
* calling convention change for px_gen_salt - return error code
* use px_strerror in pgcrypto.c
Marko Kreen
It was a bad style to begin with, and now several loops can be clearer.
* pgcrypto.c: Fix function comments
* crypt-gensalt.c, crypt-blowfish.c: stop messing with errno
* openssl.c: use px_free instead pfree
* px.h: make redefining px_alloc/px_realloc/px_free easier
Marko Kreen
libmcrypt seems to dead, maintainer address bounces,
and cast-128 fails on 2 of the 3 test vectors from RFC2144.
So I see no reason to keep around stuff I don't trust
anymore.
Support for several crypto libraries is probably only
confusing to users, although it was good for initial
developing - it helped to find hidden assumptions and
forced me to create regression tests for all functionality.
Marko Kreen
can tell whether it is being used as an aggregate or not. This allows
such a function to avoid re-pallocing a pass-by-reference transition
value; normally it would be unsafe for a function to scribble on an input,
but in the aggregate case it's safe to reuse the old transition value.
Make int8inc() do this. This gets a useful improvement in the speed of
COUNT(*), at least on narrow tables (it seems to be swamped by I/O when
the table rows are wide). Per a discussion in early December with
Neil Conway. I also fixed int_aggregate.c to check this, thereby
turning it into something approaching a supportable technique instead
of being a crude hack.
0.9.7x have EVP_DigestFinal function which which clears all of
EVP_MD_CTX. This makes pgcrypto crash in functions which
re-use one digest context several times: hmac() and crypt()
with md5 algorithm.
Following patch fixes it by carring the digest info around
EVP_DigestFinal and re-initializing cipher.
Marko Kreen.
compares two strings' soundex values for similarity, from Kris Jurka.
Also mark the text_soundex() function as STRICT, to avoid crashing
on NULL input.
regression=# select to_tsquery( '\'fotballklubber\'');
to_tsquery
------------------------------------------------
'fotball' & 'klubb' | 'fot' & 'ball' & 'klubb'
(1 row)
So, changed interface to dictionaries, lexize method of dictionary shoud return
pointer to aray of TSLexeme structs instead of char**. Last element should
have TSLexeme->lexeme == NULL.
typedef struct {
/* number of variant of split word , for example
Word 'fotballklubber' (norwegian) has two varian to split:
( fotball, klubb ) and ( fot, ball, klubb ). So, dictionary
should return:
nvariant lexeme
1 fotball
1 klubb
2 fot
2 ball
2 klubb
*/
uint16 nvariant;
/* currently unused */
uint16 flags;
/* C-string */
char *lexeme;
} TSLexeme;
typedef struct {} WordEntryPos;
to
typedef uint16 WordEntryPos
according to http://www.pgsql.ru/db/mw/msg.html?mid=2035188
Require re-fill all tsvector fields and reindex tsvector indexes.
- Dependency services may not be correctly registered when installing as
a Windows Service.
- The sleep time is changed from milliseconds to seconds as it should
be.
- Error messages during service installation/removal are logged to
stderr.
1 Report error message instead of do nothing in case of error in regex
2 Malloced storage for mask, find and repl part of Affix. This parts may be
large enough in real life (for example in czech, thanks to moje <moje@kalhotky.net>)
Also performed an initial run through of upgrading our Copyright date to
extend to 2005 ... first run here was very simple ... change everything
where: grep 1996-2004 && the word 'Copyright' ... scanned through the
generated list with 'less' first, and after, to make sure that I only
picked up the right entries ...
checks, to ensure the previous test backend has time to quit out of the
regression database. Also, allow all the checks to be run even if one
of them fails. Per suggestions from Andrew Dunstan to improve the
usefulness of buildfarm testing.
> > Windows service, it says you can use the -I and -R options.
> >
> > When I do that and I specify a password with '-P'
> (uppercase) then in
> > the registry it's saved as '-p' (lowercase) in the
> service-commandline
> > (ImagePath).
This was fixed in v1.21 of pg_autovacuum.c, That rev is tagged for
beta3, so you should not be seeing this issue unless you actually have
an older version for some reason.
http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/pg_autovacuum/p
g_autovacuum.c.diff?r1=1.20;r2=1.21;f=h
> > Also it removes the quotes I added and I'm not so sure it
> would work
> > the way it's supposed to, without it.
It's not so much that it strips them (that happens automagically), more
that it doesn't re-add them when it writes the command line in the
registry. The attached patch fixes that by simply quoting all options
that may need it.
> > If you add DependOnService (a REG_MULTI_SZ an
> array-like-thingie) and
> > have the name (in this case: pgsql-8.0-beta2-dev3) of a service it
> > depends on, it will not fail to start (it will not even try, as
> > PostgreSQL is not running), when PostgreSQL already failed.
> >
> > Maybe it's an idea to specify it on the commandline (what
> service to
> > depend on).
A -E <service> option is added in the attached patch.
Dave Page
warnings:
- remove pointless "extern" keyword from some function definitions in
contrib/tsearch2
- use "NULL" not "0" as NULL pointer in contrib/tsearch,
contrib/tsearch2, contrib/pgbench, and contrib/vacuumlo
clause implicitly whenever one is not given explicitly. Remove concept
of a schema having an associated tablespace, and simplify the rules for
selecting a default tablespace for a table or index. It's now just
(a) explicit TABLESPACE clause; (b) default_tablespace if that's not an
empty string; (c) database's default. This will allow pg_dump to use
SET commands instead of tablespace clauses to determine object locations
(but I didn't actually make it do so). All per recent discussions.
- add some additional files to the dbmirror install (approved by
ssinger)
- add a makefile for contrib/mysql, and add mysql to the list of
contribs build by default
- use xml2-config to pickup -I flags for libxml2 in contrib/xml and
contrib/xml2
Original work from Martin Pitt of Debian, minor cleanups by Neil
Conway.
snprintf(data, len, %s, NULL) crash.
The code was trying to find a connection by name when it already had an
unnamed connection and did not have a name to search with.
Kris Jurka
returning a NULL pointer (some callers remembered to check the return
value, but some did not -- it is safer to just bail out).
Also, cleanup pgstat.c to use elog(ERROR) rather than elog(LOG) followed
by exit().
http://archives.postgresql.org/pgsql-hackers/2004-10/msg00464.php.
This fix is intended to be permanent: it moves the responsibility for
calling SetBufferCommitInfoNeedsSave() into the tqual.c routines,
eliminating the requirement for callers to test whether t_infomask changed.
Also, tighten validity checking on buffer IDs in bufmgr.c --- several
routines were paranoid about out-of-range shared buffer numbers but not
about out-of-range local ones, which seems a tad pointless.
must be stale. Tweak example startup scripts to not use pg_ctl but launch
the postmaster directly, thereby ensuring that only the postmaster's direct
parent shell will be a postgres-owned process. In combination these should
fix the longstanding problem of the postmaster sometimes refusing to start
during reboot because it thinks the old lockfile is not stale.
subsequent core dump. It looks like at one time DBLINK_RES_ERROR_AS_NOTICE
didn't include a PQclear, but now it does and so these other ones are
duplicate.
mode see a fresh snapshot for each command in the function, rather than
using the latest interactive command's snapshot. Also, suppress fresh
snapshots as well as CommandCounterIncrement inside STABLE and IMMUTABLE
functions, instead using the snapshot taken for the most closely nested
regular query. (This behavior is only sane for read-only functions, so
the patch also enforces that such functions contain only SELECT commands.)
As per my proposal of 6-Sep-2004; I note that I floated essentially the
same proposal on 19-Jun-2002, but that discussion tailed off without any
action. Since 8.0 seems like the right place to be taking possibly
nontrivial backwards compatibility hits, let's get it done now.
-It fixes up some bugs with handling setval calls
-Adds upgrade instructions from prior versions
-Improved the sample config file
-Fixed some things in the clean_pending script
tablespaces correctly, and is quite restricted on objects covered (only
tables and databases, but not tablespaces and indexes).
The attached patch contributes:
- database_size(name)
- relation_size(text)
These are the well-known functions, tablespace-aware.
- pg_tablespace_size(oid)
- pg_database_size(oid)
- pg_relation_size(oid)
Tablespace-aware implementations, used by the upper functions.
pg_relation_size will report sizes of indexes as well.
- pg_size_pretty(bigint)
Formatting of sizes, to display '146MB' instead of '152885668'
Andreas Pflug
PROCLOCK structs in shared memory now have only a bitmask for held
locks, rather than counts (making them 40 bytes smaller, which is a
good thing). Multiple locks within a transaction are counted in the
local hash table instead, and we have provision for tracking which
ResourceOwner each count belongs to. Solves recently reported problem
with memory leakage within long transactions.
>
> The patch adds missing the "libpgport.a" file to the installation under
> "install-all-headers". It is needed by some contribs. I install the
> library in "pkglibdir", but I was wondering whether it should be "libdir"?
> I was wondering also whether it would make sense to have a "libpgport.so"?
>
> It fixes various macros which are used by contrib makefiles, especially
> libpq_*dir and LDFLAGS when used under PGXS. It seems to me that they are
> needed to
>
> It adds the ability to test and use PGXS with contribs, with "make
> USE_PGXS=1". Without the macro, this is exactly as before, there should be
> no difference, esp. wrt the vpath feature that seemed broken by previous
> submission. So it should not harm anybody, and it is useful at least to me.
>
> It fixes some inconsistencies in various contrib makefiles
> (useless override, ":=" instead of "=").
Fabien COELHO
as it stands - it mixes declarations in code, C++-style. The attached
patch shifts declarations to the tops of functions and enables this file
to compile cleanly as C.
Richard Poole
keep track of portal-related resources separately from transaction-related
resources. This allows cursors to work in a somewhat sane fashion with
nested transactions. For now, cursor behavior is non-subtransactional,
that is a cursor's state does not roll back if you abort a subtransaction
that fetched from the cursor. We might want to change that later.
an empty input string causes an empty output string to be returned, instead of
throwing an ERROR -- per complaint from Aaron Hillegass, and consistent with
double metaphone. Fix examples in README.soundex pointed out by James Robinson.
performance front, but with feature freeze upon us I think it's time to
drive a stake in the ground and say that this will be in 7.5.
Alvaro Herrera, with some help from Tom Lane.
--------------------------------------
The pg_trgm contrib module provides functions and index classes
for determining the similarity of text based on trigram
matching.
costing us lots more to maintain than it was worth. On shared tables
it was of exactly zero benefit because we couldn't trust it to be
up to date. On temp tables it sometimes saved an lseek, but not often
enough to be worth getting excited about. And the real problem was that
we forced an lseek on every relcache flush in order to update the field.
So all in all it seems best to lose the complexity.
conversion of basic ASCII letters. Remove all uses of strcasecmp and
strncasecmp in favor of new functions pg_strcasecmp and pg_strncasecmp;
remove most but not all direct uses of toupper and tolower in favor of
pg_toupper and pg_tolower. These functions use the same notions of
case folding already developed for identifier case conversion. I left
the straight locale-based folding in place for situations where we are
just manipulating user data and not trying to match it to built-in
strings --- for example, the SQL upper() function is still locale
dependent. Perhaps this will prove not to be what's wanted, but at
the moment we can initdb and pass regression tests in Turkish locale.
1. Fixed bug where sequences were being mirrored incorrectly if they
exceeded 127
2. Fixed a bug in the perl script with mirroring sequences(John
Burtenshaw sent an email to patches describing the bug in March but I
htink he forgot to attach his patch)
3. The dates/times in the transaction files will always use 2 digits.
Steven Singer
results with tuples as ordinary varlena Datums. This commit does not
in itself do much for us, except eliminate the horrid memory leak
associated with evaluation of whole-row variables. However, it lays the
groundwork for allowing composite types as table columns, and perhaps
some other useful features as well. Per my proposal of a few days ago.
boxes. Change interface to user-defined GiST support methods union and
picksplit. Now instead of bytea struct it used special GistEntryVector
structure.
out to be the same problem reported by Cott Lang which the previous
patch resolved) a new bug was uncovered when running with a debug level
of greater than 1.
This patch resolves this new found bug and fixes some of the other
debugging output to be more consistent.
Please apply to both HEAD and the 7.4 branch.
Matthew T. O'Connor
script for Mac OS X. I added calls to utilize the bundled apache
rotatelogs script in the DB startup for log rotation. Also modified
startup parameters file to allow using the "SystemStarter" utility to
start/stop/restart postgres with a rotating log file.
The script credits David Wheeler, 2002. I wrote him a message about
the changes an he suggested I post them here. I explain some of the
changes below.
Not sure how to submit the changes. I have 3 files, "PostgreSQL"
script, "StartupParameters.plist" file, "pg_startupscript.diff" file.
The diff file was run against the original "PostgreSQL" script file.
I'll try attaching them to this message. If they get filtered I can
resend if needed.
Thanks.
Ray A.
------------------------------------
1) Changed the "Provides" parameter in StartupParameters.plist to
"PostgreSQL" from "postgres database" simply for ease of typing. It
seems that the SystemStarter utility uses the "Provides" value if you
want to control the script. This way I did not have to enclose it in
quotes on commandline. The modified StartupParameters.plist is now an
XML document also.
2) For the startup script I added 2 user modifiable variables:
# do you want to rotate the log files, 1=true 0=false
ROTATELOGS=1
# logfile rotate in seconds
ROTATESEC="604800"
I also added a non modifiable variable:
# The apache log rotation utility
LOGUTIL="/usr/sbin/rotatelogs"
I modified the StartService and RestartService functions to execute
the new commands if the user wants log rotation.
Ray Aspeitia
I have changed the name of the new parse function to xml_valid and fixed
a reference to SortMem which meant that the code as supplied would work
against 7.3 and 7.4 but wouldn't work in CVS.
and, dblink_fetch -- allows ERROR on remote side of connection to
throw NOTICE locally instead of ERROR. Also removed documentation for
previously deprecated, now removed, functions.
have been able to significantly improve the contrib/xml XPath
integration code.
New features:
* XPath set-returning function allows multiple results from an several
XPath queries to be used as a virtual table.
* Using libxslt, XSLT transformations (with and without parameters) are
supported. (Caution: This support allows generic URL fetching from
within the backend as well).
I've removed the old code so that it is all libxml based. Rather than
attach as a patch, I've put the tar.gz (10k!) at
http://www.azuli.co.uk/pgxml-1.0.tar.gz
(all files in archive are xml/....).
I think this is worth replacing the contrib version with, even though
the function names have changed (though the same functionality is
there), because it includes a SRF and some SPI usage, in addition to
linking to an external library. And it isn't a big module! Obviously, I
understand that people might prefer to move it elsewhere, or might have
reservations about replacing an existing contrib module with an
incompatible one. I'm open to suggestions.
John Gray
exposed thereby. AFAICT these would not lead to any worse problems than
junk emitted on the backend's stdout, but we should have the option to
catch possible worse errors in future.
with ReturnSetInfo->expectedDesc. This allows custom datatypes
(e.g. from tsearch2) to be returned at runtime. Previous behavior
depended on the type oid to match between the remote and local
database, which obviously doesn't work well for custom types.
Per report from Mark Gibson.
and FreeDir routines modeled on the existing AllocateFile/FreeFile.
Like the latter, these routines will avoid failing on EMFILE/ENFILE
conditions whenever possible, and will prevent leakage of directory
descriptors if an elog() occurs while one is open.
Also, reduce PANIC to ERROR in MoveOfflineLogs() --- this is not
critical code and there is no reason to force a DB restart on failure.
All per recent trouble report from Olivier Hubaut.
Bug reference: 1081
Logged by: Aarjav Trivedi
Email address: aarjav@cc.gatech.edu
PostgreSQL version: 7.4
Operating system: Linux
Description: Spelling error in tsearch2.sql leading to problems
with
tsearch
Details:
On line 620 of tsearch2.sql which is required to install and run
TSEARCH,
REATE FUNCTION tsstat_in(cstring)
should be
CREATE FUNCTION tsstat_in(cstring)
because of this error, TSEARCH fails to work as specified,
In incorperates changes from myself and a number of contributors.
This update to dbmirror provides:
-replication of sequence operations via setval/nextval
-DBMirror.pl support for logging to syslog
-changed the names of the tables to dbmirror_* (no quotes required)
-Support for writitng SQL statements to files instead of directly to
a slave database
-More options for DBMirror.pl in the config files.
Steven Singer
Make btree index creation and initial validation of foreign-key constraints
use maintenance_work_mem rather than work_mem as their memory limit.
Add some code to guc.c to allow these variables to be referenced by their
old names in SHOW and SET commands, for backwards compatibility.
pg_autovacuum looses track of any table that's ever been truncated
(possibly other situations too). When i truncate a table it gets a
new relfilenode in pg_class. This is a problem because pg_autovacuum
assumes pg_class.relfilenode will join to pg_stats_all_tables.relid.
pg_stats_all_tables.relid is actallly the oid from pg_class, not the
relfilenode. These two values start out equal so pg_autovacuum works
initially, but it fails later on because of this incorrect assumption.
This patch fixes that problem. Applied to HEAD and 7.4.X.
Brian Hirt
not initialized if a log file is not specified on the command line. This
causes an immediate segfault on systems that fill allocated memory with some
value other than zero (my FreeBSD machine uses 0xD0).
Several crashes later I discovered that args->user, password, host, and port
are also used without being initialized.
This doesn't appear to be fixed in CVS and I came up empty on a mailing list
search -- hope it hasn't been reported already.
Craig Boston
pghackers proposal of 8-Nov. All the existing cross-type comparison
operators (int2/int4/int8 and float4/float8) have appropriate support.
The original proposal of storing the right-hand-side datatype as part of
the primary key for pg_amop and pg_amproc got modified a bit in the event;
it is easier to store zero as the 'default' case and only store a nonzero
when the operator is actually cross-type. Along the way, remove the
long-since-defunct bigbox_ops operator class.
Remove the 'strategy map' code, which was a large amount of mechanism
that no longer had any use except reverse-mapping from procedure OID to
strategy number. Passing the strategy number to the index AM in the
first place is simpler and faster.
This is a preliminary step in planned support for cross-datatype index
operations. I'm committing it now since the ScanKeyEntryInitialize()
API change touches quite a lot of files, and I want to commit those
changes before the tree drifts under me.
-References to older versions of PostgreSQL have been removed(It no
longer
compiles against older versions)
-Added a link to PgPerl at GBorg.
Steven Singer
Lars Boegild Thomsen (full email below) and also corrects the regression
expected output for a recent backend message adjustment. Please apply.
Joe Conway
areas are for the lifetime of the backend and in the interests of not breaking
something that's not broken I left alone.
Note for anyone reading this and wanting it for tsearch-v2-stable (i.e. for 7.3
backend) this patch probably will not apply cleanly to that source. It should
be simple enough to see what's going on and apply the changes by hand if need
be.
--
Nigel J. Andrews
The 'word' variable there is initialised from
the prs->words array, but immediately after,
that array may be reallocated, thus leaving
word pointing to unallocated memory.
tests) when using flex 2.5.31. The fix is to *not* try to use palloc
and pfree for allocations within the lexer; when you do that, the
yy_buffer_stack gets freed at inopportune times. The code is already
set up to do manual deallocation, so I see no particular advantage to
using palloc anyway.
This was the last piece of code that took it upon itself to reset the
random number sequence --- now we only have srandom() in postmaster start,
backend start, and explicit setseed() operations.
> patch for pg_autovacuum. This patch assumes that the stats system will
> be fixed so that all inserts, updates and deletes performed on shared
> tables reguardless of what database those commands were executed from,
> will show up in the stats shown in each database.
I had to make a further change to this to take quotes off the 'last
ANALYZE' in order for it to not overquote the relation name, so
there's a _little_ work left to get it to play well.
I have deployed it onto several boxes that should be doing some
vacuuming over the weekend, and it is now certainly hitting pg_
tables.
I would like to present a CVS-oriented patch; unfortunately, I had to
change the indentation patterns when editing some of it :-(. The
following _may_ be good; not sure...
Matthew T. O'Connor
Christopher Browne
> keywords for clarity.
Yeah, this is basically what I meant, sorry I didn't get to it quicker.
However, I tested it out a little and the patch you made doesn't work
because it produces commands like:
VACUUM ANALYZE "public.FooBar"
Which doesn't work, so I made my own patch that creates commands like:
VACUUM ANALYZE "public"."FooBar"
This allows for mixed case schema names as well as tables.
Adam, can you please give this a test as you are the person who caught
the bug in the first place.
Thanks,
Matthew T. O'Connor
ALTER TABLE mytable drop column last_column_of_table;
the timetravel trigger say on UPDATE/DELETE:
ERROR: parser: parse error at end of input
Here is the patch for this bug
B?jthe Zolt?n
getopt_long(). This is more or less the same problem as we saw earlier
with getaddrinfo() and struct addrinfo, and for the same reason: random
user-added libraries might contain the subroutine, but there's no
guarantee we will find the matching header files.
There is an option "-s oldname=newname", which changes the old field name of
the dbf-file to the newname in PostgeSQL. If the length of the new name is 0,
the field is skiped. If you want to skip the first field of the dbf-file,
you get the wildest error-messages from the backend.
dbf2pg load the dbf-file via "COPY tablename FROM STDIN". If you skip the
first field, it is an \t to much in STDIN.
A fix could be an counter j=0, which increments only, if a field is imported
(IF (strlen(fields[h].db_name)> 0) j++. And only if j > 1 (if an other field is
imported) the \t is printed.
An other small bug in the README:
-s start
Specify the first record-number in the xBase-file
we will insert.
should be
-e start
Specify the first record-number in the xBase-file
we will insert.
Thomas Behr
the new timetravel.c,
new timetravel.README (cut from spi/README and modified),
modified timetravel.sql.in
and modified timetravel.example.
Features:
- optionally 3 parameter for insert/update/delete user name
- work with CREATE UNIQUE INDEX ixxx on table xxx
(unique_field,time_off);
(the original version was work with unique index on 6.5.0-6.5.3,
and not work on 7.3.2,7.3.3)
(before 6.5.0 and between 6.5.3 and 7.3.2 I dont know)
- get_timetravel(tablename) function for check timetravel-status.
- timetravel trigger not change oid of the active record. (it is not a
good feature, because the old version is automatice prevent the paralel
update with "where oid=nnn")
B?jthe Zolt?n
>>Sounds like all that's needed for your case. But to be complete, in
>>addition to changing tablefunc.c we'd have to:
>>1) come up with a new function call signature that makes sense and does
>>not cause backward compatibility problems for other people
>>2) make needed changes to tablefunc.sql.in
>>3) adjust the README.tablefunc appropriately
>>4) adjust the regression test for new functionality
>>5) be sure we don't break any of the old cases
>>
>>If you want to submit a complete patch, it would be gratefully accepted
>>-- for review at least ;-)
>
> Here's the patch, at least for steps 1-3
Nabil Sayegh
Joe Conway
- LIKE <subtable> [ INCLUDING DEFAULTS | EXCLUDING DEFAULTS ]
- Quick cleanup of analyze.c function prototypes.
- New non-reserved keywords (INCLUDING, EXCLUDING, DEFAULTS), SQL 200X
Opted not to extend for check constraints at this time.
As per the definition that it's user defined columns, OIDs are NOT
inherited.
Doc and Source patches attached.
--
Rod Taylor <rbt@rbt.ca>
slave
servers. I haven't tested it very well, so use at your own risk (and I
recommend against using it in production).
Basically, I have a central database server that has 4 summary tables
inside
it replicated to a remote slave (these database tables are for my mail
server
authentication, so these are replicated to another server tuned for many
connections, and so I don't have postgres connections opened straight to
my
back-end database server).
Unfortunately, I also wanted to implement a replication database server
for
hot-backups. I realized, too late, that the replication process is
pretty
greedy and will try to replicate all tables marked as a
"MasterAddTable".
To make a long story, I made a patch to RServ.pm and Replicate that
allows you
to specify, on the command line, a list of tables that you want to
replicate...it'll ignore all others.
I haven't finished, since this has to be integrated with CleanLog for
instance, but this should (and does) suffice for the moment.
I have yet to test it with two slaves, but at least my mail server
replication
database now works (it was failing every time it tried to replicate, for
a
variety of reasons).
Anyone have any suggestions on how to improve on this? (or, if someone
more
familiar with this code wants to take the ball and run with it, you're
welcome to).
--
Michael A Nachbaur <mike@nachbaur.com>
- Don't attempt to convert partial or expressional unique indexes
- Don't attempt to convert unique indexes based on a non-default
opclasses
- Untested prevention of conversion of non-btree indexes unique
indexes. Untested as postgresql doesn't allow hash, gist, or rtree
based indexes to be unique.
rbt=# create unique index t on a using hash (col);
ERROR: DefineIndex: access method "hash" does not support UNIQUE
indexes
rbt=# create unique index t on a using gist (col);
ERROR: DefineIndex: access method "gist" does not support UNIQUE
indexes
rbt=# select version();
version
------------------------------------------------------------------------
PostgreSQL 7.4devel on i386-unknown-freebsd4.8, compiled by GCC 2.95.4
Rod Taylor
> Second argument to metaphone is suposed to set the limit on the
> number of characters to return, but it breaks on some phrases:
>
> usps=# select metaphone(a,3),metaphone(a,4),metaphone(a,20) from
> (select 'Hello world'::varchar AS a) a;
> HLW | HLWR | HLWRLT
>
> usps=# select metaphone(a,3),metaphone(a,4),metaphone(a,20) from
> (select 'A A COMEAUX MEMORIAL'::varchar AS a) a;
> AKM | AKMKS | AKMKSMMRL
>
> In every case I've found that does this, the 4th and 5th letters are
> always 'KS'.
Nice catch.
There was a bug in the original metaphone algorithm from CPAN. Patch
attached (while I was at it I updated my email address, changed the
copyright to PGDG, and removed an unnecessary palloc). Here's how it
looks now:
regression=# select metaphone(a,4) from (select 'A A COMEAUX
MEMORIAL'::varchar AS a) a;
metaphone
-----------
AKMK
(1 row)
regression=# select metaphone(a,5) from (select 'A A COMEAUX
MEMORIAL'::varchar AS a) a;
metaphone
-----------
AKMKS
(1 row)
Joe Conway
* A few bug fixes
* fixes solaris compile and crash issue
* decouple vacuum analyze and analyze thresholds
* detach from tty (dameonize)
* improved logging layout
* more conservative default configuration
* improved, expanded and updated README
please apply and 1st convenience, or before code freeze which ever comes
first :-)
At this point I think I have brought pg_autovacuum and its client side
design as far as I think it should go. It works, keeping file sizes in
check, helps performance and give the administrator a fair amount
flexibility in configuring it.
Next up is to do the FSM based design that is integrated into the back
end.
p.s. Thanks to Christopher Browne for his help.
Matthew T. O'Connor
1 intarray: bugfix for int[]-int[] operation
2 intarray: split _int.c to several files (_int.c now is unused)
3 ntarray (gist__intbig_ops opclass): use special type for index storage
4 ltree (gist__ltree_ops opclass), intarray (gist__intbig_ops): optimize
GiST's
penalty and picksplit interface functions, now use Hemming distance.
Teodor Sigaev
yy_fatal_error() call results in elog(ERROR) not exit(). This was
already fixed in the main lexer and plpgsql, but extend same technique
to all the other dot-l files. Also, on review of the possible calls
to yy_fatal_error(), it seems safe to use elog(ERROR) not elog(FATAL).
of an index can now be a computed expression instead of a simple variable.
Restrictions on expressions are the same as for predicates (only immutable
functions, no sub-selects). This fixes problems recently introduced with
inlining SQL functions, because the inlining transformation is applied to
both expression trees so the planner can still match them up. Along the
way, improve efficiency of handling index predicates (both predicates and
index expressions are now cached by the relcache) and fix 7.3 oversight
that didn't record dependencies of predicate expressions.
blanks, in hopes of reducing the surprise factor for newbies. Remove
redundant operators for VARCHAR (it depends wholly on TEXT operations now).
Clean up resolution of ambiguous operators/functions to avoid surprising
choices for domains: domains are treated as equivalent to their base types
and binary-coercibility is no longer considered a preference item when
choosing among multiple operators/functions. IsBinaryCoercible now correctly
reflects the notion that you need *only* relabel the type to get from type
A to type B: that is, a domain is binary-coercible to its base type, but
not vice versa. Various marginal cleanup, including merging the essentially
duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity
regression test to understand about binary compatibility (using pg_cast),
and fix a couple of small errors in the catalogs revealed thereby.
Restructure "special operator" handling to fetch operators via index opclasses
rather than hardwiring assumptions about names (cleans up the pattern_ops
stuff a little).
only remnant of this failed experiment is that the server will take
SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit
logic in libpq.
Both plannable queries and utility commands are now always executed
within Portals, which have been revamped so that they can handle the
load (they used to be good only for single SELECT queries). Restructure
code to push command-completion-tag selection logic out of postgres.c,
so that it won't have to be duplicated between simple and extended queries.
initdb forced due to addition of a field to Query nodes.
(materialization into a tuple store) discussed on pgsql-hackers earlier.
I've updated the documentation and the regression tests.
Notes on the implementation:
- I needed to change the tuple store API slightly -- it assumes that it
won't be used to hold data across transaction boundaries, so the temp
files that it uses for on-disk storage are automatically reclaimed at
end-of-transaction. I added a flag to tuplestore_begin_heap() to control
this behavior. Is changing the tuple store API in this fashion OK?
- in order to store executor results in a tuple store, I added a new
CommandDest. This works well for the most part, with one exception: the
current DestFunction API doesn't provide enough information to allow the
Executor to store results into an arbitrary tuple store (where the
particular tuple store to use is chosen by the call site of
ExecutorRun). To workaround this, I've temporarily hacked up a solution
that works, but is not ideal: since the receiveTuple DestFunction is
passed the portal name, we can use that to lookup the Portal data
structure for the cursor and then use that to get at the tuple store the
Portal is using. This unnecessarily ties the Portal code with the
tupleReceiver code, but it works...
The proper fix for this is probably to change the DestFunction API --
Tom suggested passing the full QueryDesc to the receiveTuple function.
In that case, callers of ExecutorRun could "subclass" QueryDesc to add
any additional fields that their particular CommandDest needed to get
access to. This approach would work, but I'd like to think about it for
a little bit longer before deciding which route to go. In the mean time,
the code works fine, so I don't think a fix is urgent.
- (semi-related) I added a NO SCROLL keyword to DECLARE CURSOR, and
adjusted the behavior of SCROLL in accordance with the discussion on
-hackers.
- (unrelated) Cleaned up some SGML markup in sql.sgml, copy.sgml
Neil Conway
changed as per discussion on the patches list).
This version should be a good bit better. It addresses all the issues
pointed out by Neil Conway. Vacuum and Analyze are now handled
separately. It now monitors for xid wraparound. The number of database
connections and queries has been significantly reduced compared the
previous version. I have moved it from bin to contrib. More detail on
the changes are in the TODO file.
I have not tested the xid wraparound code as I have to let my AthlonXP
1600 run select 1 in a tight loop for approx. two days in order to
perform the required 500,000,000 xacts.
Matthew T. O'Connor
version of crosstab. This fixes a major deficiency in real-world use of
the original version. Easiest to undestand with an illustration:
Data:
-------------------------------------------------------------------
select * from cth;
id | rowid | rowdt | attribute | val
----+-------+---------------------+----------------+---------------
1 | test1 | 2003-03-01 00:00:00 | temperature | 42
2 | test1 | 2003-03-01 00:00:00 | test_result | PASS
3 | test1 | 2003-03-01 00:00:00 | volts | 2.6987
4 | test2 | 2003-03-02 00:00:00 | temperature | 53
5 | test2 | 2003-03-02 00:00:00 | test_result | FAIL
6 | test2 | 2003-03-02 00:00:00 | test_startdate | 01 March 2003
7 | test2 | 2003-03-02 00:00:00 | volts | 3.1234
(7 rows)
Original crosstab:
-------------------------------------------------------------------
SELECT * FROM crosstab(
'SELECT rowid, attribute, val FROM cth ORDER BY 1,2',4)
AS c(rowid text, temperature text, test_result text, test_startdate
text, volts text);
rowid | temperature | test_result | test_startdate | volts
-------+-------------+-------------+----------------+--------
test1 | 42 | PASS | 2.6987 |
test2 | 53 | FAIL | 01 March 2003 | 3.1234
(2 rows)
Hashed crosstab:
-------------------------------------------------------------------
SELECT * FROM crosstab(
'SELECT rowid, attribute, val FROM cth ORDER BY 1',
'SELECT DISTINCT attribute FROM cth ORDER BY 1')
AS c(rowid text, temperature int4, test_result text, test_startdate
timestamp, volts float8);
rowid | temperature | test_result | test_startdate | volts
-------+-------------+-------------+---------------------+--------
test1 | 42 | PASS | | 2.6987
test2 | 53 | FAIL | 2003-03-01 00:00:00 | 3.1234
(2 rows)
Notice that the original crosstab slides data over to the left in the
result tuple when it encounters missing data. In order to work around
this you have to be make your source sql do all sorts of contortions
(cartesian join of distinct rowid with distinct attribute; left join
that back to the real source data). The new version avoids this by
building a hash table using a second distinct attribute query.
The new version also allows for "extra" columns (see the README) and
allows the result columns to be coerced into differing datatypes if they
are suitable (as shown above).
In testing a "real-world" data set (69 distinct rowid's, 27 distinct
categories/attributes, multiple missing data points) I saw about a
5-fold improvement in execution time (from about 2200 ms old, to 440 ms
new).
I left the original version intact because: 1) BC, 2) it is probably
slightly faster if you know that you have no missing attributes.
README and regression test adjustments included. If there are no
objections, please apply.
Joe Conway
. replace CREATE OR REPLACE AGGREGATE with a separate DROP and CREATE
. add DROP for all CREATE OPERATORs
. use IMMUTABLE and STRICT instead of WITH (isStrict)
. add IMMUTABLE and STRICT to int_array_aggregate's accumulator function
Gregory Stark
entire contents of the subplan into the tuplestore before we can return
any tuples. Instead, the tuplestore holds what we've already read, and
we fetch additional rows from the subplan as needed. Random access to
the previously-read rows works with the tuplestore, and doesn't affect
the state of the partially-read subplan. This is a step towards fixing
the problems with cursors over complex queries --- we don't want to
stick in Materialize nodes if they'll prevent quick startup for a cursor.
ltree_73.patch.gz - for 7.3 :
Fix ~ operation bug: eg '1.1.1' ~ '*.1'
ltree_74.patch.gz - for current CVS
Fix ~ operation bug: eg '1.1.1' ~ '*.1'
Add ? operation
Optimize index storage
Last change needs drop/create all ltree indexes, so only for 7.4
Teodor Sigaev
in one of the earth functions so that latitude and longitude to
cartesian coordinates conversion will be more accurrate. (Previously
a text string was built to provide as input which limited the accuracy
to the number of digits printed.)
The new functions were included in a recent patch to contrib/cube that has not
as yet been accepted as of yet.
I also added check constraints to the domain 'earth' since they are now
working in 7.4.
Bruno Wolff III
directly from float8 values. (As opposed to converting the values to
strings
and then parsing the strings.)
The functions are:
cube(float8) returns cube
cube(float8,float8) returns cube
cube(cube,float8) returns cube
cube(cube,float8,float8) returns cube
Bruno Wolff III
bison 1.875 and later as we did from earlier bison releases. Eventually
we will probably want to adopt the newer message spelling ... but not yet.
Per recent discussion on pgpatches.
Note: I didn't change the build rules for bootstrap, ecpg, or plpgsql
grammars, since these do not affect regression test results.