If this breaks things due to missing libxslt, then I'll have to
revert it, but let's see if it breaks the buildfarm.
Workarounds in case libxslt is missing include:
. don't configure with libxml, or
. don't build contrib modules from the contrib Makefile (use the individual module Makefiles instead), or
. change the xml2 Makefile
Smith. Along with Japanese doc updation by Tasuo Ishii.
> This patch changes the way pgbench outputs its latency log files so that
> every transaction gets a timestamp and notes which transaction type was
> executed. It's a one-line change that just dumps some additional
> information that was already sitting in that area of code. I also made a
> couple of documentation corrections and clarifications on some of the more
> confusing features of pgbench.
>
> It's straightforward to parse log files in this format to analyze what
> happened during the test at a higher level than was possible with the
> original format. You can find some rough sample code to convert this
> latency format into CVS files and then into graphs at
> http://www.westnet.com/~gsmith/content/postgresql/pgbench.htm which I'll
> be expanding on once I get all my little patches sent in here.
Also tweak README.pgbench/README.pgbench_jis:
Remove history after pgbench was added to PostgreSQL contrib module.
Those info was not only redundant since it has already been in CVS
log, but also incomplete.
--------------------------------------------------------------------------
The attached is a patch to optimize contrib/pgbench using new 8.3 features.
- Use DROP IF EXISTS to suppress errors for initial loadings.
- Use a combination of TRUNCATE and COPY to reduce WAL on creating
the accounts table.
Also, there are some cosmetic changes.
- Change the output of -v option from "starting full vacuum..."
to "starting vacuum accounts..." in reflection of the fact.
- Shape duplicated error checks into executeStatement().
There is a big performance win in "COPY with no WAL" feature.
Thanks for the efforts!
--------------------------------------------------------------------------
right, there seems precious little reason to have a pile of hand-maintained
endianness definitions in src/include/port/*.h. Get rid of those, and make
the couple of places that used them depend on WORDS_BIGENDIAN instead.
This commit breaks any code that assumes that the mere act of forming a tuple
(without writing it to disk) does not "toast" any fields. While all available
regression tests pass, I'm not totally sure that we've fixed every nook and
cranny, especially in contrib.
Greg Stark with some help from Tom Lane
pointer" in every Snapshot struct. This allows removal of the case-by-case
tests in HeapTupleSatisfiesVisibility, which should make it a bit faster
(I didn't try any performance tests though). More importantly, we are no
longer violating portable C practices by assuming that small integers are
distinct from all pointer values, and HeapTupleSatisfiesDirty no longer
has a non-reentrant API involving side-effects on a global variable.
There were a couple of places calling HeapTupleSatisfiesXXX routines
directly rather than through the HeapTupleSatisfiesVisibility macro.
Since these places had to be changed anyway, I chose to make them go
through the macro for uniformity.
Along the way I renamed HeapTupleSatisfiesSnapshot to HeapTupleSatisfiesMVCC
to emphasize that it's only used with MVCC-type snapshots. I was sorely
tempted to rename HeapTupleSatisfiesVisibility to HeapTupleSatisfiesSnapshot,
but forebore for the moment to avoid confusion and reduce the likelihood that
this patch breaks some of the pending patches. Might want to reconsider
doing that later.
uses SPI plans, this finally fixes the ancient gotcha that you can't
drop and recreate a temp table used by a plpgsql function.
Along the way, clean up SPI's API a little bit by declaring SPI plan
pointers as "SPIPlanPtr" instead of "void *". This is cosmetic but
helps to forestall simple programming mistakes. (I have changed some
but not all of the callers to match; there are still some "void *"'s
in contrib and the PL's. This is intentional so that we can see if
anyone's compiler complains about it.)
with minor editorization by me.
Hstore improvements
* add operation hstore ? text - excat equivalent of exist()
* remove undocumented behaviour of contains operation with NULL value
* now 'key'::text=>NULL returns '"key"=>NULL' instead of NULL
* Add GIN support for contains and exist operations
* Add GiST support for exist operatiion
* improve regression tests
fix it. Add macroses DatumGetNDBOX, PG_GETARG_NDBOX and PG_RETURN_NDBOX.
Backpatch for 8.2 too.
Previous versions use version 0 calling conventions. And fmgr code detoast
values for user-defined functions.
fixup various places in the tree that were clearing a StringInfo by hand.
Making this function a part of the API simplifies client code slightly,
and avoids needlessly peeking inside the StringInfo interface.
ways. I'm not totally sure that I caught everything, but at least now they pass
their regression tests with VARSIZE/SET_VARSIZE defined to reverse byte order.
Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with
VARSIZE and VARDATA, and as a consequence almost no code was using the
longer names. Rename the length fields of struct varlena and various
derived structures to catch anyplace that was accessing them directly;
and clean up various places so caught. In itself this patch doesn't
change any behavior at all, but it is necessary infrastructure if we hope
to play any games with the representation of varlena headers.
Greg Stark and Tom Lane
pg_standby is a production-ready program that can be used to
create a Warm Standby server. Other configuration is required
as well, all of which is described in the main server manual.
Simon Riggs
pg_standby is a production-ready program that can be used to
create a Warm Standby server. Other configuration is required
as well, all of which is described in the main server manual.
Simon Riggs
where possible, and fix some sites that apparently thought that fgets()
will overwrite the buffer by one byte.
Also add some strlcpy() to eliminate some weird memory handling.
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
In this case extractQuery should returns -1 as nentries. This changes
prototype of extractQuery method to use int32* instead of uint32* for
nentries argument.
Based on that gincostestimate may see two corner cases: nothing will be found
or seqscan should be used.
Per proposal at http://archives.postgresql.org/pgsql-hackers/2007-01/msg01581.php
PS tsearch_core patch should be sightly modified to support changes, but I'm
waiting a verdict about reviewing of tsearch_core patch.
suffix, to distinguish them from doubles. Make some function declarations
and definitions use the "const" qualifier for arguments consistently.
Ignore warning 4102 ("unreferenced label"), because such warnings
are always emitted by bison-generated code. Patch from Magnus Hagander.
the 8.1 SQL function definition for it. Per report from Rajesh Kumar Mallah,
such a DBA error doesn't seem at all improbable, and the cost of checking for
it is not very high compared to the cost of running this function. (It would
have been better to change the C name of the function so it wouldn't be called
by the old SQL definition, but it's too late for that now in the 8.2 branch.)
pgbench calls random() later, so it should have called srandom().
On most platforms except Windows srandom() is actually identical
to srand(), so the bug only bites Windows users.
per bug report from Akio Ishida.
cases. Operator classes now exist within "operator families". While most
families are equivalent to a single class, related classes can be grouped
into one family to represent the fact that they are semantically compatible.
Cross-type operators are now naturally adjunct parts of a family, without
having to wedge them into a particular opclass as we had done originally.
This commit restructures the catalogs and cleans up enough of the fallout so
that everything still works at least as well as before, but most of the work
needed to actually improve the planner's behavior will come later. Also,
there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way
to create a new family right now is to allow CREATE OPERATOR CLASS to make
one by default. I owe some more documentation work, too. But that can all
be done in smaller pieces once this infrastructure is in place.
Previous versions aren't affected.
Fix synonym dictionary init: string should be malloc'ed, not palloc'ed. Bug
introduced recently while fixing lowerstr().
commutator operators, and mark hash-opclass members as oprcanhash.
This is a pretty ugly, brute-force solution, but it seems that getting
rid of all these redundant-looking operators would require some tweaks
in the core operator-resolution code to behave nicely, and I'm not
willing to risk that just before RC1.
Fix string's length calculation for recoding, fix strlower() to avoid wrong
assumption about length of recoded string (was: recoded string is no greater
that source, it may not true for multibyte encodings)
Thanks to Thomas H. <me@alternize.com> and Magnus Hagander <mha@sollentuna.net>
scenarios. With multiple clinets, only the first client got the right
scaling factor and this gave a illusion of better performance in case
of the scaling factor greater than 1.
max_stack_depth is not set to an unsafe value.
This commit also provides configure-time checking for <sys/resource.h>,
and cleans up some perhaps-unportable code associated with use of that
include file and getrlimit().
static variables. This avoids any risk of potential non-reentrancy,
and in particular offers a much cleaner workaround for the Intel compiler
bug that was affecting ginutil.c.
return true for exactly the characters treated as whitespace by their flex
scanners. Per report from Victor Snezhko and subsequent investigation.
Also fix a passel of unsafe usages of <ctype.h> functions, that is, ye olde
char-vs-unsigned-char issue. I won't miss <ctype.h> when we are finally
able to stop using it.
even when a single relation requires more than max_fsm_pages pages. Also,
make VACUUM emit a warning in this case, since it likely means that VACUUM
FULL or other drastic corrective measure is needed. Per reports from Jeff
Frost and others of unexpected changes in the claimed max_fsm_pages need.
is a large enough histogram, it will use the number of matches in the
histogram to derive a selectivity estimate, rather than the admittedly
pretty bogus heuristics involving examining the pattern contents. I set
'large enough' at 100, but perhaps we should change that later. Also
apply the same technique in contrib/ltree's <@ and @> estimator. Per
discussion with Stefan Kaltenbrunner and Matteo Beccati.
alias with the old name for backwards compatibility. Per discussion,
the old name is actively wrong because validity and well-formedness
have different meanings in XML.
match the convention that foo's uninstall script is uninstall_foo.sql.
Also, stop installing lo_test.sql, which really ought to be made into
a regression test anyway (though it's unclear how to avoid a dependency
on the current OID counter...)
an earlier discussion. Centralize assumptions about what libpq depends
on in one place in Makefile.global. I am unconvinced that this list
is complete, but since ecpg seems to have gotten along with just these
entries, we'll try it this way and see what happens.
remove the old isbn_issn module which is about to be obsoleted by EAN13.
contrib/isn is by Germán Méndez Bravo. Our thanks to Garrett A. Wollman
for having written the original isbn_issn module.
(table or index) before trying to open its relcache entry. This fixes
race conditions in which someone else commits a change to the relation's
catalog entries while we are in process of doing relcache load. Problems
of that ilk have been reported sporadically for years, but it was not
really practical to fix until recently --- for instance, the recent
addition of WAL-log support for in-place updates helped.
Along the way, remove pg_am.amconcurrent: all AMs are now expected to support
concurrent update.
- predefined variable "tps"
The value of variable tps is taken from the scaling factor
specified by -s option.
- -D option
Variable values can be defined by -D option.
- \set command now allows arithmetic calculations.
Update the calling convention for all external facing functions. By
external facing, I mean all functions that are directly referenced in
cube.sql. Prior to my update, all functions used the older V0 calling
convention. They now use V1.
New Functions:
cube(float[]), which makes a zero volume cube from a float array
cube(float[], float[]), which allows the user to create a cube from
two float arrays; one for the upper right and one for the lower left
coordinate.
cube_subset(cube, int4[]), to allow you to reorder or choose a subset of
dimensions from a cube, using index values specified in the array.
Joshua Reich
Few cleanups and couple of new things:
- add SHA2 algorithm to older OpenSSL
- add BIGNUM math to have public-key cryptography work on non-OpenSSL
build.
- gen_random_bytes() function
The status of SHA2 algoritms and public-key encryption can now be
changed to 'always available.'
That makes pgcrypto functionally complete and unless there will be new
editions of AES, SHA2 or OpenPGP standards, there is no major changes
planned.
- Replace sorted array of entries in maintenance_work_mem to binary tree,
this should improve create performance.
- More precisely calculate allocated memory, eliminate leaks
with user-defined extractValue()
- Improve wordings in tsearch2
This is an extension of pgstattuple to query information from indexes.
It supports btree, hash and gist. Gin is not supported. It scans only
index pages and does not read corresponding heap tuples. Therefore,
'dead_tuple' means the number of tuples with LP_DELETE flag.
Also, I added an experimental feature for btree indexes. It checks
fragmentation factor of indexes. If an leaf has the right link on the
next adjacent page in the file, it is assumed to be continuous (not
fragmented). It will help us to decide when to REINDEX.
ITAGAKI Takahiro
> Upstream confirmed my reply in the last mail in [1]: the complete
> escaping logic in DBMirror.pl is seriously screwew.
>
> [1] http://archives.postgresql.org/pgsql-bugs/2006-06/msg00065.php
I finally found some time to debug this, and I think I found a better
patch than the one you proposed. Mine is still hackish and is still a
workaround around a proper quoting solution, but at least it repairs
the parsing without introducing the \' quoting again.
I consider this a band-aid patch to fix the recent security update.
PostgreSQL gurus, would you consider applying this until a better
solution is found for DBMirror.pl?
Olivier, can you please confirm that the patch works for you, too?
Backpatched to 8.0.X.
Martin Pitt
* new split algorithm (as proposed in http://archives.postgresql.org/pgsql-hackers/2006-06/msg00254.php)
* possible call pickSplit() for second and below columns
* add spl_(l|r)datum_exists to GIST_SPLITVEC -
pickSplit should check its values to use already defined
spl_(l|r)datum for splitting. pickSplit should set
spl_(l|r)datum_exists to 'false' (if they was 'true') to
signal to caller about using spl_(l|r)datum.
* support for old pickSplit(): not very optimal
but correct split
* remove 'bytes' field from GISTENTRY: in any case size of
value is defined by it's type.
* split GIST_SPLITVEC to two structures: one for using in picksplit
and second - for internal use.
* some code refactoring
* support of subsplit to rtree opclasses
TODO: add support of subsplit to contrib modules
tuples with less header overhead than a regular HeapTuple, per my
recent proposal. Teach TupleTableSlot code how to deal with these.
As proof of concept, change tuplestore.c to store MinimalTuples instead
of HeapTuples. Future patches will expand the concept to other places
where it is useful.
initially be 0. This is needed as a previous ABORT might have wiped out
an automatically opened transaction without maintaining the cursor count.
- Fix regression test expected file for the correct ERROR message, which
we now get given the above bug fix.
used by OpenOffice. Dictionaries are placed at
http://lingucomponent.openoffice.org/spell_dic.html
Dictionary automatically recognizes format of files.
Warning. MySpell's format has limitation with compound
word support: it's impossible to mark affix as
compound-only affix. So for norwegian, german etc
languages it's recommended to use original ispell format.
For that reason I don't want to remove my2ispell
scripts, it's has workaround at least for norwegian language.
It required some changes in lexize algorithm, but interface with
dictionaries stays compatible with old dictionaries.
Funded by Georgia Public Library Service and LibLime, Inc.
versions of OpenSSL. If your OpenSSL does not contain SHA2, then there
should be no conflict. But ofcourse, if someone upgrades OpenSSL,
server starts crashing.
Backpatched to 8.1.X.
Marko Kreen
any use in the past many years, we'd have made some effort to include
them in all executor node types; but in fact they were only in
nodeAppend.c and nodeIndexscan.c, up until I copied nodeIndexscan.c's
occurrence into the new bitmap node types. Remove some other unused
macros in execdebug.h, too. Some day the whole header probably ought to
go away in favor of better-designed facilities.
pg_freespacemap_relations --- while one could theoretically get that
number by counting rows in pg_freespacemap_pages, it's surely the hard
way to do it. Avoid expensive and inconvenient conversion to and from
text format. Minor code and docs cleanup.
tracks index pages, not free space on pages):
1/ Index free bytes set to NULL
2/ Comment added to the README briefly mentioning the index business
3/ Columns reordered more logically
4/ 'Blockid' column removed
5/ Free bytes column renamed to just 'bytes' instead of 'blockfreebytes'
Mark Kirkwood
during parse analysis, not only errors detected in the flex/bison stages.
This is per my earlier proposal. This commit includes all the basic
infrastructure, but locations are only tracked and reported for errors
involving column references, function calls, and operators. More could
be done later but this seems like a good set to start with. I've also
moved the ReportSyntaxErrorPosition logic out of psql and into libpq,
which should make it available to more people --- even within psql this
is an improvement because warnings weren't handled by ReportSyntaxErrorPosition.
Most of the changes add the mandatory USING clause to DROP OPERATOR
CLASS statements. DROP TYPE is now DROP TYPE CASCADE; without
CASCADE a DROP TYPE fails due to the circular dependency on the
type's I/O functions. The DROP FUNCTION statements for the I/O
functions have been removed, as DROP TYPE CASCADE removes them
automatically. Patch from Michael Fuhr.
similar constants if they were not previously defined. All these
constants must be defined by limits.h according to C89, so we can
safely assume they are present.
(respectively) to rename yylex and related symbols. Some were doing
it this way already, while others used not-too-reliable sed hacks in
the Makefiles. It's all nice and consistent now.
1) rank_cd now use weight of lexemes
2) rank_cd and rank can use any combination of normalization methods:
no normalization
normalization by log(length of document)
-----/------- by length of document
-----/------- by number of unique word in document
-----/------- by log(number of unique word in document)
-----/------- by number of covers (only rank_cd)
Improve cover's search.
TODO: changes in documentation
are unnecessarily allocated on the heap rather than the stack. If the
StringInfo doesn't outlive the stack frame in which it is created,
there is no need to allocate it on the heap via makeStringInfo() --
stack allocation is faster. While it's not a big deal unless the
code is in a critical path, I don't see a reason not to save a few
cycles -- using stack allocation is not less readable.
I also cleaned up a bit of code along the way: moved variable
declarations into a more tightly-enclosing scope where possible,
fixed some pointless copying of strings in dblink, etc.
more compliant with the error message style guide. In particular,
errdetail should begin with a capital letter and end with a period,
whereas errmsg should not. I also fixed a few related issues in
passing, such as fixing the repeated misspelling of "lexeme" in
contrib/tsearch2 (per Tom's suggestion).
pgcrypto crypt()/md5 and hmac() leak memory when compiled against
OpenSSL as openssl.c digest ->reset will do two DigestInit calls
against a context. This happened to work with OpenSSL 0.9.6
but not with 0.9.7+.
Reason for the messy code was that I tried to avoid creating
wrapper structure to transport algorithm info and tried to use
OpenSSL context for it. The fix is to create wrapper structure.
It also uses newer digest API to avoid memory allocations
on reset with newer OpenSSLs.
Thanks to Daniel Blaisdell for reporting it.
sorry but fix can't be applyed to previous version: it's require
refill tsvector...
2 Small optimize of load time for huge dictionaries
3 use palloc instead of malloc during load dict file
singlebyte encodings, so we should have snowball for every encodings.
I hope that finalize multibyte support work in tsearch2, but testing is needed...
sizebitvec of tsearch2, as well as identical code in several other
contrib modules. This provided about a 20X speedup in building a
large tsearch2 index ... didn't try to measure its effects for other
operations. Thanks to Stephan Vollmer for providing a test case.
the data defining the semantics of a lock method (ie, conflict resolution
table and ancillary data, which is all constant) and the hash tables
storing the current state. The only thing we give up by this is the
ability to use separate hashtables for different lock methods, but there
is no need for that anyway. Put some extra fields into the LockMethod
definition structs to clean up some other uglinesses, like hard-wired
tests for DEFAULT_LOCKMETHOD and USER_LOCKMETHOD. This commit doesn't
do anything about the performance issues we were discussing, but it clears
away some of the underbrush that's in the way of fixing that.
support for the dbf2pg contrib module.
The submitter created a patch which replaces the silent ignoring of -F
(when iconv support is disabled) with a meaningful warning.
Martin Pitt
comment line where output as too long, and update typedefs for /lib
directory. Also fix case where identifiers were used as variable names
in the backend, but as typedefs in ecpg (favor the backend for
indenting).
Backpatch to 8.1.X.
- supports multibyte encodings
- more strict rules for lexemes
- flex isn't used
Add:
- tsquery plainto_tsquery(text)
Function makes tsquery from plain text.
- &&, ||, !! operation for tsquery for combining
tsquery from it's parts: 'foo & bar' || 'asd' => 'foo & bar | asd'
functionality, but I still need to make another pass looking at places
that incidentally use arrays (such as ACL manipulation) to make sure they
are null-safe. Contrib needs work too.
I have not changed the behaviors that are still under discussion about
array comparison and what to do with lower bounds.
1 Comparison operation for tsquery
2 Btree index on tsquery
3 numnode(tsquery) - returns 'length' of tsquery
4 tsquery @ tsquery, tsquery ~ tsquery - contains, contained for tsquery.
Note: They don't gurantee exact result, only MAY BE, so it
useful only for speed up rewrite functions
5 GiST index support for @,~
6 rewrite():
select rewrite(orig, what, to);
select rewrite(ARRAY[orig, what, to]) from tsquery_table;
select rewrite(orig, 'select what, to from tsquery_table;');
7 significantly improve cover algorithm
if there isn't one already open. Upon dblink_close, only commit
the open transaction if it was started by dblink_open, and only
then when all cursors opened by dblink_open are closed. The transaction
accounting is done individually for all named connections, plus
the persistent unnamed connection.
pointers, to ensure that compilers won't rearrange accesses to occur
while we're not holding the buffer header spinlock. It's probably
not necessary to mark volatile in every single place in bufmgr.c,
but better safe than sorry. Per trouble report from Kevin Grittner.
like '23:59:60' because of fractional-second roundoff problems. Trying
to control this upstream of the actual display code was hopeless; the right
way is to explicitly round fractional seconds in the display code and then
refigure the results if the fraction rounds up to 1. Per bug #1927.
Remove unportable use of tfind/tsearch in favor of bsearch. Fix up
random number generator to use random() not rand() and to actually honor
its min/max arguments properly. That wasn't so important before, but
with exposure of capability to ask for general ranges, it will be.
argument as a 'regclass' value instead of a text string. The frontend
conversion of text string to pg_class OID is now encapsulated as an
implicitly-invocable coercion from text to regclass. This provides
backwards compatibility to the old behavior when the sequence argument
is explicitly typed as 'text'. When the argument is just an unadorned
literal string, it will be taken as 'regclass', which means that the
stored representation will be an OID. This solves longstanding problems
with renaming sequences that are referenced in default expressions, as
well as new-in-8.1 problems with renaming such sequences' schemas or
moving them to another schema. All per recent discussion.
Along the way, fix some rather serious problems in dbmirror's support
for mirroring sequence operations (int4 vs int8 confusion for instance).
a few typos in comments.
The dictionaries I checked list "altho" as a variant of "although,"
but I didn't find any other instances of the former in the source
tree so I changed it.
Michael Fuhr
the pubkey functions a bit. The actual RSA-specific code
there is tiny, most of the patch consists of reorg of the
pubkey code, as lots of it was written as elgamal-only.
---------------------------------------------------------------------------
The SHLIB section was copy-pasted from somewhere and contains
several unnecessary libs. This cleans it up a bit.
-lcrypt
we don't use system crypt()
-lssl, -lssleay32
no SSL here
-lz in win32 section
already added on previous line
-ldes
The chance anybody has it is pretty low.
And the chance pgcrypto works with it is even lower.
Also trim the win32 section.
---------------------------------------------------------------------------
It is already disabled in Makefile, remove code too.
---------------------------------------------------------------------------
I was bit hasty making the random exponent 'k' a prime. Further researh
shows that Elgamal encryption has no specific needs in respect to k,
any random number is fine.
It is bit different for signing, there it needs to be 'relatively prime'
to p - 1, that means GCD(k, p-1) == 1, which is also a lot lighter than
full primality. As we don't do signing, this can be ignored.
This brings major speedup to Elgamal encryption.
---------------------------------------------------------------------------
o pgp_mpi_free: Accept NULLs
o pgp_mpi_cksum: result should be 16bit
o Remove function name from error messages - to be similar to other
SQL functions, and it does not match anyway the called function
o remove couple junk lines
---------------------------------------------------------------------------
o Support for RSA encryption
o Big reorg to better separate generic and algorithm-specific code.
o Regression tests for RSA.
---------------------------------------------------------------------------
o Tom stuck a CVS id into file. I doubt the usefulness of it,
but if it needs to be in the file then rather at the end.
Also tag it as comment for asciidoc.
o Mention bytea vs. text difference
o Couple clarifications
---------------------------------------------------------------------------
There is a choice whether to update it with pgp functions or
remove it. I decided to remove it, updating is pointless.
I've tried to keep the core of pgcrypto relatively independent
from main PostgreSQL, to make it easy to use externally if needed,
and that is good. Eg. that made development of PGP functions much
nicer.
But I have no plans to release it as generic library, so keeping such
doc
up-to-date is waste of time. If anyone is interested in using it in
other products, he can probably bother to read the source too.
Commented source is another thing - I'll try to make another pass
over code to see if there is anything non-obvious that would need
more comments.
---------------------------------------------------------------------------
Marko Kreen
calculations for interval and time/timetz to behave sanely for both
integer and float timestamps; up to now I think it's been doing
something pretty strange...
OpenSSL 0.9.6x. The DES functions use the older 'des_'
API, but the newer 3DES functions use the 0.9.7x-only
'DES_' API.
I think I just used /usr/include/openssl/des.h for reference
when implementing them, and had upgraded OpenSSL in the
meantime.
Following patch converts DES also to newer API and provides
compatibility functions for OpenSSL < 0.9.7.
I chose this route because:
- openssl.c uses few DES functions.
- compatibility for old 'des_' API is going away at some point
of time from OpenSSL.
- as seen from macros, new API is saner
- Thus pgcrypto supports any OpenSSL version from 0.9.5 to 1.0
Tested with OpenSSL 0.9.6c and 0.9.7e.
Marko Kreen
of password-based encryption from RFC2440 (OpenPGP).
The goal of this code is to be more featureful encryption solution
than current encrypt(), which only functionality is running cipher
over data.
Compared to encrypt(), pgp_encrypt() does following:
* It uses the equvialent of random Inital Vector to get cipher
into random state before it processes user data
* Stores SHA-1 of the data into result so any modification
will be detected.
* Remembers if data was text or binary - thus it can decrypt
to/from text data. This was a major nuisance for encrypt().
* Stores info about used algorithms with result, so user needs
not remember them - more user friendly!
* Uses String2Key algorithms (similar to crypt()) with random salt
to generate full-length binary key to be used for encrypting.
* Uses standard format for data - you can feed it to GnuPG, if needed.
Optional features (off by default):
* Can use separate session key - user data will be encrypted
with totally random key, which will be encrypted with S2K
generated key and attached to result.
* Data compression with zlib.
* Can convert between CRLF<->LF line-endings - to get fully
RFC2440-compliant behaviour. This is off by default as
pgcrypto does not know the line-endings of user data.
Interface is simple:
pgp_encrypt(data text, key text) returns bytea
pgp_decrypt(data text, key text) returns text
pgp_encrypt_bytea(data bytea, key text) returns bytea
pgp_decrypt_bytea(data bytea, key text) returns bytea
To change parameters (cipher, compression, mdc):
pgp_encrypt(data text, key text, parms text) returns bytea
pgp_decrypt(data text, key text, parms text) returns text
pgp_encrypt_bytea(data bytea, key text, parms text) returns bytea
pgp_decrypt_bytea(data bytea, key text, parms text) returns bytea
Parameter names I lifted from gpg:
pgp_encrypt('message', 'key', 'compress-algo=1,cipher-algo=aes256')
For text data, pgp_encrypt simply encrypts the PostgreSQL internal data.
This maps to RFC2440 data type 't' - 'extenally specified encoding'.
But this may cause problems if data is dumped and reloaded into database
which as different internal encoding. My next goal is to implement data
type 'u' - which means data is in UTF-8 encoding by converting internal
encoding to UTF-8 and back. And there wont be any compatibility
problems with current code, I think its ok to submit this without UTF-8
encoding by converting internal encoding to UTF-8 and back. And there
wont be any compatibility problems with current code, I think its ok to
submit this without UTF-8 support.
Here is v4 of PGP encrypt. This depends on previously sent
Fortuna-patch, as it uses the px_add_entropy function.
- New function: pgp_key_id() for finding key id's.
- Add SHA1 of user data and key into RNG pools. We need to get
randomness from somewhere, and it is in user best interests
to contribute.
- Regenerate pgp-armor test for SQL_ASCII database.
- Cleanup the key handling so that the pubkey support is less
hackish.
Marko Kreen
- Move openssl random provider to openssl.c and builtin provider
to internal.c
- Make px_random_bytes use Fortuna, instead of giving error.
- Retarget random.c to aquiring system randomness, for initial seeding
of Fortuna. There is ATM 2 functions for Windows,
reader from /dev/urandom and the regular time()/getpid() silliness.
Marko Kreen
functions as STRICT, and all functions except gen_salt() as IMMUTABLE.
gen_salt() is VOLATILE.
Although the functions are now STRICT, I left their PG_ARGISNULL()
checks in place as a protective measure for users who install the
new code but use old (non-STRICT) catalog entries (e.g., restored
from a dump). Per recent discussion in pgsql-hackers.
Patch from Michael Fuhr.
to make. We ship the table file in the tarball and so this dependency
just opens file timestamp skew problems without doing anything useful.
(Not that it should hurt, either ... except for cross-compile builds.)
chdir into PGDATA and subsequently use relative paths instead of absolute
paths to access all files under PGDATA. This seems to give a small
performance improvement, and it should make the system more robust
against naive DBAs doing things like moving a database directory that
has a live postmaster in it. Per recent discussion.
(currently in beta) when cryptolib = openssl. According to the
following checkin message from several years ago, OpenSSL application
developers should no longer rely on <openssl/evp.h> to include
everything they need:
http://cvs.openssl.org/chngview?cn=9888
This patch adds the necessary header files. It doesn't appear to
break anything when building against OpenSSL 0.9.7.
BTW, core appears to build and work fine with OpenSSL 0.9.8. I've
built 7.3 through HEAD against 0.9.8-beta6 without noticing any
problems.
Michael Fuhr
- Fix wrong index results on text, char, varchar for multibyte strings
- Fix some SIGFPE signals
- Add support for infinite timestamps
- Because of locale settings, btree_gist can not be a prefix index anymore (for text).
Each node holds now just the lower and upper boundary.
current time: provide a GetCurrentTimestamp() function that returns
current time in the form of a TimestampTz, instead of separate time_t
and microseconds fields. This is what all the callers really want
anyway, and it eliminates low-level dependencies on AbsoluteTime,
which is a deprecated datatype that will have to disappear eventually.
literally.
Add GUC variables:
"escape_string_warning" - warn about backslashes in non-E strings
"escape_string_syntax" - supports E'' syntax?
"standard_compliant_strings" - treats backslashes literally in ''
Update code to use E'' when escapes are used.
to the existing X-direction tests. An rtree class now includes 4 actual
2-D tests, 4 1-D X-direction tests, and 4 1-D Y-direction tests.
This involved adding four new Y-direction test operators for each of
box and polygon; I followed the PostGIS project's lead as to the names
of these operators.
NON BACKWARDS COMPATIBLE CHANGE: the poly_overleft (&<) and poly_overright
(&>) operators now have semantics comparable to box_overleft and box_overright.
This is necessary to make r-tree indexes work correctly on polygons.
Also, I changed circle_left and circle_right to agree with box_left and
box_right --- formerly they allowed the boundaries to touch. This isn't
actually essential given the lack of any r-tree opclass for circles, but
it seems best to sync all the definitions while we are at it.
polygon operators (<<, &<, >>, &>). Per ideas originally put forward
by andrew@supernews and later rediscovered by moi. This patch just
fixes the existing opclasses, and does not add any new behavior as I
proposed earlier; that can be sorted out later. In principle this
could be back-patched, since it changes only search behavior and not
system catalog entries nor rtree index contents. I'm not currently
planning to do that, though, since I think it could use more testing.
a physically separate type. Defining 'lo' as a domain over OID works
just fine and is more efficient. Improve documentation and fix up the
test script. (Would like to turn test script into a proper regression
test, but right now its output is not constant because of numeric OIDs;
plus it makes Unix-specific assumptions about files it can import.)
unlike template0 and template1 does not have any special status in
terms of backend functionality. However, all external utilities such
as createuser and createdb now connect to "postgres" instead of
template1, and the documentation is changed to encourage people to use
"postgres" instead of template1 as a play area. This should fix some
longstanding gotchas involving unexpected propagation of database
objects by createdb (when you used template1 without understanding
the implications), as well as ameliorating the problem that CREATE
DATABASE is unhappy if anyone else is connected to template1.
Patch by Dave Page, minor editing by Tom Lane. All per recent
pghackers discussions.
includes error checking and an appropriate ereport(ERROR) message.
This gets rid of rather tedious and error-prone manipulation of errno,
as well as a Windows-specific bug workaround, at more than a dozen
call sites. After an idea in a recent patch by Heikki Linnakangas.
it is sufficient to track whether a backend holds a lock or not, and
store information about transaction vs. session locks only in the
inside-the-backend LocalLockTable. Since there can now be but one
PROCLOCK per lock per backend, LockCountMyLocks() is no longer needed,
thus eliminating some O(N^2) behavior when a backend holds many locks.
Also simplify the LockAcquire/LockRelease API by passing just a
'sessionLock' boolean instead of a transaction ID. The previous API
was designed with the idea that per-transaction lock holding would be
important for subtransactions, but now that we have subtransactions we
know that this is unwanted. While at it, add an 'isTempObject' parameter
to LockAcquire to indicate whether the lock is being taken on a temp
table. This is not used just yet, but will be needed shortly for
two-phase commit.
and RelationNameGetTupleDesc() as deprecated; remove uses of the
latter in the contrib library. Along the way, clean up crosstab()
code and documentation a little.
are now reported via elog, eliminating the need to test the result code
at most call sites. Make it possible for the caller to distinguish a
freshly acquired lock from one already held in the current transaction.
Use that capability to avoid redundant AcceptInvalidationMessages() calls
in LockRelation().
spotted by Qingqing Zhou. The HASH_ENTER action now automatically
fails with elog(ERROR) on out-of-memory --- which incidentally lets
us eliminate duplicate error checks in quite a bunch of places. If
you really need the old return-NULL-on-out-of-memory behavior, you
can ask for HASH_ENTER_NULL. But there is now an Assert in that path
checking that you aren't hoping to get that behavior in a palloc-based
hash table.
Along the way, remove the old HASH_FIND_SAVE/HASH_REMOVE_SAVED actions,
which were not being used anywhere anymore, and were surely too ugly
and unsafe to want to see revived again.
Also, remove the rather useless return value of LockReleaseAll. Change
response to detection of corruption in the shared lock tables to PANIC,
since that is the only way of cleaning up fully.
Originally an idea of Heikki Linnakangas, variously hacked on by
Alvaro Herrera and Tom Lane.
for testing PLs and contrib_regression for testing contrib, instead of
overwriting the core system's regression database as formerly done.
Andrew Dunstan
external projects, we should be careful about what parts of the GiST
API are considered implementation details, and which are part of the
public API. Therefore, I've moved internal-only declarations into
gist_private.h -- future backward-incompatible changes to gist.h should
be made with care, to avoid needlessly breaking external GiST extensions.
Also did some related header cleanup: remove some unnecessary #includes
from gist.h, and remove some unused definitions: isAttByVal(), _gistdump(),
and GISTNStrategies.
which is neither needed by nor related to that header. Remove the bogus
inclusion and instead include the header in those C files that actually
need it. Also fix unnecessary inclusions and bad inclusion order in
tsearch2 files.
that return INTERNAL without also having INTERNAL arguments. Since the
functions in question aren't meant to be called by hand anyway, I just
redeclared them to take 'internal' instead of 'text'. Also add code
to ProcedureCreate() to enforce the restriction, as I should have done
to start with :-(
Essentially, we shoehorn in a lockable-object-type field by taking
a byte away from the lockmethodid, which can surely fit in one byte
instead of two. This allows less artificial definitions of all the
other fields of LOCKTAG; we can get rid of the special pg_xactlock
pseudo-relation, and also support locks on individual tuples and
general database objects (including shared objects). None of those
possibilities are actually exploited just yet, however.
I removed pg_xactlock from pg_class, but did not force initdb for
that change. At this point, relkind 's' (SPECIAL) is unused and
could be removed entirely.
command line. We find this useful because we frequently deal with
thousands of tables in an environment where neither the databases nor
the tables are updated frequently. This helps allow us to cut down on
the overhead of updating the list for every other primary loop of
pg_autovacuum.
I chose -i as the command-line argument and documented it briefly in
the README.
The patch was applied to the 7.4.7 version of pg_autovacuum in contrib.
Thomas F.O'Connell
indexes. Replace all heap_openr and index_openr calls by heap_open
and index_open. Remove runtime lookups of catalog OID numbers in
various places. Remove relcache's support for looking up system
catalogs by name. Bulky but mostly very boring patch ...
change saves a great deal of space in pg_proc and its primary index,
and it eliminates the former requirement that INDEX_MAX_KEYS and
FUNC_MAX_ARGS have the same value. INDEX_MAX_KEYS is still embedded
in the on-disk representation (because it affects index tuple header
size), but FUNC_MAX_ARGS is not. I believe it would now be possible
to increase FUNC_MAX_ARGS at little cost, but haven't experimented yet.
There are still a lot of vestigial references to FUNC_MAX_ARGS, which
I will clean up in a separate pass. However, getting rid of it
altogether would require changing the FunctionCallInfoData struct,
and I'm not sure I want to buy into that.
* test error handling
* add tests for des, 3des, cast5
* add some tests to blowfish, rijndael
* Makefile: ability to specify different tests for different crypto
libraries, so we can skip des, 3des and cast5 for builtin.
Marko Kreen
Reserve px_get_random_bytes() for strong randomness,
add new function px_get_pseudo_random_bytes() for
weak randomness and use it in gen_salt().
On openssl case, use RAND_pseudo_bytes() for
px_get_pseudo_random_bytes().
Final result is that is user has not configured random
souce but kept the 'silly' one, gen_salt() keeps
working, but pgp_encrypt() will throw error.
Marko Kreen
* openssl.c: Add 3des and AES support
* README.pgcrypto: list only supported ciphers for openssl
OpenSSL has pre-processor symbol OPENSSL_NO_AES, which
isn't that helpful for detecting if it _does_ exist.
Thus the hack with AES_ENCRYPT.
Marko Kreen
* Use error codes instead of -1
* px_strerror for new error codes
* calling convention change for px_gen_salt - return error code
* use px_strerror in pgcrypto.c
Marko Kreen
It was a bad style to begin with, and now several loops can be clearer.
* pgcrypto.c: Fix function comments
* crypt-gensalt.c, crypt-blowfish.c: stop messing with errno
* openssl.c: use px_free instead pfree
* px.h: make redefining px_alloc/px_realloc/px_free easier
Marko Kreen
libmcrypt seems to dead, maintainer address bounces,
and cast-128 fails on 2 of the 3 test vectors from RFC2144.
So I see no reason to keep around stuff I don't trust
anymore.
Support for several crypto libraries is probably only
confusing to users, although it was good for initial
developing - it helped to find hidden assumptions and
forced me to create regression tests for all functionality.
Marko Kreen
can tell whether it is being used as an aggregate or not. This allows
such a function to avoid re-pallocing a pass-by-reference transition
value; normally it would be unsafe for a function to scribble on an input,
but in the aggregate case it's safe to reuse the old transition value.
Make int8inc() do this. This gets a useful improvement in the speed of
COUNT(*), at least on narrow tables (it seems to be swamped by I/O when
the table rows are wide). Per a discussion in early December with
Neil Conway. I also fixed int_aggregate.c to check this, thereby
turning it into something approaching a supportable technique instead
of being a crude hack.
0.9.7x have EVP_DigestFinal function which which clears all of
EVP_MD_CTX. This makes pgcrypto crash in functions which
re-use one digest context several times: hmac() and crypt()
with md5 algorithm.
Following patch fixes it by carring the digest info around
EVP_DigestFinal and re-initializing cipher.
Marko Kreen.
compares two strings' soundex values for similarity, from Kris Jurka.
Also mark the text_soundex() function as STRICT, to avoid crashing
on NULL input.
regression=# select to_tsquery( '\'fotballklubber\'');
to_tsquery
------------------------------------------------
'fotball' & 'klubb' | 'fot' & 'ball' & 'klubb'
(1 row)
So, changed interface to dictionaries, lexize method of dictionary shoud return
pointer to aray of TSLexeme structs instead of char**. Last element should
have TSLexeme->lexeme == NULL.
typedef struct {
/* number of variant of split word , for example
Word 'fotballklubber' (norwegian) has two varian to split:
( fotball, klubb ) and ( fot, ball, klubb ). So, dictionary
should return:
nvariant lexeme
1 fotball
1 klubb
2 fot
2 ball
2 klubb
*/
uint16 nvariant;
/* currently unused */
uint16 flags;
/* C-string */
char *lexeme;
} TSLexeme;
typedef struct {} WordEntryPos;
to
typedef uint16 WordEntryPos
according to http://www.pgsql.ru/db/mw/msg.html?mid=2035188
Require re-fill all tsvector fields and reindex tsvector indexes.
- Dependency services may not be correctly registered when installing as
a Windows Service.
- The sleep time is changed from milliseconds to seconds as it should
be.
- Error messages during service installation/removal are logged to
stderr.
1 Report error message instead of do nothing in case of error in regex
2 Malloced storage for mask, find and repl part of Affix. This parts may be
large enough in real life (for example in czech, thanks to moje <moje@kalhotky.net>)
Also performed an initial run through of upgrading our Copyright date to
extend to 2005 ... first run here was very simple ... change everything
where: grep 1996-2004 && the word 'Copyright' ... scanned through the
generated list with 'less' first, and after, to make sure that I only
picked up the right entries ...
checks, to ensure the previous test backend has time to quit out of the
regression database. Also, allow all the checks to be run even if one
of them fails. Per suggestions from Andrew Dunstan to improve the
usefulness of buildfarm testing.
> > Windows service, it says you can use the -I and -R options.
> >
> > When I do that and I specify a password with '-P'
> (uppercase) then in
> > the registry it's saved as '-p' (lowercase) in the
> service-commandline
> > (ImagePath).
This was fixed in v1.21 of pg_autovacuum.c, That rev is tagged for
beta3, so you should not be seeing this issue unless you actually have
an older version for some reason.
http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/pg_autovacuum/p
g_autovacuum.c.diff?r1=1.20;r2=1.21;f=h
> > Also it removes the quotes I added and I'm not so sure it
> would work
> > the way it's supposed to, without it.
It's not so much that it strips them (that happens automagically), more
that it doesn't re-add them when it writes the command line in the
registry. The attached patch fixes that by simply quoting all options
that may need it.
> > If you add DependOnService (a REG_MULTI_SZ an
> array-like-thingie) and
> > have the name (in this case: pgsql-8.0-beta2-dev3) of a service it
> > depends on, it will not fail to start (it will not even try, as
> > PostgreSQL is not running), when PostgreSQL already failed.
> >
> > Maybe it's an idea to specify it on the commandline (what
> service to
> > depend on).
A -E <service> option is added in the attached patch.
Dave Page
warnings:
- remove pointless "extern" keyword from some function definitions in
contrib/tsearch2
- use "NULL" not "0" as NULL pointer in contrib/tsearch,
contrib/tsearch2, contrib/pgbench, and contrib/vacuumlo
clause implicitly whenever one is not given explicitly. Remove concept
of a schema having an associated tablespace, and simplify the rules for
selecting a default tablespace for a table or index. It's now just
(a) explicit TABLESPACE clause; (b) default_tablespace if that's not an
empty string; (c) database's default. This will allow pg_dump to use
SET commands instead of tablespace clauses to determine object locations
(but I didn't actually make it do so). All per recent discussions.
- add some additional files to the dbmirror install (approved by
ssinger)
- add a makefile for contrib/mysql, and add mysql to the list of
contribs build by default
- use xml2-config to pickup -I flags for libxml2 in contrib/xml and
contrib/xml2
Original work from Martin Pitt of Debian, minor cleanups by Neil
Conway.
snprintf(data, len, %s, NULL) crash.
The code was trying to find a connection by name when it already had an
unnamed connection and did not have a name to search with.
Kris Jurka
returning a NULL pointer (some callers remembered to check the return
value, but some did not -- it is safer to just bail out).
Also, cleanup pgstat.c to use elog(ERROR) rather than elog(LOG) followed
by exit().
http://archives.postgresql.org/pgsql-hackers/2004-10/msg00464.php.
This fix is intended to be permanent: it moves the responsibility for
calling SetBufferCommitInfoNeedsSave() into the tqual.c routines,
eliminating the requirement for callers to test whether t_infomask changed.
Also, tighten validity checking on buffer IDs in bufmgr.c --- several
routines were paranoid about out-of-range shared buffer numbers but not
about out-of-range local ones, which seems a tad pointless.
must be stale. Tweak example startup scripts to not use pg_ctl but launch
the postmaster directly, thereby ensuring that only the postmaster's direct
parent shell will be a postgres-owned process. In combination these should
fix the longstanding problem of the postmaster sometimes refusing to start
during reboot because it thinks the old lockfile is not stale.
subsequent core dump. It looks like at one time DBLINK_RES_ERROR_AS_NOTICE
didn't include a PQclear, but now it does and so these other ones are
duplicate.
mode see a fresh snapshot for each command in the function, rather than
using the latest interactive command's snapshot. Also, suppress fresh
snapshots as well as CommandCounterIncrement inside STABLE and IMMUTABLE
functions, instead using the snapshot taken for the most closely nested
regular query. (This behavior is only sane for read-only functions, so
the patch also enforces that such functions contain only SELECT commands.)
As per my proposal of 6-Sep-2004; I note that I floated essentially the
same proposal on 19-Jun-2002, but that discussion tailed off without any
action. Since 8.0 seems like the right place to be taking possibly
nontrivial backwards compatibility hits, let's get it done now.
-It fixes up some bugs with handling setval calls
-Adds upgrade instructions from prior versions
-Improved the sample config file
-Fixed some things in the clean_pending script
tablespaces correctly, and is quite restricted on objects covered (only
tables and databases, but not tablespaces and indexes).
The attached patch contributes:
- database_size(name)
- relation_size(text)
These are the well-known functions, tablespace-aware.
- pg_tablespace_size(oid)
- pg_database_size(oid)
- pg_relation_size(oid)
Tablespace-aware implementations, used by the upper functions.
pg_relation_size will report sizes of indexes as well.
- pg_size_pretty(bigint)
Formatting of sizes, to display '146MB' instead of '152885668'
Andreas Pflug
PROCLOCK structs in shared memory now have only a bitmask for held
locks, rather than counts (making them 40 bytes smaller, which is a
good thing). Multiple locks within a transaction are counted in the
local hash table instead, and we have provision for tracking which
ResourceOwner each count belongs to. Solves recently reported problem
with memory leakage within long transactions.