Subject: [PATCHES] port patch: ultrix4
ultrix4 doesn't compile without this. this also fixes a problem
with dynamic loading (ultrix relocatable objects must be loaded
with -G 0).
Add mixed-case #define synonyms to avoid changing more source code.
Add comparison operators for boolean.
Add aggregate min() and max() for datetime and timespan.
Here are patches which should help fix timezone problems in the
datetime and abstime code. Also, I repatched varlena.c to add in
some comments and a little error checking on top of Vadim's earlier
repairs. There are slight mods to the circle data type to have the
distance operator between circles measure the distance between
closest points rather than between centers.
Subject: Re: [PATCHES] SET DateStyle patches
On Tue, 22 Apr 1997, Thomas Lockhart wrote:
> Some more patches! These (try to) finish implementing SET variable TO value
> for "DateStyle" (changed the name from simply "date" to be more descriptive).
> This is based on code from Martin and Bruce (?), which was easy to modify.
> The syntax is
>
> SET DateStyle TO 'iso'
> SET DateStyle TO 'postgres'
> SET DateStyle TO 'sql'
> SET DateStyle TO 'european'
> SET DateStyle TO 'noneuropean'
> SET DateStyle TO 'us' (same as "noneuropean")
> SET DateStyle TO 'default' (current same as "postgres,us")
>
> ("european" is just compared for the first 4 characters, and "noneuropean"
> is compared for the first 7 to allow less typing).
>
> Multiple arguments are allowed, so SET datestyle TO 'sql,euro' is valid.
>
> My mods also try to implement "SHOW variable" and "RESET variable", but
> that part just core dumps at the moment. I would guess that my errors
> are obvious to someone who knows what they are doing with the parser stuff,
> so if someone (Bruce and/or Martin??) could have it do the right thing
> we will have a more complete set of what we need.
>
> Also, I would like to have a floating point precision global variable to
> implement "SET precision TO 10" and perhaps "SET precision TO 10,2" for
> float8 and float4, but I don't know how to do that for integer types rather
> than strings. If someone is fixing the SHOW and RESET code, perhaps they can
> add some hooks for me to do the floats while they are at it.
>
> I've left some remnants of variable structures in the source code which
> I did not use in the interests of getting something working for v6.1.
> We'll have time to clean things up for the next release...
OK, here are a passel of patches for the geometric data types.
These add a "circle" data type, new operators and functions
for the existing data types, and change the default formats
for some of the existing types to make them consistant with
each other. Current formatting conventions (e.g. compatible
with v6.0 to allow dump/reload) are supported, but the new
conventions should be an improvement and we can eventually
drop the old conventions entirely.
For example, there are two kinds of paths (connected line segments),
open and closed, and the old format was
'(1,2,1,2,3,4)' for a closed path with two points (1,2) and (3,4)
'(0,2,1,2,3,4)' for an open path with two points (1,2) and (3,4)
Pretty arcane, huh? The new format for paths is
'((1,2),(3,4))' for a closed path with two points (1,2) and (3,4)
'[(1,2),(3,4)]' for an open path with two points (1,2) and (3,4)
For polygons, the old convention is
'(0,4,2,0,4,3)' for a triangle with points at (0,0),(4,4), and (2,3)
and the new convention is
'((0,0),(4,4),(2,3))' for a triangle with points at (0,0),(4,4), and (2,3)
Other data types which are also represented as lists of points
(e.g. boxes, line segments, and polygons) have similar representations
(they surround each point with parens).
For v6.1, any format which can be interpreted as the old style format
is decoded as such; we can remove that backwards compatibility but ugly
convention for v7.0. This will allow dump/reloads from v6.0.
These include some updates to the regression test files to change the test
for creating a data type from "circle" to "widget" to keep the test from
trashing the new builtin circle type.
Subject: [HACKERS] Patch: set date to euro/us postgres/iso/sql
Here a patch that implements a SET date for use by the datetime
stuff. The syntax is
SET date TO 'val[,val,...]'
where val is us (us dates), euro (european dates), postgres,
iso or sql.
Thomas is working on the integration in his datetime module.
I just needed to get the patch out before it went stale :)
Subject: [HACKERS] Money integration patches
Here are patches to integrate the money data type. I have included
some math and aggregate functions and have made the locale support optional
by #ifdef USE_LOCALE bracketing of functions.
Modules affected are:
builtins.h.patch
cash.c.patch
cash.h.patch
main.c.patch
pg_aggregate.h.patch
pg_operator.h.patch
pg_proc.h.patch
pg_type.h.patch
I changed the data type to be pass-by-reference rather than by-value
to pave the way for a larger internal representation (64-bit ints?).
Also, I changed the tabbing of cash.c and cash.h to match most of
the other Postgres source code files (4 space indent, 8 spaces == 1 tab).
The locale stuff should be tested under another convention (Russian?)
but I don't know what the correct results should be so perhaps someone
else can give them a try. Will update docs and regression tests in
the next few days.
pg_proc.h still needs modifying, but this gets it in there so that we can
get around any compiler bugs. Will try and get the pg_proc.h entries done
up later tonight...
Remove USE_LOCALE from Makefile.global.in
Add USE_LOCALE to build/configure/config.h
Add check for BUILDRUN in configure to make sure that build is run before
configure
Subject: [HACKERS] Aggregate function patches
Here are the aggregate function patches I originally sent in last December.
They fix sum() and avg() behavior for ints and floats when NULL values are
involved.
I was waiting to resubmit these until I had a chance to write a v6.0->v6.1
database upgrade script to ensure that existing v6.0 databases which have
not been reloaded for v6.1 do no break with the new aggregate behavior.
These scripts are included below. It's OK with me if someone wants to do
something different with the upgrade strategy, but something like this
was discussed a few weeks ago.
Also, there were a couple of small items which cropped up in doing a clean
install of 970403 (actually 970402 + 970403 changes since the full 970403
tar file appears to be damaged or at least suspect). They are the first
two patches below and can be omitted if desired (although I think they
aren't dangerous :).
Subject: [HACKERS] More date time functions
Here are some additional patches mostly related to the date and time
data types. It includes some type conversion routines to move between
the different date types and some other date manipulation routines such
as date_part(units,datetime).
I noticed Edmund Mergl et al's neat trick for getting function overloading
for builtin functions, so started to use that for the date and time stuff.
Later, if someone figures out how to get function overloading directly
for internal C code, then we can move to that technique.
These patches include documentation updates (don't faint!) for the built-in
man page. Doesn't yet include mention of timestamp, since I don't know
much about it and since it may change a bit to become a _real_ ANSI timestamp
which would include parser support for the declaration syntax (what do you
think, Dan?).
The patches were developed on the 970330 release, but have been rebuilt
off of the 970402 release. The first patch below is to get libpq to compile,
on my Linux box, but is not related to the rest of the patches and you can
choose not to apply that one at this time. Thanks in advance, scrappy!
Subject: [HACKERS] Patch: SET var TO 'val'
Here is a patch that adds a "SET variable TO 'somevalue'" capability
to the parser, and then calls the SetPGVariable() function (which does
just issue a elog(NOTICE) to see whether it works).
That's the framework for adding timezone/date format/language/...
stuff.
Subject: Re: [HACKERS] abstime "now" broken
Yes, I broke 'now' :( with an attempt at a bug fix involving
servers running in the UTC/GMT timezone. These patches fix
the problem, and have been tested in GMT (+00 hours),
PST (-08), and NZT (+12) timezones which exercized the code for
various cases including across day boundaries. btw, this code
fixes the same type of problem for 'today', 'yesterday', 'tomorrow',
for DATETIME, ABSTIME, DATE and TIME types.
The bugfix itself is quite small, but I have accumulated other
changes in the datetime data type and include them here also.
One set of changes involves printing ISO-formatted dates and
is in response to the helpful information from Kurt Lidl regarding
ANSI SQL dates. I'll send another e-mail sometime soon discussing
more issues he has raised...
Reply-To: hackers@hub.org, Dan McGuirk <mcguirk@indirect.com>
To: hackers@hub.org
Subject: [HACKERS] tmin writeback optimization
I was doing some profiling of the backend, and noticed that during a certain
benchmark I was running somewhere between 30% and 75% of the backend's CPU
time was being spent in calls to TransactionIdDidCommit() from
HeapTupleSatisfiesNow() or HeapTupleSatisfiesItself() to determine that
changed rows' transactions had in fact been committed even though the rows'
tmin values had not yet been set.
When a query looks at a given row, it needs to figure out whether the
transaction that changed the row has been committed and hence it should pay
attention to the row, or whether on the other hand the transaction is still
in progress or has been aborted and hence the row should be ignored. If
a tmin value is set, it is known definitively that the row's transaction
has been committed. However, if tmin is not set, the transaction
referred to in xmin must be looked up in pg_log, and this is what the
backend was spending a lot of time doing during my benchmark.
So, implementing a method suggested by Vadim, I created the following
patch that, the first time a query finds a committed row whose tmin value
is not set, sets it, and marks the buffer where the row is stored as
dirty. (It works for tmax, too.) This doesn't result in the boost in
real time performance I was hoping for, however it does decrease backend
CPU usage by up to two-thirds in certain situations, so it could be
rather beneficial in high-concurrency settings.
Subject: [HACKERS] backend/utils/adt/timestamp.c
Back to this timezone stuff. The struct tm has a field (tm_gmtoff) which
is the offset from UTC (GMT is archaic BTW) in seconds. Is this the
value you are looking for when you use timezone? Note that this applies
to NetBSD but it does not appear to be in either ANSI C or POSIX. This
looks like one of those things that is just going to have to be hand
coded for each platform.
Why not just store the values in UTC and use localtime instead of
gmtime when retrieving the value?
Also, you assume the time is returned as a 4 byte integer. In fact,
there is not even any requirement that time be an integral value. You
should use time_t here.
The input function seems unduly restrictive. Somewhere in the sources
there is an input function that allows words for months. Can't we do
the same here?
There is a standard function, difftime, for subtracting two times. It
deals with cases where time_t is not integral. There is, however, a
small performance hit since it returns a double and I don't believe
there is any system currently which uses anything but an integral for
time_t. Still, this is technically the correct and portable thing to do.
The returns from the various comparisons should probably be a bool.
of endian.h. I figure that if it exists it's pretty sure that it has
the byte order information and we may catch some other ports without
any further testing.
From: "D'Arcy J.M. Cain" <darcy@druid.net>
Subject: [HACKERS] More patches for date/time
I have accumulated several patches to add functionality to the datetime
and timespan data types as well as to fix reported porting bugs on non-BSD
machines. These patches are:
dt.c.patch - add datetime_part(), fix bugs
dt.h.patch - add quarter and timezone support, add prototypes
globals.c.patch - add time and timezone variables
miscadmin.h.patch - add time and timezone variables
nabstime.c.patch - add datetime conversion routine
nabstime.h.patch - add prototypes
pg_operator.h.patch - add datetime operators, clean up formatting
pg_proc.h.patch - add datetime functions, reassign conflicting date OIDs
pg_type.h.patch - add datetime and timespan data types
The dt.c and pg_proc.h patches are fairly large; the latter mostly because I tried
to get some columns for existing entries to line up.
Subject: [HACKERS] libpq/pqcomm stuff and Solaris byte order
I decided to go ahead with the required changes since no one else seems
to. I don't guarantee that it is perfect but with these changes the
package actually compiles. While I was at it I added to the Sparc
Solaris header to define the byte order. Note that NetBSD sets this
in the system headers so it wasn't required there.
In particular, someone may want to check whether I removed the correct
84 lines from backend/libpq/pqcomprim.c.
of common routines in pqcomprim.c (pq communication primitives).
Not all adapted to it yet, but it's a start.
- Rewritten some of those routines, to write/read bigger chunks of
data, precomputing stuff in buffers instead of sending out byte
by byte.
- As a consequence, I need to know the endianness of the machine.
Currently I rely on getting it from machine/endian.h, but this
may not be available everywhere? (Who the hell thought it was
a good idea to pass integers to the backend the other way around
than the normal network byte order? *argl*)
- Libpq looks in the environment for magic variables, and upon
establishing a connection to the backend, sends it queries
of the form "SET var_name TO 'var_value'". This needs a change
in the backend parser (Mr. Parser, are you there? :)
- Currently it looks for two Env-Vars, namely PG_DATEFORMAT
and PG_FLOATFORMAT. What else makes sense? PG_TIMEFORMAT?
PG_TIMEZONE?
From: "Martin J. Laubach" <mjl@wwx.vip.at>
Subject: [HACKERS] Patch for io routines
I am currently trying to improve on the front-backend communication
routines; and noticed that lots of code are duplicated for libpq and
the backend. This is a first patch that tries to share code between
the two, more to follow.
mjl
Subject: [HACKERS] password authentication
This patch adds support for plaintext password authentication. To use
it, you add a line like
host all 0.0.0.0 0.0.0.0 password pg_pwd.conf
to your pg_hba.conf, where 'pg_pwd.conf' is the name of a file containing
the usernames and password hashes in the format of the first two fields
of a Unix /etc/passwd file. (Of course, you can use a specific database
name or IP instead.)
Then, to connect with a password through libpq, you use the PQconnectdb()
function, specifying the "password=" tag in the connect string and also
adding the tag "authtype=password".
I also added a command-line switch '-u' to psql that tells it to prompt
for a username and password and use password authentication.
Subject: [HACKERS] linux/alpha patches
These patches lay the groundwork for a Linux/Alpha port. The port doesn't
actually work unless you tweak the linker to put all the pointers in the
first 32 bits of the address space, but it's at least a start. It
implements the test-and-set instruction in Alpha assembly, and also fixes
a lot of pointer-to-integer conversions, which is probably good anyway.
Subject: [HACKERS] better access control error messages
This patch replaces the 'no such class or insufficient privilege' with
distinct error messages that tell you whether the table really doesn't
exist or whether access was denied.
The following patches add to the backend a new debugging flag -K which prints
a debug trace of all locking operations on user relations (those with oid
greater than 20000). The code is compiled only if LOCK_MGR_DEBUG is defined,
so the patch should be harmless if not explicitly enabled.
I'm using the code to trace deadlock conditions caused by application queries
using the command "$POSTMASTER -D $PGDATA -o '-d 1 -K 1'.
The patches are for version 6.0 dated 970126.
Patches from: aoki@CS.Berkeley.EDU (Paul M. Aoki)
i gave jolly my btree bulkload code a long, long time ago but never
gave him a bunch of my bugfixes. here's a diff against the 6.0
baseline.
for some reason, this code has slowed down somewhat relative to the
insertion-build code on very small tables. don't know why -- it used
to be within about 10%. anyway, here are some (highly unscientific!)
timings on a dec 3000/300 for synthetic tables with 10k, 100k and
1000k tuples (basically, 1mb, 10mb and 100mb heaps). 'c' means
clustered (pre-sorted) inputs and 'u' means unclustered (randomly
ordered) inputs. the 10k table basically fits in the buffer pool, but
the 100k and 1000k tables don't. as you can see, insertion build is
fine if you've sorted your heaps on your index key or if your heap
fits in core, but is absolutely horrible on unordered data (yes,
that's 7.5 hours to index 100mb of data...) because of the zillions of
random i/os.
if it doesn't work for you for whatever reason, you can always turn it
back off by flipping the FastBuild flag in nbtree.c. i don't have
time to maintain it.
good luck!
baseline code:
time psql -c 'create index c10 on k10 using btree (c int4_ops)' bttest
real 8.6
time psql -c 'create index u10 on k10 using btree (b int4_ops)' bttest
real 9.1
time psql -c 'create index c100 on k100 using btree (c int4_ops)' bttest
real 59.2
time psql -c 'create index u100 on k100 using btree (b int4_ops)' bttest
real 652.4
time psql -c 'create index c1000 on k1000 using btree (c int4_ops)' bttest
real 636.1
time psql -c 'create index u1000 on k1000 using btree (b int4_ops)' bttest
real 26772.9
bulkloading code:
time psql -c 'create index c10 on k10 using btree (c int4_ops)' bttest
real 11.3
time psql -c 'create index u10 on k10 using btree (b int4_ops)' bttest
real 10.4
time psql -c 'create index c100 on k100 using btree (c int4_ops)' bttest
real 59.5
time psql -c 'create index u100 on k100 using btree (b int4_ops)' bttest
real 63.5
time psql -c 'create index c1000 on k1000 using btree (c int4_ops)' bttest
real 636.9
time psql -c 'create index u1000 on k1000 using btree (b int4_ops)' bttest
real 701.0
Essentially, config.h now includes an 'os.h', which is created via
configure by linking a "port.h" file from the port directory to the
include directory.
Going to try to merge backend/port in similar ways
#if defined(aix)
#define TERMIOS_H_LOCATION <termios.h>
#else
#define TERMIOS_H_LOCATION <sys/termios.h>
#endif
libpq/fe-exec.c modified so that location of termios.h is determined
by whether HAVE_TERMIOS_H is defined or not, in preparation for switch
to configure
/usr/include/limits.h (which quiets the costsize.c warnings)...under
FreeBSD, /usr/include/limits.h *includes* machine/limits.h, while under
Solaris, there is no such things as /usr/include/machine...
Problem with Solaris pointed out by Mark Wahl
At least the first two should be fixed before the final release of 6.0.
1) There is a mismatch between the type declared in the catalog for
the input/output attributes of pg_type and the actual type of
values stored in the table. The type of typinput, typoutput,
typsend and typreceive are declared oid (26) while the values are
regproc (24). The error was there also in previous versions but
nobody noticed it until an Assert has been added in ExecEvalVar.
The effect is that it is now impossible to replace the typoutput
of existing data types with new procs.
2) The identd hba fails after the first time because the data read
from the identd socket is not zero-terminated and strlen reports
an incorrect length if the stack contains garbage, which usually
happens after the first connection has been made.
3) The new initdb wants to create itself the data directory. This
implies that the parent directory must be writable by postgres and
this may not always be desirable. A better solution would be to
allow the directory to be created by root and then filled by initdb.
It would also nice to have some reasonable default for PGLIB and
PGDATA like the previous version did. This applies also to the
postmaster executable.
gmake of the code without interruption.
There's also some tidy-up of the MAXPATHLEN stuff based on the assumption that
all supported platforms have MAXPATHLEN defined in <sys/param.h>.
(The only unknowns for the above are AIX and IRIX5.)
as ints and longs. Touches on quite a few function args as
well. Most other files look ok as far as Oids go...still checking
though...
Since Oids are type'd as unsigned ints, they should prolly be used
with the %ud format string in elog and sprintf messages. Not sure
what kind of strangeness that could produce.
Darren King
* Wrote max(date) and min(date) aggregates
* Wrote operator "-" for date; date - date yields number of days
difference
* Wrote operator+(date,int) and operator-(date,int); the int is the
number of days. Each operator returns a new date.
By: Tom Tromey <tromey@creche.cygnus.com>
In particular, no more compiled-in default for PGDATA or LIBDIR. Commands
that need them need either invocation options or environment variables.
PGPORT default is hardcoded as 5432, but overrideable with options or
environment variables.
Changes:
* Unique index capability works using the syntax 'create unique
index'.
* Duplicate OID's in the system tables are removed. I put
little scripts called 'duplicate_oids' and 'find_oid' in
include/catalog that help to find and remove duplicate OID's.
I also moved 'unused_oids' from backend/catalog to
include/catalog, since it has to be in the same directory
as the include files in order to work.
* The backend tries converting the name of a function or aggregate
to all lowercase if the original name given doesn't work (mostly
for compatibility with ODBC).
* You can 'SELECT NULL' to your heart's content.
* I put my _bt_updateitem fix in instead, which uses
_bt_insertonpg so that even if the new key is so big that
the page has to be split, everything still works.
* All literal references to system catalog OID's have been
replaced with references to define'd constants from the catalog
header files.
* I added a couple of node copy functions. I think this was a
preliminary attempt to get rules to work.
>From the create_aggregate man page...
"The arguments to state-transition-function-1 must be
(stype1,basetype), and its return value must be stype1."
create aggregate MIN (sfunc1 = int2smaller,
basetype = int2,
stype1 = int2);
will fail becase int2smaller and int2larger are in pg_proc
as returning an int4. Can't happen since both args have to
be int2.
From: Darren King <aixssd!ceodev!darrenk@abs.net>
I'm able to get through a 'make' of the backend with no errors except
the occasional 'might not be initialized error', which is nothing major,
just annoying.
Have a few patches from D'Arcy to incorporate, but am waiting until I can
get a clean compile first, which I'm hoping to have before bed, or sometime
tomorrow.
Note. all include files that have been hit so far have had extraneous
include files cleaned out and are reduced to...the lowest common
"include file", based on 'cc -Wall -I. test.c', where test.c is:
#include "postgres.h"
#include "<top of branches>" (ie. top of branches this time was utils/fcache2.h)
*should* be intelligent enough that:
#if defined(__FreeBSD__) works, where __FreeBSD__ is actually defined
by the compiler itself.
Makefile.global used to have -DPORTNAME_<port> -D<port> as part of the flags
for gcc while all occurances of PORTNAME_<port> slowly get removed from
the source tree...
cache. I found if I manually added a line to flush the whole relation
cache, the assert error disappeared. Looking through the code, I found
that the relation cache is flushed at the end of each query if the
reference count is zero for the relation. However, printf's showed that
the rd_relcnt(reference count) for the accessed query was not returning
to zero after each query.
It turns out the parser was doing a heap_ropen in parser/analyze.c to
get information about the table's columns, but was not doing a
heap_close.
This was causing the query after the ALTER TABLE ADD to see the old
table structure, and the executor's assert was reporting the problem.
NAMEDATALEN
OIDDATALEN
EUROPEAN_DATES
HBA
DEADLOCK_TIMEOUT
OPENLINK_PATCHES
NULL_PATCH
ARRAY_PATCH
Attempting to document and centralize as many of the "defines" as possible...
kinda useless to have defines if nobody knows they exist, eh?
|
|Here's a patch for Version 2 only. It just adds an Assert to catch some
|inconsistencies in the catalog classes.
|
|--
|Bryan Henderson Phone 408-227-6803
|San Jose, California
|