Subject: [HACKERS] libpq/pqcomm stuff and Solaris byte order
I decided to go ahead with the required changes since no one else seems
to. I don't guarantee that it is perfect but with these changes the
package actually compiles. While I was at it I added to the Sparc
Solaris header to define the byte order. Note that NetBSD sets this
in the system headers so it wasn't required there.
In particular, someone may want to check whether I removed the correct
84 lines from backend/libpq/pqcomprim.c.
Subject: [HACKERS] auth.c for kerberos.
I made pgsql with eBones(international version of Kerberos4). The
following modification was needed. And I added read permition for
group to srvtab instead of running postmaster as root.
According to man page under FreeBSD for sys_errlist[], strerror() should be
used instead...not sure if this will break other systems, so only changing
two files for now, and we'll see what "errors" it turns up
of common routines in pqcomprim.c (pq communication primitives).
Not all adapted to it yet, but it's a start.
- Rewritten some of those routines, to write/read bigger chunks of
data, precomputing stuff in buffers instead of sending out byte
by byte.
- As a consequence, I need to know the endianness of the machine.
Currently I rely on getting it from machine/endian.h, but this
may not be available everywhere? (Who the hell thought it was
a good idea to pass integers to the backend the other way around
than the normal network byte order? *argl*)
- Libpq looks in the environment for magic variables, and upon
establishing a connection to the backend, sends it queries
of the form "SET var_name TO 'var_value'". This needs a change
in the backend parser (Mr. Parser, are you there? :)
- Currently it looks for two Env-Vars, namely PG_DATEFORMAT
and PG_FLOATFORMAT. What else makes sense? PG_TIMEFORMAT?
PG_TIMEZONE?
From: "Martin J. Laubach" <mjl@wwx.vip.at>
Subject: [HACKERS] Patches for 970316 compilation
I made a small pre-emptive change in the new datetime code to eliminate
calls to infnan(). Hopefully this will make Solaris (and probably other
non-GNUlib) systems happier. Didn't find fe-connect.h in the 970316
distribution, so made one up. Also, one of the test routines needs an
update for the geo-decls.h -> geo_decls.h name change.
Patches appear below...
Subject: [HACKERS] lock debug trace
This is an update to my previous patches for lock debugging, already applied
to the current sources. It adds some improvements in the output messages and
some more output in WaitOnLock(). I have used with success to trace a nasty
deadlock condition on pg_listener.
> Please apply them to the direcory "backend/optimizer/geqo".
> Two new files with different crossover techniques are included.
> Standard procedure is optimization by means of "geqo_erx.c"
> (Edge Recombination Crossover).
From: "Martin S. Utesch" <utesch@aut.tu-freiberg.de>
Subject: [HACKERS] password authentication
This patch adds support for plaintext password authentication. To use
it, you add a line like
host all 0.0.0.0 0.0.0.0 password pg_pwd.conf
to your pg_hba.conf, where 'pg_pwd.conf' is the name of a file containing
the usernames and password hashes in the format of the first two fields
of a Unix /etc/passwd file. (Of course, you can use a specific database
name or IP instead.)
Then, to connect with a password through libpq, you use the PQconnectdb()
function, specifying the "password=" tag in the connect string and also
adding the tag "authtype=password".
I also added a command-line switch '-u' to psql that tells it to prompt
for a username and password and use password authentication.
Subject: [HACKERS] linux/alpha patches
These patches lay the groundwork for a Linux/Alpha port. The port doesn't
actually work unless you tweak the linker to put all the pointers in the
first 32 bits of the address space, but it's at least a start. It
implements the test-and-set instruction in Alpha assembly, and also fixes
a lot of pointer-to-integer conversions, which is probably good anyway.
Subject: [HACKERS] linux/alpha patches
These patches lay the groundwork for a Linux/Alpha port. The port doesn't
actually work unless you tweak the linker to put all the pointers in the
first 32 bits of the address space, but it's at least a start. It
implements the test-and-set instruction in Alpha assembly, and also fixes
a lot of pointer-to-integer conversions, which is probably good anyway.
Subject: [HACKERS] equal column and table name patch
This fixes a bug where selects fail when there is a column with the same
name as the table it's a part of.
Subject: [HACKERS] better access control error messages
This patch replaces the 'no such class or insufficient privilege' with
distinct error messages that tell you whether the table really doesn't
exist or whether access was denied.
Subject: [HACKERS] backend Makefile patch
This patch cleans up backend/Makefile a little bit, and prevents it from
relinking the backend binary when no changes have been made.
Subject: [HACKERS] abort failed transaction patch
This patch allows you to end a transaction that has failed on an error
using the 'ABORT' statement without generating another error message.
(By default you get an error unless you use 'END' to terminate the
transaction, which has already been aborted anyway.)
The following patch to src/backend/libpq/pqpacket.c provides additional
checking for bad packet length data. It was tested with the Linux telnet
client, with netcat using the numbers.txt and by dumping random numbers
into the port.
Patch by: Alvaro Martinez Echevarria <alvaro@lander.es>
The following patches add to the backend a new debugging flag -K which prints
a debug trace of all locking operations on user relations (those with oid
greater than 20000). The code is compiled only if LOCK_MGR_DEBUG is defined,
so the patch should be harmless if not explicitly enabled.
I'm using the code to trace deadlock conditions caused by application queries
using the command "$POSTMASTER -D $PGDATA -o '-d 1 -K 1'.
The patches are for version 6.0 dated 970126.
Patches from: aoki@CS.Berkeley.EDU (Paul M. Aoki)
i gave jolly my btree bulkload code a long, long time ago but never
gave him a bunch of my bugfixes. here's a diff against the 6.0
baseline.
for some reason, this code has slowed down somewhat relative to the
insertion-build code on very small tables. don't know why -- it used
to be within about 10%. anyway, here are some (highly unscientific!)
timings on a dec 3000/300 for synthetic tables with 10k, 100k and
1000k tuples (basically, 1mb, 10mb and 100mb heaps). 'c' means
clustered (pre-sorted) inputs and 'u' means unclustered (randomly
ordered) inputs. the 10k table basically fits in the buffer pool, but
the 100k and 1000k tables don't. as you can see, insertion build is
fine if you've sorted your heaps on your index key or if your heap
fits in core, but is absolutely horrible on unordered data (yes,
that's 7.5 hours to index 100mb of data...) because of the zillions of
random i/os.
if it doesn't work for you for whatever reason, you can always turn it
back off by flipping the FastBuild flag in nbtree.c. i don't have
time to maintain it.
good luck!
baseline code:
time psql -c 'create index c10 on k10 using btree (c int4_ops)' bttest
real 8.6
time psql -c 'create index u10 on k10 using btree (b int4_ops)' bttest
real 9.1
time psql -c 'create index c100 on k100 using btree (c int4_ops)' bttest
real 59.2
time psql -c 'create index u100 on k100 using btree (b int4_ops)' bttest
real 652.4
time psql -c 'create index c1000 on k1000 using btree (c int4_ops)' bttest
real 636.1
time psql -c 'create index u1000 on k1000 using btree (b int4_ops)' bttest
real 26772.9
bulkloading code:
time psql -c 'create index c10 on k10 using btree (c int4_ops)' bttest
real 11.3
time psql -c 'create index u10 on k10 using btree (b int4_ops)' bttest
real 10.4
time psql -c 'create index c100 on k100 using btree (c int4_ops)' bttest
real 59.5
time psql -c 'create index u100 on k100 using btree (b int4_ops)' bttest
real 63.5
time psql -c 'create index c1000 on k1000 using btree (c int4_ops)' bttest
real 636.9
time psql -c 'create index u1000 on k1000 using btree (b int4_ops)' bttest
real 701.0
|Subject: [PATCH] adding SYS_TIME just for fun.
|
|Hi,
|
|Whilst I was playing round with the European dates patch I noticed the sysfunc()
|that allows you to do :-
|
|create table test ( da date);
|insert into test values (SYS_DATE);
|
|and have the current system date inserted.
|
|So I thought it would be nice to have the SYS_TIME facility too.
|
|I've cloned the function and changed a few things and there you have it,
|you can now do:
|
|create table test2 ( ti time);
|insert into test2 values (SYS_TIME);
The first patch changes the behavior of aclcheck for groups. Currently an user
can access a table only if he has the required permission for ALL the groups
defined for that table. With my patch he can access a table if he has the
permission for ONE of the groups, which seems to me a more useful thing.
If you think this should be the correct behavior of the acl group check feel
free to remove the #ifdef, if not please add a commented line to config.h.
2. IndexScanableOperand now uses match_indexkey_operand
instead of equal_indexkey_var (if we have some index on attribute X
then we shouldn't use it for 'where some_func(X) OP CONST').
/usr/include/limits.h (which quiets the costsize.c warnings)...under
FreeBSD, /usr/include/limits.h *includes* machine/limits.h, while under
Solaris, there is no such things as /usr/include/machine...
Problem with Solaris pointed out by Mark Wahl
1. New flag - BM_JUST_DIRTIED - added for BufferDesc;
2. All data "dirtiers" (WriteBuffer and WriteNoReleaseBuffer)
set this flag (and BM_DIRTY too);
3. All data "flushers" (FlushBuffer, BufferSync and BufferReplace)
turn this flag off just before calling smgr[blind]write/smgrflush
and check this flag after flushing buffer: if it turned ON then
BM_DIRTY will stay ON.
another one in Solaris' port-protos.h.
The following patch will bring inet_aton's prototype into scope for
Ultrix to silence a compilation warning.
If the intention is to have inet_aton's prototype in its own header
filer, the declaration in Solaris' port-protos.h should be removed.
If the declaration in port-protos.h is deemed to be the correct
place, a declaration should be added in Ultrix' port-protos.h
regards
Erik Bertelsen
included after storage/ipc.h like other similar cases that were changed
recently.
This one has popped up during the last few days.
My sources are sup'ed today, 13. jan 1996.
regards
Erik Bertelsen.
At least the first two should be fixed before the final release of 6.0.
1) There is a mismatch between the type declared in the catalog for
the input/output attributes of pg_type and the actual type of
values stored in the table. The type of typinput, typoutput,
typsend and typreceive are declared oid (26) while the values are
regproc (24). The error was there also in previous versions but
nobody noticed it until an Assert has been added in ExecEvalVar.
The effect is that it is now impossible to replace the typoutput
of existing data types with new procs.
2) The identd hba fails after the first time because the data read
from the identd socket is not zero-terminated and strlen reports
an incorrect length if the stack contains garbage, which usually
happens after the first connection has been made.
3) The new initdb wants to create itself the data directory. This
implies that the parent directory must be writable by postgres and
this may not always be desirable. A better solution would be to
allow the directory to be created by root and then filled by initdb.
It would also nice to have some reasonable default for PGLIB and
PGDATA like the previous version did. This applies also to the
postmaster executable.
gmake of the code without interruption.
There's also some tidy-up of the MAXPATHLEN stuff based on the assumption that
all supported platforms have MAXPATHLEN defined in <sys/param.h>.
(The only unknowns for the above are AIX and IRIX5.)
And now - JMP_BUF again. Is it enough, folks ?
Fixed again:
cc1: warnings being treated as errors
exc.c: In function 'ExcRaise':
exc.c:187: warning: passing arg 1 of 'Longjmp' from incompatible pointer type
gmake[3]: *** [exc.o] Error 1
%ud in a printf format strings instead of just %u.
There were three occurances of this in catalog_utils.c,
two in parser.c and one in rewriteSupport.c in the oid
patch that I submitted and was applied. They won't crash
anything, but the error messages will have a 'd' after the
Oid. Annoying, but none are db-threatening.
Sorry about that folks...I'll be more careful in the future...
Darren King
As an example I sent a bug-report on 26 Nov to tell that the fix included
below is necessary to compile pg95-current on Ultrix with Digital's
standard C compiler c89. In fact I think that this fix is needed
for any C compiler sticking very close the standard, see my discussion
in the original bug report.
Erik Bertelsen
cc1: warnings being treated as errors
datum.c: In function `DatumGetSize':
datum.c:57: warning: unsigned value >= 0 is always 1
gmake[3]: *** [datum.o] Error 1
There was:
if (byVal) {
if (len >= 0 && len <= sizeof(Datum)) {
but len has type Size (unsigned int) and so now there is:
if (byVal) {
if (len <= sizeof(Datum)) {
cc1: warnings being treated as errors
exc.c: In function 'ExcRaise':
exc.c:186: warning: passing arg 1 of 'Longjmp' from incompatible pointer type
gmake[3]: *** [exc.o] Error 1
Now we have:
#if defined (JMP_BUF)
longjmp(efp->context, 1);
#else
siglongjmp(efp->context, 1);
#endif
cc1: warnings being treated as errors
transsup.c: In function `TransBlockGetLastTransactionIdStatus':
transsup.c:122: warning: unsigned value >= 0 is always 1
gmake[3]: *** [transsup.o] Error 1
...
(old _bt_compare always returned >= 0 while comparing with P_HIKEY
on root page - it breaks root page when _bt_insertonpg tries insert
new minimal key into root page).
2. Fixed bug concerns "empty" pages: non-rightmost pages with only P_HIKEY
present on it. Such pages appear after vacuum.
as ints and longs. Touches on quite a few function args as
well. Most other files look ok as far as Oids go...still checking
though...
Since Oids are type'd as unsigned ints, they should prolly be used
with the %ud format string in elog and sprintf messages. Not sure
what kind of strangeness that could produce.
Darren King
When an acl item is added or updated the new entry is deleted if it has no
permissions and the acl array is shrinked. This is is done by decrementing
the number of items without updating the corresponding array size.
The array with the incorrect size is later read by pg_aclcheck and the entry
count is used to allocate a new array while the array size is used to copy
the old one. This causes a memory corruption and a backend crash.
This happens only to normal user as the administrator bypasses acl checks.
Massimo Dal Zotto
* Wrote max(date) and min(date) aggregates
* Wrote operator "-" for date; date - date yields number of days
difference
* Wrote operator+(date,int) and operator-(date,int); the int is the
number of days. Each operator returns a new date.
By: Tom Tromey <tromey@creche.cygnus.com>
In particular, no more compiled-in default for PGDATA or LIBDIR. Commands
that need them need either invocation options or environment variables.
PGPORT default is hardcoded as 5432, but overrideable with options or
environment variables.
Changes:
* Unique index capability works using the syntax 'create unique
index'.
* Duplicate OID's in the system tables are removed. I put
little scripts called 'duplicate_oids' and 'find_oid' in
include/catalog that help to find and remove duplicate OID's.
I also moved 'unused_oids' from backend/catalog to
include/catalog, since it has to be in the same directory
as the include files in order to work.
* The backend tries converting the name of a function or aggregate
to all lowercase if the original name given doesn't work (mostly
for compatibility with ODBC).
* You can 'SELECT NULL' to your heart's content.
* I put my _bt_updateitem fix in instead, which uses
_bt_insertonpg so that even if the new key is so big that
the page has to be split, everything still works.
* All literal references to system catalog OID's have been
replaced with references to define'd constants from the catalog
header files.
* I added a couple of node copy functions. I think this was a
preliminary attempt to get rules to work.
I found another bug in btree index. Looking at the code it seems that NULL
keys are never used to build or scan a btree index (see the explain commands
in the example). However this is not the case when a null key is retrieved
in an outer loop of a join select and used in an index scan of an inner loop.
This bug causes at least three kinds of problems:
1) the backend crashes when it tries to compare a text string with a null.
2) it is not possible to find tuples with null keys in a join.
3) null is considered equal to 0 when the datum is passed by value, see
the last query.
Submitted by: Massimo Dal Zotto <dz@cs.unitn.it>
It's bug in nodeAgg.c on lines 241, 242:
null_array = malloc(nagg);
for (i=0;i<nagg;i++)
null_array[i] = 'n';
oneTuple = heap_formtuple(tupType, tupValue, null_array);
- your query has not only aggregates but also 'group by-ed' fields and so
null_array should contain tupType->natts elements (tupType->natts > nagg in
your case).
Patch follows and it's very simple.
VAdim
My guess is that the thing had bugs, and the pfree was commented out.
The thing is probabally free'ed anyway at the end, so it was not a bad
thing.
If it does cause a bug, it will generate an error when hit, so I say
unless someone else knows, let's remove it and run the regression test.
-Bruce
2. Reap unused tuples too
3. Reap empty pages
4. Check if a page is initialized, initialize it if not
and reap it
5. Binary search in list of reapped pages/tids to check
is the heap' tid pointed by a index' tuple on this list
(it's mu-u-uch faster)
cache. I found if I manually added a line to flush the whole relation
cache, the assert error disappeared. Looking through the code, I found
that the relation cache is flushed at the end of each query if the
reference count is zero for the relation. However, printf's showed that
the rd_relcnt(reference count) for the accessed query was not returning
to zero after each query.
It turns out the parser was doing a heap_ropen in parser/analyze.c to
get information about the table's columns, but was not doing a
heap_close.
This was causing the query after the ALTER TABLE ADD to see the old
table structure, and the executor's assert was reporting the problem.
correct way to do this. Theoretically you could have a NULL
pointer that isn't represented internally as all 0 bits. This
guarantees that it convert correctly.
Submitted by: darcy@druid.com (D'Arcy J.M. Cain)