pgp_sym_encrypt's option is spelled "sess-key", not "enable-session-key".
Spotted by Jeff Janes.
In passing, improve a comment in pgp-pgsql.c to make it clearer that
the debugging options are intentionally undocumented.
This add a new pgp_armor_headers function to extract armor headers from an
ASCII-armored blob, and a new overloaded variant of the armor function, for
constructing an ASCII-armor with extra headers.
Marko Tiikkaja and me.
Instead of trying to accurately calculate the space needed, use a StringInfo
that's enlarged as needed. This is just moving things around currently - the
old code was not wrong - but this is in preparation for a patch that adds
support for extra armor headers, and would make the space calculation more
complicated.
Marko Tiikkaja
Some of the many error messages introduced in 458857cc missed 'FROM
unpackaged'. Also e016b724 and 45ffeb7e forgot to quote extension
version numbers.
Backpatch to 9.1, just like 458857cc which introduced the messages. Do
so because the error messages thrown when the wrong command is copy &
pasted aren't easy to understand.
ws2_32 is the new version of the library that should be used, as
it contains the require functionality from wsock32 as well as some
more (which is why some binaries were already using ws2_32).
Michael Paquier, reviewed by MauMau
Prominent binaries already had this metadata. A handful of minor
binaries, such as pg_regress.exe, still lack it; efforts to eliminate
such exceptions are welcome.
Michael Paquier, reviewed by MauMau.
Because of gcc -Wmissing-prototypes, all functions in dynamically
loadable modules must have a separate prototype declaration. This is
meant to detect global functions that are not declared in header files,
but in cases where the function is called via dfmgr, this is redundant.
Besides filling up space with boilerplate, this is a frequent source of
compiler warnings in extension modules.
We can fix that by creating the function prototype as part of the
PG_FUNCTION_INFO_V1 macro, which such modules have to use anyway. That
makes the code of modules cleaner, because there is one less place where
the entry points have to be listed, and creates an additional check that
functions have the right prototype.
Remove now redundant prototypes from contrib and other modules.
Specifically, on-stack memset() might be removed, so:
* Replace memset() with px_memset()
* Add px_memset to copy_crlf()
* Add px_memset to pgp-s2k.c
Patch by Marko Kreen
Report by PVS-Studio
Backpatch through 8.4.
This function provides a way of generating version 4 (pseudorandom) UUIDs
based on pgcrypto's PRNG. The main reason for doing this is that the
OSSP UUID library depended on by contrib/uuid-ossp is becoming more and
more of a porting headache, so we need an alternative for people who can't
install that. A nice side benefit though is that this implementation is
noticeably faster than uuid-ossp's uuid_generate_v4() function.
Oskari Saarenmaa, reviewed by Emre Hasegeli
pgp.h used to require including mbuf.h and px.h first. Include those in
pgp.h, so that it can be used without prerequisites. Remove mbuf.h
inclusions in .c files where mbuf.h features are not used
directly. (px.h was always used.)
Overly tight coding caused the password transformation loop to stop
examining input once it had processed a byte equal to 0x80. Thus, if the
given password string contained such a byte (which is possible though not
highly likely in UTF8, and perhaps also in other non-ASCII encodings), all
subsequent characters would not contribute to the hash, making the password
much weaker than it appears on the surface.
This would only affect cases where applications used DES crypt() to encode
passwords before storing them in the database. If a weak password has been
created in this fashion, the hash will stop matching after this update has
been applied, so it will be easy to tell if any passwords were unexpectedly
weak. Changing to a different password would be a good idea in such a case.
(Since DES has been considered inadequately secure for some time, changing
to a different encryption algorithm can also be recommended.)
This code, and the bug, are shared with at least PHP, FreeBSD, and OpenBSD.
Since the other projects have already published their fixes, there is no
point in trying to keep this commit private.
This bug has been assigned CVE-2012-2143, and credit for its discovery goes
to Rubin Xu and Joseph Bonneau.
Due to oversights, the encrypt_iv() and decrypt_iv() functions failed to
report certain types of invalid-input errors, and would instead return
random garbage values.
Marko Kreen, per report from Stefan Kaltenbrunner
We have seen one too many reports of people trying to use 9.1 extension
files in the old-fashioned way of sourcing them in psql. Not only does
that usually not work (due to failure to substitute for MODULE_PATHNAME
and/or @extschema@), but if it did work they'd get a collection of loose
objects not an extension. To prevent this, insert an \echo ... \quit
line that prints a suitable error message into each extension script file,
and teach commands/extension.c to ignore lines starting with \echo.
That should not only prevent any adverse consequences of loading a script
file the wrong way, but make it crystal clear to users that they need to
do it differently now.
Tom Lane, following an idea of Andrew Dunstan's. Back-patch into 9.1
... there is not going to be much value in this if we wait till 9.2.
This addresses only those cases that are easy to fix by adding or
moving a const qualifier or removing an unnecessary cast. There are
many more complicated cases remaining.
Add __attribute__ decorations for printf format checking to the places that
were missing them. Fix the resulting warnings. Add
-Wmissing-format-attribute to the standard set of warnings for GCC, so these
don't happen again.
The warning fixes here are relatively harmless. The one serious problem
discovered by this was already committed earlier in
cf15fb5cab.
A password containing a character with the high bit set was misprocessed
on machines where char is signed (which is most). This could cause the
preceding one to three characters to fail to affect the hashed result,
thus weakening the password. The result was also unportable, and failed
to match some other blowfish implementations such as OpenBSD's.
Since the fix changes the output for such passwords, upstream chose
to provide a compatibility hack: password salts beginning with $2x$
(instead of the usual $2a$ for blowfish) are intentionally processed
"wrong" to give the same hash as before. Stored password hashes can
thus be modified if necessary to still match, though it'd be better
to change any affected passwords.
In passing, sync a couple other upstream changes that marginally improve
performance and/or tighten error checking.
Back-patch to all supported branches. Since this issue is already
public, no reason not to commit the fix ASAP.
Added a new option --extra-install to pg_regress to arrange installing
the respective contrib directory into the temporary installation.
This is currently not yet supported for Windows MSVC builds.
Updated the .gitignore files for contrib modules to ignore the
leftovers of a temp-install check run.
Changed the exit status of "make check" in a pgxs build (which still
does nothing) to 0 from 1.
Added "make check" in contrib to top-level "make check-world".
This warning is new in gcc 4.6 and part of -Wall. This patch cleans
up most of the noise, but there are some still warnings that are
trickier to remove.
It was never terribly consistent to use OR REPLACE (because of the lack of
comparable functionality for data types, operators, etc), and
experimentation shows that it's now positively pernicious in the extension
world. We really want a failure to occur if there are any conflicts, else
it's unclear what the extension-ownership state of the conflicted object
ought to be. Most of the time, CREATE EXTENSION will fail anyway because
of conflicts on other object types, but an extension defining only
functions can succeed, with bad results.
This isn't fully tested as yet, in particular I'm not sure that the
"foo--unpackaged--1.0.sql" scripts are OK. But it's time to get some
buildfarm cycles on it.
sepgsql is not converted to an extension, mainly because it seems to
require a very nonstandard installation process.
Dimitri Fontaine and Tom Lane
in the formerly-always-blank columns just to left and right of the data.
Different marking is used for a line break caused by a newline in the data
than for a straight wraparound. A newline break is signaled by a "+" in the
right margin column in ASCII mode, or a carriage return arrow in UNICODE mode.
Wraparound is signaled by a dot in the right margin as well as the following
left margin in ASCII mode, or an ellipsis symbol in the same places in UNICODE
mode. "\pset linestyle old-ascii" is added to make the previous behavior
available if anyone really wants it.
In passing, this commit also cleans up a few regression test files that
had unintended spacing differences from the current actual output.
Roger Leigh, reviewed by Gabrielle Roth and other members of PDXPUG.
not global variables of anonymous enum types. This didn't actually hurt
much because most linkers will just merge the duplicated definitions ...
but some will complain. Per bug #4731 from Ceriel Jacobs.
Backpatch to 8.1 --- the declarations don't exist before that.
These changes assume that the varchar and xml data types are represented
the same as text. (I did not, however, accept the portions of the proposed
patch that wanted to assume bytea is the same as text --- tgl.)
Brendan Jurd
strings. This patch introduces four support functions cstring_to_text,
cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and
two macros CStringGetTextDatum and TextDatumGetCString. A number of
existing macros that provided variants on these themes were removed.
Most of the places that need to make such conversions now require just one
function or macro call, in place of the multiple notational layers that used
to be needed. There are no longer any direct calls of textout or textin,
and we got most of the places that were using handmade conversions via
memcpy (there may be a few still lurking, though).
This commit doesn't make any serious effort to eliminate transient memory
leaks caused by detoasting toasted text objects before they reach
text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few
places where it was easy, but much more could be done.
Brendan Jurd and Tom Lane
data structures and backend internal APIs. This solves problems we've seen
recently with inconsistent layout of pg_control between machines that have
32-bit time_t and those that have already migrated to 64-bit time_t. Also,
we can get out from under the problem that Windows' Unix-API emulation is not
consistent about the width of time_t.
There are a few remaining places where local time_t variables are used to hold
the current or recent result of time(NULL). I didn't bother changing these
since they do not affect any cross-module APIs and surely all platforms will
have 64-bit time_t before overflow becomes an actual risk. time_t should
be avoided for anything visible to extension modules, however.
about best practice for including the module creation scripts: to wit
that you should suppress NOTICE messages. This avoids creating
regression failures by adding or removing comment lines in the module
scripts.
remove transactions
use create or replace function
make formatting consistent
set search patch on first line
Add documentation on modifying *.sql to set the search patch, and
mention that major upgrades should still run the installation scripts.
Some of these issues were spotted by Tom today.
padded encryption scheme. Formerly it would try to access res[(unsigned) -1],
which resulted in core dumps on 64-bit machines, and was certainly trouble
waiting to happen on 32-bit machines (though in at least the known case
it was harmless because that byte would be overwritten after return).
Per report from Ken Colson; fix by Marko Kreen.
installations whose pg_config program does not appear first in the PATH.
Per gripe from Eddie Stanley and subsequent discussions with Fabien Coelho
and others.
right, there seems precious little reason to have a pile of hand-maintained
endianness definitions in src/include/port/*.h. Get rid of those, and make
the couple of places that used them depend on WORDS_BIGENDIAN instead.
Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with
VARSIZE and VARDATA, and as a consequence almost no code was using the
longer names. Rename the length fields of struct varlena and various
derived structures to catch anyplace that was accessing them directly;
and clean up various places so caught. In itself this patch doesn't
change any behavior at all, but it is necessary infrastructure if we hope
to play any games with the representation of varlena headers.
Greg Stark and Tom Lane
return true for exactly the characters treated as whitespace by their flex
scanners. Per report from Victor Snezhko and subsequent investigation.
Also fix a passel of unsafe usages of <ctype.h> functions, that is, ye olde
char-vs-unsigned-char issue. I won't miss <ctype.h> when we are finally
able to stop using it.
Few cleanups and couple of new things:
- add SHA2 algorithm to older OpenSSL
- add BIGNUM math to have public-key cryptography work on non-OpenSSL
build.
- gen_random_bytes() function
The status of SHA2 algoritms and public-key encryption can now be
changed to 'always available.'
That makes pgcrypto functionally complete and unless there will be new
editions of AES, SHA2 or OpenPGP standards, there is no major changes
planned.
versions of OpenSSL. If your OpenSSL does not contain SHA2, then there
should be no conflict. But ofcourse, if someone upgrades OpenSSL,
server starts crashing.
Backpatched to 8.1.X.
Marko Kreen
pgcrypto crypt()/md5 and hmac() leak memory when compiled against
OpenSSL as openssl.c digest ->reset will do two DigestInit calls
against a context. This happened to work with OpenSSL 0.9.6
but not with 0.9.7+.
Reason for the messy code was that I tried to avoid creating
wrapper structure to transport algorithm info and tried to use
OpenSSL context for it. The fix is to create wrapper structure.
It also uses newer digest API to avoid memory allocations
on reset with newer OpenSSLs.
Thanks to Daniel Blaisdell for reporting it.
comment line where output as too long, and update typedefs for /lib
directory. Also fix case where identifiers were used as variable names
in the backend, but as typedefs in ecpg (favor the backend for
indenting).
Backpatch to 8.1.X.
the pubkey functions a bit. The actual RSA-specific code
there is tiny, most of the patch consists of reorg of the
pubkey code, as lots of it was written as elgamal-only.
---------------------------------------------------------------------------
The SHLIB section was copy-pasted from somewhere and contains
several unnecessary libs. This cleans it up a bit.
-lcrypt
we don't use system crypt()
-lssl, -lssleay32
no SSL here
-lz in win32 section
already added on previous line
-ldes
The chance anybody has it is pretty low.
And the chance pgcrypto works with it is even lower.
Also trim the win32 section.
---------------------------------------------------------------------------
It is already disabled in Makefile, remove code too.
---------------------------------------------------------------------------
I was bit hasty making the random exponent 'k' a prime. Further researh
shows that Elgamal encryption has no specific needs in respect to k,
any random number is fine.
It is bit different for signing, there it needs to be 'relatively prime'
to p - 1, that means GCD(k, p-1) == 1, which is also a lot lighter than
full primality. As we don't do signing, this can be ignored.
This brings major speedup to Elgamal encryption.
---------------------------------------------------------------------------
o pgp_mpi_free: Accept NULLs
o pgp_mpi_cksum: result should be 16bit
o Remove function name from error messages - to be similar to other
SQL functions, and it does not match anyway the called function
o remove couple junk lines
---------------------------------------------------------------------------
o Support for RSA encryption
o Big reorg to better separate generic and algorithm-specific code.
o Regression tests for RSA.
---------------------------------------------------------------------------
o Tom stuck a CVS id into file. I doubt the usefulness of it,
but if it needs to be in the file then rather at the end.
Also tag it as comment for asciidoc.
o Mention bytea vs. text difference
o Couple clarifications
---------------------------------------------------------------------------
There is a choice whether to update it with pgp functions or
remove it. I decided to remove it, updating is pointless.
I've tried to keep the core of pgcrypto relatively independent
from main PostgreSQL, to make it easy to use externally if needed,
and that is good. Eg. that made development of PGP functions much
nicer.
But I have no plans to release it as generic library, so keeping such
doc
up-to-date is waste of time. If anyone is interested in using it in
other products, he can probably bother to read the source too.
Commented source is another thing - I'll try to make another pass
over code to see if there is anything non-obvious that would need
more comments.
---------------------------------------------------------------------------
Marko Kreen
OpenSSL 0.9.6x. The DES functions use the older 'des_'
API, but the newer 3DES functions use the 0.9.7x-only
'DES_' API.
I think I just used /usr/include/openssl/des.h for reference
when implementing them, and had upgraded OpenSSL in the
meantime.
Following patch converts DES also to newer API and provides
compatibility functions for OpenSSL < 0.9.7.
I chose this route because:
- openssl.c uses few DES functions.
- compatibility for old 'des_' API is going away at some point
of time from OpenSSL.
- as seen from macros, new API is saner
- Thus pgcrypto supports any OpenSSL version from 0.9.5 to 1.0
Tested with OpenSSL 0.9.6c and 0.9.7e.
Marko Kreen
of password-based encryption from RFC2440 (OpenPGP).
The goal of this code is to be more featureful encryption solution
than current encrypt(), which only functionality is running cipher
over data.
Compared to encrypt(), pgp_encrypt() does following:
* It uses the equvialent of random Inital Vector to get cipher
into random state before it processes user data
* Stores SHA-1 of the data into result so any modification
will be detected.
* Remembers if data was text or binary - thus it can decrypt
to/from text data. This was a major nuisance for encrypt().
* Stores info about used algorithms with result, so user needs
not remember them - more user friendly!
* Uses String2Key algorithms (similar to crypt()) with random salt
to generate full-length binary key to be used for encrypting.
* Uses standard format for data - you can feed it to GnuPG, if needed.
Optional features (off by default):
* Can use separate session key - user data will be encrypted
with totally random key, which will be encrypted with S2K
generated key and attached to result.
* Data compression with zlib.
* Can convert between CRLF<->LF line-endings - to get fully
RFC2440-compliant behaviour. This is off by default as
pgcrypto does not know the line-endings of user data.
Interface is simple:
pgp_encrypt(data text, key text) returns bytea
pgp_decrypt(data text, key text) returns text
pgp_encrypt_bytea(data bytea, key text) returns bytea
pgp_decrypt_bytea(data bytea, key text) returns bytea
To change parameters (cipher, compression, mdc):
pgp_encrypt(data text, key text, parms text) returns bytea
pgp_decrypt(data text, key text, parms text) returns text
pgp_encrypt_bytea(data bytea, key text, parms text) returns bytea
pgp_decrypt_bytea(data bytea, key text, parms text) returns bytea
Parameter names I lifted from gpg:
pgp_encrypt('message', 'key', 'compress-algo=1,cipher-algo=aes256')
For text data, pgp_encrypt simply encrypts the PostgreSQL internal data.
This maps to RFC2440 data type 't' - 'extenally specified encoding'.
But this may cause problems if data is dumped and reloaded into database
which as different internal encoding. My next goal is to implement data
type 'u' - which means data is in UTF-8 encoding by converting internal
encoding to UTF-8 and back. And there wont be any compatibility
problems with current code, I think its ok to submit this without UTF-8
encoding by converting internal encoding to UTF-8 and back. And there
wont be any compatibility problems with current code, I think its ok to
submit this without UTF-8 support.
Here is v4 of PGP encrypt. This depends on previously sent
Fortuna-patch, as it uses the px_add_entropy function.
- New function: pgp_key_id() for finding key id's.
- Add SHA1 of user data and key into RNG pools. We need to get
randomness from somewhere, and it is in user best interests
to contribute.
- Regenerate pgp-armor test for SQL_ASCII database.
- Cleanup the key handling so that the pubkey support is less
hackish.
Marko Kreen
- Move openssl random provider to openssl.c and builtin provider
to internal.c
- Make px_random_bytes use Fortuna, instead of giving error.
- Retarget random.c to aquiring system randomness, for initial seeding
of Fortuna. There is ATM 2 functions for Windows,
reader from /dev/urandom and the regular time()/getpid() silliness.
Marko Kreen
functions as STRICT, and all functions except gen_salt() as IMMUTABLE.
gen_salt() is VOLATILE.
Although the functions are now STRICT, I left their PG_ARGISNULL()
checks in place as a protective measure for users who install the
new code but use old (non-STRICT) catalog entries (e.g., restored
from a dump). Per recent discussion in pgsql-hackers.
Patch from Michael Fuhr.
to make. We ship the table file in the tarball and so this dependency
just opens file timestamp skew problems without doing anything useful.
(Not that it should hurt, either ... except for cross-compile builds.)
(currently in beta) when cryptolib = openssl. According to the
following checkin message from several years ago, OpenSSL application
developers should no longer rely on <openssl/evp.h> to include
everything they need:
http://cvs.openssl.org/chngview?cn=9888
This patch adds the necessary header files. It doesn't appear to
break anything when building against OpenSSL 0.9.7.
BTW, core appears to build and work fine with OpenSSL 0.9.8. I've
built 7.3 through HEAD against 0.9.8-beta6 without noticing any
problems.
Michael Fuhr
* test error handling
* add tests for des, 3des, cast5
* add some tests to blowfish, rijndael
* Makefile: ability to specify different tests for different crypto
libraries, so we can skip des, 3des and cast5 for builtin.
Marko Kreen
Reserve px_get_random_bytes() for strong randomness,
add new function px_get_pseudo_random_bytes() for
weak randomness and use it in gen_salt().
On openssl case, use RAND_pseudo_bytes() for
px_get_pseudo_random_bytes().
Final result is that is user has not configured random
souce but kept the 'silly' one, gen_salt() keeps
working, but pgp_encrypt() will throw error.
Marko Kreen
* openssl.c: Add 3des and AES support
* README.pgcrypto: list only supported ciphers for openssl
OpenSSL has pre-processor symbol OPENSSL_NO_AES, which
isn't that helpful for detecting if it _does_ exist.
Thus the hack with AES_ENCRYPT.
Marko Kreen
* Use error codes instead of -1
* px_strerror for new error codes
* calling convention change for px_gen_salt - return error code
* use px_strerror in pgcrypto.c
Marko Kreen
It was a bad style to begin with, and now several loops can be clearer.
* pgcrypto.c: Fix function comments
* crypt-gensalt.c, crypt-blowfish.c: stop messing with errno
* openssl.c: use px_free instead pfree
* px.h: make redefining px_alloc/px_realloc/px_free easier
Marko Kreen
libmcrypt seems to dead, maintainer address bounces,
and cast-128 fails on 2 of the 3 test vectors from RFC2144.
So I see no reason to keep around stuff I don't trust
anymore.
Support for several crypto libraries is probably only
confusing to users, although it was good for initial
developing - it helped to find hidden assumptions and
forced me to create regression tests for all functionality.
Marko Kreen
0.9.7x have EVP_DigestFinal function which which clears all of
EVP_MD_CTX. This makes pgcrypto crash in functions which
re-use one digest context several times: hmac() and crypt()
with md5 algorithm.
Following patch fixes it by carring the digest info around
EVP_DigestFinal and re-initializing cipher.
Marko Kreen.
>
> The patch adds missing the "libpgport.a" file to the installation under
> "install-all-headers". It is needed by some contribs. I install the
> library in "pkglibdir", but I was wondering whether it should be "libdir"?
> I was wondering also whether it would make sense to have a "libpgport.so"?
>
> It fixes various macros which are used by contrib makefiles, especially
> libpq_*dir and LDFLAGS when used under PGXS. It seems to me that they are
> needed to
>
> It adds the ability to test and use PGXS with contribs, with "make
> USE_PGXS=1". Without the macro, this is exactly as before, there should be
> no difference, esp. wrt the vpath feature that seemed broken by previous
> submission. So it should not harm anybody, and it is useful at least to me.
>
> It fixes some inconsistencies in various contrib makefiles
> (useless override, ":=" instead of "=").
Fabien COELHO
conversion of basic ASCII letters. Remove all uses of strcasecmp and
strncasecmp in favor of new functions pg_strcasecmp and pg_strncasecmp;
remove most but not all direct uses of toupper and tolower in favor of
pg_toupper and pg_tolower. These functions use the same notions of
case folding already developed for identifier case conversion. I left
the straight locale-based folding in place for situations where we are
just manipulating user data and not trying to match it to built-in
strings --- for example, the SQL upper() function is still locale
dependent. Perhaps this will prove not to be what's wanted, but at
the moment we can initdb and pass regression tests in Turkish locale.
only remnant of this failed experiment is that the server will take
SET AUTOCOMMIT TO ON. Still TODO: provide some client-side autocommit
logic in libpq.
is pgcrypto bug as it assumed too much about inner workings of OpenSSL.
Following patch stops pgcrypto using EVP* functions for ciphers and lets
it manage ciphers itself.
This patch supports Blowfish, DES and CAST5 algorithms.
Marko Kreen
Create objects in public schema.
Make spacing/capitalization consistent.
Remove transaction block use for object creation.
Remove unneeded function GRANTs.