Release 9.0 Release date 2010-??-?? Overview ADD HERE The above items are explained in more detail in the sections below. Migration to Version 9.0 CURRENT AS OF 2010-03-15 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities: Server Settings Remove server variable add_missing_from, which was defaulted to off for many years (Tom) Remove server variable regex_flavor, which was defaulted to advanced (e.g. Perl-regex compatible) for many years (Tom) Queries When querying a parent table, do not do additional permission checks on child tables returned as part of the query (Peter) The SQL standard specifies this behavior. Have fractional seconds conversion truncate rather than round when using float-based dates/times (Tom) String Handling Fix SIMILAR TO to match the SQL standard-specified behavior by processing ? and {} the same way they are processed in regular expressions (Tom) Properly treat ^ and $ as literals in SIMILAR TO patterns, to match the SQL standard (Tom) Previously these were treated using regular expression syntax. This change breaks backward compatibility. This also affects substring()'s interpretation of regular expressions. Process parentheses as literals in SIMILAR TO expressions; also make character class handling more standards-compliant (Tom) This also affects substring()'s handling of regular expressions. Do not allow substring() to have a negative third length, per the SQL standard (Tom) Object Renaming Tighten enforcement of column renaming when a child table inherits the renamed column from an unrelated parent (KaiGai Kohei) No longer rename index names and index column names when table columns are renamed (Tom) Administrators still can rename such columns manually. Expression index names are also not renamed. WHY ARE EXPRESSION INDEXES SIGNIFICANT? PL/pgSQL Variables Have PL/pgSQL throw an error if a variable name conflicts with a column name used in a query (Tom) This behavior can be changed via the server variable plpgsql.variable_conflict, or by the per-function option #variable_conflict. The former behavior was to bind to variable names over column names. Remove PL/pgSQL's RENAME declaration option (Tom) Instead, use ALIAS, which can now alias any variable, not just dollar sign variables, e.g. $1. PL/pgSQL no longer allows unquoted variables names that match SQL reserved words (Tom) Variables can be double-quoted to avoid this restriction. Changes Server Continuous Archiving Allow continuous archive standby systems to accept read-only queries (Simon, Heikki) This feature is called Hot Standby. There are new postgresql.conf and recovery.conf settings to enable this feature, as well as extensive documentation. Allow continuous archive (WAL) files to be streamed to a standby system (Fujii Masao, Heikki) Previously WAL files could be sent to standby systems only as 16 megabytes files; this allows master changes to be sent to the slave with very little delay. There are new postgresql.conf and recovery.conf settings to enable this feature, as well as extensive documentation. Add pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), which can be used to monitor standby server WAL activity (Fujii Masao, Heikki) Performance Improve performance of finding inherited child tables (Tom) Allow per-tablespace sequential and random page cost variables (seq_page_cost/(random_page_cost)) via ALTER TABLESPACE ... SET/RESET (Robert Haas) Improve performance of TRUNCATE when used in the same transaction as table creation (Tom) Optimizer Allow IS NOT NULL restrictions to use indexes (Tom) This is particularly useful for finding MAX()/MIN() values in indexes that also contain NULLs. Remove unnecessary outer joins (Robert Haas) Outer joins where the inner side is unique and not referenced in the query are unnecessary and are therefore now removed. Improve optimizer equivalence detection of <> boolean tests (Tom) <link linkend="geqo">GEQO</link> Use the same random seed everytime GEQO plans a query (Andres Freund) While the Genetic Query Optimizer (GEQO) still selects random plans, it now selects the same random plans for identical queries. You can modify to randomize the starting value of the random plan generator. Improve GEQO plan selection (Tom). This avoids the rare error, "failed to make a valid plan". Optimizer Statistics Improve ANALYZE to support inheritance-tree statistics (Tom) This is particularly useful for partitioned tables. Improve autovacuum detection of when re-analyze is necessary (Tom) Improve optimizer statistics for greater/less-than comparisons (Tom) When looking up optimizer statistics for greater/less-than comparisons, if the comparison value is in the first or last histogram bucket, use an index to generate the actual statistics. This is particularly useful for columns that are always increasing, and hence often have inaccurate statistics. Allow setting of distinct statistics using ALTER TABLE (Robert Haas) This allows user-override of the number or percentage of distinct values for a column and optionally its child tables. This value is normally computed by ANALYZE. Authentication Add support for RADIUS (Remote Authentication Dial In User Service) authentication (Magnus) Allow LDAP (Lightweight Directory Access Protocol) authentication to operate in "search/bind" mode (Robert Fleming, Magnus) This allows the user to be looked up first, then the system uses the DN (Distinguished Name) returned for that user. Add samehost and samenet designations to pg_hba.conf (Stef Walter) These match the server's IP address and network address respectively. Monitoring Add the ability for clients to set an application name, which is displayed in pg_stat_activity (Dave Page) Add an SQL state option (%e) to log_line_prefix (Guillaume Smet) Write to the Windows event log in UTF16 encoding (Itagaki Takahiro) Statistics Counters Add pg_stat_reset_shared('bgwriter') to reset the cluster-wide shared statistics of the bgwriter (Greg Smith) Add pg_stat_reset_single_table_counters() and pg_stat_reset_single_function_counters() to allow the reseting of statistics counters for individual tables and indexes (Magnus) Server Settings Allow setting of configuration variables based on database/role combinations (Alvaro) Previously only per-database and per-role setting were possible, not combinations. All role and database settings are now stored in the new pg_db_role_setting system table. A new psql \drds command shows these settings. Backwards-compatible system views do not show this information. Add boolean variable bonjour, which controls whether a Bonjour-enabled binary advertises itself via Bonjour (Tom) The default is off, meaning it does not advertise. Log changed parameter values when postgresql.conf is reloaded (Peter) Queries Do SELECT FOR UPDATE/SHARE processing after applying LIMIT, so the number of rows returned is always predictable (Tom) Previously, concurrent transactions could potentially cause SELECT to return fewer rows than specified by LIMIT. FOR UPDATE can still affect ORDER BY ordering, but this can be corrected by using FOR UPDATE in a subquery. Allow mixing of traditional and SQL-standard LIMIT/OFFSET syntax in the same query (Tom) Increase the supported frame options in window functions (Hitoshi Harada) This allows frames (RANGE or ROWS) to start with CURRENT ROW, and to use the ROWS n PRECEDING/FOLLOWING clause. Have SELECT and CREATE TABLE AS return row counts to the client (Boszormenyi Zoltan) psql does not display these counts. Unicode Strings Add Unicode surrogate pair (dual 16-bit) support to U& strings and identifiers (Peter) Allow Unicode escapes in E'...' strings (Marko Kreen) Object Manipulation Speed up CREATE DATABASE by deferring flushes to disk (Andres Freund, Greg Stark) Allow comments only on columns of tables, views, and composite types, not other objects like indexes and TOAST tables (Tom) Allow the creation of enumerate types with no labels (Bruce) This is useful for supporting binary upgrades. Have columns defined with storage type MAIN remain on the main heap page unless it cannot fit (Kevin Grittner) Previously MAIN values were forced to TOAST tables until the row size was one-quarter of the page size. <command>ALTER</> Add ALTER DEFAULT PRIVILEGES command to control privileges of all newly-created objects (Petr Jelinek) It currently supports tables, views, sequences, and functions, and also has per-schema control. Implement IF EXISTS for DROP COLUMN and DROP CONSTRAINT (Andres Freund) Allow ALTER TABLE commands which rewrite tables to skip WAL logging (Itagaki Takahiro) Such operations either complete fully or are rolled back, so WAL archiving can be skipped, unless running in continuous archiving mode. <link linkend="SQL-CREATETABLE"><command>CREATE TABLE</></link> Add support for copying COMMENTS and STORAGE to the CREATE TABLE ... LIKE INCLUDING command (Itagaki Takahiro) Add support for copying all attributes to the CREATE TABLE LIKE INCLUDING command (Itagaki Takahiro) Add the SQL-standard CREATE TABLE ... OF type command (Peter) This allows the creation of a table to match an existing composite type. Additional constraints and defaults can be specified in the command. Constraints Add deferrable unique constraints (Dean Rasheed) This allows UPDATE tab SET col = col + 1 to work on columns that have a unique indexes or are marked as primary keys, but DEFERRABLE INITIALLY DEFERRED must be used to mark the constraint as deferred. Generalize uniqueness constraints by allowing arbitrary operator comparisons, not just equality (Jeff Davis) This is enabled with the CREATE TABLE CONSTRAINT ... EXCLUDE clause. While uniqueness checks could be specified using this syntax, the real value of this features is in using complex operators that do not have built-in constraints. Improve the constraint violation error message to report the values causing the failure (Itagaki Takahiro) For example, a uniqueness constraint violation now reports Key (x)=(2) already exists. Object Permissions Add the ability to make mass permission changes per schema using the new GRANT/REVOKE IN SCHEMA clause (Petr Jelinek) Add the ability to control large object permissions with GRANT/REVOKE(KaiGai Kohei) Utility Operations Have LISTEN/NOTIFY store events in a memory queue, rather than a system table (Joachim Wieland) This greatly improves performance for these operations. Allow NOTIFY to pass an optional string to listeners (Joachim Wieland) Allow CLUSTER on all system tables (Tom) Global system tables still cannot be clustered. <link linkend="SQL-COPY"><command>COPY</></link> Allow * as a parameter in FORCE QUOTE for COPY CSV (Itagaki Takahiro) This forces quotes for all CSV output columns. Add new COPY syntax that allows parameters to be specified in parentheses (Robert Haas, Emmanuel Cecchet) This allows greater flexibility for future COPY options. The old syntax is still supported. <link linkend="SQL-EXPLAIN"><command>EXPLAIN</></link> Add new EXPLAIN (BUFFERS) to report query buffer activity (Itagaki Takahiro) log_*_stats log output, e.g. , no longer shows this information. Add hash usage information to EXPLAIN output (Robert Haas) Allow EXPLAIN output in XML, JSON, and YAML formats (Robert Haas, Greg Sabino Mullane) Allow EXPLAIN options to be specified inside parentheses (Robert Haas) This allows for the expansion of EXPLAIN options. The old syntax is still supported. <link linkend="SQL-VACUUM"><command>VACUUM</></link> Change VACUUM FULL to rewrite the entire table and indexes, rather than moving around single rows to compact space (Itagaki Takahiro, Tom) The previous method was usually slower and caused index bloat. Add new VACUUM syntax that allows parameters to be specified in parentheses (Itagaki Takahiro) This allows greater flexibility for future VACUUM options. The old syntax is still supported. Indexes Allow an index to be auto-named by not supplying an index name to CREATE INDEX (Tom) Allow REINDEX on system indexes (Tom) WAS THIS POSSIBLE ON ANY SYSTEM TABLE BEFORE? NON-HARDWIRED ONES? Add point_ops opclass for GiST (Teodor) DETAILS? Use red-black trees for GIN index creation (Teodor) Red-black trees are self-balanced so allow for faster index creation. Data Types Allow hex values to be specified in bytea strings (Peter) The variable controls if hex (default) or octal escapes are used for bytea output. (SWITCH DEFAULT FOR BETA? PETER) Libpq's PQescapeByteaConn() now uses the hex format for PostgreSQL 9.0 servers. Allow to be increased to 3 (Tom) The previous maximum extra_float_digits was 2. <link linkend="textsearch">Full Text Search</link> Add prefix support for the full text search synonym dictionary (Teodor) Add full text search filtering dictionaries (Teodor) Filtering dictionaries allow tokens to be modified and passed to subsequent dictionaries. Allow underscores in full text email addresses (Teodor) Functions Allow case-insensitive regular expression matching with UTF-8 server encodings. Previously, only ASCII characters and single-byte encodings worked properly. Other multi-byte, non-UTF-8 encodings are still broken for case-insensitive regular expression matching. Allow function calls to supply parameter names and match them to named parameters in the function definition (Pavel Stehule) For example, if a function is defined to take parameters a and b, it can be called with func(7 AS a, 12 AS b) or func(12 AS b, 7 AS a). Add support for to_char() scientific notation output ('EEEE') (Pavel Stehule, Brendan Jurd) Have to_char() honor 'FM' (fill mode) in 'Y', 'YY', and 'YYY' specifications (Bruce) It was already honored by 'YYYY'. Correct calculations of "overlap" and "contains" operations over polygons (Teodor) Aggregates Allow aggregate functions to use ORDER BY (Andrew Gierth) For example, this is now supported, array_agg(a ORDER BY b). This is useful for aggregates where the order of values is significant. Add the string_agg() aggregate function which aggregates values into a single string (Pavel Stehule) An optional second argument allows specification of a delimiter. Aggregate functions that are called with DISTINCT are now passed NULL values if the aggregate transition function is not marked as STRICT (Andrew Gierth) For example, agg(DISTINCT x) might pass NULL x values to agg(). Bit Strings Add get_bit() and set_bit() functions for bit strings, mirroring those for bytea (Leonardo F) Implement OVERLAY() (replace) for bit strings and bytea (Leonardo F) Object Information Functions Add pg_table_size() and pg_indexes_size() to provide a more user-friendly interface to the pg_relation_size() function (Bernd Helmle) Add has_sequence_privilege() for sequence permission checking (Abhijit Menon-Sen) Have information schema properly display date type octet lengths (Peter) The reported length is now the maximum octet length; previously, a huge value was reported. Speed up information schema privilege views (Joachim Wieland) Function and Trigger Creation Implement anonymous functions using the DO statement (Petr Jelinek, Joshua Tolley, Hannu Valtonen) This allows execution of server-side code without the need to create a new function and execute it. Allow SQL-compliant per-column triggers (Itagaki Takahiro) Such triggers are fired only if the specified columns are affected by the query, e.g. in UPDATE's SET list. information_schema now also shows this information. Add WHERE clause to CREATE TRIGGER to allow control over whether a trigger is fired (Takahiro Itagaki) While a check can be performed inside the trigger, doing it in an external WHERE clause has performance benefits. Server-Side Languages Add the OR REPLACE clause to CREATE LANGUAGE (Tom) This is helpful to optionally install a language if it does not already exist, and is particularly helpful now that PL/pgSQL is installed by default. <link linkend="plpgsql">PL/PgSQL</link> Server-Side Language Install server-side language PL/pgSQL by default (Bruce) Allow PL/pgSQL to handle row types with dropped columns (Pavel Stehule) Allow IN parameters to be assigned values within PL/pgSQL functions (Steve Prentice) Improve error location reporting in PL/pgSQL (Tom) Have PL/pgSQL use the main lexer, rather than a custom version (Tom) <link linkend="plpgsql-cursors">PL/pgSQL Cursors</link> Add count and ALL options to MOVE FORWARD/BACKWARD in PL/pgSQL (Pavel Stehule) Allow PL/pgSQL's WHERE CURRENT OF to use a cursor variable (Tom) Add PL/pgSQL's OPEN cursor FOR EXECUTE to use parameters (Pavel Stehule, Itagaki Takahiro) This is accomplished with a new USING clause. <link linkend="plperl">PL/Perl</link> Server-Side Language Add new PL/Perl functions: quote_literal(), quote_nullable(), quote_ident(), encode_bytea(), decode_bytea(), looks_like_number(), encode_array_literal(), encode_array_constructor() (Tim Bunce) Add server variable to specify a PL/Perl Perl initialization function (Tim Bunce) and plperl.on_plperlu_init are also available for trusted/untrusted-specific initialization. Improve error context support in PL/Perl (Alexey Klyukin) Support END blocks in PL/Perl (Tim Bunce) END blocks do not currently allow database access. Allow use strict in PL/Perl (Tim Bunce) This can be enabled with the server variable . Allow require in PL/Perl (Tim Bunce) This basically tests to see if the module is loaded, and if not, generates an error. Allow use feature in PL/Perl if Perl version 5.10 or later is used (Tim Bunce) Verify that PL/Perl return values are valid in the server encoding (Andrew) Improve PL/Perl code structure (Tim Bunce) PL/Perl subroutines are now given names (Tim Bunce) This is for the use of profiling and code coverage tools. DIDN'T THEY HAVE NAMES BEFORE? PL/Perl no longer allows Safe version 2.20 because it is unsafe (Tim Bunce) <link linkend="plpython">PL/Python</link> Server-Side Language Add Unicode support in PL/Python (Peter) Strings are automatically converted from/to the server encoding as necessary. Improve bytea support in PL/Python (Caleb Welton) Bytea values passed into PL/Python now are represented as binary, rather than the Postgres bytea text format. Null bytes are now also output properly from PL/Python. Boolean and numeric value passing in PL/Python was also improved. Add array parameter/return support to PL/Python (Peter) Improve mapping of domains to Python base types in PL/Python (Peter) Add Python 3 support to PL/Python (Peter) The new server-side language is called plpython3u. Improve error location and exception reporting in PL/Python (Peter) Client Applications Add vacuumdb <link linkend="APP-PSQL"><application>psql</></link> Properly escape psql variables and identifiers (Pavel Stehule, Robert Haas) For example, :'var' will be escaped as a literal string, and :"var" will be escaped as an SQL identifier. Ignore leading UTF-8-encoded Unicode byte-order marker in psql (Itagaki Takahiro) This is enabled when the client encoding is UTF-8. Fix psql --file - to properly honor Prevent overwriting of psql's command-line history if two psql sessions are run simultaneously (Tom) Improve psql's tab completion support (Itagaki Takahiro) <application>psql</> Display Allow psql to use fancy Unicode line-drawing characters via \pset linestyle unicode (Roger Leigh) Improve display of wrapped columns in psql (Roger Leigh) The previous format is available by using \pset linestyle old-ascii. <application>psql</> <link linkend="APP-PSQL-meta-commands"><command>\d</></link> Commands Have \d show child tables that inherit from the specified parent (Damien Clochard) \d shows only the number of child tables, while \d+ shows the names of all child tables. Show definition of indexes in \d index_name (Khee Chin) The definition is useful for expression indexes. In psql, show the view definition only with \d+, not with \d (Peter) <link linkend="APP-PGDUMP"><application>pg_dump</></link> Have pg_dump/pg_restore Fix pg_dump to properly dump large objects if standard_conforming_strings is enabled (Tom) Large objects dumps now use hex format for output. (SWITCH DEFAULT FOR BETA? TOM) Allow pg_dump to dump comments attached to columns of composite types (Taro Minowa (Higepon)) Have pg_dump These were already present in custom output mode. <link linkend="app-pg-ctl"><application>pg_ctl</></link> Allow pg_ctl to be safely used to start the postmaster at boot-time (Tom) Previously the pg_ctl process could have been mistakenly identified as a running postmaster based on a stale postmaster lock file. Give pg_ctl the ability to initialize the database (like initdb) (Zdenek Kotala) <application>Development Tools</> <link linkend="libpq"><application>libpq</></link> Add new libpq functions PQconnectdbParams() and PQconnectStartParams() (Guillaume Lelarge) These functions are similar to PQconnectdb() and PQconnectStart() except they allow a null-terminated array of connection options, rather than requiring all options to be sent in a single string. Add libpq functions PQescapeLiteral() and PQescapeIdentifier() (Robert Haas) These functions return appropriately quoted and escaped literal strings and identifiers. The caller is not required to pre-allocate the string result, as is required by PQescapeStringConn(). Add checking for a per-user service file (.pg_service.conf), which is checked before the site-wide service file (Peter) The file .pg_service.conf is assumed to be in the user's home directory. Properly report an error if the specified libpq service cannot be found (Peter) Issue a warning if the .pgpass-retrieved password fails (Bruce) <link linkend="ecpg"><application>ecpg</></link> Add SQLDA (SQL Descriptor Area) support to ecpg (Boszormenyi Zoltan) Add the DESCRIBE [OUTPUT] statement to ecpg (Boszormenyi Zoltan) Add an ecpg function to return the current transaction status (Bernd Helmle) Add the string data type in ecpg Informix-compatibility mode (Zoltan Boszormenyi) Allow ecpg to use new and old variable names without restriction (Michael) Have ecpg return zero for non-SQL3 data types (Michael) Previously it returned the negative of the data type oid. <application>ecpg</> Cursors Add ecpg out-of-scope cursor support in native mode (Boszormenyi Zoltan) This allows DECLARE to use variables that are not in scope when OPEN is called. This facility already existed in ecpg Informix-compatibility mode. Allow dynamic ecpg cursor names (Boszormenyi Zoltan) Allow ecpg to use noise words FROM and IN in FETCH and MOVE (Zoltan Boszormenyi) Build Options Enable client thread safety by default (Bruce) Thread-safe builds can be disabled with configure Add a compile-time option to allow the Linux out-of-memory killer to kill backends (Alex Hunsaker, Tom) Now that /proc/self/oom_adj allows disabling of the Linux out-of-memory (OOM) killer for the postmaster and its children, the new compile-time option -DLINUX_OOM_ADJ=0 allows the killer to be enabled for postmaster children. pg_config shows if this flag was used during compilation. Use DocBook XSL stylesheets for man page building (Peter) Makefiles New Makefile targets world, install-world, and installcheck-world (Andrew) These are similar to the existing all, install, and installcheck targets, but they build HTML documentation, build and test /contrib, and test server-side languages and ecpg. Add data and documentation location control to PGXS Makefiles (Mark Cave-Ayland) Restructure the HTML documentation build Makefile rules (Peter) New Requirements Require Autoconf 2.63 for building from source (Peter) Require Flex 2.5.31 or later to build from source (Tom) Require Perl version 5.8 or greater to build the server from a CVS copy (John Naylor, Andrew) Windows Add support for compiling on 64-bit Windows and running in 64-bit mode (Tsutomu Yamada, Magnus) This allows for large shared memory sizes on Windows. Allow server builds using Visual Studio 2008 (Magnus) Allow multi-processor compilation using Microsoft Visual C (Magnus) Source Code Distribute documentation in a proper directory tree, rather than as tar archive files inside the main distribution tarball (Peter) For example, the HTML documentation is now in doc/src/sgml/html; the manual pages are packaged similarly. Enable the server lexer to be reentrant (Tom) This was needed for use of the lexer by PL/pgSQL. Improve speed of memory allocation (Tom, Greg Stark) Add system columns to better document the use of indexes for constraint enforcement (Tom) Allow multiple actions to be communicated using the same operating system signal (Fujii Masao) This allows improved backend communication as new features are added. Improve source code test coverage, including /contrib, PL/Python, and PL/Perl (Peter, Andrew) Remove the use of flat files for system table bootstrapping (Tom, Alvaro) This also improves performance when using millions of users and databases. Improve the ability to translate psql strings (Peter) Reduce the length of some file names so file paths are less than 100 characters (Tom) Some decompression programs have problems with long file names. Tighten input requirements for int2 vector input (Caleb Welton) Add a new ERRCODE_INVALID_PASSWORD SQLSTATE error code (Bruce) Properly remove the few remaining personal source code copyright entries (Bruce) The personal copyright notices were insignificant but the community occasionally had to answer questions about them. Feature Support Use a more modern API for Bonjour (Tom) Bonjour now requires OS X 10.3 or later. Add processor test-and-test lock support for the SuperH architecture (Nobuhiro Iwamatsu) Allow non-GCC compilers to use inline functions if they support them (Kurt Harriman) Remove support for platforms that don't have a working 64-bit integer data types (Tom) It is believed all supported platforms have working 64-bit integer data types. Server Programming Allow use of C++ functions in backend code (Kurt Harriman, Peter) This removes keyword conflicts that previously made C++ usage difficult in backend code. extern "C" { } might still be necessary. Add AggCheckCallContext() for use in detecting if a C function is being called as an aggregate (Hitoshi Harada) Require fastgetattr() and heap_getattr() backend macros to use a non-NULL fourth argument (Robert Haas) KEEP? Server Hooks Add parser hooks to access column and parameter references in queries (Tom) Add a hook so loadable modules can control utility commands (Itagaki Takahiro) Allow the calling of parser hooks from SPI and cached plans (Tom) Binary Upgrade Support Add support for preservation of all relfilenodes, for use during binary upgrades (Bruce) Add support for binary upgrades to preserve pg_type and pg_enum oids (Bruce) This is needed to allow binary upgrades of user-defined composite types, arrays, and enums (enumerated types). Move tablespace data directories into their own PostgreSQL version-specific subdirectory (Bruce) This simplifies binary upgrades. Contrib Add multi-threaded option ( This allows multiple CPUs to be used for pgbench tests. Add \shell and \setshell meta commands to /contrib/pgbench (Michael Paquier) New features for /contrib/dict_xsyn (Sergey Karpov) The new options are matchorig, matchsynonyms, and keepsynonyms. Add full text dictionary /contrib/unaccent (Teodor) This filter dictionary removes accents from tokens. Add dblink_get_notify() to /contrib/dblink (Marcus Kempe) This allows async notifications in dblink. Greatly increase /contrib/hstore's length limit and add btree and hash abilities so GROUP BY and DISTINCT operations are possible (Andrew Gierth) New functions and operators were also added. Add /contrib/passwordcheck which can check the strength of assigned passwords (Laurenz Albe) The source code of this module should be modified to implement site-specific password policies. Add query text to /contrib/auto_explain output (Andrew) Add buffer access counters to /contrib/pg_stat_statements (Itagaki Takahiro) Update /contrib/start-scripts/linux to use /proc/self/oom_adj to disable the Linux out-of-memory (OOM) killer (Alex Hunsaker, Tom) Release 9.0alpha4 Overview PostgreSQL alpha releases are snapshots of development code. They are intended to preview and test upcoming features and to provide the possibility for early feedback. They should not be used in production installations or active development projects. While the PostgreSQL code is continuously subject to a number of automated and manual tests, alpha releases might have serious bugs. Also features may be changed incompatibly or removed at any time during the development cycle. The development cycle of a PostgreSQL major release alternates between periods of development and periods of integration work, called commit fests, normally one month each. Alpha releases are planned to be produced at the end of every commit fest, thus every two months. Since the first commit fest starts within a month from the beginning of development altogether, early alpha releases are not indicative of the likely feature set of the final release. The release notes below highlight user visible changes and new features. There are normally numerous bug fixes and performance improvements in every new snapshot of PostgreSQL, and it would be too bulky to attempt to list them all. Note that many bug fixes are also backported to stable releases of PostgreSQL, and you should be using those if you are looking for bug-fix-only upgrades for your current installations. The release notes are cumulative over all alpha releases. Items that are new in the latest alpha release are emphasized. Migration To upgrade from any release to an alpha release or from an alpha release to any other release will most likely require a dump/restore upgrade procedure. It may happen that this is not necessary in particular cases, but that is not verified beforehand. (The server will warn you in any case when a dump/restore is necessary if you attempt to use it with an old data directory.) Note, however, that the dump/restore upgrade procedure is expected to work for alpha releases, and problems in this area should be reported. Testing The primary reason we release alphas is to get users to test new features as early as possible. If you are interested in helping with organized testing, please see the testing information page. Changes SQL Features Disallow comments on columns of relation types other than tables, views, and composite types, that is, sequences and TOAST tables. Adjust naming of indexes and their columns. Index expression columns are now named after the FigureColname result for their expressions, rather than always being "pg_expression_N". Digits are appended to this name if needed to make the column name unique within the index. Default names for indexes and associated constraints now include the column names of all their columns, not only the first one as in previous practice. Allow the index name to be omitted in CREATE INDEX, causing the system to choose an index name the same as it would do for an unnamed index constraint. A necessary side-effect is to promote the word CONCURRENTLY to a more reserved status; it can't be a table/column/index name anymore unless quoted. Improve the handling of SET CONSTRAINTS commands by having them search pg_constraint before searching pg_trigger. This allows saner handling of corner cases. Typed table feature; this adds the CREATE TABLE name OF type command. Tighten integrity checks on ALTER TABLE ... RENAME COLUMN in inheritance hierarchies. Replace the pg_listener-based LISTEN/NOTIFY mechanism with an in-memory queue. In addition, add support for a "payload" string to be passed along with each notify event. This implementation should be significantly more efficient than the old one, and is also more compatible with Hot Standby usage. There is not yet any facility for HS slaves to receive notifications generated on the master, although such a thing is possible in future. Have SELECT and CREATE TABLE AS queries return a row count. While this is invisible in psql, other interfaces, like libpq, make this value visible. Extend the set of frame options supported for window functions. This allows the frame to start from CURRENT ROW (in either RANGE or ROWS mode), and it also adds support for ROWS n PRECEDING and ROWS n FOLLOWING start and end points. Add DO statement to support execution of procedural language code without having to create a function for it. Support use of function argument names to identify which actual arguments match which function parameters. The syntax uses AS, for example funcname(value AS arg1, anothervalue AS arg2). Remove previously deprecated configuration parameter add_missing_from. The system now always behaves as though add_missing_from were off. Performance and behavioral improvements in UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries with joins. Fix various corner cases that could have resulted in duplicated output rows. Set-returning functions are now prohibited in the target list of SELECT FOR UPDATE/SHARE. FOR UPDATE does not propagate into a WITH query anymore. Fix SELECT FOR UPDATE/SHARE in conjunction with LIMIT. Previously, it could return fewer rows than the limit specified. Add a WHEN clause to CREATE TRIGGER, allowing a boolean expression to be checked to determine whether the trigger should be fired. For BEFORE triggers this is mostly a matter of spec compliance; but for AFTER triggers it can provide a noticeable performance improvement, since queuing of a deferred trigger event and re-fetching of the row(s) at end of statement can be short-circuited if the trigger does not need to be fired. Add exclusion constraints, which generalize the concept of uniqueness to support any indexable commutative operator, not just equality. Two rows violate the exclusion constraint if "row1.col OP row2.col" is TRUE for each of the columns in the constraint. Support ORDER BY within aggregate function calls, at long last providing a non-kluge method for controlling the order in which values are fed to an aggregate function. At the same time eliminate the old implementation restriction that DISTINCT was only supported for single-argument aggregates. Behavioral change: formerly, agg(DISTINCT x) dropped null values of x unconditionally. Now, it does so only if the aggregate's transition function is strict; otherwise nulls are treated as DISTINCT normally would, i.e., you get one copy. Add SQL-compliant triggers on columns, which fire only if certain columns are named in the UPDATE's SET list. Add CREATE LIKE INCLUDING COMMENTS and STORAGE, and INCLUDING ALL shortcut to allow users to make an exact copy of a table including all options and features. Define a new, more extensible syntax for COPY options in order to support additional COPY options in the future. Modify the definition of window-function PARTITION BY and ORDER BY clauses so that their elements are always taken as simple expressions over the query's input columns. This fixes a bug. Fix bug with nested WITH RECURSIVE statements. Add surrogate pair support for U& string and identifier syntax. Add Unicode escapes in E'...' strings. DROP COLUMN and DROP CONSTRAINT now support an IF EXISTS clause so that users can avoid fatal errors when running repeatable scripts. UNIQUE constraints can now be DEFERRABLE. This is primarily useful for incremental updates of numerical keys, e.g. "ID = ID + 1" Allow parentheses around the query expression that follows a WITH clause. INFORMATION_SCHEMA, a catalog of standard views of database objects, has been updated to the SQL:2008 standard. Change character_octet_length to more sensible values in INFORMATION_SCHEMA. Allow * as parameter for FORCE QUOTE for COPY CSV, so that all columns will be quoted. Performance Support "x IS NOT NULL" clauses as index-scan conditions. When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Generic implementation of red-black binary tree. For now, only GIN uses it during index creation. Using self-balanced tree greatly speeds up index creation in corner cases with preordered data. Allow rewriting forms of ALTER TABLE to skip WAL logging. Speed up INFORMATION_SCHEMA's privilege views. Make TRUNCATE do truncate-in-place when processing a relation that was created or previously truncated in the current (sub)transaction. Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. This should speed up many ORM-generated and reporting tool queries. Remove the use of the flat files pg_auth and pg_database in order to improve performance. (Warning: pgbouncer and possibly other tools currently suggest referring to the pg_auth file for its user database. Such schemes will no longer work.) EXPLAIN allows output of plans in XML, JSON, or YAML format for automated processing of explain plans by analysis or visualization tools. EXPLAIN now supports the use of generic options in EXPLAIN ( option value, ... ) format, which permits the creation of additional EXPLAIN options. ALTER TABLE ... ALTER COLUMN ... SET STATISTICS DISTINCT allows users to manually tweak the number of distinct values estimated for a column, to fix cases where ANALYZE estimates are incorrect. Make GEQO's planning deterministic by having it start from a predictable random number seed each time. Rewrite GEQO's gimme_tree function so that it always finds a legal join sequence. Previously, it could have failed to produce a plan in some cases. Tweak TOAST code so that columns marked with MAIN storage strategy are not forced out-of-line unless that is necessary to make the row fit on a page. Previously, they were forced out-of-line if needed to get the row down to the default target size (1/4th page). Simplify the forms foo <> true and foo <> false to foo = false and foo = true during query optimization. Avoid per-send() system calls to manage SIGPIPE in libpq, if the platform provides either sockopt(SO_NOSIGPIPE) or the MSG_NOSIGNAL flag to send(). Administration and Monitoring Add the ability to store inheritance-tree statistics in pg_statistic, and teach ANALYZE to compute such stats for tables that have subclasses. Revise pgstat's tracking of tuple changes to improve the reliability of decisions about when to auto-analyze. The previous code depended on n_live_tuples + n_dead_tuples - last_anl_tuples, where all three of these numbers could be bad estimates from ANALYZE itself. Even worse, in the presence of a steady flow of HOT updates and matching HOT-tuple reclamations, auto-analyze might never trigger at all, even if all three numbers are exactly right, because n_dead_tuples could hold steady. Support ALTER TABLESPACE name SET/RESET ( tablespace_options ), currently supporting seq_page_cost and random_page_cost as parameters. New faster implementation of VACUUM FULL. The old idiom of using CLUSTER instead of VACUUM FULL for speed is now obsolete. Add buffer access counters to pg_stat_statements. Add some simple support and documentation for using process-specific oom_adj settings to prevent the postmaster from being OOM-killed on Linux systems. Place tablespace directories in their own subdirectory so pg_migrator can upgrade clusters without renaming the tablespace directories. Introduce Streaming Replication. Add pg_table_size() and pg_indexes_size() to provide more user-friendly wrappers around the pg_relation_size() function. Add pg_stat_reset_shared('bgwriter') to reset the cluster-wide shared statistics of the bgwriter. Replace ALTER TABLE ... SET STATISTICS DISTINCT with a more general mechanism. Attributes can now have options, just as relations and tablespaces do, and the reloptions code is used to parse, validate, and store them. Fix longstanding gripe that we check for 0000000001.history at start of archive recovery, even when we know it is never present. Make standby server continuously retry restoring the next WAL segment with restore_command, if the connection to the primary server is lost. This ensures that the standby can recover automatically, if the connection is lost for a long time and standby falls behind so much that the required WAL segments have been archived and deleted in the master. This also makes standby_mode useful without streaming replication; the server will keep retrying restore_command every few seconds until the trigger file is found. That's the same basic functionality pg_standby offers, but without the bells and whistles. Add functions to reset the statistics counter for a single table/index or a single function. Augment EXPLAIN output with more details on Hash nodes. Reindexing shared system catalogs is now fully transactional and crash-safe. Allow read-only connections during recovery, also known as Hot Standby. This provides a built-in master-slave replication solution. Provide a parenthesized-options syntax for VACUUM, analogous to that recently adopted for EXPLAIN. This will allow additional options to be implemented in future without having to make them fully-reserved keywords. The old syntax remains available for existing options, however. Add support for an application_name parameter, which is displayed in pg_stat_activity and recorded in log entries. Fix longstanding problems in VACUUM caused by untimely interruptions. Add YAML to list of EXPLAIN formats. Add an EXPLAIN (BUFFERS) option to show buffer-usage statistics. Add a hook to let loadable modules get control at ProcessUtility execution, and use it to extend contrib/pg_stat_statements to track utility commands. Add a Boolean server configuration parameter "bonjour" to control whether a Bonjour-enabled build actually attempts to advertise itself via Bonjour. When reloading postgresql.conf, log what parameters actually changed. Make it possibly to specify server configuration parameters per user-database combination. Add a \drds command to psql to display the settings. Allow the collection of statistics on sequences. Increase the maximum value of extra_float_digits to 3, and have pg_dump use that value when the backend is new enough to allow it, because it is possible to need 3 extra digits for float4 values (but not for float8 values). Add the ability to include the SQLSTATE error code of any error messages in the PostgreSQL activity log with the new log_line_prefix placeholder %e. Show the exact value being complained of in unique-constraint-violation error messages, including unique-index build failures. Security Add support for RADIUS authentication. When querying a table with child tables, do not check permissions on the child tables, only on the parent table. The old behavior was found to be useless and confusing in virtually all cases, and also contrary to the SQL standard. Add a hook to CREATE/ALTER ROLE to allow an external module to check the strength of database passwords, and create a sample implementation of such a hook as a new contrib module "passwordcheck". Add large object access control. Allow LDAP authentication to operate in search+bind mode, meaning it does a search for the user in the directory first, and then binds with the DN found for this user. This allows for LDAP logins in scenarios where the DN of the user cannot be determined simply by prefix and suffix, such as the case where different users are located in different containers. The old way of authentication can be significantly faster, so it's kept as an option. Add ALTER DEFAULT PRIVILEGES command, which allows users to adjust the privileges that will be applied to subsequently-created objects. Add GRANT/REVOKE ON ALL TABLES/SEQUENCES/FUNCTIONS IN SCHEMA. This makes it easier to manage permissions on database objects. Support "samehost" and "samenet" specifications in pg_hba.conf. This allows users with dynamic server addresses to run PostgreSQL without frequently modifying pg_hba.conf. New has_sequence_privilege() functions allow you to check sequence privileges for a given ROLE. Built-In Functions Fix 3-parameter form of bit substring() to throw error for negative length, as required by SQL standard. Add get_bit/set_bit functions for bit strings, paralleling those for bytea, and implement OVERLAY() for bit strings and bytea. Add string_agg aggregate functions. The one-argument version concatenates the input values into a string. The two-argument version also does the same thing, but inserts delimiters between elements. Honor to_char() "FM" specification in YYY, YY, and Y; it was already honored by YYYY. Remove configuration parameter regex_flavor. It is now always "advanced". Teach the regular expression functions to do case-insensitive matching and locale-dependent character classification properly when the database encoding is UTF8. This previously only worked correctly for single-byte encodings. It is still broken for other multibyte encodings. Support POSIX-compatible interpretation of ? as well as {m,n} and related constructs in SIMILAR TO, per SQL:2008. The to_char() formatting functions now supports EEEE (scientific notation). Use floor() not rint() when reducing precision of fractional seconds in timestamp_trunc, timestamptz_trunc, and interval_trunc() for the float-datetime case. This improves accuracy of time calculations. Fix ancient bug in handling of to_char() modifier 'TH', when used with HH. Data Types Allow enums to be created with zero labels, for use during binary upgrade. When doing "ARRAY[...]::domain", where domain is a domain over an array type, we need to check domain constraints. Regression introduced in 8.4. Fix integer-to-bit-string conversions to handle the first fractional byte correctly when the output bit width is wider than the given integer by something other than a multiple of 8 bits. Fix encoding handling in binary input function of xml type. Tighten binary receive functions so that they reject values that the text input functions don't accept either. New hex-string input and output format options for type bytea. Hex output format is enabled by default, which is an incompatible change. See the new bytea_output parameter if you need to restore compatibility. Adds prefix support for text search synonym dictionary, allowing creation of synonyms on partial matches. Server Tools initdb now selects the encoding SQL_ASCII when the locale specifies US-ASCII. This case previously failed. Add init[db] option to pg_ctl. It is now reasonably safe to use pg_ctl to start the postmaster from a boot-time script. pg_dump/pg_restore --clean now drops large objects. Modify parallel pg_restore ordering logic to avoid a potential O(N^2) slowdown for some complex databases. libpq Add user-specific .pg_service.conf file. Add new escaping functions PQescapeLiteral and PQescapeIdentifier. Introduce two new libpq connection functions, PQconnectdbParams and PQconnectStartParams, which are analogous to PQconnectdb and PQconnectStart respectively, but accept two NULL-terminated arrays, keywords and values, rather than conninfo strings. This avoids the need to build the conninfo string in cases where it might be inconvenient to do so. Throw error when a specified connection service name is not found in pg_service.conf, instead of ignoring it. Make libpq reject non-numeric and out-of-range port numbers with a suitable error message. psql Allow psql variables to be interpolated with literal or identifier escaping. In psql, show view definition only with \d+, not with \d. In psql, ignore UTF-8-encoded Unicode byte-order mark at the beginning of a file if the client encoding is UTF-8. Improve psql's tabular display of wrapped-around data by inserting markers in the formerly-always-blank columns just to left and right of the data. "pset linestyle old-ascii" is added to make the previous behavior available if anyone really wants it. Simplify psql's new linestyle behavior to default to linestyle=ascii all the time, rather than hoping we can tell whether the terminal supports UTF-8 characters. Implement significantly saner behavior when two or more psql sessions overlap in their use of the history file. Add "pset linestyle ascii/unicode" option to psql, allowing our traditional ASCII-art style of table output to be upgraded to use Unicode box drawing characters if desired. Have \d show child tables that inherit from the specified parent psql now shows the index methods in \di Procedural Languages PL/pgSQL Add "USING expressions" option to PL/pgSQL's OPEN cursor FOR EXECUTE. PL/pgSQL is installed by default. Remove PL/pgSQL's RENAME declaration, which has bizarre and mostly nonfunctional behavior, and is so little used that no one has been interested in fixing it. To ensure that possible uses are covered, remove the ALIAS declaration's arbitrary restriction that only $n identifiers can be aliased. The PL/pgSQL parser and scanner were reworked to behave much more sanely: A variable reference will only be recognized where it can validly go, ie, a place where a column value or parameter would be legal, instead of the former behavior that would replace any textual match including table names and column aliases (leading to syntax errors later on). When a name could refer either to a PL/pgSQL variable or a table column, PL/pgSQL formerly always assumed the variable was meant, sometimes resulting in surprising behavior. Now, PL/pgSQL can assume the variable is meant, or assume the table column is meant, or throw an error in ambiguous cases. For safety the default is to throw error. To configure this see the PL/pgSQL documentation. Error reporting is much nicer: it no longer shows edited versions of statements that look significantly different from what you wrote. Note that this change affects the set of keywords that are reserved in PL/pgSQL (i.e., cannot be the name of a PL/pgSQL variable). Now, all keywords shown as reserved in Appendix C are reserved for PL/pgSQL purposes as well. However, many PL/pgSQL-only keywords that were formerly treated as reserved no longer are. As in regular SQL, you can double-quote a variable's name if you want to use a name that conflicts with a reserved keyword. PL/pgSQL IN parameters now accept value assignments. Allow cursor commands MOVE FORWARD n, MOVE BACKWARD n, MOVE FORWARD ALL, MOVE BACKWARD ALL in PL/pgSQL. PL/pgSQL functions can now better cope with row types containing dropped columns. PL/Perl Add utility functions to PL/Perl: quote_literal, quote_nullable, quote_ident, encode_bytea, decode_bytea, looks_like_number, encode_array_literal, encode_array_constructor. Add plperl.on_plperl_init, plperl.on_plperlu_init, and plperl.on_init settings for startup code. Allow (ineffective) use of "require". If the required module is not already loaded then it dies. So "use strict;" now works in PL/Perl. Pre-load the feature module if Perl >= 5.10. So "use feature :5.10;" now works in PL/Perl. Stored procedure subs are now given names. The names are not visible in ordinary use, but they make tools like Devel::NYTProf and Devel::Cover much more useful. Handle END blocks in PL/Perl. Clean up package namespace use and use of Safe. Change log level of Perl warnings from NOTICE to WARNING. Add support for anonymous code blocks (DO blocks) to PL/Perl. Improve error context reporting in PL/Perl, for easier debugging. Convert a Perl array to a PostgreSQL array when returned by set-returning functions as well as non-SRFs. PL/Python Improved printing of Python exceptions in PL/Python. Add PL/Python DO handler. Support arrays as parameters and return values of PL/Python functions. Add Python 3 support to PL/Python. It behaves more or less unchanged compared to Python 2, but the new language variant is called plpython3u. Fix/improve bytea and boolean support in PL/Python. Data type conversion into and out of PL/Python previously went through an intermediate string representation, which caused various discrepancies especially with bytea and boolean data. This is now fixed by converting the values directly. PL/Python now accepts Unicode objects where it previously only accepted string objects (for example, as return value). Unicode objects are converted to the PostgreSQL server encoding as necessary. Improve error context reporting in PL/Python, for easier debugging. Greatly expand the regression testing for PL/Python. Additional Supplied Modules Add query text to auto_explain output. Add \shell and \setshell meta commands to pgbench. \shell runs an external shell command. \setshell does the same and assigns the result to a variable. Multiple improvements in contrib/hstore, including raising limits on keys and values, conversions to and from records and arrays, and support for GROUP BY and DISTINCT. pgbench is now multi-threaded, allowing it to use multiple CPU's for its client connections, and to do more realistic workload testing. Added the ability to retrieve asynchronous notifications using dblink, via the addition of the function dblink_get_notify(). Added matchorig, matchsynonyms, and keepsynonyms options to contrib/dict_xsyn. Added contrib/unaccent. ECPG Add sqlda support in both native and compatibility mode. Fix SQL3 type return value: For non-SQL3 types ECPG used to return -OID. This will break if there are enough OIDs to fill the namespace. Therefore we play it safe and return 0 if there is no OID->SQL3 type mapping available. Make char the default sqlda type. Add DESCRIBE [OUTPUT] statement. Add out-of-scope cursor support to native mode. Refactor ECPG grammar so that it uses the core grammar's unreserved_keyword list, minus a few specific words that have to be treated specially. This de-reserves a number of words in ECPG. Added dynamic cursor names to ECPG. Add ECPG function that returns the current transaction status. Make ECPG more robust against applications freeing strings. ECPG now includes a STRING datatype for Informix compatibility mode. Ports Add basic build support for Visual Studio 2008, without resorting to generating the build files for 2005 and then converting them. Added support for 64-bit Windows platforms. Disable triggering failover with a signal in pg_standby on Windows. It never really worked before anyway. Change the WIN32 API version to be 5.01 (Windows XP), to bring in the proper IPv6 headers in newer SDKs. Write to the Windows eventlog in UTF-16, converting the message encoding as necessary. Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. The new code will fail on Mac OS X releases before 10.3. Reserve the shared memory region during backend startup on Windows, so that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Add s_lock support for SuperH architecture (not well tested). Source Code, Build Options Improve PGXS makefile system to allow the module's makefile to specify where to install DATA and DOCS files. This is mainly intended to allow versioned installation, eg, install into contrib/fooM.N/ rather than directly into contrib/. Get rid of the need for manual maintenance of the initial contents of pg_attribute, by having genbki.pl derive the information from the various catalog header files. This greatly simplifies modification of the "bootstrapped" catalogs. This patch finally kills genbki.sh and Gen_fmgrtab.sh; we now rely entirely on Perl scripts for those build steps. To avoid creating a Perl build dependency where there was not one before, they will be built and shipped in tarballs. Move the default configuration for the MSVC build system to config_default.pl, and allow using config.pl to override the defaults. Make configure check the version of Perl we're building with, and reject versions < 5.8. Remove all support for working around on systems that don't provide a working 64-bit integer type. Add new make targets "world", "install-world" and "installcheck-world" to build, install and check just about everything. In addition to everything built installed and tested by all, install and installcheck targets, these build HTML documentation, build and test contrib, and test procedural languages and ECPG. configure --enable-thread-safety is now the default (but can still be turned off). Fix inclusions of readline/editline header files so that we only attempt to #include the version of history.h that is in the same directory as the readline.h we are using. This avoids problems in some scenarios where both readline and editline are installed. Derived files that are shipped in the distribution used to be built in the source directory even for out-of-tree builds. They are now built in the build tree. This should be more convenient for certain developers' workflows. Functions which conflict with C++ reserved words have been renamed, making backend header files now safe to use with C++ libraries. Add man pages for SPI functions. Upgrade to Autoconf 2.63 (not relevant to users of distribution tarballs). Minimum version of Flex is now 2.5.31, to support reentrant scanners (not relevant to users of distribution tarballs). Documentation build rules have been improved. The generated documentation files are now shipped "loose", not in sub-tarballs. New toolchain to generate man pages. From now on, the man pages will be current in every release. Code-coverage testing support now extends to the entire source tree, not only src/backend/. Make the core scanner re-entrant, along with additional fixes that will let it be used directly by PL/pgSQL.