From 39387228c8b7043d168bada0c64e9f59e83285f5 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Fri, 5 Nov 2021 20:26:18 -0400 Subject: [PATCH] First-draft release notes for 14.1. As usual, the release notes for other branches will be made by cutting these down, but put them up for community review first. Also as usual for a .1 release, there are some entries here that are not really relevant for v14 because they already appeared in 14.0. Those'll be removed later. --- doc/src/sgml/release-14.sgml | 1950 ++++++++++++++++++++++++++++++++++ 1 file changed, 1950 insertions(+) diff --git a/doc/src/sgml/release-14.sgml b/doc/src/sgml/release-14.sgml index 51820eac1a..d95734e91f 100644 --- a/doc/src/sgml/release-14.sgml +++ b/doc/src/sgml/release-14.sgml @@ -1,6 +1,1956 @@ + + Release 14.1 + + + Release date: + 2021-11-11 + + + + This release contains a variety of fixes from 14.0. + For information about new features in major release 14, see + . + + + + Migration to Version 14.1 + + + A dump/restore is not required for those running 14.X. + + + + However, note that installations using physical replication should + update standby servers before the primary server, as explained in + the first changelog entry below. + + + + Also, several bugs have been found that may have resulted in corrupted + indexes, as explained in the next several changelog entries. If any + of those cases apply to you, it's recommended to reindex + possibly-affected indexes after updating. + + + + + Changes + + + + + + + Fix physical replication for cases where the primary crashes + after shipping a WAL segment that ends with a partial WAL record + (Álvaro Herrera) + + + + If the primary did not survive long enough to finish writing the + rest of the incomplete WAL record, then the previous crash-recovery + logic had it back up and overwrite WAL starting from the beginning + of the incomplete WAL record. This is problematic since standby + servers may already have copies of that WAL segment. They will then + see an inconsistent next segment, and will not be able to recover + without manual intervention. To fix, do not back up over a WAL + segment boundary when restarting after a crash. Instead write a new + type of WAL record at the start of the next WAL segment, informing + readers that the incomplete WAL record will never be finished and + must be disregarded. + + + + When applying this update, it's best to update standby servers + before the primary, so that they will be ready to handle this new + WAL record type if the primary happens to crash. + + + + + + + Ensure that parallel VACUUM doesn't miss any + indexes (Peter Geoghegan, Masahiko Sawada) + + + + A parallel VACUUM would fail to process indexes + that are below the min_parallel_index_scan_size + cutoff, if the table also has at least two indexes that are above + that size. This could result in those indexes becoming corrupt, + since they'd still contain references to any heap entries removed by + the VACUUM; subsequent queries using such indexes + would be likely to return rows they shouldn't. + This problem does not affect autovacuum, since it doesn't use + parallel vacuuming. However, it is advisable to reindex any + manually-vacuumed tables that have the right mix of index sizes. + + + + + + + Fix CREATE INDEX CONCURRENTLY to wait for + the latest prepared transactions (Andrey Borodin) + + + + Rows inserted by just-prepared transactions might be omitted from + the new index, causing queries relying on the index to miss such + rows. The previous fix for this type of problem failed to account + for PREPARE TRANSACTION commands that were still + in progress when CREATE INDEX CONCURRENTLY + checked for them. As before, in installations that have enabled + prepared transactions (max_prepared_transactions + > 0), it's recommended to reindex any concurrently-built indexes + in case this problem occurred when they were built. + + + + + + + Avoid race condition that can cause backends to fail to add entries + for new rows to an index being built concurrently (Noah Misch, + Andrey Borodin) + + + + While it's apparently rare in the field, this case could potentially + affect any index built or reindexed with + the CONCURRENTLY option. It is recommended to + reindex any such indexes to make sure they are correct. + + + + + + + Fix float4 and float8 hash functions to + produce uniform results for NaNs (Tom Lane) + + + + Since PostgreSQL's floating-point types + deem all NaNs to be equal, it's important for the hash functions to + produce the same hash code for all bit-patterns that are NaNs + according to the IEEE 754 standard. This failed to happen before, + meaning that hash indexes and hash-based query plans might produce + incorrect results for non-canonical NaN values. + ('-NaN'::float8 is one way to produce such a + value on most machines.) It is advisable to reindex hash indexes + on floating-point columns, if there is any possibility that they + might contain such values. + + + + + + + Fix REINDEX CONCURRENTLY to preserve operator + class parameters that were attached to the target index + (Michael Paquier) + + + + + + + Prevent data loss during crash recovery of CREATE + TABLESPACE, when wal_level + = minimal (Noah Misch) + + + + If the server crashed between CREATE TABLESPACE + and the next checkpoint, replay would fully remove the contents of + the new tablespace's directory, relying on subsequent WAL replay + to restore everything within that directory. This interacts badly + with optimizations that skip writing WAL (one example + is COPY into a just-created table). Such + optimizations are applied only when wal_level + is minimal, which is not the default in v10 and + later. + + + + + + + Fix incorrect creation of shared dependencies when cloning a + database that contains non-builtin objects (Aleksander Alekseev) + + + + The effects of this error are probably limited in practice. In + principle, it could allow a role to be dropped while it still owns + objects; but most installations would never want to drop a role + that had been used for objects they'd added + to template1. + + + + + + + Ensure that the relation cache is invalidated for a table being + attached to or detached from a partitioned table (Amit Langote, + Álvaro Herrera) + + + + This oversight could allow misbehavior of subsequent inserts/updates + addressed directly to the partition, but only in currently-existing + sessions. + + + + + + + Ensure that the relation cache is invalidated for all partitions + of a partitioned table that is being added to or removed from a + publication (Hou Zhijie, Vignesh C) + + + + This oversight could lead to improper replication behavior until all + currently-existing sessions have exited. + + + + + + + Ensure that the relation cache is invalidated when creating or + dropping a FOR ALL TABLES publication + (Hou Zhijie, Vignesh C) + + + + This oversight could lead to improper replication behavior until all + currently-existing sessions have exited. + + + + + + + Fix corruption of parse tree while creating a range type (Alex + Kozhemyakin, Sergey Shinderuk) + + + + CREATE TYPE incorrectly freed an element of the + parse tree, which could cause problems for a later event trigger, or + if the CREATE TYPE command was stored in the plan + cache and used again later. + + + + + + + Don't discard a cast to the same type with unspecified type modifier + (Tom Lane) + + + + For example, if column f1 is of + type numeric(18,3), the parser used to simply + discard a cast like f1::numeric, on the grounds + that it would have no run-time effect. That's true, but the exposed + type of the expression should still be considered to be + plain numeric, + not numeric(18,3). This is important for + correctly resolving the type of larger constructs, such + as recursive UNIONs. + + + + + + + Fix updates of element fields in arrays of domain over composite + (Tom Lane) + + + + A command such as UPDATE tab SET fld[1].subfld = + value failed if the array elements were domains rather + than plain composites. + + + + + + + Disallow the combination of FETCH FIRST WITH TIES + and FOR UPDATE SKIP LOCKED (David Christensen) + + + + FETCH FIRST WITH TIES necessarily fetches + one more row than requested, so tha it can determine whether the + next row is a tie or not. In our current implementation, + if FOR UPDATE is used then that row will also get + locked even though it is not returned. That results in undesirable + behavior if the SKIP LOCKED option is specified. + It's difficult to change this without introducing a different set of + undesirable behaviors, so for now, forbid the combination. + + + + + + + Disallow creating an ICU collation if the current database's + encoding won't support it (Tom Lane) + + + + Previously this was allowed, but the collation effectively vanished + into the ether because of the way collation lookup works: you could + not use the collation, nor even drop it. + + + + + + + Disallow ALTER INDEX index ALTER COLUMN col SET + (options) (Nathan Bossart, Michael Paquier) + + + + While the parser accepts this, it's undocumented and doesn't + actually work. + + + + + + + Fix corner-case loss of precision in + numeric power() (Dean Rasheed) + + + + The result could be inaccurate when the first argument is very close + to 1. + + + + + + + Avoid regular expression errors with capturing parentheses + inside {0} (Tom Lane) + + + + Regular expressions like (.){0}...\1 drew + invalid backreference number. Other regexp engines + such as Perl don't complain, though, and for that matter ours + doesn't either in some closely related cases. Worse, it could throw + an assertion failure instead. Fix it so that no error is thrown and + instead the back-reference is silently deemed to never match. + + + + + + + Prevent regular expression back-references from sometimes matching + when they shouldn't (Tom Lane) + + + + The regexp engine was careless about clearing match data + for capturing parentheses after rejecting a partial match. This + could allow a later back-reference to succeed when by rights it + should fail for lack of a defined referent. + + + + + + + Fix regular expression performance bug with back-references inside + iteration nodes (Tom Lane) + + + + Unnecessarily stupid back-tracking logic could result in exponential + time spent looking for a match. Fortunately the problem is masked + in most cases by other optimizations; but it is possible to + demonstrate it with fairly simple, if contrived, regexps. + + + + + + + Fix incorrect results from AT TIME ZONE applied + to a time with time zone value (Tom Lane) + + + + The results were incorrect if the target time zone was specified by + a dynamic timezone abbreviation (that is, one that is defined as + equivalent to a full time zone name, rather than a fixed UTC offset). + + + + + + + Use the CLDR project's data to map Windows time zone names to IANA + time zones (Tom Lane) + + + + When running on Windows, initdb attempts + to set the new cluster's timezone parameter to + the IANA time zone matching the system's prevailing time zone. + We were using a mapping table that we'd generated years ago and + updated only fitfully; unsurprisingly, it contained a number of + errors as well as omissions of recently-added zones. It turns out + that CLDR has been tracking the most appropriate mappings, so start + using their data. This change will not affect any existing + installation, only newly-initialized clusters. + + + + + + + Fix planner error with pulling up subquery expressions into function + rangetable entries (Tom Lane) + + + + If a function in FROM laterally references the + output of some sub-SELECT earlier in + the FROM clause, and we are able to flatten that + sub-SELECT into the outer query, the + expression(s) copied into the function expression were not fully + processed. This could lead to crashes at execution. + + + + + + + Fix mistranslation of PlaceHolderVars to inheritance child relations + (Tom Lane) + + + + This error could result in assertion failures, or in mis-planning of + queries having partitioned or inherited tables on the nullable side + of an outer join. + + + + + + + Avoid using MCV-only statistics to estimate the range of a column + (Tom Lane) + + + + There are corner cases in which ANALYZE will + build a most-common-values (MCV) list but not a histogram, even + though the MCV list does not account for all the observed values. + In such cases, keep the planner from using the MCV list alone to + estimate the range of column values. + + + + + + + Fix restoration of a Portal's snapshot inside a subtransaction + (Bertrand Drouvot) + + + + If a procedure commits or rolls back a transaction, and then its + next significant action is inside a new subtransaction, snapshot + management went wrong, leading to a dangling pointer and probable + crash. A typical example in PL/pgSQL is a COMMIT + immediately followed by a BEGIN ... EXCEPTION + block that performs a query. + + + + + + + Clean up correctly if a transaction fails after exporting its + snapshot (Dilip Kumar) + + + + This oversight would only cause a problem if the same session + attempted to export a snapshot again. The most likely scenario for + that is creation of a replication slot (followed by rollback) + and then creation of another replication slot. + + + + + + + Ensure prepared transactions are properly accounted for during + promotion of a standby server (Michael Paquier, Andres Freund) + + + + There was a narrow window where a prepared transaction could be + omitted from a snapshot taken by a concurrently-running session. + If that session then used the snapshot to perform data updates, + erroneous results or data corruption could occur. + + + + + + + Disallow LISTEN in background workers (Tom Lane) + + + + There's no infrastructure to support this, so if someone did + it, it would only result in preventing cleanup of + the NOTIFY queue. + + + + + + + Send NOTIFY signals to other backends during + transaction commit, not in the server's idle loop (Artur Zakirov, + Tom Lane) + + + + This change allows notifications to be delivered immediately after + an intra-procedure COMMIT. It also allows + logical replication workers to send notifications. + + + + + + + Fix could not find RecursiveUnion error + when EXPLAIN tries to print a filter condition + attached to a WorkTableScan node (Tom Lane) + + + + + + + Refuse to rewind a cursor marked NO SCROLL + if it has been held over from a previous transaction due + to the WITH HOLD option (Tom Lane) + + + + We have long forbidden fetching backwards from a NO + SCROLL cursor, but for historical reasons the prohibition + didn't extend to cases in which we rewind the query altogether and + then re-fetch forwards. That exception leads to inconsistencies, + particularly for held-over cursors which may not have stored all the + data necessary to rewind. Disallow rewinding for non-scrollable + held-over cursors to block the worst inconsistencies. (v15 will + remove the exception altogether.) + + + + + + + Fix possible failure while saving a WITH HOLD + cursor at transaction end, if it had already been read to completion + (Tom Lane) + + + + + + + Fix detection of a relation that has grown to the maximum allowed + length (Tom Lane) + + + + An attempt to extend a table or index past the limit of 2^32-1 + blocks was rejected, but not soon enough to prevent inconsistent + internal state from being created. + + + + + + + Correctly track the presence of data-modifying CTEs when expanding + a DO INSTEAD rule (Greg Nancarrow, Tom Lane) + + + + The previous failure to do this could lead to problems such as + unsafely choosing a parallel plan. + + + + + + + Fix incorrect reporting of permissions failures on extended + statistics objects (Tomas Vondra) + + + + The code typically produced cache lookup error rather + than the intended message. + + + + + + + Fix incorrect snapshot handling in parallel workers (Greg Nancarrow) + + + + This oversight could lead to misbehavior in parallel queries if the + transaction isolation level is less than REPEATABLE READ. + + + + + + + Fix logical decoding to correctly ignore toast-table changes for + transient tables (Bertrand Drouvot) + + + + Logical decoding normally ignores changes in transient tables such + as those created during an ALTER TABLE heap + rewrite. But that filtering wasn't applied to the associated toast + table if any, leading to possible errors when rewriting a table + that's being published. + + + + + + + Fix logical decoding's memory usage accounting to handle TOAST data + correctly (Bertrand Drouvot) + + + + + + + Fix walreceiver to ensure that it creates an archive notification + file before exiting (Fujii Masao) + + + + This failed to happen if the walreceiver exited exactly at a WAL + segment boundary, thus delaying archiving of the new segment on the + standby. + + + + + + + Compute the correct WAL range to include in a backup manifest when a + timeline change is involved (Kyotaro Horiguchi) + + + + + + + Avoid trying to lock the OLD + and NEW pseudo-relations in a rule + that uses SELECT FOR UPDATE + (Masahiko Sawada, Tom Lane) + + + + + + + Fix parser's processing of aggregate FILTER + clauses (Tom Lane) + + + + If the FILTER expression is a plain boolean column, + the semantic level of the aggregate could be mis-determined, leading + to not-per-spec behavior. If the FILTER + expression is itself a boolean-returning aggregate, an error should + be thrown but was not, likely resulting in a crash at execution. + + + + + + + Ensure correct lock level is used when renaming a table (Nathan + Bossart, Álvaro Herrera) + + + + For historical reasons, ALTER INDEX ... RENAME + can be applied to any sort of relation. The lock level required to + rename an index is lower than that required to rename a table or + other kind of relation, but the code got this wrong and would use + the weaker lock level whenever the command is spelled ALTER + INDEX. + + + + + + + Prevent ALTER TYPE/DOMAIN/OPERATOR ... SET + from changing extension membership (Tom Lane) + + + + ALTER ... SET executed by an extension script + would cause the target object to become a member of the extension if + it was not already. In itself this isn't too troubling, since + there's little reason for an extension script to touch an object not + belonging to the extension. But ALTER TYPE SET + will recurse to dependent domains, thus causing them to also become + extension members. This causes unwanted side-effects from + extension upgrade scripts that use that command to adjust the + properties of a base type belonging to the extension. Fix by + redefining these ALTER cases to never change + extension membership. + + + + + + + Avoid trying to clean up LLVM state after an error within LLVM + (Andres Freund, Justin Pryzby) + + + + This prevents a likely crash during backend exit after a fatal LLVM + error. + + + + + + + Avoid null-pointer-dereference crash when dropping a role that owns + objects being dropped concurrently (Álvaro Herrera) + + + + + + + Prevent snapshot reference leak warning + when lo_export() or a related function fails + (Heikki Linnakangas) + + + + + + + Ensure that scans of SP-GiST indexes are counted in the statistics + views (Tom Lane) + + + + Incrementing the number-of-index-scans counter was overlooked in the + SP-GiST code, although per-tuple counters were advanced correctly. + + + + + + + Fix inefficient code generation for CoerceToDomain expression nodes + (Ranier Vilela) + + + + + + + Recalculate relevant wait intervals + if recovery_min_apply_delay is changed during + recovery (Soumyadeep Chakraborty, Ashwin Agrawal) + + + + + + + Fix infinite loop if a simplehash.h hash table + reaches 2^32 elements (Yura Sokolov) + + + + It seems unlikely that this bug has yet been hit in practice, as it + would require work_mem settings of hundreds of + gigabytes for existing uses of simplehash.h. + + + + + + + Avoid O(N^2) behavior in some list-manipulation operations + (Nathan Bossart, Tom Lane) + + + + These changes fix slow processing in several scenarios, including: + when a standby replays a transaction that held many exclusive locks + on the primary; when many files are due to be unlinked after a + checkpoint; when hash aggregation involves many batches; and when + pg_trgm extracts indexable conditions from a + complex regular expression. Only the first of these scenarios has + actually been reported from the field, but they all seem like + plausible consequences of inefficient list deletions. + + + + + + + Reduce memory consumption during calculation of extended statistics + (Justin Pryzby, Tomas Vondra) + + + + + + + Add more defensive checks around B-tree posting list splits + (Peter Geoghegan) + + + + This change should help detect index corruption involving duplicate + table TIDs. + + + + + + + Allow the autovacuum launcher process to respond + to pg_log_backend_memory_contexts() requests + more quickly (Koyu Tanigawa) + + + + + + + Fix memory leak in HMAC hash calculations (Sergey Shinderuk) + + + + + + + Disallow setting huge_pages + to on when shared_memory_type + is sysv (Thomas Munro) + + + + Previously, this setting was accepted, but it did nothing for lack + of any implementation. + + + + + + + Fix missing libpq functions on AIX + (Tony Reix) + + + + Code reorganization led to the following documented functions not + being exported from libpq on AIX: + pg_encoding_to_char(), + pg_utf_mblen(), + pg_char_to_encoding(), + pg_valid_server_encoding(), and + pg_valid_server_encoding_id(). + Restore them to visibility. + + + + + + + Fix ecpg to recover correctly + after malloc() failure while establishing a + connection (Michael Paquier) + + + + + + + Fix checking of query type in PL/pgSQL's RETURN + QUERY statement (Tom Lane) + + + + RETURN QUERY should accept any query that can + return tuples, e.g. UPDATE RETURNING. v14 + accidentally disallowed anything but SELECT; + moreover, the RETURN QUERY EXECUTE variant + failed to apply any query-type check at all. + + + + + + + Fix misevaluation of stable functions called in the arguments of a + PL/pgSQL CALL statement (Tom Lane) + + + + They were being called with an out-of-date snapshot, so that they + would not see any database changes made since the start of the + session's top-level command. + + + + + + + Allow EXIT out of the outermost block in a + PL/pgSQL routine (Tom Lane) + + + + If the routine does not require an explicit RETURN, + this usage should be valid, but it was rejected. + + + + + + + Remove pg_ctl's hard-coded limits on the + total length of generated commands (Phil Krylov) + + + + For example, this removes a restriction on how many command-line + options can be passed through to the postmaster. Individual path + names that pg_ctl deals with, such as the + postmaster executable's name or the data directory name, are still + limited to MAXPGPATH bytes in most cases. + + + + + + + Fix pg_dump to dump non-global default + privileges correctly (Neil Chen, Masahiko Sawada) + + + + If a global (unrestricted) ALTER DEFAULT + PRIVILEGES command revoked some present-by-default + privilege, for example EXECUTE for functions, and + then a restricted ALTER DEFAULT PRIVILEGES + command granted that privilege back for a selected role or + schema, pg_dump failed to dump the + restricted privilege grant correctly. + + + + + + + Make pg_dump acquire shared lock on + partitioned tables that are to be dumped (Tom Lane) + + + + This oversight was usually pretty harmless, since + once pg_dump has locked any of the leaf + partitions, that would suffice to prevent significant DDL on the + partitioned table itself. However problems could ensue when dumping + a childless partitioned table, since no relevant lock would be held. + + + + + + + Improve pg_dump's performance by avoiding + making per-table queries for RLS policies, and by avoiding repetitive + calls to format_type() (Tom Lane) + + + + These changes provide only marginal improvement when dumping from a + local server, but a dump from a remote server can benefit + substantially. + + + + + + + Fix crash in pg_dump when attempting to + dump trigger definitions from a pre-8.3 server (Tom Lane) + + + + + + + Fix incorrect filename in pg_restore's + error message about an invalid large object TOC file (Daniel + Gustafsson) + + + + + + + Ensure that pgbench exits with non-zero + status after a socket-level failure (Yugo Nagata, Fabien Coelho) + + + + The desired behavior is to finish out the run but then exit with + status 2. Also, fix the reporting of such errors. + + + + + + + Prevent pg_amcheck from checking + temporary relations, as well as indexes that are invalid or not + ready (Mark Dilger) + + + + This avoids unhelpful checks of relations that will almost certainly + appear inconsistent. + + + + + + + Make contrib/amcheck skip unlogged tables + when running on a standby server (Mark Dilger) + + + + It's appropriate to do this since such tables will be empty, + and unlogged indexes were already handled similarly. + + + + + + + Fix failure of contrib/btree_gin indexes + on "char" + (not char(n)) columns, + when an indexscan using the < + or <= operator is performed (Tom Lane) + + + + Such an indexscan failed to return all the entries it should. + + + + + + + Change contrib/pg_stat_statements to read + its query texts file in units of at most 1GB + (Tom Lane) + + + + Such large query text files are very unusual, but if they do occur, + the previous coding would fail on Windows 64 (which rejects + individual read requests of more than 2GB). + + + + + + + Fix null-pointer crash + when contrib/postgres_fdw tries to report a + data conversion error (Tom Lane) + + + + + + + Add spinlock support for the RISC-V architecture (Marek Szuba) + + + + This is essential for reasonable performance on that platform. + + + + + + + Support OpenSSL 3.0.0 + (Peter Eisentraut, Daniel Gustafsson, Michael Paquier) + + + + + + + Set correct type identifier on OpenSSL BIO (I/O abstraction) + objects created by PostgreSQL + (Itamar Gafni) + + + + This oversight probably only matters for code that is doing + tasks like auditing the OpenSSL installation. But it's + nominally a violation of the OpenSSL API, so fix it. + + + + + + + Fix our pkg-config files to again support static + linking of libpq (Peter Eisentraut) + + + + + + + Make pg_regexec() robust against an + out-of-range search_start parameter + (Tom Lane) + + + + Return REG_NOMATCH, instead of possibly crashing, + when search_start is past the end of the + string. This case is probably unreachable within + core PostgreSQL, but extensions might be + more careless about the parameter value. + + + + + + + Ensure that GetSharedSecurityLabel() can be + used in a newly-started session that has not yet built its critical + relation cache entries (Jeff Davis) + + + + + + + When running a TAP test, include the module's own directory + in PATH (Andrew Dunstan) + + + + This allows tests to find built programs that are not installed, + such as custom test drivers. + + + + + + + Update time zone data files to tzdata + release 2021e for DST law changes in Fiji, Jordan, Palestine, and + Samoa, plus historical corrections for Barbados, Cook Islands, + Guyana, Niue, Portugal, and Tonga. + + + + Also, the Pacific/Enderbury zone has been renamed to Pacific/Kanton. + Also, the following zones have been merged into nearby, more-populous + zones whose clocks have agreed with them since 1970: Africa/Accra, + America/Atikokan, America/Blanc-Sablon, America/Creston, + America/Curacao, America/Nassau, America/Port_of_Spain, + Antarctica/DumontDUrville, and Antarctica/Syowa. + In all these cases, the previous zone name remains as an alias. + + + + + + + + Release 14