Rethink definition of pg_attribute.attcompression.

Redefine '\0' (InvalidCompressionMethod) as meaning "if we need to
compress, use the current setting of default_toast_compression".
This allows '\0' to be a suitable default choice regardless of
datatype, greatly simplifying code paths that initialize tupledescs
and the like.  It seems like a more user-friendly approach as well,
because now the default compression choice doesn't migrate into table
definitions, meaning that changing default_toast_compression is
usually sufficient to flip an installation's behavior; one needn't
tediously issue per-column ALTER SET COMPRESSION commands.

Along the way, fix a few minor bugs and documentation issues
with the per-column-compression feature.  Adopt more robust
APIs for SetIndexStorageProperties and GetAttributeCompression.

Bump catversion because typical contents of attcompression will now
be different.  We could get away without doing that, but it seems
better to ensure v14 installations all agree on this.  (We already
forced initdb for beta2, anyway.)

Discussion: https://postgr.es/m/626613.1621787110@sss.pgh.pa.us
This commit is contained in:
Tom Lane 2021-05-27 13:24:24 -04:00
parent a717e5c771
commit e6241d8e03
29 changed files with 261 additions and 384 deletions

View File

@ -1261,10 +1261,14 @@
<structfield>attcompression</structfield> <type>char</type> <structfield>attcompression</structfield> <type>char</type>
</para> </para>
<para> <para>
The current compression method of the column. If it is an invalid The current compression method of the column. Typically this is
compression method (<literal>'\0'</literal>) then column data will not <literal>'\0'</literal> to specify use of the current default setting
be compressed. Otherwise, <literal>'p'</literal> = pglz compression or (see <xref linkend="guc-default-toast-compression"/>). Otherwise,
<literal>'l'</literal> = <productname>LZ4</productname> compression. <literal>'p'</literal> selects pglz compression, while
<literal>'l'</literal> selects <productname>LZ4</productname>
compression. However, this field is ignored
whenever <structfield>attstorage</structfield> does not allow
compression.
</para></entry> </para></entry>
</row> </row>

View File

@ -8256,13 +8256,14 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
<para> <para>
This variable sets the default This variable sets the default
<link linkend="storage-toast">TOAST</link> <link linkend="storage-toast">TOAST</link>
compression method for columns of newly-created tables. The compression method for values of compressible columns.
<command>CREATE TABLE</command> statement can override this default (This can be overridden for individual columns by setting
by specifying the <literal>COMPRESSION</literal> column option. the <literal>COMPRESSION</literal> column option in
<command>CREATE TABLE</command> or
The supported compression methods are <literal>pglz</literal> and, <command>ALTER TABLE</command>.)
if <productname>PostgreSQL</productname> was compiled with The supported compression methods are <literal>pglz</literal> and
<literal>--with-lz4</literal>, <literal>lz4</literal>. (if <productname>PostgreSQL</productname> was compiled with
<option>--with-lz4</option>) <literal>lz4</literal>.
The default is <literal>pglz</literal>. The default is <literal>pglz</literal>.
</para> </para>
</listitem> </listitem>

View File

@ -26253,10 +26253,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_column_compression</primary> <primary>pg_column_compression</primary>
</indexterm> </indexterm>
<function>pg_column_compression</function> ( <type>"any"</type> ) <function>pg_column_compression</function> ( <type>"any"</type> )
<returnvalue>integer</returnvalue> <returnvalue>text</returnvalue>
</para> </para>
<para> <para>
Shows the compression algorithm that was used to compress a Shows the compression algorithm that was used to compress
an individual variable-length value. Returns <literal>NULL</literal> an individual variable-length value. Returns <literal>NULL</literal>
if the value is not compressed. if the value is not compressed.
</para></entry> </para></entry>

View File

@ -104,7 +104,6 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] | GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> | UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> | PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] } [ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@ -391,24 +390,27 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term> </term>
<listitem> <listitem>
<para> <para>
This sets the compression method to be used for data inserted into a column. This form sets the compression method for a column, determining how
values inserted in future will be compressed (if the storage mode
permits compression at all).
This does not cause the table to be rewritten, so existing data may still This does not cause the table to be rewritten, so existing data may still
be compressed with other compression methods. If the table is rewritten with be compressed with other compression methods. If the table is rewritten with
<command>VACUUM FULL</command> or <command>CLUSTER</command>, or restored <command>VACUUM FULL</command> or <command>CLUSTER</command>, or restored
with <application>pg_restore</application>, then all tuples are rewritten with <application>pg_restore</application>, then all values are rewritten
with the configured compression methods. with the configured compression method.
However, when data is inserted from another relation (for example,
Also, note that when data is inserted from another relation (for example, by <command>INSERT ... SELECT</command>), values from the source table are
by <command>INSERT ... SELECT</command>), tuples from the source data are not necessarily detoasted, so any previously compressed data may retain
not necessarily detoasted, and any previously compressed data is retained its existing compression method, rather than being recompressed with the
with its existing compression method, rather than recompressing with the compression method of the target column.
compression methods of the target columns.
The supported compression The supported compression
methods are <literal>pglz</literal> and <literal>lz4</literal>. methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal> (<literal>lz4</literal> is available only if <option>--with-lz4</option>
was used when building <productname>PostgreSQL</productname>. was used when building <productname>PostgreSQL</productname>.) In
addition, <replaceable class="parameter">compression_method</replaceable>
can be <literal>default</literal>, which selects the default behavior of
consulting the <xref linkend="guc-default-toast-compression"/> setting
at the time of data insertion to determine the method to use.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>

View File

@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv> <refsynopsisdiv>
<synopsis> <synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
{ <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ] { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COMPRESSION <replaceable>compression_method</replaceable> ] [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable> | <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] } | LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ] [, ... ]
@ -293,17 +293,22 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem> <listitem>
<para> <para>
The <literal>COMPRESSION</literal> clause sets the compression method The <literal>COMPRESSION</literal> clause sets the compression method
for a column. Compression is supported only for variable-width data for the column. Compression is supported only for variable-width data
types, and is used only for columns whose storage type is main or types, and is used only when the column's storage mode
extended. (See <xref linkend="sql-altertable"/> for information on is <literal>main</literal> or <literal>extended</literal>.
column storage types.) Setting this property for a partitioned table (See <xref linkend="sql-altertable"/> for information on
column storage modes.) Setting this property for a partitioned table
has no direct effect, because such tables have no storage of their own, has no direct effect, because such tables have no storage of their own,
but the configured value is inherited by newly-created partitions. but the configured value will be inherited by newly-created partitions.
The supported compression methods are <literal>pglz</literal> and The supported compression methods are <literal>pglz</literal> and
<literal>lz4</literal>. <literal>lz4</literal> is available only if <literal>lz4</literal>. (<literal>lz4</literal> is available only if
<literal>--with-lz4</literal> was used when building <option>--with-lz4</option> was used when building
<productname>PostgreSQL</productname>. The default <productname>PostgreSQL</productname>.) In addition,
is <literal>pglz</literal>. <replaceable class="parameter">compression_method</replaceable>
can be <literal>default</literal> to explicitly specify the default
behavior, which is to consult the
<xref linkend="guc-default-toast-compression"/> setting at the time of
data insertion to determine the method to use.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>

View File

@ -975,8 +975,8 @@ PostgreSQL documentation
<para> <para>
Do not output commands to set <acronym>TOAST</acronym> compression Do not output commands to set <acronym>TOAST</acronym> compression
methods. methods.
With this option, all objects will be created using whichever With this option, all columns will be restored with the default
compression method is the default during restore. compression setting.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>

View File

@ -464,12 +464,12 @@ PostgreSQL documentation
<para> <para>
Do not output commands to set <acronym>TOAST</acronym> compression Do not output commands to set <acronym>TOAST</acronym> compression
methods. methods.
With this option, all objects will be created using whichever With this option, all columns will be restored with the default
compression method is the default during restore. compression setting.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
<varlistentry> <varlistentry>
<term><option>--no-unlogged-table-data</option></term> <term><option>--no-unlogged-table-data</option></term>
<listitem> <listitem>

View File

@ -376,6 +376,16 @@ but the varlena header does not tell whether it has occurred &mdash;
the content of the <acronym>TOAST</acronym> pointer tells that, instead. the content of the <acronym>TOAST</acronym> pointer tells that, instead.
</para> </para>
<para>
The compression technique used for either in-line or out-of-line compressed
data can be selected for each column by setting
the <literal>COMPRESSION</literal> column option in <command>CREATE
TABLE</command> or <command>ALTER TABLE</command>. The default for columns
with no explicit setting is to consult the
<xref linkend="guc-default-toast-compression"/> parameter at the time data is
inserted.
</para>
<para> <para>
As mentioned, there are multiple types of <acronym>TOAST</acronym> pointer datums. As mentioned, there are multiple types of <acronym>TOAST</acronym> pointer datums.
The oldest and most common type is a pointer to out-of-line data stored in The oldest and most common type is a pointer to out-of-line data stored in
@ -392,13 +402,6 @@ useful for avoiding copying and redundant processing of large data values.
Further details appear in <xref linkend="storage-toast-inmemory"/>. Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para> </para>
<para>
The compression technique used for either in-line or out-of-line compressed
data can be selected using the <literal>COMPRESSION</literal> option on a per-column
basis when creating a table. The default for columns with no explicit setting
is taken from the value of <xref linkend="guc-default-toast-compression" />.
</para>
<sect2 id="storage-toast-ondisk"> <sect2 id="storage-toast-ondisk">
<title>Out-of-Line, On-Disk TOAST Storage</title> <title>Out-of-Line, On-Disk TOAST Storage</title>

View File

@ -232,11 +232,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
* same compression method. Otherwise we have to use the * same compression method. Otherwise we have to use the
* default method. * default method.
*/ */
if (att->atttypid == atttype->type_id && if (att->atttypid == atttype->type_id)
CompressionMethodIsValid(att->attcompression))
compression = att->attcompression; compression = att->attcompression;
else else
compression = GetDefaultToastCompression(); compression = InvalidCompressionMethod;
cvalue = toast_compress_datum(value, compression); cvalue = toast_compress_datum(value, compression);

View File

@ -104,18 +104,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
att->attstorage == TYPSTORAGE_MAIN)) att->attstorage == TYPSTORAGE_MAIN))
{ {
Datum cvalue; Datum cvalue;
char compression = att->attcompression;
/* cvalue = toast_compress_datum(untoasted_values[i],
* If the compression method is not valid, use the default. We att->attcompression);
* don't expect this to happen for regular index columns, which
* inherit the setting from the corresponding table column, but we
* do expect it to happen whenever an expression is indexed.
*/
if (!CompressionMethodIsValid(compression))
compression = GetDefaultToastCompression();
cvalue = toast_compress_datum(untoasted_values[i], compression);
if (DatumGetPointer(cvalue) != NULL) if (DatumGetPointer(cvalue) != NULL)
{ {

View File

@ -53,10 +53,12 @@ toast_compress_datum(Datum value, char cmethod)
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value))); Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value))); Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
Assert(CompressionMethodIsValid(cmethod));
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value)); valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
/* If the compression method is not valid, use the current default */
if (!CompressionMethodIsValid(cmethod))
cmethod = default_toast_compression;
/* /*
* Call appropriate compression routine for the compression method. * Call appropriate compression routine for the compression method.
*/ */

View File

@ -642,10 +642,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attbyval = typeForm->typbyval; att->attbyval = typeForm->typbyval;
att->attalign = typeForm->typalign; att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage; att->attstorage = typeForm->typstorage;
if (IsStorageCompressible(typeForm->typstorage)) att->attcompression = InvalidCompressionMethod;
att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
att->attcollation = typeForm->typcollation; att->attcollation = typeForm->typcollation;
ReleaseSysCache(tuple); ReleaseSysCache(tuple);
@ -711,7 +708,7 @@ TupleDescInitBuiltinEntry(TupleDesc desc,
att->attbyval = false; att->attbyval = false;
att->attalign = TYPALIGN_INT; att->attalign = TYPALIGN_INT;
att->attstorage = TYPSTORAGE_EXTENDED; att->attstorage = TYPSTORAGE_EXTENDED;
att->attcompression = GetDefaultToastCompression(); att->attcompression = InvalidCompressionMethod;
att->attcollation = DEFAULT_COLLATION_OID; att->attcollation = DEFAULT_COLLATION_OID;
break; break;

View File

@ -2483,10 +2483,10 @@ reform_and_rewrite_tuple(HeapTuple tuple,
* perform the compression here; we just need to decompress. That * perform the compression here; we just need to decompress. That
* will trigger recompression later on. * will trigger recompression later on.
*/ */
struct varlena *new_value; struct varlena *new_value;
ToastCompressionId cmid; ToastCompressionId cmid;
char cmethod; char cmethod;
char targetmethod;
new_value = (struct varlena *) DatumGetPointer(values[i]); new_value = (struct varlena *) DatumGetPointer(values[i]);
cmid = toast_get_compression_id(new_value); cmid = toast_get_compression_id(new_value);
@ -2495,7 +2495,7 @@ reform_and_rewrite_tuple(HeapTuple tuple,
if (cmid == TOAST_INVALID_COMPRESSION_ID) if (cmid == TOAST_INVALID_COMPRESSION_ID)
continue; continue;
/* convert compression id to compression method */ /* convert existing compression id to compression method */
switch (cmid) switch (cmid)
{ {
case TOAST_PGLZ_COMPRESSION_ID: case TOAST_PGLZ_COMPRESSION_ID:
@ -2506,10 +2506,16 @@ reform_and_rewrite_tuple(HeapTuple tuple,
break; break;
default: default:
elog(ERROR, "invalid compression method id %d", cmid); elog(ERROR, "invalid compression method id %d", cmid);
cmethod = '\0'; /* keep compiler quiet */
} }
/* figure out what the target method is */
targetmethod = TupleDescAttr(newTupDesc, i)->attcompression;
if (!CompressionMethodIsValid(targetmethod))
targetmethod = default_toast_compression;
/* if compression method doesn't match then detoast the value */ /* if compression method doesn't match then detoast the value */
if (TupleDescAttr(newTupDesc, i)->attcompression != cmethod) if (targetmethod != cmethod)
{ {
values[i] = PointerGetDatum(detoast_attr(new_value)); values[i] = PointerGetDatum(detoast_attr(new_value));
values_free[i] = true; values_free[i] = true;

View File

@ -701,6 +701,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attbyval = Ap->am_typ.typbyval; attrtypes[attnum]->attbyval = Ap->am_typ.typbyval;
attrtypes[attnum]->attalign = Ap->am_typ.typalign; attrtypes[attnum]->attalign = Ap->am_typ.typalign;
attrtypes[attnum]->attstorage = Ap->am_typ.typstorage; attrtypes[attnum]->attstorage = Ap->am_typ.typstorage;
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
attrtypes[attnum]->attcollation = Ap->am_typ.typcollation; attrtypes[attnum]->attcollation = Ap->am_typ.typcollation;
/* if an array type, assume 1-dimensional attribute */ /* if an array type, assume 1-dimensional attribute */
if (Ap->am_typ.typelem != InvalidOid && Ap->am_typ.typlen < 0) if (Ap->am_typ.typelem != InvalidOid && Ap->am_typ.typlen < 0)
@ -715,6 +716,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attbyval = TypInfo[typeoid].byval; attrtypes[attnum]->attbyval = TypInfo[typeoid].byval;
attrtypes[attnum]->attalign = TypInfo[typeoid].align; attrtypes[attnum]->attalign = TypInfo[typeoid].align;
attrtypes[attnum]->attstorage = TypInfo[typeoid].storage; attrtypes[attnum]->attstorage = TypInfo[typeoid].storage;
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
attrtypes[attnum]->attcollation = TypInfo[typeoid].collation; attrtypes[attnum]->attcollation = TypInfo[typeoid].collation;
/* if an array type, assume 1-dimensional attribute */ /* if an array type, assume 1-dimensional attribute */
if (TypInfo[typeoid].elem != InvalidOid && if (TypInfo[typeoid].elem != InvalidOid &&
@ -724,11 +726,6 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attndims = 0; attrtypes[attnum]->attndims = 0;
} }
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
/* /*
* If a system catalog column is collation-aware, force it to use C * If a system catalog column is collation-aware, force it to use C
* collation, so that its behavior is independent of the database's * collation, so that its behavior is independent of the database's

View File

@ -899,9 +899,7 @@ sub morph_row_for_pgattr
$row->{attbyval} = $type->{typbyval}; $row->{attbyval} = $type->{typbyval};
$row->{attalign} = $type->{typalign}; $row->{attalign} = $type->{typalign};
$row->{attstorage} = $type->{typstorage}; $row->{attstorage} = $type->{typstorage};
$row->{attcompression} = '\0';
$row->{attcompression} =
$type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
# set attndims if it's an array type # set attndims if it's an array type
$row->{attndims} = $type->{typcategory} eq 'A' ? '1' : '0'; $row->{attndims} = $type->{typcategory} eq 'A' ? '1' : '0';

View File

@ -1719,8 +1719,6 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */ /* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0'; attStruct->attgenerated = '\0';
attStruct->attcompression = InvalidCompressionMethod;
/* /*
* Change the column name to something that isn't likely to conflict * Change the column name to something that isn't likely to conflict
*/ */

View File

@ -601,7 +601,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl); Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition); static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition); static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static char GetAttributeCompression(Form_pg_attribute att, char *compression); static char GetAttributeCompression(Oid atttypid, char *compression);
/* ---------------------------------------------------------------- /* ----------------------------------------------------------------
@ -897,17 +897,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated) if (colDef->generated)
attr->attgenerated = colDef->generated; attr->attgenerated = colDef->generated;
/* if (colDef->compression)
* lookup attribute's compression method and store it in the attr->attcompression = GetAttributeCompression(attr->atttypid,
* attr->attcompression. colDef->compression);
*/
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
GetAttributeCompression(attr, colDef->compression);
else
attr->attcompression = InvalidCompressionMethod;
} }
/* /*
@ -6602,13 +6594,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attbyval = tform->typbyval; attribute.attbyval = tform->typbyval;
attribute.attalign = tform->typalign; attribute.attalign = tform->typalign;
attribute.attstorage = tform->typstorage; attribute.attstorage = tform->typstorage;
/* do not set compression in views etc */ attribute.attcompression = GetAttributeCompression(typeOid,
if (rel->rd_rel->relkind == RELKIND_RELATION || colDef->compression);
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression);
else
attribute.attcompression = InvalidCompressionMethod;
attribute.attnotnull = colDef->is_not_null; attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false; attribute.atthasdef = false;
attribute.atthasmissing = false; attribute.atthasmissing = false;
@ -7995,23 +7982,24 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
/* /*
* Helper function for ATExecSetStorage and ATExecSetCompression * Helper function for ATExecSetStorage and ATExecSetCompression
* *
* Set the attcompression and/or attstorage for the respective index attribute * Set the attstorage and/or attcompression fields for index columns
* if the respective input values are valid. * associated with the specified table column.
*/ */
static void static void
SetIndexStorageProperties(Relation rel, Relation attrelation, SetIndexStorageProperties(Relation rel, Relation attrelation,
AttrNumber attnum, char newcompression, AttrNumber attnum,
char newstorage, LOCKMODE lockmode) bool setstorage, char newstorage,
bool setcompression, char newcompression,
LOCKMODE lockmode)
{ {
HeapTuple tuple;
ListCell *lc; ListCell *lc;
Form_pg_attribute attrtuple;
foreach(lc, RelationGetIndexList(rel)) foreach(lc, RelationGetIndexList(rel))
{ {
Oid indexoid = lfirst_oid(lc); Oid indexoid = lfirst_oid(lc);
Relation indrel; Relation indrel;
AttrNumber indattnum = 0; AttrNumber indattnum = 0;
HeapTuple tuple;
indrel = index_open(indexoid, lockmode); indrel = index_open(indexoid, lockmode);
@ -8034,14 +8022,14 @@ SetIndexStorageProperties(Relation rel, Relation attrelation,
if (HeapTupleIsValid(tuple)) if (HeapTupleIsValid(tuple))
{ {
attrtuple = (Form_pg_attribute) GETSTRUCT(tuple); Form_pg_attribute attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
if (CompressionMethodIsValid(newcompression)) if (setstorage)
attrtuple->attcompression = newcompression;
if (newstorage != '\0')
attrtuple->attstorage = newstorage; attrtuple->attstorage = newstorage;
if (setcompression)
attrtuple->attcompression = newcompression;
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple); CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId, InvokeObjectPostAlterHook(RelationRelationId,
@ -8134,8 +8122,9 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()). * matching behavior of index.c ConstructTupleDescriptor()).
*/ */
SetIndexStorageProperties(rel, attrelation, attnum, SetIndexStorageProperties(rel, attrelation, attnum,
InvalidCompressionMethod, true, newstorage,
newstorage, lockmode); false, 0,
lockmode);
table_close(attrelation, RowExclusiveLock); table_close(attrelation, RowExclusiveLock);
@ -12299,23 +12288,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attbyval = tform->typbyval; attTup->attbyval = tform->typbyval;
attTup->attalign = tform->typalign; attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage; attTup->attstorage = tform->typstorage;
attTup->attcompression = InvalidCompressionMethod;
/* Setup attribute compression */
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
{
/*
* No compression for plain/external storage, otherwise, default
* compression method if it is not already set, refer comments atop
* attcompression parameter in pg_attribute.h.
*/
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
ReleaseSysCache(typeTuple); ReleaseSysCache(typeTuple);
@ -15613,7 +15586,6 @@ ATExecSetCompression(AlteredTableInfo *tab,
Form_pg_attribute atttableform; Form_pg_attribute atttableform;
AttrNumber attnum; AttrNumber attnum;
char *compression; char *compression;
char typstorage;
char cmethod; char cmethod;
ObjectAddress address; ObjectAddress address;
@ -15638,17 +15610,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED), (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot alter system column \"%s\"", column))); errmsg("cannot alter system column \"%s\"", column)));
typstorage = get_typstorage(atttableform->atttypid); /*
* Check that column type is compressible, then get the attribute
/* prevent from setting compression methods for uncompressible type */ * compression method code
if (!IsStorageCompressible(typstorage)) */
ereport(ERROR, cmethod = GetAttributeCompression(atttableform->atttypid, compression);
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("column data type %s does not support compression",
format_type_be(atttableform->atttypid))));
/* get the attribute compression method. */
cmethod = GetAttributeCompression(atttableform, compression);
/* update pg_attribute entry */ /* update pg_attribute entry */
atttableform->attcompression = cmethod; atttableform->attcompression = cmethod;
@ -15662,7 +15628,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
* Apply the change to indexes as well (only for simple index columns, * Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()). * matching behavior of index.c ConstructTupleDescriptor()).
*/ */
SetIndexStorageProperties(rel, attrel, attnum, cmethod, '\0', lockmode); SetIndexStorageProperties(rel, attrel, attnum,
false, 0,
true, cmethod,
lockmode);
heap_freetuple(tuple); heap_freetuple(tuple);
@ -18612,29 +18581,30 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
* resolve column compression specification to compression method. * resolve column compression specification to compression method.
*/ */
static char static char
GetAttributeCompression(Form_pg_attribute att, char *compression) GetAttributeCompression(Oid atttypid, char *compression)
{ {
char typstorage = get_typstorage(att->atttypid);
char cmethod; char cmethod;
/* if (compression == NULL || strcmp(compression, "default") == 0)
* No compression for plain/external storage, refer comments atop return InvalidCompressionMethod;
* attcompression parameter in pg_attribute.h
*/
if (!IsStorageCompressible(typstorage))
{
if (compression == NULL)
return InvalidCompressionMethod;
/*
* To specify a nondefault method, the column data type must be toastable.
* Note this says nothing about whether the column's attstorage setting
* permits compression; we intentionally allow attstorage and
* attcompression to be independent. But with a non-toastable type,
* attstorage could not be set to a value that would permit compression.
*
* We don't actually need to enforce this, since nothing bad would happen
* if attcompression were non-default; it would never be consulted. But
* it seems more user-friendly to complain about a certainly-useless
* attempt to set the property.
*/
if (!TypeIsToastable(atttypid))
ereport(ERROR, ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED), (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("column data type %s does not support compression", errmsg("column data type %s does not support compression",
format_type_be(att->atttypid)))); format_type_be(atttypid))));
}
/* fallback to default compression if it's not specified */
if (compression == NULL)
return GetDefaultToastCompression();
cmethod = CompressionNameToMethod(compression); cmethod = CompressionNameToMethod(compression);
if (!CompressionMethodIsValid(cmethod)) if (!CompressionMethodIsValid(cmethod))

View File

@ -561,6 +561,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <node> TableConstraint TableLikeClause %type <node> TableConstraint TableLikeClause
%type <ival> TableLikeOptionList TableLikeOption %type <ival> TableLikeOptionList TableLikeOption
%type <str> column_compression opt_column_compression
%type <list> ColQualList %type <list> ColQualList
%type <node> ColConstraint ColConstraintElem ConstraintAttr %type <node> ColConstraint ColConstraintElem ConstraintAttr
%type <ival> key_actions key_delete key_match key_update key_action %type <ival> key_actions key_delete key_match key_update key_action
@ -609,7 +610,6 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound %type <list> hash_partbound
%type <defelt> hash_partbound_elem %type <defelt> hash_partbound_elem
%type <str> optColumnCompression
/* /*
* Non-keyword token types. These are hard-wired into the "flex" lexer. * Non-keyword token types. These are hard-wired into the "flex" lexer.
@ -2302,6 +2302,15 @@ alter_table_cmd:
n->def = (Node *) makeString($6); n->def = (Node *) makeString($6);
$$ = (Node *)n; $$ = (Node *)n;
} }
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET COMPRESSION <cm> */
| ALTER opt_column ColId SET column_compression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
n->def = (Node *) makeString($5);
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ... AS IDENTITY ... */ /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ... AS IDENTITY ... */
| ALTER opt_column ColId ADD_P GENERATED generated_when AS IDENTITY_P OptParenthesizedSeqOptList | ALTER opt_column ColId ADD_P GENERATED generated_when AS IDENTITY_P OptParenthesizedSeqOptList
{ {
@ -2346,15 +2355,6 @@ alter_table_cmd:
n->missing_ok = true; n->missing_ok = true;
$$ = (Node *)n; $$ = (Node *)n;
} }
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
| ALTER opt_column ColId SET optColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
n->def = (Node *) makeString($5);
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */ /* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior | DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{ {
@ -3462,7 +3462,7 @@ TypedTableElement:
| TableConstraint { $$ = $1; } | TableConstraint { $$ = $1; }
; ;
columnDef: ColId Typename optColumnCompression create_generic_options ColQualList columnDef: ColId Typename opt_column_compression create_generic_options ColQualList
{ {
ColumnDef *n = makeNode(ColumnDef); ColumnDef *n = makeNode(ColumnDef);
n->colname = $1; n->colname = $1;
@ -3522,13 +3522,15 @@ columnOptions: ColId ColQualList
} }
; ;
optColumnCompression: column_compression:
COMPRESSION name COMPRESSION ColId { $$ = $2; }
{ | COMPRESSION DEFAULT { $$ = pstrdup("default"); }
$$ = $2; ;
}
| /*EMPTY*/ { $$ = NULL; } opt_column_compression:
; column_compression { $$ = $1; }
| /*EMPTY*/ { $$ = NULL; }
;
ColQualList: ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); } ColQualList ColConstraint { $$ = lappend($1, $2); }

View File

@ -4651,13 +4651,13 @@ static struct config_enum ConfigureNamesEnum[] =
{ {
{"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT, {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default compression for new columns."), gettext_noop("Sets the default compression method for compressible values."),
NULL, NULL
GUC_IS_NAME
}, },
&default_toast_compression, &default_toast_compression,
TOAST_PGLZ_COMPRESSION, TOAST_PGLZ_COMPRESSION,
default_toast_compression_options, NULL, NULL default_toast_compression_options,
NULL, NULL, NULL
}, },
{ {

View File

@ -208,8 +208,6 @@ typedef struct Archive
/* other important stuff */ /* other important stuff */
char *searchpath; /* search_path to set during restore */ char *searchpath; /* search_path to set during restore */
char *default_toast_compression; /* default TOAST compression to
* set during restore */
char *use_role; /* Issue SET ROLE to this */ char *use_role; /* Issue SET ROLE to this */
/* error handling */ /* error handling */

View File

@ -86,7 +86,6 @@ static void _selectTableAccessMethod(ArchiveHandle *AH, const char *tableam);
static void processEncodingEntry(ArchiveHandle *AH, TocEntry *te); static void processEncodingEntry(ArchiveHandle *AH, TocEntry *te);
static void processStdStringsEntry(ArchiveHandle *AH, TocEntry *te); static void processStdStringsEntry(ArchiveHandle *AH, TocEntry *te);
static void processSearchPathEntry(ArchiveHandle *AH, TocEntry *te); static void processSearchPathEntry(ArchiveHandle *AH, TocEntry *te);
static void processToastCompressionEntry(ArchiveHandle *AH, TocEntry *te);
static int _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH); static int _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH);
static RestorePass _tocEntryRestorePass(TocEntry *te); static RestorePass _tocEntryRestorePass(TocEntry *te);
static bool _tocEntryIsACL(TocEntry *te); static bool _tocEntryIsACL(TocEntry *te);
@ -2638,8 +2637,6 @@ ReadToc(ArchiveHandle *AH)
processStdStringsEntry(AH, te); processStdStringsEntry(AH, te);
else if (strcmp(te->desc, "SEARCHPATH") == 0) else if (strcmp(te->desc, "SEARCHPATH") == 0)
processSearchPathEntry(AH, te); processSearchPathEntry(AH, te);
else if (strcmp(te->desc, "TOASTCOMPRESSION") == 0)
processToastCompressionEntry(AH, te);
} }
} }
@ -2697,29 +2694,6 @@ processSearchPathEntry(ArchiveHandle *AH, TocEntry *te)
AH->public.searchpath = pg_strdup(te->defn); AH->public.searchpath = pg_strdup(te->defn);
} }
static void
processToastCompressionEntry(ArchiveHandle *AH, TocEntry *te)
{
/* te->defn should have the form SET default_toast_compression = 'x'; */
char *defn = pg_strdup(te->defn);
char *ptr1;
char *ptr2 = NULL;
ptr1 = strchr(defn, '\'');
if (ptr1)
ptr2 = strchr(++ptr1, '\'');
if (ptr2)
{
*ptr2 = '\0';
AH->public.default_toast_compression = pg_strdup(ptr1);
}
else
fatal("invalid TOASTCOMPRESSION item: %s",
te->defn);
free(defn);
}
static void static void
StrictNamesCheck(RestoreOptions *ropt) StrictNamesCheck(RestoreOptions *ropt)
{ {
@ -2779,8 +2753,7 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
/* These items are treated specially */ /* These items are treated specially */
if (strcmp(te->desc, "ENCODING") == 0 || if (strcmp(te->desc, "ENCODING") == 0 ||
strcmp(te->desc, "STDSTRINGS") == 0 || strcmp(te->desc, "STDSTRINGS") == 0 ||
strcmp(te->desc, "SEARCHPATH") == 0 || strcmp(te->desc, "SEARCHPATH") == 0)
strcmp(te->desc, "TOASTCOMPRESSION") == 0)
return REQ_SPECIAL; return REQ_SPECIAL;
/* /*
@ -3103,11 +3076,6 @@ _doSetFixedOutputState(ArchiveHandle *AH)
if (AH->public.searchpath) if (AH->public.searchpath)
ahprintf(AH, "%s", AH->public.searchpath); ahprintf(AH, "%s", AH->public.searchpath);
/* Select the dump-time default_toast_compression */
if (AH->public.default_toast_compression)
ahprintf(AH, "SET default_toast_compression = '%s';\n",
AH->public.default_toast_compression);
/* Make sure function checking is disabled */ /* Make sure function checking is disabled */
ahprintf(AH, "SET check_function_bodies = false;\n"); ahprintf(AH, "SET check_function_bodies = false;\n");

View File

@ -276,7 +276,6 @@ static void dumpDatabaseConfig(Archive *AH, PQExpBuffer outbuf,
static void dumpEncoding(Archive *AH); static void dumpEncoding(Archive *AH);
static void dumpStdStrings(Archive *AH); static void dumpStdStrings(Archive *AH);
static void dumpSearchPath(Archive *AH); static void dumpSearchPath(Archive *AH);
static void dumpToastCompression(Archive *AH);
static void binary_upgrade_set_type_oids_by_type_oid(Archive *fout, static void binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer, PQExpBuffer upgrade_buffer,
Oid pg_type_oid, Oid pg_type_oid,
@ -925,13 +924,11 @@ main(int argc, char **argv)
*/ */
/* /*
* First the special entries for ENCODING, STDSTRINGS, SEARCHPATH and * First the special entries for ENCODING, STDSTRINGS, and SEARCHPATH.
* TOASTCOMPRESSION.
*/ */
dumpEncoding(fout); dumpEncoding(fout);
dumpStdStrings(fout); dumpStdStrings(fout);
dumpSearchPath(fout); dumpSearchPath(fout);
dumpToastCompression(fout);
/* The database items are always next, unless we don't want them at all */ /* The database items are always next, unless we don't want them at all */
if (dopt.outputCreateDB) if (dopt.outputCreateDB)
@ -3398,58 +3395,6 @@ dumpSearchPath(Archive *AH)
destroyPQExpBuffer(path); destroyPQExpBuffer(path);
} }
/*
* dumpToastCompression: save the dump-time default TOAST compression in the
* archive
*/
static void
dumpToastCompression(Archive *AH)
{
char *toast_compression;
PQExpBuffer qry;
if (AH->dopt->no_toast_compression)
{
/* we don't intend to dump the info, so no need to fetch it either */
return;
}
if (AH->remoteVersion < 140000)
{
/* pre-v14, the only method was pglz */
toast_compression = pg_strdup("pglz");
}
else
{
PGresult *res;
res = ExecuteSqlQueryForSingleRow(AH, "SHOW default_toast_compression");
toast_compression = pg_strdup(PQgetvalue(res, 0, 0));
PQclear(res);
}
qry = createPQExpBuffer();
appendPQExpBufferStr(qry, "SET default_toast_compression = ");
appendStringLiteralAH(qry, toast_compression, AH);
appendPQExpBufferStr(qry, ";\n");
pg_log_info("saving default_toast_compression = %s", toast_compression);
ArchiveEntry(AH, nilCatalogId, createDumpId(),
ARCHIVE_OPTS(.tag = "TOASTCOMPRESSION",
.description = "TOASTCOMPRESSION",
.section = SECTION_PRE_DATA,
.createStmt = qry->data));
/*
* Also save it in AH->default_toast_compression, in case we're doing
* plain text dump.
*/
AH->default_toast_compression = toast_compression;
destroyPQExpBuffer(qry);
}
/* /*
* getBlobs: * getBlobs:
@ -16398,6 +16343,33 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
storage); storage);
} }
/*
* Dump per-column compression, if it's been set.
*/
if (!dopt->no_toast_compression)
{
const char *cmname;
switch (tbinfo->attcompression[j])
{
case 'p':
cmname = "pglz";
break;
case 'l':
cmname = "lz4";
break;
default:
cmname = NULL;
break;
}
if (cmname != NULL)
appendPQExpBuffer(q, "ALTER %sTABLE ONLY %s ALTER COLUMN %s SET COMPRESSION %s;\n",
foreign, qualrelname,
fmtId(tbinfo->attnames[j]),
cmname);
}
/* /*
* Dump per-column attributes. * Dump per-column attributes.
*/ */
@ -16419,35 +16391,6 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
qualrelname, qualrelname,
fmtId(tbinfo->attnames[j]), fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]); tbinfo->attfdwoptions[j]);
/*
* Dump per-column compression, if different from default.
*/
if (!dopt->no_toast_compression)
{
const char *cmname;
switch (tbinfo->attcompression[j])
{
case 'p':
cmname = "pglz";
break;
case 'l':
cmname = "lz4";
break;
default:
cmname = NULL;
break;
}
if (cmname != NULL &&
(fout->default_toast_compression == NULL ||
strcmp(cmname, fout->default_toast_compression) != 0))
appendPQExpBuffer(q, "ALTER %sTABLE ONLY %s ALTER COLUMN %s SET COMPRESSION %s;\n",
foreign, qualrelname,
fmtId(tbinfo->attnames[j]),
cmname);
}
} /* end loop over columns */ } /* end loop over columns */
if (ftoptions) if (ftoptions)

View File

@ -1636,9 +1636,9 @@ describeOneTableDetails(const char *schemaname,
indexdef_col = -1, indexdef_col = -1,
fdwopts_col = -1, fdwopts_col = -1,
attstorage_col = -1, attstorage_col = -1,
attcompression_col = -1,
attstattarget_col = -1, attstattarget_col = -1,
attdescr_col = -1, attdescr_col = -1;
attcompression_col = -1;
int numrows; int numrows;
struct struct
{ {
@ -2055,7 +2055,7 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage"); appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++; attstorage_col = cols++;
/* compression info */ /* compression info, if relevant to relkind */
if (pset.sversion >= 140000 && if (pset.sversion >= 140000 &&
!pset.hide_compression && !pset.hide_compression &&
(tableinfo.relkind == RELKIND_RELATION || (tableinfo.relkind == RELKIND_RELATION ||
@ -2259,7 +2259,7 @@ describeOneTableDetails(const char *schemaname,
if (fdwopts_col >= 0) if (fdwopts_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, fdwopts_col), false, false); printTableAddCell(&cont, PQgetvalue(res, i, fdwopts_col), false, false);
/* Storage and Description */ /* Storage mode, if relevant */
if (attstorage_col >= 0) if (attstorage_col >= 0)
{ {
char *storage = PQgetvalue(res, i, attstorage_col); char *storage = PQgetvalue(res, i, attstorage_col);
@ -2273,7 +2273,7 @@ describeOneTableDetails(const char *schemaname,
false, false); false, false);
} }
/* Column compression. */ /* Column compression, if relevant */
if (attcompression_col >= 0) if (attcompression_col >= 0)
{ {
char *compression = PQgetvalue(res, i, attcompression_col); char *compression = PQgetvalue(res, i, attcompression_col);

View File

@ -23,16 +23,16 @@
extern int default_toast_compression; extern int default_toast_compression;
/* /*
* Built-in compression method-id. The toast compression header will store * Built-in compression method ID. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression * this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression methods. * method IDs are directly mapped to the built-in compression methods.
* *
* Don't use these values for anything other than understanding the meaning * Don't use these values for anything other than understanding the meaning
* of the raw bits from a varlena; in particular, if the goal is to identify * of the raw bits from a varlena; in particular, if the goal is to identify
* a compression method, use the constants TOAST_PGLZ_COMPRESSION, etc. * a compression method, use the constants TOAST_PGLZ_COMPRESSION, etc.
* below. We might someday support more than 4 compression methods, but * below. We might someday support more than 4 compression methods, but
* we can never have more than 4 values in this enum, because there are * we can never have more than 4 values in this enum, because there are
* only 2 bits available in the places where this is used. * only 2 bits available in the places where this is stored.
*/ */
typedef enum ToastCompressionId typedef enum ToastCompressionId
{ {
@ -42,8 +42,9 @@ typedef enum ToastCompressionId
} ToastCompressionId; } ToastCompressionId;
/* /*
* Built-in compression methods. pg_attribute will store this in the * Built-in compression methods. pg_attribute will store these in the
* attcompression column. * attcompression column. In attcompression, InvalidCompressionMethod
* denotes the default behavior.
*/ */
#define TOAST_PGLZ_COMPRESSION 'p' #define TOAST_PGLZ_COMPRESSION 'p'
#define TOAST_LZ4_COMPRESSION 'l' #define TOAST_LZ4_COMPRESSION 'l'
@ -51,19 +52,6 @@ typedef enum ToastCompressionId
#define CompressionMethodIsValid(cm) ((cm) != InvalidCompressionMethod) #define CompressionMethodIsValid(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/*
* GetDefaultToastCompression -- get the default toast compression method
*
* This exists to hide the use of the default_toast_compression GUC variable.
*/
static inline char
GetDefaultToastCompression(void)
{
return (char) default_toast_compression;
}
/* pglz compression/decompression routines */ /* pglz compression/decompression routines */
extern struct varlena *pglz_compress_datum(const struct varlena *value); extern struct varlena *pglz_compress_datum(const struct varlena *value);

View File

@ -53,6 +53,6 @@
*/ */
/* yyyymmddN */ /* yyyymmddN */
#define CATALOG_VERSION_NO 202105231 #define CATALOG_VERSION_NO 202105271
#endif #endif

View File

@ -126,8 +126,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
char attstorage; char attstorage;
/* /*
* Compression method. Must be InvalidCompressionMethod if and only if * attcompression sets the current compression method of the attribute.
* typstorage is 'plain' or 'external'. * Typically this is InvalidCompressionMethod ('\0') to specify use of the
* current default setting (see default_toast_compression). Otherwise,
* 'p' selects pglz compression, while 'l' selects LZ4 compression.
* However, this field is ignored whenever attstorage does not allow
* compression.
*/ */
char attcompression BKI_DEFAULT('\0'); char attcompression BKI_DEFAULT('\0');

View File

@ -53,7 +53,7 @@ SELECT * INTO cmmove1 FROM cmdata;
Table "public.cmmove1" Table "public.cmmove1"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+------+-----------+----------+---------+----------+-------------+--------------+------------- --------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | | f1 | text | | | | extended | | |
SELECT pg_column_compression(f1) FROM cmmove1; SELECT pg_column_compression(f1) FROM cmmove1;
pg_column_compression pg_column_compression
@ -146,7 +146,7 @@ ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
Table "public.cmdata2" Table "public.cmdata2"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+-------------------+-----------+----------+---------+----------+-------------+--------------+------------- --------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | character varying | | | | extended | pglz | | f1 | character varying | | | | extended | | |
ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer; ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
\d+ cmdata2 \d+ cmdata2
@ -158,6 +158,7 @@ ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
--changing column storage should not impact the compression method --changing column storage should not impact the compression method
--but the data should not be compressed --but the data should not be compressed
ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar; ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
ALTER TABLE cmdata2 ALTER COLUMN f1 SET COMPRESSION pglz;
\d+ cmdata2 \d+ cmdata2
Table "public.cmdata2" Table "public.cmdata2"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
@ -179,12 +180,12 @@ SELECT pg_column_compression(f1) FROM cmdata2;
(1 row) (1 row)
-- test compression with materialized view -- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1; CREATE MATERIALIZED VIEW compressmv(x) AS SELECT * FROM cmdata1;
\d+ mv \d+ compressmv
Materialized view "public.mv" Materialized view "public.compressmv"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+------+-----------+----------+---------+----------+-------------+--------------+------------- --------+------+-----------+----------+---------+----------+-------------+--------------+-------------
x | text | | | | extended | pglz | | x | text | | | | extended | | |
View definition: View definition:
SELECT cmdata1.f1 AS x SELECT cmdata1.f1 AS x
FROM cmdata1; FROM cmdata1;
@ -196,7 +197,7 @@ SELECT pg_column_compression(f1) FROM cmdata1;
lz4 lz4
(2 rows) (2 rows)
SELECT pg_column_compression(x) FROM mv; SELECT pg_column_compression(x) FROM compressmv;
pg_column_compression pg_column_compression
----------------------- -----------------------
lz4 lz4
@ -222,7 +223,7 @@ SELECT pg_column_compression(f1) FROM cmpart2;
pglz pglz
(1 row) (1 row)
-- test compression with inheritence, error -- test compression with inheritance, error
CREATE TABLE cminh() INHERITS(cmdata, cmdata1); CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
NOTICE: merging multiple inherited definitions of column "f1" NOTICE: merging multiple inherited definitions of column "f1"
ERROR: column "f1" has a compression method conflict ERROR: column "f1" has a compression method conflict
@ -239,14 +240,6 @@ SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression" ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
HINT: Available values: pglz, lz4. HINT: Available values: pglz, lz4.
SET default_toast_compression = 'lz4'; SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
Table "public.cmdata2"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
SET default_toast_compression = 'pglz'; SET default_toast_compression = 'pglz';
-- test alter compression method -- test alter compression method
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4; ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
@ -266,10 +259,17 @@ SELECT pg_column_compression(f1) FROM cmdata;
lz4 lz4
(2 rows) (2 rows)
-- test alter compression method for the materialized view ALTER TABLE cmdata2 ALTER COLUMN f1 SET COMPRESSION default;
ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4; \d+ cmdata2
\d+ mv Table "public.cmdata2"
Materialized view "public.mv" Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
f1 | character varying | | | | plain | | |
-- test alter compression method for materialized views
ALTER MATERIALIZED VIEW compressmv ALTER COLUMN x SET COMPRESSION lz4;
\d+ compressmv
Materialized view "public.compressmv"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+------+-----------+----------+---------+----------+-------------+--------------+------------- --------+------+-----------+----------+---------+----------+-------------+--------------+-------------
x | text | | | | extended | lz4 | | x | text | | | | extended | lz4 | |
@ -277,7 +277,7 @@ View definition:
SELECT cmdata1.f1 AS x SELECT cmdata1.f1 AS x
FROM cmdata1; FROM cmdata1;
-- test alter compression method for the partitioned table -- test alter compression method for partitioned tables
ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz; ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4; ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
-- new data should be compressed with the current compression method -- new data should be compressed with the current compression method

View File

@ -50,7 +50,7 @@ SELECT * INTO cmmove1 FROM cmdata;
Table "public.cmmove1" Table "public.cmmove1"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+------+-----------+----------+---------+----------+-------------+--------------+------------- --------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | | f1 | text | | | | extended | | |
SELECT pg_column_compression(f1) FROM cmmove1; SELECT pg_column_compression(f1) FROM cmmove1;
pg_column_compression pg_column_compression
@ -144,7 +144,7 @@ ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
Table "public.cmdata2" Table "public.cmdata2"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+-------------------+-----------+----------+---------+----------+-------------+--------------+------------- --------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | character varying | | | | extended | pglz | | f1 | character varying | | | | extended | | |
ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer; ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
\d+ cmdata2 \d+ cmdata2
@ -156,6 +156,7 @@ ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
--changing column storage should not impact the compression method --changing column storage should not impact the compression method
--but the data should not be compressed --but the data should not be compressed
ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar; ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
ALTER TABLE cmdata2 ALTER COLUMN f1 SET COMPRESSION pglz;
\d+ cmdata2 \d+ cmdata2
Table "public.cmdata2" Table "public.cmdata2"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
@ -177,18 +178,18 @@ SELECT pg_column_compression(f1) FROM cmdata2;
(1 row) (1 row)
-- test compression with materialized view -- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1; CREATE MATERIALIZED VIEW compressmv(x) AS SELECT * FROM cmdata1;
ERROR: relation "cmdata1" does not exist ERROR: relation "cmdata1" does not exist
LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1; LINE 1: ...TE MATERIALIZED VIEW compressmv(x) AS SELECT * FROM cmdata1;
^ ^
\d+ mv \d+ compressmv
SELECT pg_column_compression(f1) FROM cmdata1; SELECT pg_column_compression(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist ERROR: relation "cmdata1" does not exist
LINE 1: SELECT pg_column_compression(f1) FROM cmdata1; LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
^ ^
SELECT pg_column_compression(x) FROM mv; SELECT pg_column_compression(x) FROM compressmv;
ERROR: relation "mv" does not exist ERROR: relation "compressmv" does not exist
LINE 1: SELECT pg_column_compression(x) FROM mv; LINE 1: SELECT pg_column_compression(x) FROM compressmv;
^ ^
-- test compression with partition -- test compression with partition
CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1); CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
@ -217,7 +218,7 @@ SELECT pg_column_compression(f1) FROM cmpart2;
----------------------- -----------------------
(0 rows) (0 rows)
-- test compression with inheritence, error -- test compression with inheritance, error
CREATE TABLE cminh() INHERITS(cmdata, cmdata1); CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
ERROR: relation "cmdata1" does not exist ERROR: relation "cmdata1" does not exist
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata); CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
@ -234,14 +235,6 @@ HINT: Available values: pglz.
SET default_toast_compression = 'lz4'; SET default_toast_compression = 'lz4';
ERROR: invalid value for parameter "default_toast_compression": "lz4" ERROR: invalid value for parameter "default_toast_compression": "lz4"
HINT: Available values: pglz. HINT: Available values: pglz.
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
Table "public.cmdata2"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
SET default_toast_compression = 'pglz'; SET default_toast_compression = 'pglz';
-- test alter compression method -- test alter compression method
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4; ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
@ -264,11 +257,18 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz pglz
(2 rows) (2 rows)
-- test alter compression method for the materialized view ALTER TABLE cmdata2 ALTER COLUMN f1 SET COMPRESSION default;
ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4; \d+ cmdata2
ERROR: relation "mv" does not exist Table "public.cmdata2"
\d+ mv Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
-- test alter compression method for the partitioned table --------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
f1 | character varying | | | | plain | | |
-- test alter compression method for materialized views
ALTER MATERIALIZED VIEW compressmv ALTER COLUMN x SET COMPRESSION lz4;
ERROR: relation "compressmv" does not exist
\d+ compressmv
-- test alter compression method for partitioned tables
ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz; ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ERROR: relation "cmpart1" does not exist ERROR: relation "cmpart1" does not exist
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4; ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;

View File

@ -69,6 +69,7 @@ ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
--changing column storage should not impact the compression method --changing column storage should not impact the compression method
--but the data should not be compressed --but the data should not be compressed
ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar; ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
ALTER TABLE cmdata2 ALTER COLUMN f1 SET COMPRESSION pglz;
\d+ cmdata2 \d+ cmdata2
ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain; ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
\d+ cmdata2 \d+ cmdata2
@ -76,10 +77,10 @@ INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
SELECT pg_column_compression(f1) FROM cmdata2; SELECT pg_column_compression(f1) FROM cmdata2;
-- test compression with materialized view -- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1; CREATE MATERIALIZED VIEW compressmv(x) AS SELECT * FROM cmdata1;
\d+ mv \d+ compressmv
SELECT pg_column_compression(f1) FROM cmdata1; SELECT pg_column_compression(f1) FROM cmdata1;
SELECT pg_column_compression(x) FROM mv; SELECT pg_column_compression(x) FROM compressmv;
-- test compression with partition -- test compression with partition
CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1); CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
@ -92,7 +93,7 @@ INSERT INTO cmpart VALUES (repeat('123456789', 4004));
SELECT pg_column_compression(f1) FROM cmpart1; SELECT pg_column_compression(f1) FROM cmpart1;
SELECT pg_column_compression(f1) FROM cmpart2; SELECT pg_column_compression(f1) FROM cmpart2;
-- test compression with inheritence, error -- test compression with inheritance, error
CREATE TABLE cminh() INHERITS(cmdata, cmdata1); CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata); CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
@ -100,9 +101,6 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
SET default_toast_compression = ''; SET default_toast_compression = '';
SET default_toast_compression = 'I do not exist compression'; SET default_toast_compression = 'I do not exist compression';
SET default_toast_compression = 'lz4'; SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
SET default_toast_compression = 'pglz'; SET default_toast_compression = 'pglz';
-- test alter compression method -- test alter compression method
@ -111,11 +109,14 @@ INSERT INTO cmdata VALUES (repeat('123456789', 4004));
\d+ cmdata \d+ cmdata
SELECT pg_column_compression(f1) FROM cmdata; SELECT pg_column_compression(f1) FROM cmdata;
-- test alter compression method for the materialized view ALTER TABLE cmdata2 ALTER COLUMN f1 SET COMPRESSION default;
ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4; \d+ cmdata2
\d+ mv
-- test alter compression method for the partitioned table -- test alter compression method for materialized views
ALTER MATERIALIZED VIEW compressmv ALTER COLUMN x SET COMPRESSION lz4;
\d+ compressmv
-- test alter compression method for partitioned tables
ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz; ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4; ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;