postgresql/doc/src/sgml/pgwalinspect.sgml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

214 lines
7.7 KiB
Plaintext
Raw Normal View History

<!-- doc/src/sgml/pgwalinspect.sgml -->
<sect1 id="pgwalinspect" xreflabel="pg_walinspect">
<title>pg_walinspect &mdash; low-level WAL inspection</title>
<indexterm zone="pgwalinspect">
<primary>pg_walinspect</primary>
</indexterm>
<para>
The <filename>pg_walinspect</filename> module provides SQL functions that
allow you to inspect the contents of write-ahead log of
a running <productname>PostgreSQL</productname> database cluster at a low
level, which is useful for debugging, analytical, reporting or
educational purposes. It is similar to <xref linkend="pgwaldump"/>, but
accessible through SQL rather than a separate utility.
</para>
<para>
All the functions of this module will provide the WAL information using the
current server's timeline ID.
</para>
<para>
All the functions of this module will try to find the first valid WAL record
that is at or after the given <replaceable>in_lsn</replaceable> or
<replaceable>start_lsn</replaceable> and will emit error if no such record
is available. Similarly, the <replaceable>end_lsn</replaceable> must be
available, and if it falls in the middle of a record, the entire record must
be available.
</para>
<note>
<para>
Some functions, such as <function><link
linkend="pg-logical-emit-message">pg_logical_emit_message</link></function>,
return the LSN <emphasis>after</emphasis> the record just
inserted. Therefore, if you pass that LSN as
<replaceable>in_lsn</replaceable> or <replaceable>start_lsn</replaceable>
to one of these functions, it will return the <emphasis>next</emphasis>
record.
</para>
</note>
<para>
By default, use of these functions is restricted to superusers and members of
the <literal>pg_read_server_files</literal> role. Access may be granted by
superusers to others using <command>GRANT</command>.
</para>
<sect2 id="pgwalinspect-funcs">
<title>General Functions</title>
<variablelist>
<varlistentry id="pgwalinspect-funcs-pg-get-wal-record-info">
<term>
<function>pg_get_wal_record_info(in_lsn pg_lsn) returns record</function>
</term>
<listitem>
<para>
Gets WAL record information of a given LSN. If the given LSN isn't
at the start of a WAL record, it gives the information of the next
available valid WAL record; or an error if no such record is found.
For example, usage of the function is as
follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_record_info('0/1E826E98');
-[ RECORD 1 ]----+----------------------------------------------------
start_lsn | 0/1E826F20
end_lsn | 0/1E826F60
prev_lsn | 0/1E826C80
xid | 0
resource_manager | Heap2
record_type | PRUNE
record_length | 58
main_data_length | 8
fpi_length | 0
description | snapshotConflictHorizon 33748 nredirected 0 ndead 2
block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
</screen>
</para>
</listitem>
</varlistentry>
<varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info">
<term>
<function>
pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
returns setof record
</function>
</term>
<listitem>
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
Returns one row per WAL record. If a future
<replaceable>end_lsn</replaceable> (i.e. ahead of the current LSN of
the server) is specified, it returns information until the end of WAL.
The function raises an error if <replaceable>start_lsn</replaceable>
is not available. For example, usage of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
start_lsn | 0/1E913618
end_lsn | 0/1E913650
prev_lsn | 0/1E9135A0
xid | 0
resource_manager | Standby
record_type | RUNNING_XACTS
record_length | 50
main_data_length | 24
fpi_length | 0
description | nextXid 33775 latestCompletedXid 33774 oldestRunningXid 33775
block_ref |
</screen>
</para>
</listitem>
</varlistentry>
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
returns setof record
</function>
</term>
<listitem>
<para>
Gets statistics of all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. By default, it returns one row per
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
it returns one row per <replaceable>record_type</replaceable>. If a
future <replaceable>end_lsn</replaceable> (i.e. ahead of the current
LSN of the server) is specified, it returns statistics until the end
of WAL. An error is raised if <replaceable>start_lsn</replaceable> is
not available. For example, usage of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
"resource_manager/record_type" = 'Transaction';
-[ RECORD 1 ]----------------+-------------------
resource_manager/record_type | Transaction
count | 2
count_percentage | 8
record_size | 875
record_size_percentage | 41.23468426013195
fpi_size | 0
fpi_size_percentage | 0
combined_size | 875
combined_size_percentage | 2.8634072910530795
</screen>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 02:09:07 +01:00
<function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
</term>
<listitem>
<para>
pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 02:09:07 +01:00
Gets a copy of the block information stored in WAL records. This includes
copies of the block data (<literal>NULL</literal> if none) and full page
images as <type>bytea</type> values (after
applying decompression when necessary, or <literal>NULL</literal> if none)
and their information associated with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 02:09:07 +01:00
<replaceable>end_lsn</replaceable>. Returns one row per block registered
in a WAL record. If a future <replaceable>end_lsn</replaceable> (i.e.
ahead of the current LSN of the server) is specified, it returns
statistics until the end of WAL. An error is raised if
<replaceable>start_lsn</replaceable> is not available. For example,
usage of the function is as follows:
<screen>
pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 02:09:07 +01:00
postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
relblocknumber, forkname,
substring(blockdata for 24) as block_trimmed,
substring(fpi for 24) as fpi_trimmed, fpilen, fpiinfo
FROM pg_get_wal_block_info('0/1871080', '0/1871440');
-[ RECORD 1 ]--+---------------------------------------------------
pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 02:09:07 +01:00
lsn | 0/18712F8
blockid | 0
reltablespace | 1663
pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 02:09:07 +01:00
reldatabase | 16384
relfilenode | 16392
relblocknumber | 0
forkname | main
pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 02:09:07 +01:00
block_trimmed | \x02800128180164000000
fpi_trimmed | \x0000000050108701000000002c00601f00200420e0020000
fpilen | 204
fpiinfo | {HAS_HOLE,APPLY}
</screen>
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2 id="pgwalinspect-author">
<title>Author</title>
<para>
Bharath Rupireddy <email>bharath.rupireddyforpostgres@gmail.com</email>
</para>
</sect2>
</sect1>