rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/test/isolation/specs/detach-partition-concurrent...

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

85 lines
4.4 KiB
RPMSpec
Raw Normal View History

ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
# Try various things to happen to a partition with an incomplete detach
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
#
# Note: Always keep "s2noop" right after "s1cancel" in permutations. This
# reduces the probability of the timing problem that the cancel error report
# is shown together with the next query instead of with the cancel query.
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
setup
{
CREATE TABLE d3_listp (a int) PARTITION BY LIST(a);
CREATE TABLE d3_listp1 PARTITION OF d3_listp FOR VALUES IN (1);
Allow a partdesc-omitting-partitions to be cached Makes partition descriptor acquisition faster during the transient period in which a partition is in the process of being detached. This also adds the restriction that only one partition can be in pending-detach state for a partitioned table. While at it, return find_inheritance_children() API to what it was before 71f4c8c6f74b, and create a separate find_inheritance_children_extended() that returns detailed info about detached partitions. (This incidentally fixes a bug in 8aba9322511 whereby a memory context holding a transient partdesc is reparented to a NULL PortalContext, leading to permanent leak of that memory. The fix is to no longer rely on reparenting contexts to PortalContext. Reported by Amit Langote.) Per gripe from Amit Langote Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
2021-04-28 21:44:35 +02:00
CREATE TABLE d3_listp2 PARTITION OF d3_listp FOR VALUES IN (2);
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
CREATE TABLE d3_pid (pid int);
INSERT INTO d3_listp VALUES (1);
}
teardown {
Allow a partdesc-omitting-partitions to be cached Makes partition descriptor acquisition faster during the transient period in which a partition is in the process of being detached. This also adds the restriction that only one partition can be in pending-detach state for a partitioned table. While at it, return find_inheritance_children() API to what it was before 71f4c8c6f74b, and create a separate find_inheritance_children_extended() that returns detailed info about detached partitions. (This incidentally fixes a bug in 8aba9322511 whereby a memory context holding a transient partdesc is reparented to a NULL PortalContext, leading to permanent leak of that memory. The fix is to no longer rely on reparenting contexts to PortalContext. Reported by Amit Langote.) Per gripe from Amit Langote Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
2021-04-28 21:44:35 +02:00
DROP TABLE IF EXISTS d3_listp, d3_listp1, d3_listp2, d3_pid;
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
}
session "s1"
step "s1b" { BEGIN; }
step "s1brr" { BEGIN ISOLATION LEVEL REPEATABLE READ; }
step "s1s" { SELECT * FROM d3_listp; }
step "s1spart" { SELECT * FROM d3_listp1; }
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
# Sleep 0.1s after sending cancel, to give s2 time to react
step "s1cancel" { SELECT pg_cancel_backend(pid), pg_sleep(0.1) FROM d3_pid; }
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
step "s1c" { COMMIT; }
step "s1alter" { ALTER TABLE d3_listp1 ALTER a DROP NOT NULL; }
step "s1insert" { INSERT INTO d3_listp VALUES (1); }
step "s1insertpart" { INSERT INTO d3_listp1 VALUES (1); }
step "s1drop" { DROP TABLE d3_listp; }
step "s1droppart" { DROP TABLE d3_listp1; }
step "s1trunc" { TRUNCATE TABLE d3_listp; }
step "s1list" { SELECT relname FROM pg_catalog.pg_class
WHERE relname LIKE 'd3_listp%' ORDER BY 1; }
step "s1describe" { SELECT 'd3_listp' AS root, * FROM pg_partition_tree('d3_listp')
UNION ALL SELECT 'd3_listp1', * FROM pg_partition_tree('d3_listp1'); }
session "s2"
step "s2begin" { BEGIN; }
step "s2snitch" { INSERT INTO d3_pid SELECT pg_backend_pid(); }
step "s2detach" { ALTER TABLE d3_listp DETACH PARTITION d3_listp1 CONCURRENTLY; }
Allow a partdesc-omitting-partitions to be cached Makes partition descriptor acquisition faster during the transient period in which a partition is in the process of being detached. This also adds the restriction that only one partition can be in pending-detach state for a partitioned table. While at it, return find_inheritance_children() API to what it was before 71f4c8c6f74b, and create a separate find_inheritance_children_extended() that returns detailed info about detached partitions. (This incidentally fixes a bug in 8aba9322511 whereby a memory context holding a transient partdesc is reparented to a NULL PortalContext, leading to permanent leak of that memory. The fix is to no longer rely on reparenting contexts to PortalContext. Reported by Amit Langote.) Per gripe from Amit Langote Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
2021-04-28 21:44:35 +02:00
step "s2detach2" { ALTER TABLE d3_listp DETACH PARTITION d3_listp2 CONCURRENTLY; }
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
step "s2noop" { UNLISTEN noop; }
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
step "s2detachfinal" { ALTER TABLE d3_listp DETACH PARTITION d3_listp1 FINALIZE; }
step "s2drop" { DROP TABLE d3_listp1; }
step "s2commit" { COMMIT; }
# Try various things while the partition is in "being detached" state, with
# no session waiting.
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1describe" "s1alter"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1insert" "s1c"
permutation "s2snitch" "s1brr" "s1s" "s2detach" "s1cancel" "s2noop" "s1insert" "s1c" "s1spart"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1insertpart"
Allow a partdesc-omitting-partitions to be cached Makes partition descriptor acquisition faster during the transient period in which a partition is in the process of being detached. This also adds the restriction that only one partition can be in pending-detach state for a partitioned table. While at it, return find_inheritance_children() API to what it was before 71f4c8c6f74b, and create a separate find_inheritance_children_extended() that returns detailed info about detached partitions. (This incidentally fixes a bug in 8aba9322511 whereby a memory context holding a transient partdesc is reparented to a NULL PortalContext, leading to permanent leak of that memory. The fix is to no longer rely on reparenting contexts to PortalContext. Reported by Amit Langote.) Per gripe from Amit Langote Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
2021-04-28 21:44:35 +02:00
# Test partition descriptor caching
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach2" "s1cancel" "s2noop" "s1c" "s1brr" "s1insert" "s1s" "s1insert" "s1c"
permutation "s2snitch" "s1b" "s1s" "s2detach2" "s1cancel" "s2noop" "s1c" "s1brr" "s1s" "s1insert" "s1s" "s1c"
Allow a partdesc-omitting-partitions to be cached Makes partition descriptor acquisition faster during the transient period in which a partition is in the process of being detached. This also adds the restriction that only one partition can be in pending-detach state for a partitioned table. While at it, return find_inheritance_children() API to what it was before 71f4c8c6f74b, and create a separate find_inheritance_children_extended() that returns detailed info about detached partitions. (This incidentally fixes a bug in 8aba9322511 whereby a memory context holding a transient partdesc is reparented to a NULL PortalContext, leading to permanent leak of that memory. The fix is to no longer rely on reparenting contexts to PortalContext. Reported by Amit Langote.) Per gripe from Amit Langote Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
2021-04-28 21:44:35 +02:00
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
# "drop" here does both tables
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1drop" "s1list"
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
# "truncate" only does parent, not partition
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1trunc" "s1spart"
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
Allow a partdesc-omitting-partitions to be cached Makes partition descriptor acquisition faster during the transient period in which a partition is in the process of being detached. This also adds the restriction that only one partition can be in pending-detach state for a partitioned table. While at it, return find_inheritance_children() API to what it was before 71f4c8c6f74b, and create a separate find_inheritance_children_extended() that returns detailed info about detached partitions. (This incidentally fixes a bug in 8aba9322511 whereby a memory context holding a transient partdesc is reparented to a NULL PortalContext, leading to permanent leak of that memory. The fix is to no longer rely on reparenting contexts to PortalContext. Reported by Amit Langote.) Per gripe from Amit Langote Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
2021-04-28 21:44:35 +02:00
# If a partition pending detach exists, we cannot drop another one
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s2detach2" "s1c"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s2detachfinal" "s1c" "s2detach2"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1droppart" "s2detach2"
Allow a partdesc-omitting-partitions to be cached Makes partition descriptor acquisition faster during the transient period in which a partition is in the process of being detached. This also adds the restriction that only one partition can be in pending-detach state for a partitioned table. While at it, return find_inheritance_children() API to what it was before 71f4c8c6f74b, and create a separate find_inheritance_children_extended() that returns detailed info about detached partitions. (This incidentally fixes a bug in 8aba9322511 whereby a memory context holding a transient partdesc is reparented to a NULL PortalContext, leading to permanent leak of that memory. The fix is to no longer rely on reparenting contexts to PortalContext. Reported by Amit Langote.) Per gripe from Amit Langote Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
2021-04-28 21:44:35 +02:00
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
# When a partition with incomplete detach is dropped, we grab lock on parent too.
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s2begin" "s2drop" "s1s" "s2commit"
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
# Partially detach, then select and try to complete the detach. Reading
# from partition blocks (AEL is required on partition); reading from parent
# does not block.
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1b" "s1spart" "s2detachfinal" "s1c"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1b" "s1s" "s2detachfinal" "s1c"
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY Allow a partition be detached from its partitioned table without blocking concurrent queries, by running in two transactions and only requiring ShareUpdateExclusive in the partitioned table. Because it runs in two transactions, it cannot be used in a transaction block. This is the main reason to use dedicated syntax: so that users can choose to use the original mode if they need it. But also, it doesn't work when a default partition exists (because an exclusive lock would still need to be obtained on it, in order to change its partition constraint.) In case the second transaction is cancelled or a crash occurs, there's ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final steps. The main trick to make this work is the addition of column pg_inherits.inhdetachpending, initially false; can only be set true in the first part of this command. Once that is committed, concurrent transactions that use a PartitionDirectory will include or ignore partitions so marked: in optimizer they are ignored if the row is marked committed for the snapshot; in executor they are always included. As a result, and because of the way PartitionDirectory caches partition descriptors, queries that were planned before the detach will see the rows in the detached partition and queries that are planned after the detach, won't. A CHECK constraint is created that duplicates the partition constraint. This is probably not strictly necessary, and some users will prefer to remove it afterwards, but if the partition is re-attached to a partitioned table, the constraint needn't be rechecked. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
2021-03-25 22:00:28 +01:00
# DETACH FINALIZE in a transaction block. No insert/select on the partition
# is allowed concurrently with that.
Make detach-partition-concurrently-3 less timing-sensitive This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com> Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
2021-05-25 18:53:29 +02:00
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s1b" "s1spart" "s2detachfinal" "s1c"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s2begin" "s2detachfinal" "s2commit"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s2begin" "s2detachfinal" "s1spart" "s2commit"
permutation "s2snitch" "s1b" "s1s" "s2detach" "s1cancel" "s2noop" "s1c" "s2begin" "s2detachfinal" "s1insertpart" "s2commit"