rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/test/regress/sql/create_index_spgist.sql

421 lines
16 KiB
MySQL
Raw Normal View History

Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
--
-- SP-GiST index tests
--
CREATE TABLE quad_point_tbl AS
SELECT point(unique1,unique2) AS p FROM tenk1;
INSERT INTO quad_point_tbl
SELECT '(333.0,400.0)'::point FROM generate_series(1,1000);
INSERT INTO quad_point_tbl VALUES (NULL), (NULL), (NULL);
CREATE INDEX sp_quad_ind ON quad_point_tbl USING spgist (p);
CREATE TABLE kd_point_tbl AS SELECT * FROM quad_point_tbl;
CREATE INDEX sp_kd_ind ON kd_point_tbl USING spgist (p kd_point_ops);
CREATE TABLE radix_text_tbl AS
SELECT name AS t FROM road WHERE name !~ '^[0-9]';
INSERT INTO radix_text_tbl
SELECT 'P0123456789abcdef' FROM generate_series(1,1000);
INSERT INTO radix_text_tbl VALUES ('P0123456789abcde');
INSERT INTO radix_text_tbl VALUES ('P0123456789abcdefF');
CREATE INDEX sp_radix_ind ON radix_text_tbl USING spgist (t);
-- get non-indexed results for comparison purposes
SET enable_seqscan = ON;
SET enable_indexscan = OFF;
SET enable_bitmapscan = OFF;
SELECT count(*) FROM quad_point_tbl WHERE p IS NULL;
SELECT count(*) FROM quad_point_tbl WHERE p IS NOT NULL;
SELECT count(*) FROM quad_point_tbl;
SELECT count(*) FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
SELECT count(*) FROM quad_point_tbl WHERE box '(200,200,1000,1000)' @> p;
SELECT count(*) FROM quad_point_tbl WHERE p << '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p >> '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p <^ '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p >^ '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p ~= '(4585, 365)';
CREATE TEMP TABLE quad_point_tbl_ord_seq1 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl;
CREATE TEMP TABLE quad_point_tbl_ord_seq2 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
CREATE TEMP TABLE quad_point_tbl_ord_seq3 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '333,400') n, p <-> '333,400' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl WHERE p IS NOT NULL;
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdef';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcde';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdefF';
SELECT count(*) FROM radix_text_tbl WHERE t < 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t ~<~ 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t <= 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t ~<=~ 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t = 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t = 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t >= 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t ~>=~ 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t > 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t ~>~ 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t ^@ 'Worth';
-- Now check the results from plain indexscan
SET enable_seqscan = OFF;
SET enable_indexscan = ON;
SET enable_bitmapscan = OFF;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p IS NULL;
SELECT count(*) FROM quad_point_tbl WHERE p IS NULL;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p IS NOT NULL;
SELECT count(*) FROM quad_point_tbl WHERE p IS NOT NULL;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl;
SELECT count(*) FROM quad_point_tbl;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
SELECT count(*) FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE box '(200,200,1000,1000)' @> p;
SELECT count(*) FROM quad_point_tbl WHERE box '(200,200,1000,1000)' @> p;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p << '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p << '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p >> '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p >> '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p <^ '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p <^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p >^ '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p >^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p ~= '(4585, 365)';
SELECT count(*) FROM quad_point_tbl WHERE p ~= '(4585, 365)';
EXPLAIN (COSTS OFF)
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl;
CREATE TEMP TABLE quad_point_tbl_ord_idx1 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl;
SELECT * FROM quad_point_tbl_ord_seq1 seq FULL JOIN quad_point_tbl_ord_idx1 idx
ON seq.n = idx.n
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
WHERE seq.dist IS DISTINCT FROM idx.dist;
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
EXPLAIN (COSTS OFF)
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
CREATE TEMP TABLE quad_point_tbl_ord_idx2 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
SELECT * FROM quad_point_tbl_ord_seq2 seq FULL JOIN quad_point_tbl_ord_idx2 idx
ON seq.n = idx.n
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
WHERE seq.dist IS DISTINCT FROM idx.dist;
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
EXPLAIN (COSTS OFF)
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '333,400') n, p <-> '333,400' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl WHERE p IS NOT NULL;
CREATE TEMP TABLE quad_point_tbl_ord_idx3 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '333,400') n, p <-> '333,400' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM quad_point_tbl WHERE p IS NOT NULL;
SELECT * FROM quad_point_tbl_ord_seq3 seq FULL JOIN quad_point_tbl_ord_idx3 idx
ON seq.n = idx.n
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
WHERE seq.dist IS DISTINCT FROM idx.dist;
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p <@ box '(200,200,1000,1000)';
SELECT count(*) FROM kd_point_tbl WHERE p <@ box '(200,200,1000,1000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE box '(200,200,1000,1000)' @> p;
SELECT count(*) FROM kd_point_tbl WHERE box '(200,200,1000,1000)' @> p;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p << '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p << '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p >> '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p >> '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p <^ '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p <^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p >^ '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p >^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p ~= '(4585, 365)';
SELECT count(*) FROM kd_point_tbl WHERE p ~= '(4585, 365)';
EXPLAIN (COSTS OFF)
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM kd_point_tbl;
CREATE TEMP TABLE kd_point_tbl_ord_idx1 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM kd_point_tbl;
SELECT * FROM quad_point_tbl_ord_seq1 seq FULL JOIN kd_point_tbl_ord_idx1 idx
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
ON seq.n = idx.n
WHERE seq.dist IS DISTINCT FROM idx.dist;
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
EXPLAIN (COSTS OFF)
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM kd_point_tbl WHERE p <@ box '(200,200,1000,1000)';
CREATE TEMP TABLE kd_point_tbl_ord_idx2 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '0,0') n, p <-> '0,0' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM kd_point_tbl WHERE p <@ box '(200,200,1000,1000)';
SELECT * FROM quad_point_tbl_ord_seq2 seq FULL JOIN kd_point_tbl_ord_idx2 idx
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
ON seq.n = idx.n
WHERE seq.dist IS DISTINCT FROM idx.dist;
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
EXPLAIN (COSTS OFF)
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '333,400') n, p <-> '333,400' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM kd_point_tbl WHERE p IS NOT NULL;
CREATE TEMP TABLE kd_point_tbl_ord_idx3 AS
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
SELECT row_number() OVER (ORDER BY p <-> '333,400') n, p <-> '333,400' dist, p
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
FROM kd_point_tbl WHERE p IS NOT NULL;
SELECT * FROM quad_point_tbl_ord_seq3 seq FULL JOIN kd_point_tbl_ord_idx3 idx
Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 23:01:35 +02:00
ON seq.n = idx.n
WHERE seq.dist IS DISTINCT FROM idx.dist;
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
Provide stable test for NULL-values in KNN SP-GiST f5f084fc3e has removed test because of its instability. This commit provides alternative test with determined ordering using extra ORDER BY expression. Backpatch-through: 12
2019-09-20 14:31:12 +02:00
-- check ORDER BY distance to NULL
SELECT (SELECT p FROM kd_point_tbl ORDER BY p <-> pt, p <-> '0,0' LIMIT 1)
FROM (VALUES (point '1,2'), (NULL), ('1234,5678')) pts(pt);
Improve handling of NULLs in KNN-GiST and KNN-SP-GiST This commit improves subject in two ways: * It removes ugliness of 02f90879e7, which stores distance values and null flags in two separate arrays after GISTSearchItem struct. Instead we pack both distance value and null flag in IndexOrderByDistance struct. Alignment overhead should be negligible, because we typically deal with at most few "col op const" expressions in ORDER BY clause. * It fixes handling of "col op NULL" expression in KNN-SP-GiST. Now, these expression are not passed to support functions, which can't deal with them. Instead, NULL result is implicitly assumed. It future we may decide to teach support functions to deal with NULL arguments, but current solution is bugfix suitable for backpatch. Reported-by: Nikita Glukhov Discussion: https://postgr.es/m/826f57ee-afc7-8977-c44c-6111d18b02ec%40postgrespro.ru Author: Nikita Glukhov Reviewed-by: Alexander Korotkov Backpatch-through: 9.4
2019-09-19 20:30:19 +02:00
Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
2019-04-11 22:15:54 +02:00
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdef';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdef';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcde';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcde';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdefF';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdefF';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t < 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t < 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~<~ 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t ~<~ 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t <= 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t <= 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~<=~ 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t ~<=~ 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t = 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t = 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t >= 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t >= 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~>=~ 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t ~>=~ 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t > 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t > 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~>~ 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t ~>~ 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ^@ 'Worth';
SELECT count(*) FROM radix_text_tbl WHERE t ^@ 'Worth';
-- Now check the results from bitmap indexscan
SET enable_seqscan = OFF;
SET enable_indexscan = OFF;
SET enable_bitmapscan = ON;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p IS NULL;
SELECT count(*) FROM quad_point_tbl WHERE p IS NULL;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p IS NOT NULL;
SELECT count(*) FROM quad_point_tbl WHERE p IS NOT NULL;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl;
SELECT count(*) FROM quad_point_tbl;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
SELECT count(*) FROM quad_point_tbl WHERE p <@ box '(200,200,1000,1000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE box '(200,200,1000,1000)' @> p;
SELECT count(*) FROM quad_point_tbl WHERE box '(200,200,1000,1000)' @> p;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p << '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p << '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p >> '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p >> '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p <^ '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p <^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p >^ '(5000, 4000)';
SELECT count(*) FROM quad_point_tbl WHERE p >^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM quad_point_tbl WHERE p ~= '(4585, 365)';
SELECT count(*) FROM quad_point_tbl WHERE p ~= '(4585, 365)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p <@ box '(200,200,1000,1000)';
SELECT count(*) FROM kd_point_tbl WHERE p <@ box '(200,200,1000,1000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE box '(200,200,1000,1000)' @> p;
SELECT count(*) FROM kd_point_tbl WHERE box '(200,200,1000,1000)' @> p;
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p << '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p << '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p >> '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p >> '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p <^ '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p <^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p >^ '(5000, 4000)';
SELECT count(*) FROM kd_point_tbl WHERE p >^ '(5000, 4000)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM kd_point_tbl WHERE p ~= '(4585, 365)';
SELECT count(*) FROM kd_point_tbl WHERE p ~= '(4585, 365)';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdef';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdef';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcde';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcde';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdefF';
SELECT count(*) FROM radix_text_tbl WHERE t = 'P0123456789abcdefF';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t < 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t < 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~<~ 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t ~<~ 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t <= 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t <= 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~<=~ 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t ~<=~ 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'Aztec Ct ';
SELECT count(*) FROM radix_text_tbl WHERE t = 'Aztec Ct ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t = 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t = 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t >= 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t >= 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~>=~ 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t ~>=~ 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t > 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t > 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ~>~ 'Worth St ';
SELECT count(*) FROM radix_text_tbl WHERE t ~>~ 'Worth St ';
EXPLAIN (COSTS OFF)
SELECT count(*) FROM radix_text_tbl WHERE t ^@ 'Worth';
SELECT count(*) FROM radix_text_tbl WHERE t ^@ 'Worth';
RESET enable_seqscan;
RESET enable_indexscan;
RESET enable_bitmapscan;